Skip to main content

Investigate rate limit hits

When an integration reaches a limit of an external system, the automator waits until the limit has passed and then continues (see Auto-retry for API calls). The integration keeps working, but it slows down, and other integrations that use the same credentials have to wait as well. The package log shows a warning each time a call is paused by a rate limit.

This guide shows how to use the account usage to find out why a limit is reached, and what to do about it.

1. Find the account​

Select View account usage in the actions menu in the header, and select the environment and the period to investigate. Select 30d to see whether the limit is reached regularly.

On the Accounts tab, sort the table by Limit hits or Waiting on rate limits, and select the account at the top. This opens the Usage tab of the account.

2. Find the credentials and the limit​

On the Usage tab, look for the section of the credentials with limit hits. The Max limit used tile shows which limit was reached. For Xurrent, the request limit and the GraphQL query cost limit are independent of each other, and they each call for a different solution, so check which one is reached.

If the section mentions that other accounts use these credentials too, the calls of those accounts count towards the same limit. Their usage is included in the section, so the cause can also be found in another environment or profile.

3. Find out when the limit is reached​

The chart of the limit shows how the share of the limit that was used developed over time, and the heatmap below shows when the limit is reached during the week. Select % of limit in the heatmap to see the highest share used in each hour.

The pattern tells what kind of problem it is:

  • The limit is reached at the same time on every day, or on specific days.
    This is usually caused by scheduled work, such as a nightly import or a weekly synchronization. Zoom in on one of the peaks to see which packages are active at that moment.
  • The limit is reached during office hours on every weekday.
    This means the integrations are consistently running at a high load, which usually follows the activity of the users of the external system.
  • The limit is reached once, or only on a few occasions.
    This is usually caused by a one-off event, such as a bulk update in the external system that triggered many webhook events, or an import of an unusually large file. This may be acceptable, since the automator waits until the limit has passed.

4. Find the package and the function​

Below the heatmap, the Packages in this period table shows which packages called the account. When you are zoomed in on a peak, the table only shows the calls in that part of the period. Sort it by Calls, Requests or Points, depending on the limit that was reached, and select the package at the top.

On the Usage tab of the package, the Functions in this period table shows which functions made the calls. This narrows the search down to the part of the script that's responsible.

5. Take action​

Depending on what you found, there are several options:

  • Make fewer calls.
    Look for calls inside loops, for example a fetch per record, where a single fetchFilter or fetchAll could retrieve all records at once. Avoid updating records that haven't changed, and use database snapshots for data that rarely changes.
  • Watch out for unexpected auto-fetches. A package that loops over a long list of people and auto-fetches the manager and organization of each one quickly adds up. Such loops can often be avoided by using the GraphQL API instead.
  • Reduce the cost of GraphQL queries.
    Request only the fields you need, and reduce the size of the connections in the query.
  • Spread out scheduled work.
    If several schedules run at the same moment, start them at different times. For a large import, consider processing it in smaller parts.
  • Add additional credentials.
    If the integrations are consistently running at a high load, add an additional set of credentials to the account. The automator distributes the calls over all credentials of an account, so each token adds to the available capacity.