Auto-retry for API calls

Many functions automatically retry failed API calls before giving up:

• The HTTP functions
• The REST functions
• The Xurrent REST API functions
• The Xurrent Graphql API functions

This is useful for transient network errors, rate limiting, and so on. Auto-retry works for all accounts.

Retrying is done according to the following rules:

• Rate limit
  When the retry-after header is present in the response, retry after the specified delay.
• GraphQL cost limit
  When the response indicates that the query cost exceeds the remaining budget, retry after the delay specified in the response body.

A rate limit applies to all calls using the same credentials, so they all wait until it has passed. The account usage shows how often limits were hit and how long calls had to wait.

A call is not retried when the requested delay is longer than 1 hour, or when it has been rate limited for more than 1 hour.

• AWS WAF block
  When the call is blocked or challenged by AWS Web Application Firewall (which protects e.g. Xurrent), retry up to 3 times, waiting 10, 20 and 40 minutes respectively.
• Connection error
  Retry based on error type:
  • ETIMEDOUT (1 retry)
  • ECONNREFUSED / ECONNRESET / ENOTFOUND (5 retries)
  • Other (10 retries)
• Transient error response
  When the response has no retry-after header, the following status codes are retried up to 2 times, 2 seconds apart:
  • 408, 425 and 429
  • All 5xx, except 501 and 505