Auto-retry for API calls

Many functions automatically retry failed API calls before giving up:

• The HTTP functions
• The REST functions
• The Xurrent REST API functions
• The Xurrent Graphql API functions

This is useful for transient network errors, rate limiting, and so on. Auto-retry works for all accounts.

Retrying is done according to the following rules:

• Rate limit
  When the retry-after header is present in the response, retry after the specified delay.
• GraphQL cost limit
  When the response indicates that the query cost exceeds the remaining budget, retry after the delay specified in the response body.

A rate limit applies to all calls using the same credentials, so they all wait until it has passed.

A call is not retried when the requested delay is longer than 1 hour, or when it has been rate limited for more than 1 hour.

• AWS WAF block
  When the call is blocked or challenged by AWS Web Application Firewall (which protects e.g. Xurrent), retry up to 3 times, waiting 10, 20 and 40 minutes respectively.
• Connection error
  Retry based on error type:
  • ETIMEDOUT (1 retry)
  • ECONNREFUSED / ECONNRESET / ENOTFOUND (5 retries)
  • Other (10 retries)
• Transient error response
  When the response has no retry-after header, the following status codes are retried up to 2 times, 2 seconds apart:
  • 408, 425 and 429
  • All 5xx, except 501 and 505