Handling 429 Too Many Requests as a client: backoff, jitter, retry budgets

The advice on handling 429 Too Many Requests is always the same. You read Retry-After, you back off exponentially, you add jitter, you cap the attempts. Every article says it, this one included. It is correct as far as it goes.

What none of them tell you is what it buys. So I built a small harness: a real HTTP server running a real sliding-window limiter, a few hundred real clients, and six different retry policies pointed at it. Two of the results genuinely surprised me, and one of them contradicts the advice above.

The short version: jitter does not make your requests faster, it makes them fewer, and it makes your slowest client slower. And retrying harder against a saturated backend does not get more work done at all. Not "a bit less than you'd hope." None.

This is the client-side half of a pair. The other half, your 429 response is probably wrong, is about what to send. This one is about what to do when you receive one.

What to do with a 429, in order

Four of those five steps are decisions, not code. Most retry bugs I have chased were a missing decision rather than a wrong formula.

Step 1: read Retry-After properly, because it has two shapes

Here is the part that bites people. RFC 9110 §10.2.3 defines the field like this:

Retry-After = HTTP-date / delay-seconds

delay-seconds = 1*DIGIT

Two forms. Both legal. The spec's own examples are Retry-After: 120 and Retry-After: Fri, 31 Dec 1999 23:59:59 GMT. A parser that does parseInt(header, 10) and moves on will read that date as NaN, or worse, read a header beginning with a digit as a number of seconds when it was a date.

Two more details worth knowing. First, delay-seconds is an integer, so a server that wants you to wait 300ms has to round. Round down and it sends 0, which is a hot loop with extra steps; most servers floor at 1 instead. Second, RFC 9110 describes Retry-After in terms of 503 and 3xx responses. The 429 code comes from RFC 6585 §4, which says a 429 "MAY include a Retry-After header." May. Plan for its absence.

The header that means the opposite of what you think

The nastier version of this problem is not Retry-After at all, it is the reset header sitting next to it. @fastify/rate-limit sends x-ratelimit-reset as seconds remaining. GitHub publishes its equivalent as a Unix epoch second. Same header name, opposite units.

Feed GitHub's value into a client written for Fastify's convention and you sleep until the year 3000. Feed Fastify's value into a client written for GitHub's and you compute a reset date in January 1970, conclude the window already expired, and retry immediately. Forever.

So: prefer Retry-After, treat reset headers as a hint, and never let a parse failure produce a zero delay.

const MAX_WAIT_MS = 5 * 60_000;

/** Returns ms to wait, or null if the server didn't say. */
function parseRetryAfter(header: string | null, now = Date.now()): number | null {
  if (!header) return null;
  const raw = header.trim();

  // delay-seconds: 1*DIGIT, and nothing else.
  if (/^\d+$/.test(raw)) {
    return Math.min(Number(raw) * 1000, MAX_WAIT_MS);
  }

  // HTTP-date. Date.parse handles IMF-fixdate; clock skew is on us.
  const at = Date.parse(raw);
  if (Number.isNaN(at)) return null;          // unparseable: fall back to backoff
  return Math.min(Math.max(0, at - now), MAX_WAIT_MS);
}

parseRetryAfter("120");                                  // 120000
parseRetryAfter("Fri, 31 Dec 1999 23:59:59 GMT");        // 0   (already past)
parseRetryAfter("2 minutes");                            // null
parseRetryAfter("999999999");                            // 300000 (clamped)

The clamp matters more than it looks. A misconfigured upstream once handed us a Retry-After in the hundreds of thousands of seconds. Without a ceiling, a worker sleeps for a day and you find out on Monday. Ask me how I know.

The HTTP-date form has a second failure mode that no amount of parsing fixes: it is computed against the server's clock and evaluated against yours. If your container's clock is 30 seconds fast, you retry 30 seconds early, every time. That is a good reason to prefer the integer form when you are the one sending it, which is the argument the provider-side post makes at length.

Step 2: backoff and jitter, and what they actually cost

The canonical formulas come from Marc Brooker's Exponential Backoff And Jitter (AWS Architecture Blog, March 2015), which is still the best thing written on the subject:

Strategy	Next delay
No jitter	min(cap, base * 2^attempt)
Full jitter	random(0, min(cap, base * 2^attempt))
Equal jitter	b/2 + random(0, b/2) where b = base * 2^attempt
Decorrelated jitter	min(cap, random(base, prev * 3))

Brooker's simulation found full jitter did less total work than the alternatives, and decorrelated jitter finished slightly sooner while sending slightly more. That was measured on contended writes, though, not on a rate limiter that is deliberately refusing you. So I measured the rate-limiter case.

The harness

A Node HTTP server with a sliding-window log limiter: 20 requests per second, 429 plus a correct integer Retry-After on rejection. Against it, 150 concurrent clients, each needing 2 successful requests, so 300 units of real work. Seeded RNG, so the run repeats. I tracked requests sent, per-job latency percentiles, and peak arrivals in any 50ms bucket after the cold start, which is the number that tells you whether clients are moving in a herd.

Client strategy	Requests sent	Per success	p50	p95	Peak arrivals / 50ms
Retry-After, honored exactly	1,511	5.0	2,041ms	13,121ms	140
Retry-After as a floor, plus full jitter	1,099	3.7	1,035ms	15,256ms	73
Full jitter, Retry-After ignored	1,388	4.6	284ms	14,590ms	38

Figures are from one run; repeating it moved totals by a few percent without changing the ordering.

Read the first and second rows together, because that is the whole argument. Adding jitter on top of Retry-After cut requests sent by 27% and cut the peak burst roughly in half. The server had a materially easier time.

And the p95 got worse. 13.1 seconds to 15.3 seconds.

Why jitter makes your unluckiest client slower

This is the part I had wrong going in, and it is not a quirk of my harness. It falls out of what jitter is.

Honoring Retry-After exactly is fair. Every rejected client is told the same thing, waits the same interval, and comes back together. Painful for the server, but nobody is singled out: the wait is uniform, so the tail is tight. Notice the exact-honoring row has a p95 of 13.1s and a max, in the raw data, of 13.13s. Almost no spread.

Jitter is unfair by construction. It replaces one shared wait with a random draw per client. Most clients draw short and get through early, which is why p50 halves. But a client can draw long repeatedly, and each long draw pushes it behind the clients that drew short. Randomness has no memory of who has already been waiting.

So the honest framing, which I have not seen written down anywhere: jitter is a transfer. You trade tail latency for a smoother arrival curve and fewer wasted calls. If you are a background job, take that trade every time. If a human is watching a spinner, know that you just made the worst case worse, and consider capping the jitter range rather than using the full 2^n window.

Should I honor Retry-After or use my own backoff?

Both, in that order. Treat Retry-After as a floor and add jitter on top:

function nextDelay(attempt: number, retryAfterMs: number | null): number {
  const base = 100, cap = 8_000;
  const window = Math.min(cap, base * 2 ** (attempt - 1));
  return (retryAfterMs ?? 0) + Math.random() * window;   // floor + full jitter
}

Retrying before the floor is just a guaranteed 429, and on some providers it extends the penalty. Retrying at exactly the floor puts you in the herd. The floor tells you when it becomes possible to succeed; the jitter decides where in the queue you stand. That combination was the cheapest of everything I measured, at 3.7 requests per unit of work.

Step 3: retry budgets, or why your retry loop is not your decision alone

Everything above is per-request. A per-request policy cannot see the thing that actually causes outages, which is every request retrying at once.

Capped attempts do not solve this, and the arithmetic is why. Google's SRE book points out that retries compound through layers: if three services in a call chain each retry three times, one user request can become 27 backend calls. Your "max 3 attempts" is multiplied by everyone else's.

The fix the big infrastructure projects converged on is a retry budget: a cap on retries expressed as a fraction of your normal traffic, shared across all requests, rather than a count per request.

Implementation	Budget	Floor	Notes
Finagle RetryBudget	20% of requests (percentCanRetry = 0.2)	minRetriesPerSec = 10	Token bucket, credits expire after a 10s ttl
Envoy RetryBudget	budget_percent defaults to 20% of active requests	min_retry_concurrency defaults to 3	Circuit-breaker threshold on concurrent retries
Google SRE	Retries kept below 10% of requests	Per-request cap of 3 attempts	Layered: a per-request cap and a per-client ratio

Note the shape they all share: a percentage, plus a small absolute floor so a low-traffic client can still retry at all. Note also that they cluster around 10-20%. If your retries exceed a fifth of your traffic, the industry consensus is that you are the problem.

What a budget is worth, measured

Second harness. A backend that serves 25 requests per second and nothing more. 120 clients each offering a unit of work every 120ms for 10 seconds, which is roughly 40x the capacity available. This is the bad afternoon: demand far above what the backend can absorb, and no amount of client cleverness changes that ceiling.

I measured amplification, meaning HTTP requests the backend had to handle per unit of work offered, against goodput, meaning units actually completed per second.

Client policy	Requests sent	Amplification	Completed	Goodput
No retries at all	9,960	1.00x	250	24.8/s
Backoff + jitter, 3 attempts max	29,721	2.98x	275	27.0/s
Backoff + jitter, retry until success	82,956	8.28x	350	25.0/s
Backoff + jitter + 10% retry budget	10,015	1.01x	250	24.8/s
Backoff + jitter + adaptive throttling	2,554	0.26x	250	24.7/s

Look at the goodput column. It never moves. Every policy completed between 24.7 and 27.0 units per second against a backend rated for 25.

Now look at the requests column. The unbounded retry policy sent 83,000 requests where the no-retry policy sent 10,000, and finished the same amount of work. Eight times the load for nothing. Those extra 73,000 requests were not wasted in the sense of being slightly inefficient. They were wasted in the sense of consuming connections, CPU and log volume on a server that was already underwater, in order to accomplish nothing at all.

That is the bold claim I will defend: against a capacity-bound backend, retry aggressiveness has no effect on throughput. It only decides how much damage you do on the way to the same number. The ceiling is the server's, not yours, and no client-side policy can raise it.

The last row is the interesting one. Adaptive throttling, from the Google SRE book, has each client track its own recent accept rate and reject outbound requests locally with probability:

max(0, (requests - K * accepts) / (requests + 1))

with K = 2 recommended, meaning a client starts self-limiting once it is sending twice what is getting accepted. It sent a quarter of the requests the no-retry policy sent and completed exactly the same work, because it stopped putting doomed requests on the wire in the first place.

A retry budget in about twenty lines

class RetryBudget {
  private deposits: number[] = [];
  private spent = 0;

  constructor(
    private percentCanRetry = 0.1,   // retries as a fraction of successes
    private minRetriesPerSec = 3,    // floor, so low traffic can still retry
    private ttlMs = 10_000,
  ) {}

  /** Call on every successful response. */
  deposit(): void {
    this.deposits.push(Date.now());
  }

  private balance(): number {
    const cutoff = Date.now() - this.ttlMs;
    const before = this.deposits.length;
    this.deposits = this.deposits.filter((t) => t > cutoff);
    this.spent = Math.max(0, this.spent - (before - this.deposits.length) * this.percentCanRetry);
    return this.deposits.length * this.percentCanRetry
      + (this.minRetriesPerSec * this.ttlMs) / 1000
      - this.spent;
  }

  /** Returns false when you've spent your share. Do not retry. */
  tryWithdraw(): boolean {
    if (this.balance() < 1) return false;
    this.spent += 1;
    return true;
  }
}

One budget per upstream, shared by every request to it. The floor is what keeps a service that only handles a few requests per minute from being unable to retry anything, which is the failure mode you hit if you implement the percentage alone.

Does a retry budget mean I drop work I could have completed?

Sometimes, yes, and you should decide that consciously rather than discover it.

A budget is tuned for the case where the backend is genuinely failing, and there it costs nothing, as the table shows. But a 429 is not a failure, it is a scheduling instruction, and waiting really does work. In an earlier version of the first experiment I put a tight budget on a workload that was merely rate limited, and it refused retries for work that would have succeeded a second later.

The practical rule I would give: run the budget against the error classes (5xx, timeouts, connection failures) where retrying is a gamble, and let 429s with a valid Retry-After spend from a separate, more generous allowance. The server told you when to come back. That is information a 500 never gives you.

When not to retry at all

Backoff is the answer to "when should I retry." The prior question is whether you should retry at all, and for a large fraction of failures the answer is no.

The status code is not enough on its own

429 means "too fast, come back later" and is retryable. But two responses that look adjacent are not:

• 402 Payment Required means you are out of quota. Retrying does not help until something refills or somebody pays. Backing off exponentially against a 402 is a loop that ends when your budget does.
• 403 Forbidden means the credential is disabled or lacks access. That is a state change on the server, not a timing problem. Retrying is pure noise.
• 401 is worth exactly one retry, after refreshing the credential, and never more.

This distinction is why ReqKey's error documentation says a 500 is safe to retry with backoff while 402 and 403 should be surfaced to your customer, because "retrying won't help until the underlying issue is fixed." A client that treats every non-2xx as retryable will hammer an upstream over a billing problem it cannot fix. We wrote about the quota-versus-speed distinction in more depth in what are API credits.

The response body can override the status

Some APIs, ReqKey among them, return 200 with a negative decision in the body rather than an error status, because the call succeeded even though the answer was no. POST /key/validate returns 200 with {"valid": false, "message": "Key not found"} for an unknown key. A retry loop branching on res.ok alone will treat that as success, and a loop branching on status alone will never see it. Branch on the decision field.

Non-idempotent requests

A retried GET is free. A retried POST that charges a card is a support ticket. If the first request timed out you do not know whether it was received, and "the response never arrived" is indistinguishable from "the request never landed."

Retry those only behind an idempotency key: a client-generated identifier the server uses to deduplicate within a window. That is a whole topic on its own, and it interacts badly with usage metering, which we picked apart in your API credit system bills before it knows the request worked.

When the deadline has already passed

The retry nobody thinks about. If the caller waiting on you has a 5 second timeout and you have burned 4.8 seconds across two attempts, the third attempt cannot possibly help. Its result arrives after everyone has given up, but it still costs the backend a full request. Pass a deadline down and check it before sleeping, not just an attempt count.

if (Date.now() + delay > deadline) throw new DeadlineExceeded();
await sleep(delay);

Putting it together: handling 429 Too Many Requests end to end

Every piece above, in one loop. This is close to what I would actually ship.

const RETRYABLE = new Set([429, 500, 502, 503, 504]);

async function requestWithRetry(
  url: string,
  init: RequestInit,
  opts: { budget: RetryBudget; deadline: number; maxAttempts?: number },
): Promise<Response> {
  const { budget, deadline, maxAttempts = 5 } = opts;
  let lastError: unknown;

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    let res: Response;
    try {
      res = await fetch(url, { ...init, signal: AbortSignal.timeout(deadline - Date.now()) });
    } catch (err) {
      lastError = err;                       // network error or timeout: retryable
      if (attempt === maxAttempts) throw err;
      if (!budget.tryWithdraw()) throw new Error("retry budget exhausted");
      await sleep(backoff(attempt, null, deadline));
      continue;
    }

    if (res.ok) { budget.deposit(); return res; }
    if (!RETRYABLE.has(res.status)) return res;   // 402/403/400: caller's problem, not a timing one
    if (attempt === maxAttempts) return res;

    // 429s with a server-supplied floor don't spend from the error budget.
    const retryAfterMs = parseRetryAfter(res.headers.get("retry-after"));
    if (res.status !== 429 && !budget.tryWithdraw()) return res;

    const delay = backoff(attempt, retryAfterMs, deadline);
    if (delay === null) return res;               // wouldn't finish before the deadline
    await sleep(delay);
  }

  throw lastError ?? new Error("exhausted retries");
}

/** Retry-After as a floor, full jitter on top, deadline-aware. */
function backoff(attempt: number, retryAfterMs: number | null, deadline: number): number | null {
  const window = Math.min(8_000, 100 * 2 ** (attempt - 1));
  const delay = (retryAfterMs ?? 0) + Math.random() * window;
  return Date.now() + delay > deadline ? null : delay;
}

Five decisions in forty lines: is it retryable, did the server tell me when, where in the queue do I stand, can I afford it, and will the answer still matter when it arrives.

If you are calling an API through an SDK, check what it already does

Before you write any of this, find out whether your client library got there first, because two layers of retries is the 27x problem in miniature. ReqKey's own SDKs normalize the part people get wrong: the Node and Python clients both read retryAfter from the response body and fall back to the Retry-After header when the body omits it, so you get one number regardless of which the server sent.

import { ReqKey } from "reqkey";

const client = new ReqKey({ rootKey: process.env.REQKEY_ROOT_KEY! });
const result = await client.verify(userKey, { credits: 1 });

if (!result.valid) {
  switch (result.reason) {
    case "rate_limited":
      // result.retryAfter is already normalized from body or header
      return tooManyRequests({ retryAfter: result.retryAfter });
    case "insufficient_credits":
      return paymentRequired();          // do NOT retry: quota, not timing
    case "invalid_key":
    case "forbidden":
      return unauthorized();             // do NOT retry: state, not timing
  }
}

The reason field exists precisely so the retry decision does not have to be reverse-engineered from a status code. Its values are valid, invalid_key, insufficient_credits, forbidden, rate_limited and denied. Exactly one of those is worth waiting on.

The other side of the connection

Everything here assumes the server is telling you the truth. Plenty do not: they send a 429 with no Retry-After, or a reset header in ambiguous units, or a 403 where they mean 429. Your client then has to guess, and guessing is what produces the 8x amplification above.

If you also run an API, that cuts both ways. The reason ReqKey returns Retry-After alongside X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Window on a rate-limited validation is that a client which knows when to come back is a client that stops guessing. A rate-limited request also consumes no credits and no rate-limit quota, so a customer who backs off correctly is not additionally punished for having been throttled. That is the behavior that makes well-behaved clients possible, and it is worth checking whether your own limiter offers it. If it does not, what your limiter does when Redis dies is probably also worth an afternoon.

Key takeaways

• Parse Retry-After for both legal forms, and clamp it. It is HTTP-date or an integer number of seconds, and a bare parseInt silently mishandles half of them. An unclamped value from a misconfigured upstream will park a worker for a day.
• Treat Retry-After as a floor and put jitter on top. That combination was the cheapest thing I measured, at 3.7 requests per unit of work against 5.0 for honoring the header exactly, and it halved the peak arrival burst.
• Know that jitter is a transfer, not a free win. It cut requests by 27% and made p95 worse. Take that trade for background work; think twice when a human is waiting.
• Against a saturated backend, retrying harder completes no additional work. Goodput stayed at the server's ceiling whether clients sent 1.00x or 8.28x the offered load. The extra 8x is pure damage.
• Add a retry budget of 10-20% of traffic with a small absolute floor. Finagle, Envoy and Google all landed in that range independently. A per-request attempt cap cannot see the herd; a budget can.
• Decide what is retryable before you decide when. 402 and 403 are not timing problems, non-idempotent writes need an idempotency key, and a request that cannot beat its deadline should never leave the process.

Try it against a real limiter

Retry code is hard to trust until you have watched it get refused. If you want a rate limiter to point this at, ReqKey's free tier is $0/month and includes 100,000 requests, which is more than enough to run the harness above against real 429s with real Retry-After headers and see how your client behaves. Set a consumer's rateLimit deliberately low, run your retry loop at it, and count the requests you send per unit of work. If that number is above about four, your backoff has a bug in it.

The docs cover the response shapes, and the product page covers what sits behind them.