Retries are the most popular form of optimism in distributed systems. The code is three lines, the intent is good, and in development it works. The trouble starts when the thing being retried is not a single request but a shared dependency that is already struggling.
Amplification
Suppose a service handles 1,000 requests per second and one dependency starts failing 10% of the time. With three attempts per request, the failing dependency now receives roughly 1,300 requests per second instead of 1,000 — and the extra 300 arrive precisely when it can least afford them.
Worse, the retries are not evenly spread. Most clients implement the same interval, often a flat one second. The first wave fails together, the second wave departs together, and the dependency sees a sawtooth of load instead of a curve it could absorb.
A retry policy that ignores time is just a traffic amplifier with extra steps.
Backoff is not enough on its own
Exponential backoff reduces the frequency of waves, but if every client computes the same delay, the waves stay synchronized. Add jitter so the attempts spread out:
delay = random(0, min(cap, base × 2^attempt))
base = 100 ms, cap = 5 s
Full jitter — a uniform random value between zero and the exponential ceiling — is usually the simplest policy that behaves well under load. It gives up a little determinism and buys a lot of smoothness.
Budgets, not infinite patience
Per-request retry limits are necessary but not sufficient, because they scale with traffic. A retry budget caps retries relative to successful requests, so the total amplification stays bounded even when the error rate rises.
- Allow retries only for idempotent operations, or require an idempotency key.
- Propagate deadlines. A retry that cannot finish before the caller's deadline is wasted work.
- Honor
Retry-Afterwhen the server sends it; the server knows more than the client. - Do not retry the errors that will never succeed: malformed requests, authentication failures, and most 4xx responses.
The server's half of the contract
Clients retry because failure is ambiguous. A server can make it less ambiguous: fail fast with a clear status, return 503 with a realistic Retry-After, and shed load before the queue turns into a timeout farm. A dependency that protects itself is easier to retry against than one that quietly accepts everything.
The goal is not to eliminate retries. It is to make the total load they create predictable, so a partial failure stays partial.