Backoff and jitter
Retries add load during an outage. If 1,000 callers/s all retry at the same time, the recovering service overloads again. Backoff with jitter spreads the retries over time, so the service can recover.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Immediate retries | Exponential backoff with jitter |
|---|---|---|---|
| Error rate | under 0.1% | 15% Fail | 0% Pass |
| p99 latency | under 5 s | 7.9 s Fail | 4 s Pass |
| Monthly cost | 4 USD/mo | 4 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Default wait before the first retry | 100 ms |
| Default jitter | 0.2 ratio |
| Default call timeout | 10,000 ms |
3 retries per call · no jitter · peak 6.0× normal load
How it works
A retry is a second chance for a request. It works well when a failure is brief. But a retry is also more load, and it arrives when the service is weak. If the service is down, every caller retries, and the retry rate can be a multiple of the normal rate.
In the mini board 1,000 callers each second use a one-core service. Its link fails from 0:02 to 0:04. With plain retries, all callers retry at once. When the link returns, the service sees a wall of retries on top of the new requests. It overloads a second time. This is a retry storm.
Backoff makes each retry wait longer than the one before it, for example 750 ms, then 1.5 s, then 3 s, up to a cap. Fewer retries arrive in each moment. Jitter adds a random part to each wait, so callers do not move together. Without jitter, callers that failed at the same instant retry at the same instant.
With backoff and jitter, the retries spread over seconds, and the service recovers. At least 99.9 percent of requests finish, and no second wave appears.
Set a limit on tries and a timeout for each try. Retry only requests that are safe to repeat.
In the levels you set the backoff and the jitter and watch the load during recovery.