Deadlines and timeouts
A timeout stops the wait for a call when a set time passes, and a deadline is the time budget for the whole request. The default timeout here is 10,000 ms, but users stop waiting much sooner. A timeout a little above the normal p99 frees a worker fast and keeps healthy calls.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Timeout of 10,000 ms on the slow call | Timeout of 100 ms on the slow call |
|---|---|---|---|
| p99 latency | under 350 ms | 2 s Fail | 102 ms Pass |
| Availability | at least 85% | 100% Pass | 90% Pass |
| Monthly cost | 8 USD/mo | 8 USD/mo | |
| Error rate | 0% | 10% |
Key numbers
| Number | Value |
|---|---|
| Timeout on a call, default | 10,000 ms |
| Call to an outside API, median | 150 ms |
| Call to an outside API, p99 | 300 ms |
| Workers in one service, default | 64 workers |
Timeout 10.0 s · 64 of 64 workers wait in a fault · 0% of normal calls stop
How it works
Without a limit, a call to a slow dependency waits as long as the dependency takes. Each waiting call holds a worker, a connection and memory. When the dependency slows down, the waiting calls pile up and the worker pool fills. Then the whole service stops, also for requests that do not need that dependency.
A timeout puts a limit on each wait. Set it a little above the normal p99 of the call. If it is too long, the service waits through the fault. If it is too short, healthy calls fail on a normal day.
A deadline is a budget for the whole request. If the user waits at most 350 ms, and the first step used 200 ms, the next call has only 150 ms left. Send the remaining time with each call, so that no part works on a request that the caller already gave up.
A timeout changes a slow failure into a fast one. The request still fails, but the worker is free for the next one. Give the user a clear error or a fallback answer, or retry within a budget.
In the levels you set timeouts so that a slow dependency cannot hold the whole service.