Visibility timeout
A visibility timeout hides a claimed job for a fixed time. If the timeout is shorter than the job, a second worker repeats the job. Here a 5 s timeout and 30 s jobs make repeats, and heartbeats that extend the claim stop them.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Short lease, no heartbeat | Short lease with heartbeats |
|---|---|---|---|
| Duplicates | none | 56 Fail | 0 Pass |
| Completion time | under 60 s | 30 s Pass | 30 s Pass |
| Lost writes | none | 0 Pass | 0 Pass |
| Monthly cost | 20 USD/mo | 20 USD/mo | |
| Error rate | 0% | 0% |
Key numbers
| Number | Value |
|---|---|
| Default time a claimed job stays hidden | 30 s |
| Default delivery tries | 5 tries |
| Messages per second, one queue | 20,000 ops/s |
Heartbeats · runs of one 30 s job: 1
How it works
A queue gives a job to a worker and hides it from the other workers for a time. If the worker confirms the job before the time ends, the job is done. If not, the queue assumes that the worker died, and it shows the job again. This lease is the visibility timeout.
The lease is a deadline for the worker. Set it too short, and a slow but healthy worker loses its job while it still works on it. In the mini board each job takes 30 s, and the visibility timeout is 5 s. After 5 s the queue offers the job to a second worker, and both run it. The effect happens twice.
You can set a long timeout, for example 60 s. Then a crashed worker holds its job for up to 60 s before anyone else can take it. Recovery gets slow.
Heartbeats give a better answer. A short lease, and the worker renews it every few seconds while it is alive. A healthy worker keeps its job for as long as it needs. A dead worker stops renewing, and the job returns after one short lease. In the mini board the heartbeats remove all repeats, and the 200 jobs finish within the limit of 60 s.
Also make the effect safe to repeat, because a repeat can still happen after a crash.
In the levels you set the lease and the heartbeat for the real job duration.