Dead-letter queue
A dead-letter queue holds a job that fails on every try, after a set number of deliveries. One malformed job among 100 can block all the jobs behind it. After 3 tries the queue moves it aside, and the 99 valid jobs finish.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Poison job tried 100 times | Moved aside after 3 tries |
|---|---|---|---|
| Completion time | under 150 s | 220 s Fail | 102.08 s Pass |
| Lost writes | none | 0 Pass | 0 Pass |
| Monthly cost | 8 USD/mo | 8 USD/mo | |
| Error rate | 0% | 0% | |
| Level rules | none broken | Board is not complete Fail | Pass |
Key numbers
| Number | Value |
|---|---|
| Default delivery tries | 5 tries |
| Default time a claimed job stays hidden | 30 s |
| Messages per second, one queue | 20,000 ops/s |
Dead-letter queue · the bad job gets 3 tries · 0 valid jobs wait behind it
How it works
A queue hands a job to a worker. If the worker fails, the queue offers the job again. For a temporary error this is right. For a poison job, such as a malformed message, it is not. The job fails on every try and returns to the queue each time.
With a single consumer, a poison job can hold up everything. In the mini board one malformed job among 100 is allowed 100 deliveries. Each failed try takes time, and the valid jobs behind it wait. The batch does not finish inside 150 s.
A dead-letter queue sets a limit. After a number of failed deliveries, here 3, the queue moves the job to a separate queue. The valid jobs go on, and all of them finish. The bad job is kept, so nobody loses it. A person can read it, fix the cause and send it again.
Set the limit to match the kind of error. A few tries cover a temporary fault. Many tries only hide a poison job. Alert on the size of the dead-letter queue, because a growing queue means a bug, not bad luck.
A dead-letter queue is not a trash can. Every job in it still needs an owner.
In the levels you set the retry limit and the dead-letter queue and check that no valid job is lost.