Idempotent consumer
At-least-once delivery can repeat a job. A lost acknowledgment delivered 1 payment job twice in the mini board. An idempotency key, saved in the same transaction as the effect, keeps the charge count at 1.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Every delivery runs the effect | Key checked before the effect |
|---|---|---|---|
| Duplicates | none | 200 Fail | 0 Pass |
| Completion time | under 40 s | 10.1 s Pass | 10.1 s Pass |
| Lost writes | none | 0 Pass | 0 Pass |
| Monthly cost | 12 USD/mo | 12 USD/mo | |
| Error rate | 0% | 0% |
Key numbers
| Number | Value |
|---|---|
| Payment provider call, median | 150 ms |
| Default delivery tries | 5 tries |
| Acknowledgment round trips for AMQP | 1 RTT |
13 deliveries · 10 payments applied · 0 charged twice
How it works
A queue promises that a message is not lost. To keep that promise, it repeats a message when it does not hear an acknowledgment. The acknowledgment can be lost even when the work was done. So at-least-once delivery means some messages arrive twice.
The consumer must handle that. For a payment, a second run charges the customer twice. The queue cannot prevent this, because it cannot see inside the effect.
An idempotent consumer keeps a record of the messages that it has handled. Each message carries a unique ID. Before it acts, the consumer checks the record. If the ID is there, it skips the effect and sends the acknowledgment again. If not, it does the effect and records the ID.
The record and the effect must change together. If the consumer charges the card and then crashes before it saves the ID, the next try charges again. Save the ID in the same transaction as the effect when you can. When the effect is a call to another service, pass the ID on as that service's idempotency key.
In the mini board a lost acknowledgment delivers one payment job twice. With the key, the charge count is 1.
In the levels you make the consumer safe for repeats and check the count of duplicate effects.