Queueing
A queue holds work that arrives while the server is busy. A log write takes 2 ms, but if consumers finish fewer events per second than arrive, the backlog grows every second. Only spare service capacity drains it.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | One consumer | Two consumers |
|---|---|---|---|
| Queue lag | under 30 s | 9,765 s Fail | 0 s Pass |
| Lost writes | none | 0 Pass | 0 Pass |
| Monthly cost | 40 USD/mo | 48 USD/mo | |
| p99 latency | 54 ms | 54 ms | |
| Error rate | 43.5% | 43.5% |
Key numbers
| Number | Value |
|---|---|
| Append to a log (Kafka), median | 2 ms |
| Events per second, one log part | 100,000 ops/s |
| Send to a queue (RabbitMQ), median | 0.5 ms |
| Messages per second, one queue part | 20,000 ops/s |
| CPU time for one simple job | 0.3 ms |
4 workers · 3 jobs waiting · wait 33 ms
How it works
Work seldom arrives at an even rate. A queue absorbs short bursts: requests wait for a moment, then the server catches up. This works only when the service rate is higher than the arrival rate on average.
If 4,000 events arrive each second and the consumers finish 3,000, the backlog grows by 1,000 events every second. It never drains on its own. After an hour, the oldest event is far behind, and each new event waits behind it. To drain a backlog, you need capacity above the arrival rate, not only equal to it.
Wait time also grows long before a server is 100 percent busy. At 50 percent busy, a new job seldom waits. At 90 percent, it often waits behind several others. At 99 percent, the wait becomes very long. This is why p99 rises sharply under high load.
A log with partitions adds one more limit. Each partition goes to one consumer in a group, so consumers above the partition count stay idle.
In the levels you compare the arrival rate with the service rate, add consumers or partitions, and keep queue lag below the target.