Publish and subscribe
A work queue gives each job to one worker. Publish and subscribe gives every subscriber its own copy. When each of 2,000 order events must reach 4 services, a slow analytics consumer must not stop billing.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | One work queue for four services | Each service has its own copy |
|---|---|---|---|
| Completion time | under 35 s | 40 s Fail | 20.01 s Pass |
| Lost writes | none | 6,000 Fail | 0 Pass |
| p99 latency | under 150 ms | 91 ms Pass | 91 ms Pass |
| Monthly cost | 24 USD/mo | 24 USD/mo | |
| Error rate | 0% | 0% | |
| Level rules | none broken | Board is not complete Fail | Pass |
Key numbers
| Number | Value |
|---|---|
| Messages per second, one managed bus | 20,000 ops/s |
| Bus hop, median | 0.5 ms |
| Queue request price | 0.4 USD/M requests |
13 deliveries · 10 payments applied · 0 charged twice
How it works
A work queue spreads jobs over a pool of workers. Each job goes to one worker, which is what you want for image resizing. It is wrong when one event must reach several different services. Billing, search, email and analytics each need every order event, and each does a different task.
Publish and subscribe separates the sender from the receivers. The service that creates the event publishes it once. Each subscriber has its own queue, and the bus puts a copy in every one. A subscriber reads at its own speed.
In the mini board each of 2,000 order events must reach four services. With one shared work queue, each event goes to only one service, so 6,000 of the 8,000 effects never happen. With a bus, each service gets every event. Analytics stops from 0:05 to 0:10, and its queue fills for 5 s while billing, search and email carry on. When analytics comes back, it reads its backlog, and all four effects finish within 35 s.
The design has a cost. The bus stores one copy for each subscriber, so storage and traffic grow with their number. A new subscriber can also read only events that arrive after it joins, unless the bus keeps history.
Add a dead-letter queue for each subscriber, so one bad event cannot stop its consumer.
In the levels you give each service its own copy and check that a slow one does not block the others.