Fan-out delivery
Fan-out cost follows the size of the audience. One goal update to 200,000 viewers is 200,000 deliveries. The broker publishes once, but the gateways must make every delivery.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | One gateway delivers to everyone | Four gateways share the viewers |
|---|---|---|---|
| p99 latency | under 1 s | 416 ms Pass | 49 ms Pass |
| Error rate | under 0.1% | 46.4% Fail | 0% Pass |
| Monthly cost | 136 USD/mo | 520 USD/mo |
Key numbers
| Number | Value |
|---|---|
| CPU time for one delivery | 0.3 ms |
| Memory for one open connection | 20 KB |
| Deliveries per second, one gateway part | 10,000 ops/s |
4 workers · 3 jobs waiting · wait 33 ms
How it works
A live score is one small message. The work comes from the number of viewers. A broker accepts the update once and hands it to the gateways. Each gateway holds the open connections of its viewers, and it must write the update to each of them.
In the mini board one update each second goes to 200,000 viewers. Each delivery takes about 0.3 ms of CPU. That is 60 CPU-seconds of work for each update. One gateway with 32 vCPU needs about 1.9 s to do it, so it falls behind, and 46 percent of the goal requests fail.
The fix is more delivery capacity, not a faster broker. Four gateways with 32 vCPU each share the viewers, and the broker sends each update to all of them. The work for each gateway falls to about one quarter, and p99 stays under one second.
Open connections also need memory, here about 20 KB for each. A gateway with 50,000 connections holds about 1 GB of connection state before it sends anything.
A queue between the broker and the gateways lets a slow gateway fall behind without stopping the others.
In the levels you add gateways until the delivery work fits the time limit.