Response shaping
A backend for frontend answers one phone call with one screen payload. It replaces four 20 KB calls, 660 Mbps on a 120 Mbps cell, with one compressed answer of about 87 Mbps. The cost is its own CPU.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Phones call four services | One shaped, compressed answer |
|---|---|---|---|
| p99 latency | under 140 ms | 10.1 s Fail | 79 ms Pass |
| Error rate | under 0.1% | 89.6% Fail | 0% Pass |
| Monthly cost | 32 USD/mo | 40 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Default network link of a server | 1,000 Mbps |
| CPU time for one backend for frontend call | 0.3 ms |
| Calls per second, one backend for frontend | 10,000 ops/s |
100 images/s · 1,600 of 1,000 Mbps · it does not fit
How it works
A phone on a shared cell has a small share of a small link. A screen that needs data from four services, with 20 KB from each, puts 80 KB on the cell. When 1,000 phones refresh once a second, that is 660 Mbps on a link of 120 Mbps. The cell is full, and every call waits.
A backend for frontend sits in the data center, near the services. The phone makes one call. The backend calls the four services over fast links, builds one answer for the screen, and compresses it. It also removes fields that the screen does not show.
In the mini board one 20 KB answer still needs 165 Mbps, which is more than the cell. Compression at level 3 brings it to about 87 Mbps, and the cell has room. p99 falls under 140 ms.
The backend does work that the phone used to do, so it needs CPU. Compression and fan-out take about 1.5 vCPU at 1,000 screens/s, so the mini board gives it 2. Each screen also needs its own shaped answer, so the backend becomes one more service to keep.
In the levels you cut the bytes on the slow link and add the CPU where the work moves.