Parallel fan-out
Calls that wait for each other add their times. Six services at 20 ms each cost 120 ms in a chain and about 20 ms in parallel. Keep the order only for calls that need an earlier answer.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Six calls one after another | Six calls at the same time |
|---|---|---|---|
| p99 latency | under 160 ms | 340 ms Fail | 142 ms Pass |
| Error rate | under 0.1% | 0% Pass | 0% Pass |
| Monthly cost | 56 USD/mo | 56 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Mobile 4G round trip, slow end | 90 ms |
| Round trip inside one zone | 0.5 ms |
| Node API call, median | 1 ms |
All at once · phone round trip 65 ms + calls 20 ms = 85 ms
How it works
A screen often needs data from several services: session, profile, inbox, orders, offers. A simple implementation asks the first service, waits for the answer, then asks the next. The wait times add up.
In the mini board the home screen API calls six services one after another. Each call costs 20 ms of CPU, and the phone also pays a 65 ms round trip. p99 is 340 ms against a limit of 160 ms.
Most of the six calls do not depend on each other. The API can send them all at once and wait for the slowest. The path then costs about one service, not six. In the mini board p99 falls to 142 ms.
Parallel calls use more at once. The API needs enough workers to hold six calls for each request, and the six services see the load at the same time. If one call is slow, the whole request waits for it, so set a timeout for each call and decide what to show when one is missing.
Keep a chain only where the second call needs the answer of the first.
In the levels you find which calls can run together and send them at the same time.