Latency
Latency is the time from the moment a request starts to the moment its answer arrives. A round trip in one zone takes 0.5 ms, but a cold start can add 350 ms. Users feel the slowest requests, so Cogore measures p99, the time that 99 of 100 requests beat.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Cold function, no warm pool | Two warm workers |
|---|---|---|---|
| p99 latency | under 100 ms | 276 ms Fail | 74 ms Pass |
| Error rate | under 0.1% | 0% Pass | 0% Pass |
| Monthly cost | 2 USD/mo | 2 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Call on the same host | 0.05 ms |
| Round trip in one zone | 0.5 ms |
| Round trip between zones | 1.5 ms |
| Round trip between continents, up to | 160 ms |
| Start a new API server | 90 s |
Calls per page: 20 · 18% of pages wait 2.0 s
How it works
Every hop adds time: the network between parts, the queue in front of a busy server, the work itself, and setup work such as a cold start or a new connection. Latency is the sum of these delays along the slowest path that a request takes.
The average hides the problem. If 1 request in 100 waits 2 s, the average still looks good, but a page that makes 20 calls sees at least one slow call on about 18 percent of loads (1 - 0.99^20). That is why the levels set targets on p99 and sometimes p50.
To cut latency, remove hops, keep work close to the user, keep workers warm, and keep queues short. Each fix has a price: a warm pool costs money while it waits, and a copy close to the user can be out of date.
In the levels you trace where the time goes, then remove the largest delay first.