Loss-limited throughput
Loss and long round trips limit TCP far below the link rate. At 140 ms RTT and 1 percent loss, one TCP connection slows down after each loss. QUIC or more connections keep more data in flight.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | One TCP connection | HTTP/3 on QUIC |
|---|---|---|---|
| p99 latency | under 500 ms | 10.1 s Fail | 284 ms Pass |
| Error rate | under 1% | 96.1% Fail | 0% Pass |
| Monthly cost | 8 USD/mo | 8 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Round trip between continents, fast end | 70 ms |
| Round trip between continents, slow end | 160 ms |
| Largest segment of one packet | 1,460 bytes |
HTTP/1.1 · connections: 6 · 30 files in 7 round trips, 630 ms
How it works
A TCP connection sends a window of data and waits for acknowledgments. When it sees a lost packet, it cuts the window in half and grows it slowly again. On a short path, the window grows back in a few milliseconds. On a long path it takes many round trips.
A simple model shows the effect. The rate of one connection is about the segment size divided by the round trip time and by the square root of the loss. With 1,460 bytes, a 140 ms round trip and 1 percent loss, the model gives a rate near 1 Mbps. The link may be 1 Gbps. The connection cannot use it.
In the mini board a cross-ocean path has 140 ms RTT and 1 percent loss. One TCP connection must carry 100 objects/s of 50 KB each, which is 40 Mbps. The connection falls far below that, so requests queue, and p99 and errors rise past their limits.
There are two ways to fix it. More connections add their rates together, but each pays its own setup. QUIC recovers loss for each stream and changes how the window reacts, so a loss costs less. In the mini board the QUIC-based protocol carries the load, and p99 stays under 500 ms.
For bulk transfer over long distances, use several streams, tune the window, or choose a protocol that handles loss better.
In the levels you pick the transport for a long, lossy path and read the p99 and the errors.