Bandwidth
Bandwidth is the number of bits that a link can carry each second. A host has a 1,000 Mbps network card, so 100 images per second at 2 MB each, which need 1,600 Mbps, do not fit. The extra bytes wait or fail, and a faster CPU does not help.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | API sends every image | Object storage sends the images |
|---|---|---|---|
| Error rate | under 0.1% | 49.2% Fail | 0% Pass |
| p99 latency | under 300 ms | 10.1 s Fail | 110 ms Pass |
| Monthly cost | 16 USD/mo | 31,225 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Network card of a standard host | 1,000 Mbps |
| Network card of a dedicated host | 10,000 Mbps |
| Data out from the origin, for each GB | 0.06 USD/GB |
| Data out from the CDN, for each GB | 0.01 USD/GB |
| Most data in one TCP packet | 1,460 bytes |
100 images/s · 1,600 of 1,000 Mbps · it does not fit
How it works
To find the bandwidth that a service needs, multiply the requests per second by the bytes in each response. Then multiply by 8 to get bits. Small API answers seldom fill a link. Images, video, downloads and backups fill it fast.
When a link is full, packets wait in buffers. Then they drop, and the sender sends them again. Latency rises first, then errors start. The CPU can stay almost idle all the time, so a CPU graph does not show the problem.
There are three main fixes. Send fewer bytes: compress responses and resize images. Send the bytes from another place: object storage and a CDN have their own network capacity, so large files do not pass through the API server. Or get a larger link. Bytes also cost money when they leave the cloud. In the price table, data from the origin costs 0.06 USD per GB, and data from the CDN costs 0.01 USD per GB.
In the levels you compute the bits per second on each link, and you move large files off a path that cannot carry them.