Capacity
Capacity is the most work that a part can do each second. One API request uses 0.3 ms of CPU, so 1 vCPU serves at most about 3,300 requests per second. Above that rate, requests wait in a queue and latency grows fast.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | API and database on one host | API and database on separate hosts |
|---|---|---|---|
| p99 latency | under 200 ms | 69 ms Pass | 70 ms Pass |
| Monthly cost | 32 USD/mo | 32 USD/mo | |
| Error rate | 0% | 0% | |
| Level rules | none broken | Host capacity Fail | Pass |
Key numbers
| Number | Value |
|---|---|
| CPU time for one API request | 0.3 ms |
| Database reads per second, for each vCPU | 15,000 reads/s |
| Database writes per second, for each vCPU | 2,500 writes/s |
| vCPU on one small host | 2 vCPU |
| Price of one 2 vCPU, 8 GB host | 16 USD/mo |
280 req/s · CPU 70% · mean 22 ms
How it works
Every part has limits: CPU time, memory, disk operations and network bandwidth. The first limit that the load reaches sets the capacity of the part. To find it, multiply the work for one request by the request rate. At 0.3 ms of CPU for each request, 2,000 requests per second need 0.6 vCPU.
Parts on one host share that host. If the API and the database run on the same 2 vCPU box, each one takes CPU from the other. More replicas on the same box do not help, because they divide the same cores. A second host, or a larger one, adds real capacity.
Keep headroom. Near 100 percent use, a small burst makes a long queue, and p99 grows much faster than the average. Many teams aim for about 70 percent use at peak, so that a spike or the loss of one server does not stop the service. Headroom costs money, so size for the peak that you expect, not for every possible spike.
In the levels you read the load on each host and part, find the first limit, and add or move capacity where it is needed.