CPU cost per request
The CPU time of one request sets how many requests a server can take. In this run, JSON needs 144 percent of the half core of the worker at 2,400 calls/s. gRPC needs 58 percent, so one worker is enough.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | JSON, one worker | gRPC, one worker |
|---|---|---|---|
| p99 latency | under 180 ms | 10.1 s Fail | 25 ms Pass |
| Error rate | under 0.1% | 35.3% Fail | 0% Pass |
| Monthly cost | 12 USD/mo | 12 USD/mo |
Key numbers
| Number | Value |
|---|---|
| CPU cost of gRPC compared with JSON | 0.4 ratio |
| Payload size of gRPC compared with JSON | 0.4 ratio |
| CPU time for one worker call | 0.3 ms |
280 req/s · CPU 70% · mean 22 ms
How it works
Capacity is a division. A server has so many CPU milliseconds each second, and each request uses a share. When requests arrive faster than the CPU can serve them, the queue grows without limit.
Encoding and decoding use CPU on every request. In the mini board a front service calls a worker 2,400 times each second. Each call sends 2 KB and gets 12 KB back. The worker has half a core. With JSON over HTTP/2 it needs 144 percent of that half core, so the queue fills and p99 reaches 10,112 ms.
You have two ways to fix it. You can add a second worker, which brings each worker to 72 percent of CPU, but the cost rises to 16 USD/month. Or you can make each request cheaper. A binary format like gRPC needs about 0.4 times the CPU and 0.4 times the bytes, so one worker runs at 58 percent for 12 USD/month, with p99 at 25 ms.
A cheaper format has costs too. You need a schema, tools and a way to debug binary messages. Count those costs with the servers you save.
In the levels you compare the CPU cost of each design before you add servers.