N+1 queries
An N+1 pattern makes 1 call to list N items, then N more calls to read each item. With 50 records in each query, one batched call replaces 50 calls and removes 49 round trips. Latency then no longer grows with N.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | 50 calls for each query | 1 batched call for each query |
|---|---|---|---|
| p99 latency | under 180 ms | 10.1 s Fail | 25 ms Pass |
| Error rate | under 0.1% | 70.9% Fail | 0% Pass |
| Monthly cost | 8 USD/mo | 8 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Round trip inside one zone | 0.5 ms |
| CPU time for one API call | 0.3 ms |
| Calls per second, one API server | 10,000 ops/s |
Batched · 2 calls in a row · round trips 1.0 ms
How it works
The pattern starts small. Code loads a list of 50 records, then loops over the list and loads the details of each record with its own call. On a laptop with 5 records it feels fast. In production the list has 50 records, and each call has a round trip and some setup work.
In the mini board a GraphQL gateway resolves each query with 50 separate calls. At 200 queries/s that is 10,000 calls/s. Every call pays a round trip of 0.5 ms in the same zone and 0.3 ms of CPU time, and the waits add up inside one request. p99 goes past the 180 ms limit.
A batch call takes many keys at once and returns all records. The gateway sends 1 call for each query. The records still need work, but the setup and the round trips happen once.
Batching needs care. Set an upper limit for the batch size, so one huge query cannot hold the service. Check access rights for each record in the batch. Map each result back to its key, and handle records that are missing.
In the levels you batch the calls of a resolver and watch p99 fall.