When a cache does not help
A cache helps only when its hits save more than its hop costs. With 98 percent writes, a 40 ms cache hop saves almost nothing. Remove it, and p99 falls below 45 ms.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Cache in front of the database | Calls go straight to the database |
|---|---|---|---|
| p99 latency | under 45 ms | 10.1 s Fail | 34 ms Pass |
| Monthly cost | 8 USD/mo | 4 USD/mo | |
| Error rate | 85.3% | 0% |
Key numbers
| Number | Value |
|---|---|
| Round trip on one continent, slow end | 40 ms |
| Cache read, median | 0.1 ms |
| Database read, median | 0.5 ms |
Without the cache · added to every call: 0 ms · 98 of 100 calls are writes
How it works
A cache is an extra step. Every read goes to the cache first. A hit saves the cost of the slower source. A miss pays for the cache hop and then for the source. The cache pays off only when the saved time on the hits is larger than the time added on all requests.
The reads decide that. A cache can help only reads. A write must still reach the source, and in a write-through or write-aside design it often touches the cache too. If most requests are writes, the cache has few reads to serve.
In the mini board 98 percent of the calls are writes. The remote cache sits 40 ms away. Every call pays the hop, but only 2 percent are reads, and few of those hit. The cache adds work without enough reuse to repay it. p99 passes the limit of 45 ms.
Removing the cache sends the calls straight to the database. The extra hop is gone, and p99 falls below 45 ms. The cost falls too, because there is one part less to run.
This does not make caches bad. A separate read-heavy path of the same system can still use one, with a write policy that suits it. Measure the ratio of reads to writes and the hit ratio before you add a cache, and measure again after.
In the levels you check the read share of a path and remove a cache that does not repay its hop.