Cache key normalization
A cache key must name the response, not the request noise. When tracking parameters split 1 product into 100 keys, the hit ratio falls to about 3 percent. Remove them from the key, and the same small cache reaches 90 percent.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Raw request as key | Normalized key |
|---|---|---|---|
| Hit ratio | at least 90% | 3.2% Fail | 99.962% Pass |
| p99 latency | under 150 ms | 26 ms Pass | 23 ms Pass |
| Monthly cost | 8 USD/mo | 8 USD/mo | |
| Error rate | 0% | 0% |
Key numbers
| Number | Value |
|---|---|
| Memory overhead for each cache key | 60 bytes |
| Cache read, median | 0.1 ms |
| Database read, median | 0.5 ms |
Normalized keys · hit ratio 90% · 10 of 100 reads reach the database
How it works
Two requests can ask for the same data in different words. One URL has a tracking parameter. Another has its parameters in a different order. A third has a host name in other case. A cache that uses the raw URL as the key stores one entry for each variant, and each variant is a miss until it comes again.
In the mini board each product has 100 raw keys, and the cache is small. The hit ratio is about 3 percent. The cache holds copies that almost never repeat, so about 97 reads in 100 reach the database. Each extra key also costs 60 bytes of overhead, on top of its value.
Normalization maps equal requests to one key. Drop the parameters that do not change the response. Sort the others. Lower the case of the host. Then the 100 variants share 1 entry.
Be careful with what you drop. Some inputs change the answer: language, currency, user role, API version. If you drop one of them, the cache returns the wrong data to a user. That is worse than a miss. A safe rule is to list the inputs that count, not the inputs to remove.
In the levels you normalize the key and watch the hit ratio and p99.