Caching
A cache keeps copies of recent answers close to the caller. A Redis read takes 0.1 ms and one cache serves 100,000 reads per second, so each hit removes work from the database. A miss still goes to the database and pays the full cost.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | No cache | Cache in front of the database |
|---|---|---|---|
| p99 latency | under 200 ms | 10.1 s Fail | 68 ms Pass |
| Error rate | under 0.1% | 58.7% Fail | 0% Pass |
| Monthly cost | 16 USD/mo | 16 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Cache read (Redis), median | 0.1 ms |
| Database read (Postgres), median | 0.5 ms |
| Reads per second, one cache | 100,000 ops/s |
| Queries per second, one database | 15,000 ops/s |
| Cache price | 4 USD/mo |
1,600 keys · hit 81% · mean 3.2 ms
How it works
Most systems read the same data again and again. A product page, a user profile or a price list changes rarely but is read thousands of times per second. Without a cache, each of those reads goes to the database, and the database queue grows until requests time out.
A cache sits between the service and the database. On a hit it answers at once. On a miss the service reads the database and stores the answer in the cache for the next caller. The hit ratio decides how much load the cache removes: at a 95 percent hit ratio, only 1 read in 20 reaches the database.
A cache has costs too. Its copy can be older than the database, so you must choose how long an entry lives and when a write removes it. It uses memory, so it keeps only the hottest keys. When it fails or restarts empty, every read goes to the database at the same time.
In the levels you add a cache, size it, set its time to live and protect the database when many keys expire together.