Change bus invalidation
A local cache goes stale when another instance writes. With 4 instances, a write through one leaves 3 caches stale. A change bus sends each update to every cache within 1 s.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Each instance keeps its own copy | A bus sends each update to all caches |
|---|---|---|---|
| Data age | under 1 s | 59.8 s Fail | 0 ms Pass |
| Stale reads | under 3,000 | 124,414 Fail | 0 Pass |
| p99 latency | under 150 ms | 38 ms Pass | 38 ms Pass |
| Monthly cost | 28 USD/mo | 32 USD/mo | |
| Error rate | 0% | 0% |
Key numbers
| Number | Value |
|---|---|
| Local cache read, median | 0.1 ms |
| Change bus hop, median | 0.5 ms |
| Default time to live of a local cache | 60 s |
TTL 5 minutes · 12 lookups/h · old answers up to 5 minutes
How it works
A local cache is fast, because it lives inside the instance. But each instance has its own copy. When instance A writes a new price, the caches in instances B, C and D still hold the old one. They keep it until their time to live ends, here up to 60 s by default.
A short time to live reduces the problem but costs hit ratio. A change bus solves it from the other side. The database publishes each update to a bus. Every instance subscribes, and each one removes or replaces the key when it hears an update. The copies then become fresh within the delay of the bus, about 1 s in the mini board.
In the mini board 4 API instances keep local price caches. Without the bus, the other 3 caches serve old prices after each write, and the number of stale reads passes the limit. With the bus, every price update reaches every cache within one second.
The bus needs care. It should be durable, so an instance that restarts does not miss updates. A message can arrive late or twice, so apply updates by version. And keep a time to live as a backup, in case a message is lost.
In the levels you connect the writes to every cache and check how old a read can be.