Data placement
Data placement decides where each piece of state lives. A shared store in the same zone answers in less than 1 ms, and a store on another continent can take 160 ms. State that lives in the memory of one host also stops when that host stops.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Sessions in the memory of each server | Sessions in a shared Redis store |
|---|---|---|---|
| Availability | at least 95% | 74.5% Fail | 99.924% Pass |
| Error rate | under 5% | 25.5% Fail | 0.076% Pass |
| Monthly cost | 32 USD/mo | 32 USD/mo | |
| p99 latency | 68 ms | 69 ms |
Key numbers
| Number | Value |
|---|---|
| Call on the same host | 0.05 ms |
| Shared store read (Redis), median | 0.1 ms |
| Round trip in one zone | 0.5 ms |
| Round trip between zones | 1.5 ms |
| Round trip between continents, up to | 160 ms |
Lasers in orbit, New York to London · 7,151 km · 24 ms one way
How it works
Every service keeps some state: sessions, carts, counters and files. The place where that state lives decides two things. The first is speed. Data in local memory is the fastest, data on the network adds a round trip, and data on another continent adds many milliseconds. The second is survival. Data in local memory exists only on that host.
Sessions show the trade-off. If each API server keeps sessions in its own memory, a user must always come back to the same server. When that host fails, its users lose their sessions, and the load balancer cannot move them. If sessions live in a shared store such as Redis, any server can serve any user. Each request pays about 0.6 ms more, but a host can fail and no user must sign in again.
The same choice comes back at every scale: a process or a shared cache, one zone or two, one region or many. Keep data close to the work that reads it most. Keep state off the servers that you want to replace, scale or lose without harm.
In the levels you move state out of servers, put copies near their readers, and make servers easy to replace.