Online serving copy
An online serving copy keeps the latest values where the reader needs them, in a store built for fast lookups. A model that reads 10,000 features/s cannot read them from the order database that also keeps history. A change feed keeps the copy within 0.1 s.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Lookups read the order database | Lookups read an online copy |
|---|---|---|---|
| p99 latency (Lookups) | under 10 ms | 6.3 s Fail | 3.5 ms Pass |
| Error rate (Lookups) | under 0.1% | 35.7% Fail | 0% Pass |
| Data age | under 60 s | 0 ms Pass | 100 ms Pass |
| Monthly cost | 24 USD/mo | 44 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Reads per second, one cache | 100,000 ops/s |
| Queries per second, one database | 15,000 ops/s |
| Cache read, median | 0.1 ms |
Lasers in orbit, New York to London · 7,151 km · 24 ms one way
How it works
Where data lives decides who can read it fast. The order database keeps every order for training and audits. It is built for history and for 300 new orders each second. A fraud model that needs 10,000 feature lookups each second, each under 10 ms, cannot share that database.
The answer is a second copy, placed for the reader. An online store holds only the latest value of each feature. It is a small in-memory store, so a lookup takes about 0.1 ms and one node serves 100,000 lookups each second. The database stays as the source of truth.
The copy must follow the source. A change feed reads the commits of the order table and updates the online store. In the mini board it keeps the copy within about 0.1 s, far inside the limit of 60 s. If the feed stops, the copy gets old, so watch its lag.
A copy has costs. You pay for the second store and the feed. The two copies can disagree for a short time. Decide which copy answers which question, and never write to the copy directly.
In the levels you decide where each copy lives and how it stays current.