Delete propagation
A delete must reach every copy of the data. 20 percent of 1,000 records are deleted at 0:10, and caches and search must drop them within 1 s. Send the delete marker through the same change path as updates.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Delete reaches only the database | Delete sent to cache and search |
|---|---|---|---|
| Data age | under 1 s | 29.9 s Fail | 200 ms Pass |
| Data mismatches | under 100 | 2,990 Fail | 26 Pass |
| Monthly cost | 36 USD/mo | 44 USD/mo | |
| p99 latency | 25 ms | 25 ms | |
| Error rate | 0% | 0% |
Key numbers
| Number | Value |
|---|---|
| Default checkpoint interval | 30 s |
| Default time to live of a cache entry | 60 s |
| Default refresh interval of a search index | 1 s |
2 workers · 1 TB moves in 1.1 h · 1,000 Mbps per card
How it works
When a user asks you to delete their data, the database row is only one copy. The same data lives in a cache, in a search index, in exports and in logs. If a delete reaches only the database, the other copies still show the data. For a privacy request this is a failure, and it can break the law.
In the mini board the source deletes 20 percent of 1,000 user records at 0:10. Cached replies and search results must stop exposing them within 1 s, and the total number of mismatched replies must stay below 100. The cache keeps entries for 600 s by default, so without help it serves deleted records for up to 10 minutes.
A delete is also a change. The database log records it like an update. A change feed reads that log and sends a delete marker, called a tombstone, to every derived store. The cache removes the key, and the search index removes the document, both within about a second.
Do not rely on time to live to remove deleted data. And do not forget the stores that you rarely look at: backups, analytics tables, copies in other regions. Keep a list of every place that holds user data, and test the delete path for each. Backups need a plan too.
In the levels you carry deletes through the same path as updates and count the replies that still show deleted data.