Event sourcing snapshots

Event sourcing rebuilds state from saved events. A replay of all 10,050,000 events takes too long. A snapshot with its log position lets recovery load the snapshot and replay only the rest, in under 1 minute.

Play a level

See it on the board

The same traffic runs on two boards. The simulator measured both when this page was built.

Board to show
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker4 vCPU · 4 GB
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker4 vCPU · 4 GB
Replay all events: misses a goal
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker4 vCPU · 4 GB
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker4 vCPU · 4 GB
Load a snapshot, replay the rest: meets the goals
See it on the board
Measure Goal Replay all events Load a snapshot, replay the rest
Replay time under 60 s 502.77 s Fail 3.8 s Pass
Monthly cost 24 USD/mo 24 USD/mo
p99 latency 100 ms 100 ms
Error rate 0% 0%

Key numbers

Default part values from the catalog. The mini board has 10,050,000 events and a snapshot of 160 MB.
NumberValue
Events per second, one log100,000 ops/s
Disk throughput of a log500 MB/s
Default retention of a log168 h
Tables join, documents copyTap or press Space to switch between tables and documents

Tables · a read touches 3 places in 0.5 ms · a shared change writes 1 place

How it works

An event sourced system stores each change as an event: deposit, withdrawal, transfer. The current state is not stored as the truth. It is computed by replaying the events in order. This gives a full history, and you can rebuild the state at any past time.

Replay has a cost. In the mini board the log holds 10,050,000 events of 128 bytes. After a crash, a service that replays all events from the start needs about 500 s to be ready.

A snapshot fixes this. Now and then the service saves its state together with the position in the log, for example the number of the last event it applied. Recovery loads the newest snapshot, which is 160 MB here, and replays only the events after its position. The log after the snapshot is short, so recovery ends in under one minute.

A snapshot is a cache of the log, not a second truth. If a snapshot is lost or wrong, you can build it again from the events. Take snapshots at a steady rate, for example every 100,000 events, so the replay stays short.

Keep events immutable. If the event format must change, write an upgrade step that reads old events, and do not edit them.

In the levels you set the snapshot rule and check the replay time.