Storage
Storage keeps data when a process stops, on disks, in object stores or in archives. A disk fills in free space divided by write rate, so 10 GB free at 5 MB/s lasts about 33 minutes. When the disk is full, writes fail and the service on it can stop.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Logs kept on the host for 1 day | Logs shipped off the host, 20 s kept locally |
|---|---|---|---|
| Error rate | under 0.1% | 16.7% Fail | 0% Pass |
| p99 latency | under 200 ms | 68 ms Pass | 69 ms Pass |
| Monthly cost | 16 USD/mo | 16 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Time a service keeps local logs, default | 86,400 s |
| Disk throughput, one part | 500 MB/s |
| Disk operations per second, one part | 50,000 IOPS |
| Object storage price for each GB | 0.015 USD/GB-mo |
| Cold storage price for each GB | 0.004 USD/GB-mo |
5 MB/s · logs kept on the host · disk full in 33 minutes
How it works
Each kind of storage has a different price, speed and life. Memory is fast but empty after a restart. A local disk keeps data after a restart, but only while the host lives, and its size is fixed. A network block disk can move to a new host. Object storage keeps files of any size at a low price for each GB, with a higher latency for each request. Cold storage costs even less, but a read can take minutes or hours.
A full disk is a frequent cause of outages. Logs, temporary files and old versions grow every second. When the disk is full, the database cannot write, the service cannot log, and requests fail. Monitor the free space and the rate at which it falls, and alert on the time left, not only on the percent used.
Keep in each place only what it needs. Rotate local logs after a short time and send them to a separate log store. Move old files from hot storage to cheaper tiers. Delete data when its retention time ends, which also limits the personal data that you keep.
Throughput and operations per second are limits too. A disk that writes 500 MB/s can still slow a busy database.
In the levels you size disks, ship logs off the host and choose the storage tier for each kind of data.