Compact small files

  • Fix
  • Flow 09

Difficulty 3 of 5

Streams write 2,000 batches/s of 64 KB into the events store. The daily report reads 20 GB and now times out. Bring the report p99 under 20 s with under 1 percent errors, and keep ingest errors under 0.1 percent. Use at most 3 edits.

09:05 The daily report times out after 60 s. Each run opens 312,500 files. Ingest writes queue behind it.

Play level

HTTP/2SQLUsers2,000 req/sUsers0.05 req/sAnalytics DB8 vCPU · 16 GB
HTTP/2SQLUsers2,000 req/sUsers0.05 req/sAnalytics…8 vCPU · 16 GB
The board at the start of the level
A disk that fillsArrow keys change the write rate. Tap to ship the logs.

5 MB/s · logs kept on the host · disk full in 33 minutes

Goals and limits

p99 latency (Daily)
under 20 s
Error rate (Daily)
under 1%
Error rate (Ingest)
under 0.1%
Monthly cost
under 100 USD/mo
Edit limit
3
Traffic
0 req/s

Concepts

How players did

Player results load from the server.

All levels