Bulkhead against a cascade

  • Fix
  • Guard 13

Difficulty 4 of 5

At 0:20 the locked database runs at half speed for 40 s. Checkout then times out together with search. Keep checkout p99 under 150 ms and availability at 99.9 percent, and search at 99 percent. Budget: 40 USD/month. At most 4 edits.

11:20 UTC. Checkout p99 jumps from 20 ms to 10 s. Graph: the database queue climbs while both request types wait in it.

Play level

HTTP/2HTTP/2SQLUsers300 req/sUsers400 req/sNode API2 vCPU · 2 GBPostgres2 vCPU · 4 GB
HTTP/2HTTP/2SQLUsers300 req/sUsers400 req/sNode API2 vCPU · 2 GBPostgres2 vCPU · 4 GB
The board at the start of the level
Failure domains on the floorTap for the next fault. Arrow keys move the replicas.

Two rows · 3 of 3 replicas up · load served

Goals and limits

p99 latency (Checkout)
under 150 ms
Availability (Checkout)
at least 99.9%
Availability (Search)
at least 99%
Monthly cost
under 40 USD/mo
Parts you can add
Node API, Postgres, Load balancer (L7)
Part limit
3
Edit limit
4
Traffic
300 req/s

Concepts

How players did

Player results load from the server.

All levels