Chaos day: four failure drills
Guard 19Keep every goal green in each wave.
- Traffic
- Browse1,080 req/s · reads onlyOrders120 req/s · writes only
- Limit
- Node API answers about 1,000 req/s
- Bill
- Load balancer (L7)4 USD/mo
- Node API × 28 USD/mo
- Postgres8 USD/mo
20 of 24 USD/mo
Goals
Not tested yet
Four 50 s waves each inject one failure, in order: an API instance crashes, the database runs 12 times slower, zone A fails, an API instance runs 6 times slower. Keep browse p99 under 300 ms and errors under 2 percent for 95 percent of the run. Each wave adds budget.
Parts you can add: Node API, Load balancer (L7), Redis cache. At most 10 parts. Budget: 24 USD/mo.
Tap a part to add it, or drag it onto a trace.Click a part to add it, or drag it onto a trace.