Protect a fixed GPU fleet at launch
Difficulty 5 of 5
A chat app launches on 2 H100 GPUs (about 24 chats/s). Demand rises from 16 to 64 chats/s in 4 waves, and the last wave has a 10 s burst at 6 times the rate. Keep p99 under 7 s and errors under 15 percent for 90 percent of the run. Budget: 3,812 USD/month, plus 4 per wave.
280 req/s · CPU 70% · mean 22 ms
Goals and limits
- Time within SLO
- at least 90%
- p99 latency
- under 7 s
- Error rate
- under 15%
- Monthly cost
- under 3,812 USD/mo
- Waves
- 4
- Time within SLO
- at least 90%
- Parts you can add
- Semantic cache, Redis cache, Rate limiter
- Part limit
- 2
- Hosts
- 2 × 1 vCPU · 80 GB
- Traffic
- 16 req/s
Concepts
How players did
Player results load from the server.