Model provider failover

  • Fix
  • Agent 08

Difficulty 3 of 5

Chats call one hosted model provider through a router. The provider goes down at 0:30 for 60 s, and every chat fails until it returns. Keep availability above 98 percent and p99 below 5 s. You may make 3 edits.

0:30. Chat errors jump from 0 to 100 percent. The model provider sends no answers.

Play level

HTTP/2Users10 req/sAPI server1 vCPU · 1 GBLoad balanc…1 vCPU · 1 GBLLM API1 vCPU · 1 GB
HTTP/2HTTP/2HTTP/2Users10 req/sAPI server1 vCPU · 1 GBLoad balanc…1 vCPU · 1 GBLLM API1 vCPU · 1 GB
The board at the start of the level
Three failure domainsTap to fail one replica. The fourth tap heals all.

3 of 3 up · availability 99.9999%

Goals and limits

Availability
at least 98%
p99 latency
under 5 s
Monthly cost
under 40 USD/mo
Parts you can add
LLM API
Part limit
2
Edit limit
3
Traffic
10 req/s

Concepts

How players did

Player results load from the server.

All levels