Drift
Difficulty 3 of 5
A model predicts which orders will ship late. It was trained once, before day 0. Over 90 days the order mix changes twice. Keep accuracy above 90 percent over the run and find each change within 3 days. You can make 3 edits.
Day 41, 09:00. Late-shipping model accuracy is 0.90 and has fallen for 26 days. No alert fired. Graph: daily accuracy.
Two write paths · 4 of 16 served features differ from training
Goals and limits
- Accuracy
- at least 92%
- Detection time
- under 259,200 s
- Error rate
- under 0.1%
- Monthly cost
- under 200 USD/mo
- Edit limit
- 3
- Traffic
- 20 req/s
Concepts
How players did
Player results load from the server.