From the future
Difficulty 2 of 5
A fraud model passed its offline test with 99 percent recall. Live, it catches 79 percent of fraud and flags 21 percent of good orders. Its training rows and live requests read the same features. Keep recall above 90 percent and false alarms under 5 percent. 3 edits.
Day 3 after launch. Offline recall was 0.99; live recall is 0.79 and 21 percent of good orders are flagged. Graph: live recall.
2 workers · 1 TB moves in 1.1 h · 1,000 Mbps per card
Goals and limits
- Recall (Orders)
- at least 90%
- False positives (Orders)
- under 5%
- Error rate (Orders)
- under 0.1%
- Monthly cost
- under 200 USD/mo
- Edit limit
- 3
- Traffic
- 1,000 req/s
Concepts
How players did
Player results load from the server.