CPU or GPU for boosted trees

  • Choose
  • Model 10

Difficulty 3 of 5

A fraud model with 400 boosted trees scores 20,000 payments/s. One score takes about 1 ms on one CPU core. Each score must return within 25 ms at p99. Budget 600 USD/mo. Choose how to serve the model.

Play level

Options

  • One GPU server with batches of 256
  • One GPU server, one request at a time
  • Eight CPU replicas with 4 vCPU each
A batch shares one read of the weightsDrag or use arrow keys to change the batch size

Goals and limits

p99 latency
under 25 ms
Error rate
under 0.1%
Monthly cost
under 600 USD/mo
Traffic
20,000 req/s

Concepts

How players did

Player results load from the server.

All levels