Live ranking vs nightly scores
Live ranking scores 500 items on a CPU for each page view. Nightly scoring computes the picks once and stores them for a key lookup. At 3,000 views/s, live ranking fails 88 percent of views and p99 passes 10 s. Nightly scores give p99 4.1 ms for 24 USD/mo against 40 USD/mo.
Batch 32 · 2,500 tokens/s · a 200 token answer takes 2.6 s
1,500 req/s · Node API · 3 hosts · 18 USD/mo
Case 1
A shop shows 20 picks per user at up to 3,000 views/s. The picks use only yesterday's orders. p99 must stay below 50 ms and cost below 60 USD/mo.
| Measure | Goal | Rank 500 items on CPU per view | Score all users each night |
|---|---|---|---|
| p99 latency | under 50 ms | 10.1 s Fail | 4.1 ms Pass |
| Error rate | under 0.1% | 88.4% Fail | 0% Pass |
| Monthly cost | 40 USD/mo | 24 USD/mo |