Missing metrics

  • Fix
  • Watch 01

Difficulty 2 of 5

A sale doubles traffic to 2,400 pages/s at 0:30. Page p99 rises from 26 ms to over 10 s. The only graph on the dashboard is API CPU, and it stays near 36 percent. Find the cause. Keep p99 under 100 ms and errors under 0.1 percent. Budget: 36 USD/mo. At most 4 edits.

00:30 UTC. Page p99 jumps from 26 ms to 10 s. Dashboard: requests and API CPU (36 percent). No disk or queue graph.

Play level

HTTP/2SQLgRPCUsers1,200 req/sAPI server1 vCPU · 1 GB × 2Postgres1 vCPU · 2 GBChange capt…
HTTP/2SQLgRPCUsers1,200 req/sAPI serverPostgres1 vCPU · 2 GBChange ca…
The board at the start of the level
A graph for each resourceTap to add or remove the probes

One CPU graph · CPU 40% · disk no graph · the fault is hidden

Goals and limits

p99 latency
under 100 ms
Error rate
under 0.1%
Monthly cost
under 36 USD/mo
Parts you can add
Redis cache, Memory cache, API server
Part limit
6
Edit limit
4
Traffic
1,200 req/s

Concepts

How players did

Player results load from the server.

All levels