Shadow traffic to a new model

Model 07
HTTP/2gRPCUsers500 req/sAPI server2 vCPU · 2 GBModel (CPU)4 vCPU · 4 GBModel (CPU)12 vCPU · 12 GB
HTTP/2gRPCUsers500 req/sAPI serverModel (CPU)Model (CPU)

Turn every goal green.

Traffic
500 req/s · writes only
Limit
API server answers about 6,700 req/s
Bill
  • API server8 USD/mo
  • Model (CPU)16 USD/mo
  • Model (CPU)48 USD/mo
72 of 100 USD/mo

Goals

Not tested yet

99 of 100 requests finish faster than this.The share of requests that fail or get no answer.How long a message waits before a worker takes it.What the servers and services cost each month.

A new fraud model must score a copy of every live request for a week before it can replace the live model. It is slower and not yet stable. The API sends each request to every model link. Keep user p99 below 30 ms with no errors, and keep the copies at most 30 s behind.

A canary before the whole fleetTap to change the deployment method

Canary · 5% of requests fail for 15 seconds · 750 failed requests

Parts you can add: Message broker, Managed queue, Kafka log, Worker. At most 4 parts. Budget: 100 USD/mo.

Tap a part to add it, or drag it onto a trace.Click a part to add it, or drag it onto a trace.