Uneven work

Scale 02

Choose

A balancer sends 300 requests/s to two backends with 2 vCPU each. Measured CPU per request is 1 ms on one backend and 50 ms on the other. Choose the routing plan. Keep p99 below 350 ms and errors below 0.1 percent.

Goals

Not tested yet

99 of 100 requests finish faster than this.The share of requests that fail or get no answer.What the servers and services cost each month.
Choose a design
Send requests in round-robin order
Round robin with a 300 ms backend timeout
Send requests to the less busy backend