Hedged requests

Scale 16
HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2Users3,200 req/sLoad balanc…2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GBNode API2 vCPU · 2 GB
HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2HTTP/2UsersLoad…Node…Node…Node…Node…Node…Node…Node…Node…Node…Node…Node…Node…Node…Node…Node…Node…

Tune the parts. Turn every goal green.

Traffic
3,200 req/s · reads only
Limit
Node API answers about 500 req/s
Bill
  • Load balancer (L7)8 USD/mo
  • Node API × 16128 USD/mo
136 of 200 USD/mo

Goals

Not tested yet

99 of 100 requests finish faster than this.The share of requests that fail or get no answer.How busy the busiest CPU is. Keep room for spikes.What the servers and services cost each month.

One of 16 replicas answers 200 ms late. After the timeout, the client sends the request again to the pool, up to 2 times. Tune the timeout. Keep p99 under 100 ms, errors under 0.1 percent and CPU use under 56 percent.

A page waits for its slowest callDrag or use arrow keys to change the calls per page

Calls per page: 20 · 18% of pages wait 2.0 s