Recall vs speed

Model 13
HTTP/2gRPCUsers3,000 req/sAPI server4 vCPU · 4 GBVector DB8 vCPU · 64 GB
HTTP/2gRPCUsers3,000 req/sAPI server4 vCPU · 4 GBVector DB8 vCPU · 64 GB

Tune the parts. Turn every goal green.

Traffic
3,000 req/s · reads only
Limit
API server answers about 13,000 req/s
Bill
  • API server16 USD/mo
  • Vector DB256 USD/mo
272 of 400 USD/mo

Goals

Not tested yet

99 of 100 requests finish faster than this.The share of requests that fail or get no answer.What the servers and services cost each month.

A vector index holds 10 million items with 768 dimensions on 8 fixed vCPU. It must answer 3,000 searches/s. At least 95 percent of searches must find the right item, at p99 under 20 ms. You can change only the search width.

An index against a full scanTap to add or remove the index

Index · 4 pages read per query · disk writes per insert: 2