Embed on write, retrieve on read

Agent 10
GPU server (L4)520 USD/moGPU server (L4)520 USD/moUsers25 req/sUsers20 req/sAPI server1 vCPU · 1 GBPostgres1 vCPU · 2 GBSearch index2 vCPU · 4 GBLLM API1 vCPU · 1 GB
520 USD/mo520 USD/moUsers25 req/sUsers20 req/sAPI serv…PostgresSearch i…LLM API

Turn every goal green.

Traffic
Ask25 req/s · reads onlyEdits20 req/s · writes only
Limit
API server answers about 3,300 req/s
Bill
  • GPU server (L4) × 21,040 USD/mo
  • API server4 USD/mo
  • Postgres8 USD/mo
  • 2 more20 USD/mo
1,072 of 1,100 USD/mo

Goals

Not tested yet

99 of 100 requests finish faster than this.How old the data is when a reader gets it.The share of requests that fail or get no answer.What the servers and services cost each month.

A help bot answers 25 questions/s from 600,000 article chunks. Editors change 20 articles/s, and a bulk import at 0:30 raises that to 300/s for 30 s. Keep answer p99 under 1.6 s. A changed article must be searchable within 15 s. You have two L4 GPU hosts.

An index against a full scanTap to add or remove the index

Index · 4 pages read per query · disk writes per insert: 2

Parts you can add: Embedding model. At most 2 parts. Budget: 1,100 USD/mo.

Tap a part to add it, or drag it onto a trace.Click a part to add it, or drag it onto a trace.