Learn
Latency, throughput and prices
Every number on this page comes from the data that runs the simulator. The levels use the same values.
Start a new server · 90 s
Latency ladder
Each step sits lower by the log of its time. Each step also shows how many times slower it is than the step above.
| Step | Time | Than step above |
|---|---|---|
| Nanoseconds | ||
| L1 cache read | 1 nsFastest | Fastest |
| L2 cache read | 4 ns4× slower | 4× slower |
| Light in fiber across a rack (2 m) | 9.79 ns2.4× slower | 2.4× slower |
| L3 cache read | 15 ns1.5× slower | 1.5× slower |
| Hop to the other CPU socket | 50 ns3.3× slower | 3.3× slower |
| Main memory read | 90 ns1.8× slower | 1.8× slower |
| Top-of-rack switch hop | 250 ns2.8× slower | 2.8× slower |
| Light in fiber across a hall (100 m) | 490 ns2× slower | 2× slower |
| Microseconds | ||
| Network card send | 1 µs2× slower | 2× slower |
| Light in fiber across a campus (2 km) | 9.79 µs9.8× slower | 9.8× slower |
| Call to another process on the same host | 50 µs5.1× slower | 5.1× slower |
| Flash (NVMe) read | 75 µs1.5× slower | 1.5× slower |
| Cache read (Redis) | 100 µs1.3× slower | 1.3× slower |
| Round trip in one zone | 500 µs5× slower | 5× slower |
| Milliseconds | ||
| Round trip between zones | 1.5 ms3× slower | 3× slower |
| Round trip on one continent | 20 to 40 ms19× slower | 19× slower |
| Object storage read | 30 ms1.1× slower | 1.1× slower |
| Phone on 4G, round trip | 40 to 90 ms2× slower | 2× slower |
| Round trip between continents | 70 to 160 ms1.8× slower | 1.8× slower |
| Call to an outside API | 150 ms1.4× slower | 1.4× slower |
| Light in fiber around the planet | 196 ms1.3× slower | 1.3× slower |
| Seconds | ||
| A failed part comes back | 30 s153× slower | 153× slower |
| Cold storage read | 60 s2× slower | 2× slower |
| Start a new server | 90 s1.5× slower | 1.5× slower |
Tap two steps to compare them.
Throughput
Requests that one vCPU serves each second, as the simulator counts them.
| Part | Req/s per vCPU |
|---|---|
| Layer 4 load balancer | 1,000,000 |
| Cache (Redis) | 100,000 |
| Event log (Kafka) | 100,000 |
| Layer 7 load balancer | 30,000 |
| Key-value store reads | 30,000 |
| Message queue (RabbitMQ) | 20,000 |
| SQL database reads | 15,000 |
| Key-value store writes | 10,000 |
| API service, compiled | 3,333 |
| SQL database writes | 2,500 |
| API service, Node.js | 1,000 |
| API service, Python | 333 |
Links and buses
How fast data moves over each link. Memory rates are in bytes, network rates in bits.
| Link | Rate |
|---|---|
| GPU memory (H100) | 3.35 TB/s |
| GPU to GPU link (H100) | 450 GB/s |
| Network card, 400G | 400 Gb/s |
| InfiniBand link (NDR) | 400 Gb/s |
| Network card, 100G | 100 Gb/s |
| PCIe 5.0, one lane | 32 Gb/s |
| Network card, 25G | 25 Gb/s |
| VPS network card | 1 Gb/s |
| Event log broker (Kafka) | 300 Mb/s |
Prices
The price table that the levels use. A design costs the sum of its parts each month.
| Item | Price |
|---|---|
| Servers | |
| VPS 1 vCPU 2 GB | 6 USD/mo |
| VPS 2 vCPU 4 GB | 10 USD/mo |
| VPS 2 vCPU 8 GB | 16 USD/mo |
| VPS 4 vCPU 8 GB | 24 USD/mo |
| VPS 4 vCPU 16 GB | 32 USD/mo |
| Dedicated 2 vCPU 8 GB | 45 USD/mo |
| VPS 8 vCPU 32 GB | 60 USD/mo |
| Dedicated 8 vCPU 32 GB | 160 USD/mo |
| GPU server (L4) | 520 USD/mo |
| GPU server (H100) | 1,900 USD/mo |
| Most other parts | 4 USD/mo |
| Usage | |
| Data out from a CDN | 0.01 USD per GB |
| Data out from servers | 0.06 USD per GB |
| Large model, input | 3 USD per million input tokens |
| Large model, output | 12 USD per million output tokens |
| Small model, input | 0.1 USD per million input tokens |
| Small model, output | 0.4 USD per million output tokens |
| Logs and metrics ingest | 0.12 USD per GB |
| Object storage reads | 0.0004 USD per 1,000 requests |
| Object storage writes | 0.005 USD per 1,000 requests |
| Queue messages | 0.4 USD per million requests |
| Cold storage | 0.004 USD per GB per month |
| Object storage | 0.015 USD per GB per month |