Layer 7 path routing
A layer 4 balancer routes by address and port, and it can handle about 1,000,000 requests/s. A layer 7 balancer reads the HTTP request, so it can route /api and /images to different backends and end TLS, at about 10,000 requests/s for each part.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | Layer 4 pass-through | Layer 7 with TLS and path routing |
|---|---|---|---|
| p99 latency (Api) | under 50 ms | 10.1 s Fail | 32 ms Pass |
| p99 latency (Images) | under 200 ms | 10.1 s Fail | 92 ms Pass |
| Error rate (Api) | under 0.1% | 4.4% Fail | 0% Pass |
| Error rate (Images) | under 0.1% | 4.6% Fail | 0% Pass |
| Monthly cost | 76 USD/mo | 76 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Requests per second, one layer 4 balancer | 1,000,000 ops/s |
| Requests per second, one layer 7 balancer | 10,000 ops/s |
| Layer 7 balancer hop, median | 0.3 ms |
5 servers · 72% busy · p99 66 ms
How it works
A load balancer works at one layer of the network. A layer 4 balancer sees addresses and ports, and it passes the encrypted bytes through. It does little work for each packet, so one part can carry about 1,000,000 requests/s in the catalog.
A layer 7 balancer reads the HTTP request. It can see the path, the host and the headers. So it can send /api to one group of servers and /images to another. It can also end TLS, so the backends receive plain traffic and need no certificates.
In the mini board the gateway must end TLS and route by path. A layer 4 balancer cannot do either, because it cannot read the request. Its p99 misses the 180 ms limit in the run. The layer 7 balancer reads the request and routes it, and p99 stays inside the limit.
Layer 7 costs more work for each request. It decrypts and parses every request, so a part carries about 10,000 requests/s. At high load you need more balancers. A common design puts a layer 4 balancer in front, to spread the load, and layer 7 balancers behind it.
In the levels you choose the layer for the routing that the design needs.