Connection pooling
A connection pooler shares a small set of database connections among many clients. Twelve replicas with 20 connections each open 240 connections, but the database allows 200. A pooler caps the database side at 80 connections.
See it on the board
The same traffic runs on two boards. The simulator measured both when this page was built.
| Measure | Goal | 12 replicas connect straight to the database | A pooler holds 80 connections |
|---|---|---|---|
| Error rate | none | 16.8% Fail | 0% Pass |
| p99 latency | under 200 ms | 27 ms Pass | 28 ms Pass |
| Monthly cost | 60 USD/mo | 60 USD/mo |
Key numbers
| Number | Value |
|---|---|
| Default pool size, one API replica | 20 connections |
| Connections a database accepts | 200 connections |
| Memory for each open connection | 1 KB |
Pooler · 80 connections asked · 80 open, limit 200 · 0 refused
How it works
A database connection is a costly resource. Each one holds memory and a process on the database side, and the database accepts only so many. A client that needs a connection for every request would be slow, so apps keep a pool of open connections.
The problem comes with scale. Each replica has its own pool. In the mini board there are 12 replicas with 20 connections each, which is 240 connections. The database allows 200, so some replicas get connection errors. Adding replicas makes it worse.
A pooler sits between the replicas and the database. The replicas connect to the pooler, and the pooler holds a small number of real connections, here 80. Requests share these connections for short turns. The database sees 80 connections, however many replicas you add.
Pooling has limits. A request that holds a connection for a long time blocks others, so keep transactions short. The pooler is one more part, and it can fail. Smaller pools in each replica also work if their total stays under the limit.
In the levels you cut the connection count at the database and keep p99 low.