Random vs keyed partitions

A random partition spreads events evenly, but two events of one user can reach different consumers. A keyed partition sends all events of one user to one consumer, in order. At 1,000 events/s over 8 consumers, random partitions deliver 18,463 events out of order. Keyed partitions deliver none.

Play the level

Delivered twice, charged onceTap to turn the ID check on or off

13 deliveries · 10 payments applied · 0 charged twice

Case 1

1,000 events/s go to 8 consumers, and each event needs 2 ms of CPU. The events of one user must finish in the order sent. Queue lag must stay below 5 s.

Case 1
Measure Goal Random partitions One partition for each user
Out of order none 18,463 Fail 0 Pass
Queue lag under 5 s 0 s Pass 0 s Pass
Monthly cost 72 USD/mo 72 USD/mo
p99 latency 56 ms 56 ms
Error rate 0% 0%
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker1 vCPU · 2 GB × 8
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker1 vCPU · 2 GB × 8
Random partitions: misses a goal
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker1 vCPU · 2 GB × 8
KafkaTCPUsersKafka log2 vCPU · 2 GBWorker1 vCPU · 2 GB × 8
One partition for each user: meets the goals