Learn
Concepts
Each concept page gives a short definition, key numbers from the simulator, two boards that show the effect, and the levels that teach it.
Core concepts
- Async work Async work moves a slow task out of the request path, so the caller gets an answer before the task ends.
- Authentication Authentication proves who a user or a service is, with a password, a code, a key or a token.
- Authorization Authorization decides what an identity may do after it proves who it is.
- Availability Availability is the share of time, or of requests, that a system serves correctly.
- Backpressure Backpressure is a signal that tells a producer to slow down when consumers cannot keep up.
- Bandwidth Bandwidth is the number of bits that a link can carry each second.
- Caching A cache keeps copies of recent answers close to the caller.
- Capacity Capacity is the most work that a part can do each second.
- Concurrency Concurrency is the number of requests that a service works on at the same time.
- Consistency A consistency rule says which version of the data a read can return.
- Coordination Coordination lets several machines agree on one fact, such as which runner owns a job.
- Cost Cost is the monthly price of the hosts, parts and traffic in a design.
- Cryptography Cryptography protects data with keys, so that only a holder of the right key can read or change it.
- Data models A data model is the shape in which a database keeps data, such as tables, documents or keys.
- Data pipelines A data pipeline moves data through steps such as extract, transform and load, often in large batches.
- Data placement Data placement decides where each piece of state lives.
- Deadlines and timeouts A timeout stops the wait for a call when a set time passes, and a deadline is the time budget for the whole request.
- Delivery guarantees A delivery guarantee says how many times a message can reach its consumer.
- Deployment Deployment is the way a new version of the software reaches users.
- Distributed compute Distributed compute splits one large job across many machines or GPUs, so that it ends sooner.
- Failure domains A failure domain is a group of parts that one fault can stop at the same time.
- Freshness Freshness is the age of the data that a reader sees.
- Idempotency An operation is idempotent when it has the same effect if it runs twice or once.
- Indexing An index is a sorted structure that lets a query find rows without a read of the whole table.
- Inference Inference is the work of running a trained model to answer a request.
- Latency Latency is the time from the moment a request starts to the moment its answer arrives.
- Load balancing A load balancer spreads requests across several servers and stops sending to a server that fails.
- Memory hierarchy The memory hierarchy is the ladder of places that keep data, from CPU caches and RAM to disks and remote storage.
- Model quality Model quality is how often a model gives the right answer on real traffic, not only on test data.
- Observability Observability is the ability to see what a system does inside from the data it sends out: metrics, logs and traces.
- Partitioning Partitioning splits data across several nodes, so each node stores and serves only part of it.
- Propagation delay Propagation delay is the time that a signal needs to travel between two points.
- Queueing A queue holds work that arrives while the server is busy.
- Recovery Recovery is the work that brings a system and its data back after a loss.
- Replication Replication keeps copies of the same data on more than one machine.
- Retries A retry sends a failed call again, because many faults last less than 1 s.
- Storage Storage keeps data when a process stops, on disks, in object stores or in archives.
- Transactions A transaction groups several writes so that all of them happen or none of them do.
- Transport Transport is how bytes move between two programs: connections, handshakes, streams and the protocols on top.
- Trust boundaries A trust boundary is a line where data from a less trusted source enters a part with more power.
Comparisons
- Account model vs coin model In the account model, each payment updates one balance row, so all payments for one merchant wait in line.
- B-tree vs LSM tree A B-tree updates pages in place, so each write can cost random disk work.
- Cache-aside vs write-through With cache-aside the service writes to the database and the cache keeps its old copy until the entry expires.
- Database rows vs object storage for images A database row that holds image bytes makes every read use database CPU and network.
- Date shard key vs hash shard key A date key sends all of today's orders to one shard.
- Direct calls vs backend for frontend With direct calls, each phone calls four services and gets four responses over the cell link.
- Direct origin vs CDN for images A CDN keeps copies of popular files near users, so most reads never reach the origin.
- Dual write vs outbox A dual write saves the row and then sends the event as a second step, so a failure between the steps loses or delays the event.
- ETL vs ELT ETL cleans the data before it loads.
- Fan-out on read vs fan-out on write Fan-out on read builds each feed when a user asks for it.
- Full logs vs sampled logs A managed log intake bills each line.
- GeoDNS vs anycast for failover GeoDNS picks an edge by name lookup, and resolvers keep the old answer until the TTL ends.
- Global limit vs per-key limit A global limit counts all requests together, so a botnet and real people share one budget.
- GPU vs CPU for boosted trees A GPU pays off with large batches, and a batch adds waiting time.
- Hash partition vs time partition A hash partition spreads rows by key, so a query for one day reads every partition.
- HTTP/1.1 vs HTTP/2 HTTP/1.1 sends one request at a time on a connection, so requests wait in line.
- HTTP/2 vs HTTP/3 on a lossy link HTTP/2 runs on TCP, so one lost packet stops every stream on the connection.
- Immediate retries vs exponential backoff Immediate retries send a failed request again at once, so 1,000 callers/s add a second wave to a service that tries to recover.
- JSON over HTTP vs gRPC JSON text costs CPU to write and read, and it is larger on the wire.
- Key call per record vs envelope encryption A key service call for each record hits the rate cap of the service.
- Lease vs fencing token A lease gives one runner the right to work for a short time, but a paused runner can still send work later.
- Live ranking vs nightly scores Live ranking scores 500 items on a CPU for each page view.
- Local sessions vs shared sessions Local sessions live in the memory of one server, so a user who lands on another server loses the session.
- LRU vs LFU LRU keeps the keys that were used last.
- Every message vs customer facts as memory Storing every message keeps all detail, but the index must hold 120 million items.
- Modulo hash vs consistent hash Modulo hashing picks a node with the key hash modulo the node count, so a new node moves most keys.
- Monolith vs microservices: the latency price A monolith calls its modules in memory.
- More API servers vs a faster disk More servers help only when the servers are the limit.
- N+1 calls vs batched calls An N+1 resolver makes one call for the list and one more call for each record.
- No breaker vs circuit breaker Without a breaker, every call waits for the failed payment API, and the waiting calls pile up.
- No coalescing vs request coalescing Without coalescing, every reader that misses a key starts its own fill from the database.
- No prefix cache vs prefix cache Without a prefix cache, the GPU reads the same 5,400 system prompt tokens again for every chat.
- No shedding vs load shedding Without shedding, a sale spike queues every request at the API.
- Offline database vs online feature store The offline order database keeps full history for training and is slow for point lookups.
- OFFSET vs keyset pagination OFFSET reads the rows it skips, so deep pages cost more.
- One big server vs three small servers One big server has no spare when it fails.
- One replica vs majority quorum With one replica for reads and writes, a read can reach a follower that has not applied the latest write.
- Optimistic vs ZK rollup An optimistic rollup lets anyone challenge a batch for a set time before a withdrawal is final.
- Polled flags vs pushed flags A polled flag is cached in each instance and refreshed when its time ends.
- Polling vs server-sent events With polling each browser asks for news every second, even when there is none.
- Random vs keyed partitions A random partition spreads events evenly, but two events of one user can reach different consumers.
- Round robin vs least connections Round robin gives each backend the same number of requests.
- Row store vs column store A row store keeps each row together, so a sum over 2 columns still reads every column of every row.
- Replicas in one rack vs in different racks Two replicas in one rack fail together when the rack switch fails.
- Server rendering vs static pages Server rendering builds the page on a server for every view.
- SHA-256 vs Argon2id for passwords SHA-256 is fast, so a thief with the password table can test 10,000 guesses/s on one core.
- Shared fleet vs cells In a shared fleet, every user group runs on the same servers, so one failure reaches all of them.
- Shared tables vs database per tenant Shared tables keep every tenant in one database and filter by tenant ID.
- One agent vs a planner with executors One agent does every step with one large model.
- Single database vs read replicas One database serves every read and write.
- One zone vs two zones Two servers in one zone fail together when the zone fails.
- SMS codes vs passkeys A look-alike site can relay an SMS code at once, and an attacker can take over a phone number.
- Strict expiry vs stale-while-revalidate Strict expiry drops an entry when its time ends.
- Sync vs async replication Sync replication confirms a write only after a follower has it.
- Synchronous link vs outbox A synchronous write makes the chat wait for the archive on Earth, 1.28 s away one way.
- Tumbling vs sliding windows A tumbling window closes and a new one starts, so each event lands in one window.
- Wait for the server vs optimistic update Waiting shows a like only after the server answers.
- Warm standby vs active-active A warm standby keeps a small copy in a second region and grows it after a failure.
- Work queue vs log A work queue gives each message to one worker and then deletes it.
- Write-back vs write-through Write-back confirms a write as soon as the cache has it and sends it to the database later.