Case studies
Work through requirements, estimates and trade-offs, then run the failure scenarios.
- URL shortener
Turn long URLs into 7-character codes and redirect at 100× the write rate. The interesting parts are code generation without collisions and a read path that the cache, not the database, carries.
- Pastebin
Store text snippets and serve them by ID. It looks like the URL shortener, but the payload is 20× bigger, so the real design question is where the bytes live: metadata in a database, content in an object store, hot pastes in a cache.
- Rate limiter
Protect an API from overload and abuse by rejecting excess requests early with 429. The design choices are the algorithm, where the counters live, and what happens when the limiter itself is slow or down.
- Key-value store
Design a Dynamo-style store: partition keys over a consistent-hash ring, replicate each key N times and let R and W quorums trade consistency for latency and availability.
- Distributed cache
Design a Memcached/Redis-style cache tier that absorbs most reads for a database. The hard parts are not the hash map: they are partitioning, hot keys, invalidation races and what happens when a cache node restarts empty.
- News feed
Show each user a feed of recent posts from the people they follow. The central decision is when to do the work: at write time (fan-out on write, cheap reads) or at read time (fan-out on read, cheap writes).
- Chat (WhatsApp / Messenger)
One-to-one and group messaging with delivery receipts. The core problems are keeping millions of connections open, storing messages once even when clients retry, and delivering them in order to devices that come and go.
- Notification system
Accept notification requests from many internal services and deliver them through push, SMS and email providers. The design is a durable queue between fast producers and slow, flaky third parties, with retries, rate limits and user preferences in between.
- Twitter home timeline
Serve home timelines at hundreds of thousands of reads per second. Twitter's answer was fan-out on write into in-memory timelines, with celebrities merged in at read time — a hybrid that bounds both read cost and write amplification.
- Instagram (photo sharing)
Upload photos, store them durably and serve them to a global audience. Most bytes never touch your servers: the CDN serves views, an object store keeps the originals, and asynchronous workers produce the sizes clients ask for.
- YouTube (video streaming)
Accept uploads, transcode them into an adaptive bitrate ladder and stream segments to millions of concurrent viewers. Bandwidth, not requests, is the constraint: the CDN carries the bytes, and transcoding is a large batch pipeline.
- Uber (ride matching)
Track a million moving drivers and match each ride request to a nearby free driver in about a second. The write path is a firehose of location pings; the correctness problem is never assigning one driver to two riders.
- Proximity service (Yelp nearby)
Find businesses near a location. The data barely changes and the queries are heavily concentrated on dense areas, so the design is a read-optimised geo index — geohash, quadtree or S2/H3 cells — replicated and cached, fed asynchronously from the source of truth.
- Web crawler
Download billions of web pages politely and fresh without getting trapped in infinite URL spaces or hammering small hosts. The architecture revolves around a two-tier frontier (priority + per-host politeness queues), URL canonicalisation with Bloom-filter deduplication, SimHash content deduplication, and aggressive DNS/robots.txt caching.
- Typeahead / autocomplete
Return the top k query completions on every keystroke in under 50 ms so suggestions feel instantaneous. The trick is precomputing the top-k list at every trie node offline or in a streaming window, serving reads entirely from memory, and never mutating the trie synchronously on the read path.
- Full-text search engine
Index billions of documents into an inverted index sharded by document ID, scatter queries across shards in parallel, score candidates with BM25 + static signals (PageRank) + neural re-ranking, and gather the top results with snippets under a tight tail-latency budget.
- Collaborative editor (Google Docs)
Let dozens of people edit the same document concurrently in real time, offline or online, and converge to the exact same state. The core design combines Operational Transformation (OT) or sequence CRDTs with an append-only operation log, background snapshot compaction, and ephemeral cursor/presence fan-out.
- File sync & storage (Dropbox)
Keep files synchronised across a user's devices without re-uploading an entire 1 GB file when one paragraph changes. Files are split into content-addressed chunks (~4 MB or Rabin-fingerprinted variable chunks) stored in an immutable object store, while a transactional metadata database tracks versioned chunk lists and notifies linked devices.
- Payment system (Stripe-like ledger)
Move money without ever charging a customer twice or losing a cent when networks time out. Every mutation carries an end-to-end Idempotency-Key, records immutable double-entry ledger postings (sum of debits = sum of credits) in an ACID transaction with an outbox, and reconciles daily against external card networks.
- Ticket booking (Ticketmaster)
Sell 50,000 concert seats to 2 million fans waiting at 10:00:00 AM without ever selling the same seat twice. The design pairs a virtual waiting room at the edge with two-phase seat reservation (an atomic bounded-TTL hold followed by payment capture) and conditional atomic updates (`decrementIfPositive` / CAS) on seat rows.
- Hotel booking (Booking.com / Airbnb)
Browse availability across millions of rooms and book a stay for a date range. Browsing is 95%+ of traffic and tolerates a few seconds of staleness; booking is a small transactional core where two guests must never claim the same last room and a retried checkout must never book twice.
- Real-time leaderboard
Rank millions of players by score in real time: update a player's score when a match ends, fetch the global top 100, and show any player their exact rank and neighbours. SQL `COUNT(*)` rank queries collapse at scale; augmented skip lists (Redis sorted sets) turn both updates and rank lookups into O(log N) memory operations.
- Ad click event aggregation
Ingest tens of thousands of ad clicks per second, deduplicate retries and fraudulent double-clicks, and produce per-minute billing and dashboard counts within seconds. Because clicks are money, the pipeline needs event-time windowing, watermarks for late mobile events, and end-to-end exactly-once accounting.
- Metrics monitoring and alerting (Prometheus / Datadog)
Collect tens of millions of time-series samples per second from hosts and services, compress them in memory, serve fast dashboard range queries, and evaluate alerting rules without failing when the fleet being monitored is itself on fire.
- Distributed job scheduler
Schedule one-off delayed tasks and recurring cron jobs across a fleet of workers with at-least-once execution, lease heartbeats, fencing tokens so zombie workers cannot corrupt downstream state, and isolation when every tenant schedules a job for 00:00 UTC.
- Stock exchange (order matching engine)
Accept limit and market orders, enforce pre-trade risk limits, and match buys against sells by strict price-time priority. Traditional database locks are orders of magnitude too slow and non-deterministic; a real exchange sequences commands into a total order per symbol and runs a single-threaded, in-memory deterministic state machine.
Planet-scale references
Illustrative models with stated assumptions. Scale mode estimates load; correctness stays unknown.