News feed
Show each user a feed of recent posts from the people they follow. The central decision is when to do the work: at write time (fan-out on write, cheap reads) or at read time (fan-out on read, cheap writes).
Component labels and connections
- Clients (client) · calls: Load balancer
- Load balancer (loadBalancer) · calls: Feed API 1, Feed API 2
- Feed API 1 (service) · calls: Feed cache, Posts DB, Feed store, Fan-out queue
- Feed API 2 (service) · calls: Feed cache, Posts DB, Feed store, Fan-out queue
- Feed cache (cache) · calls: none
- Posts DB (database) · calls: none
- Fan-out queue (queue) · calls: none
- Fan-out workers (worker) · calls: Feed store
- Feed store (kvStore) · calls: none
Requirements
Functional
- Publish a post (text, media references).
- Read a feed of recent posts from followed accounts, newest first (ranking is out of scope here), paginated.
- Follow and unfollow.
Non-functional
- Feed reads p99 under 200 ms; reads outnumber posts ~100:1.
- A new post appears in followers' feeds within a few seconds (eventual consistency is fine).
- Posting must stay fast and available even when fan-out lags.
Worked estimates
Feed reads
≈ 35k reads/s average, ≈ 70k/s peak
300M DAU × 10 feed loads/day ÷ 86,400
Posts
≈ 350 posts/s average
300M × 0.1 posts/day ÷ 86,400
Fan-out writes
≈ 70k feed inserts/s — write amplification of 200×
350 posts/s × 200 followers (median-ish)
Feed storage
≈ 1.2 TB of IDs — feeds store IDs, not post bodies
300M feeds × 500 post IDs × 8 B
Celebrity post
50M inserts for one post: at 70k/s, ~12 minutes of the whole fan-out tier — the reason for the hybrid model
one account with 50M followers
API
POST /v1/posts
Request: { text, mediaIds[] } + Idempotency-Key
Response: 201 { postId }
GET /v1/feed?cursor=
Request: cursor (opaque, encodes last seen post ID)
Response: 200 { posts[], nextCursor }
Cursor pagination stays stable while new posts arrive; offsets skip or repeat items.
POST /v1/follows/{userId}
Request: —
Response: 204
Optionally backfills recent posts into the follower's feed.
Data model
- posts — store: db
- feeds — store: feedstore
posts is the source of truth, sharded by author so 'posts by author X' is one shard.
feeds holds one precomputed list of post IDs per user in a KV store; post bodies are hydrated from a post cache at read time.
The follow graph (follower → followee, both directions indexed) lives in its own store and is read by the fan-out workers.
High-level design
Trace permitted calls in the architecture above, then open the reference canvas to inspect routing.
Deep dives
Push, pull or hybrid
Fan-out on write costs (posts/s × average followers) writes but makes a feed read one key lookup. Fan-out on read costs (feed reads/s × accounts followed) lookups but posting is free. With 100 reads per post and a median of a few hundred followers, push wins for ordinary accounts. A celebrity with tens of millions of followers breaks push — one post is minutes of fan-out — so hybrid systems skip fan-out above a threshold and merge celebrity posts at read time.
Feed storage and hydration
Store post IDs only, capped at a few hundred per user; older pages are computed on demand. A feed read fetches the ID list, then hydrates posts and authors with batched multi-gets from caches. Inactive users can be skipped during fan-out and rebuilt on their next visit, which removes a large share of fan-out writes.
Idempotent fan-out and ordering
The queue delivers at least once, so fan-out must be idempotent: add the post ID to a set or sorted set keyed by post ID rather than appending. Feeds sort by post timestamp or a time-ordered ID (Snowflake-style), so out-of-order fan-out does not matter. Deleting a post removes it from the source; readers filter deleted IDs at hydration instead of un-fanning out.
Ranking
Ranked feeds keep the same pipeline but replace 'newest first' with a scoring pass at read time over a few hundred candidates (from the precomputed list plus other sources). The candidate generation is the system-design problem; the scoring model is not.
Failure scenarios
Posting surges 4× during a live event
Scenario and expected result
From t = 1.2 s to 2.2 s all traffic quadruples; posts go from 150/s to 600/s, beyond the fan-out workers' ~400 posts/s.
Expected: Posting stays fast because it only writes the post and enqueues one message. The fan-out queue absorbs the excess — its depth and consumer lag grow and drain after the surge — so new posts reach feeds late but nothing fails. The queue converts a load spike into delay.
The fan-out workers crash
Scenario and expected result
All fan-out workers die from t = 1.3 s to 1.9 s. Messages they were processing are not acknowledged and become visible again after the 500 ms visibility timeout.
Expected: Feeds stop updating for the outage, then the backlog drains. In-flight messages are redelivered; because the reference worker writes feed entries idempotently, redelivery creates no duplicates. Users never see an error, only staleness.
The feed cache is flushed
Scenario and expected result
The feed cache restarts empty at t = 1.5 s.
Expected: Feed reads fall through to the KV feed store, which is sized to take them: latency rises a little and the cache refills. Precomputed feeds make a cache loss cheap — in a pull design a cache loss means recomputing every feed.
Alternative designs
Reference
- Inspect the architecture above and run the same scenario.
Fan-out on read (pull)
- No fan-out tier and no write amplification: a feed read queries recent posts for every followed account and merges them. Posting is trivial, but every read becomes a scatter-gather over hundreds of authors, so read latency and database load grow with the follow count. In the simulator each feed read becomes a scatter-gather of 8 indexed queries, one per posts shard, so the posts database takes 8 queries per read and every read waits for the slowest shard. At this read rate that is more than the database can serve, and most reads fail — the same hardware that comfortably serves the push design.
Reference
- Inspect the architecture above and run the same scenario.
Append post IDs instead of writing idempotently
- Appending is natural ('push post ID onto the list'), but a fan-out worker that crashes after writing and before acknowledging the message gets it redelivered and appends the same post twice. Write fan-out entries keyed by post ID (a set or sorted set) so a redelivery is a no-op.
Reference
- Inspect the architecture above and run the same scenario.
Hybrid: push for most, pull for celebrities
- Fan out on write for accounts below a follower threshold; for celebrities, skip fan-out and merge their recent posts at read time. Twitter's timeline service works this way. See the Twitter timeline case study.
Clarifying questions
- Chronological or ranked feed?
- Follower count distribution — are there accounts with millions of followers?
- How fresh must a feed be?
- Media in posts, and is hydration in scope?
Common mistakes
- Pure fan-out on write with no celebrity strategy.
- Storing full post bodies in every follower's feed.
- Non-idempotent fan-out with at-least-once delivery.
- Offset pagination on a live feed.
- Doing fan-out synchronously in the post request.
Self-assessment rubric
requirements / levels 1-4
- 1/4 Post and read.
- 2/4 Adds follows and pagination.
- 3/4 Read/write ratio and freshness target.
- 4/4 Distinguishes availability of posting from freshness of feeds.
estimates / levels 1-4
- 1/4 None.
- 2/4 QPS only.
- 3/4 QPS, fan-out write rate and feed storage.
- 4/4 Celebrity fan-out cost and its consequence.
api / levels 1-4
- 1/4 Endpoints only.
- 2/4 Post/feed/follow with codes.
- 3/4 Cursor pagination and idempotent posting.
- 4/4 Explains why offset pagination breaks with live data.
data model / levels 1-4
- 1/4 One table.
- 2/4 Posts and follows.
- 3/4 Posts by author, precomputed feeds of IDs.
- 4/4 Caps, hydration and storage per user justified.
high level / levels 1-4
- 1/4 Query at read time without analysis.
- 2/4 Push or pull chosen.
- 3/4 Push with queue, workers and feed store.
- 4/4 Hybrid with celebrity handling and caches on hydration.
deep dive / levels 1-4
- 1/4 None.
- 2/4 Mentions caching.
- 3/4 Push vs pull with numbers.
- 4/4 Idempotent fan-out, inactive users and deletes.
wrap-up / levels 1-4
- 1/4 Stops.
- 2/4 Metrics.
- 3/4 Consumer lag and hot accounts as risks.
- 4/4 Freshness SLO on fan-out lag with alerting and backfill plans.