Skip to content
All case studies

Instagram (photo sharing)

Upload photos, store them durably and serve them to a global audience. Most bytes never touch your servers: the CDN serves views, an object store keeps the originals, and asynchronous workers produce the sizes clients ask for.

Open reference designInteractive lessonPractice an interview
ClientsDNS (CDN, o…CDN edgeLoad balanc…Media API 1Media API 2Media metad…Photo storeResize jobsResize work…
Arrows show permitted calls. Open the canvas to inspect route programs.
Component labels and connections
  • Clients (client) · calls: DNS (CDN, origin fallback)
  • DNS (CDN, origin fallback) (dns) · calls: CDN edge, Load balancer
  • CDN edge (cdn) · calls: Load balancer
  • Load balancer (loadBalancer) · calls: Media API 1, Media API 2
  • Media API 1 (service) · calls: Media metadata (sharded Postgres), Photo store, Resize jobs
  • Media API 2 (service) · calls: Media metadata (sharded Postgres), Photo store, Resize jobs
  • Media metadata (sharded Postgres) (database) · calls: none
  • Photo store (objectStore) · calls: none
  • Resize jobs (queue) · calls: none
  • Resize workers (worker) · calls: Photo store

Requirements

Functional

Non-functional

Worked estimates

  • Uploads

    ≈ 1,160 uploads/s average, ≈ 3k/s peak

    100M photos/day ÷ 86,400

  • Storage

    ≈ 90 PB/year — object storage with tiering for old photos

    100M × (2 MB original + ~0.5 MB of resized versions) × 365

  • Views

    ≈ 580k image requests/s

    500M DAU × 100 image views/day ÷ 86,400

  • Egress

    ≈ 87 GB/s ≈ 700 Gbit/s — only a CDN serves this economically

    580k/s × ~150 KB average served size

  • Origin traffic

    ≈ 29k/s to the origin; the hit rate decides origin size

    580k/s × (1 − 95% CDN hit rate)

  • Resize workers

    ≈ 140 cores busy on average; ≈ 400 at peak

    Little's law: 1,160 uploads/s × 120 ms CPU

API

POST /v1/media

Request: multipart image + { caption }; resumable for large files

Response: 201 { mediaId, urls{} } (sizes may be pending)

GET /p/{photoKey}_{size}.jpg

Request: served by the CDN

Response: 200 image · 404

Keys are immutable: an edit creates a new key, so CDN caching never needs invalidation.

GET /v1/users/{id}/media?cursor=

Request: cursor = last mediaId

Response: 200 { media[] }

Data model

media rows are sharded by userId (Instagram used thousands of logical shards on PostgreSQL), so a profile grid is one shard.

mediaId embeds a timestamp and the shard ID (41 bits ms + 13 bits shard + 10 bits sequence), so IDs sort by time and locate their shard without a lookup.

Image bytes live in the object store under immutable keys; the database only stores the key.

High-level design

Trace permitted calls in the architecture above, then open the reference canvas to inspect routing.

Deep dives

The upload path

Clients upload directly to the object store with a pre-signed URL (or through an upload service for small files), resumable in chunks so a dropped mobile connection does not restart a 10 MB upload. The API then writes the metadata row and enqueues processing: resizing, content moderation, and feature extraction. The response returns before processing finishes; clients show a local preview meanwhile.

Serving images

Image URLs are immutable (content-addressed or versioned), so CDN entries never need invalidation and can be cached for a year. Popularity is heavily skewed, so the CDN's hit rate is high for recent popular photos and low for the long tail of old ones; an origin shield (a regional mid-tier cache) collapses misses from many edges into one origin read. Choose image sizes per device and use modern formats to cut egress.

IDs and sharding

Instagram generated 64-bit IDs inside PostgreSQL: 41 bits of milliseconds since a custom epoch, 13 bits of logical shard ID, 10 bits of per-shard sequence. Thousands of logical shards were mapped onto fewer physical servers, so rebalancing moves whole logical shards. IDs sort by time, which makes 'newest first' queries and cursors cheap.

Failure scenarios

The CDN edge goes down

Scenario and expected result

The CDN fails from t = 1.5 s on. DNS still points clients at it until the cached record expires (TTL 300 ms here, scaled down from the usual 30–300 s), then fails over to the origin.

Expected: Requests fail until the DNS record expires; after that, the origin takes every image view. It survives here because the cell's origin was sized generously, but note what changed: object-store reads jump from the miss rate to the full rate, and egress now leaves your origin. DNS TTL is your failover time — and real resolvers often hold records longer than the TTL.

Open this scenario

Uploads spike 3× (New Year's Eve)

Scenario and expected result

From t = 1.2 s to 2.2 s all traffic is 3× higher; uploads go from 40/s to 120/s, above the resize workers' ~60/s (each job is ~120 ms of CPU plus ~60 ms to store the results).

Expected: Uploads are acknowledged as fast as before because resizing is asynchronous; the resize queue grows and thumbnails lag behind. Views keep hitting the CDN. Scale workers on queue depth.

Open this scenario

The object store slows down 6×

Scenario and expected result

From t = 1.5 s to 2.3 s object-store reads take ~150 ms instead of ~25 ms.

Expected: Most views are CDN hits and do not notice. CDN misses and uploads slow down, and the slowest misses exceed the API's 200 ms storage timeout and fail. The CDN is shielding the storage tier — the hit rate is a reliability feature, not just a cost one.

Open this scenario

Alternative designs

Reference

  • Inspect the architecture above and run the same scenario.

Serve images from your own servers

  • Full control and no CDN bill, but 700 Gbit/s of egress from a few regions means high latency far from them and enormous origin capacity. A CDN (or your own edge network, as Facebook and Netflix built) is the only practical option at this scale.

Reference

  • Inspect the architecture above and run the same scenario.

Resize synchronously during upload

  • The client gets every size immediately, but uploads now take hundreds of milliseconds of CPU each and a spike of uploads saturates the API tier. Asynchronous workers let uploads stay fast and absorb spikes as queue depth.

Reference

  • Inspect the architecture above and run the same scenario.

Store photos in a filesystem-like blob store you run

  • Facebook's Haystack packed many photos into large files to avoid per-file metadata reads, then f4 moved older 'warm' photos to erasure-coded storage at lower cost. Worth it at Facebook's scale; most teams should use a managed object store and lifecycle tiers.

Clarifying questions

Common mistakes

Self-assessment rubric

requirements / levels 1-4
  1. 1/4 Upload and view.
  2. 2/4 Sizes and durability.
  3. 3/4 Read-heavy ratio, latency targets, async processing.
  4. 4/4 Defines what 'uploaded' means (acknowledged vs processed).
estimates / levels 1-4
  1. 1/4 None.
  2. 2/4 QPS.
  3. 3/4 Storage, egress and origin traffic from hit rate.
  4. 4/4 Worker sizing for processing via Little's law.
api / levels 1-4
  1. 1/4 Endpoints.
  2. 2/4 Upload/view.
  3. 3/4 Resumable upload and immutable image URLs.
  4. 4/4 Pre-signed direct upload and cursor pagination by time-ordered ID.
data model / levels 1-4
  1. 1/4 Photos in DB.
  2. 2/4 Metadata + blob store.
  3. 3/4 Metadata sharded by user, blobs keyed immutably.
  4. 4/4 ID scheme that encodes time and shard.
high level / levels 1-4
  1. 1/4 Servers serve images.
  2. 2/4 CDN + object store.
  3. 3/4 Async processing pipeline.
  4. 4/4 Origin shield, DNS failover and how each tier fails.
deep dive / levels 1-4
  1. 1/4 None.
  2. 2/4 Mentions CDN.
  3. 3/4 Upload path and serving path in detail.
  4. 4/4 Storage tiering, ID generation and rebalancing.
wrap-up / levels 1-4
  1. 1/4 Stops.
  2. 2/4 Metrics.
  3. 3/4 CDN hit rate and processing lag as signals.
  4. 4/4 Cost model (egress and storage) and failure drills.