Case study: Pastebin
Design a paste service where the payload, not the request rate, drives the design: metadata in a database, content in an object store, hot pastes in a cache.
30 min9 steps
By the end you can
- Estimate content and metadata storage separately and size a cache in bytes
- Split metadata and content across stores and order the writes safely
- Design expiry, burn-after-read and cleanup without full scans or races
- Predict the effect of a slow or lossy object store and choose where retries are safe
Requirements
Share immutable text; list/delete owned pastes
- Expiry and burn-after-read
10 KB typical; 10 MB maximum
- p99 <200 ms; 99.9% reads; durable acknowledgements
Estimates
New pastes/day
10M
10 KB each; 10 reads/paste
Writes/s
116
230 peak
Reads/s
1,160
2,300 peak
Content, 5 years
183 TB
10M / 10 KB / 365 / 5
Metadata
5.5 TB
18B / 300 B
Peak egress
23 MB/s
2,300 / 10 KB
Hot cache
200 GB
20M / 10 KB; illustrative upper bound
API
POST /v1/pastes
- content, syntax, expiresIn, burnAfterRead; Idempotency-Key; 201/413/429
GET /p/{id}; DELETE /v1/pastes/{id}
- 200/404/410; owner deletion invalidates caches
Data model: two stores
Metadata database
- id, userId, blobKey, syntax, expiresAt; indexed owner/expiry
Object storage
- blobKey / content; write blob before metadata; sweep orphans
Order the steps of creating a paste so that a crash at any point never leaves a paste that cannot be read.
- 1Insert the metadata row that points at blobKey
- 2Reject the request with 413 if it exceeds the size limit
- 3Write the content to the object store under a new blobKey
- 4Return 201 with the paste ID
High-level design
Deep dive: expiry and burn-after-read
Expiry
- Check expiresAt on reads; return 410; indexed batch cleanup
Burn-after-read
- Atomic conditional metadata deletion selects one reader; bypass shared caches
Alternatives
Database content
- One transaction; simple at small scale; expensive large replicas
Public CDN
- Fast immutable reads; bypass private/burn pastes; purge deletes
No blob retries
- Lost packets become failed reads
Failure scenarios you can run
Inject
Object store 8x slower
Observe
120 ms misses occupy API workers
Result
Cache hits stay fast
Inject
10% object-link loss
Observe
Retry reads with jitter
Result
Creates without safe retry can fail
Inject
Viral paste: 40%
Observe
Hot content stays cached
Result
Long-tail misses reach object storage
Wrap up
Watch
- Object latency, cache bytes, egress
Protect
- Expiry checks, atomic burn, safe retries
Then score each stage from 1 to 4 against the rubric.
Answer this step to continue.