Case study: Collaborative editor (Google Docs)
Design a real-time collaborative document editor: Operational Transformation vs sequence CRDTs, append-only operation logs with idempotent opIds, and snapshot compaction.
35 min9 steps
By the end you can
- Estimate operation rates, op-log storage growth, snapshot compaction cadence, and cursor broadcast fan-out
- Compare server-ordered Operational Transformation (OT) and sequence CRDTs (RGA) for concurrent text edits
- Combine an append-only operation log with periodic object-store snapshots and client opId deduplication
- Predict system behaviour under dropped WebSocket acks, compactor outages, and hot all-hands documents
Pin down what you are building
Features
- Concurrent editing, cursors, offline merge, version restore
Targets
- 0 ms local wait; remote propagation <150 ms
Safety
- Eventual convergence; no lost/duplicate ops; about 100 writers, thousands of viewers
Work out the numbers
Operations
2.5M/s
5M editors / 2 s; 150 B/op
Log
32 TB/day
2.5M * 150 B * 86,400
Load
Snapshot + tail
Compact every 100-500 ops; 50 KB snapshot; target <100 tail ops
Cursor fan-out
12,500/s/doc
5 * 50^2; ephemeral, never written to disk
Design the protocol and API
Edit
WS op: docId, opId, baseRev, delta
Commit
Transform since baseRev; append serverRev; ack + broadcast
Load
GET /v1/documents/{docId} -> snapshot + tailOps
Presence
WS cursor/selection; best effort; one unacked OT batch/client
Model the data
oplog
docId + rev -> opId, authorId, delta, timestamp
Snapshots
docId + snapshotRev -> compacted state
opdedup
docId + opId; 24-hour TTL
Draw the high-level architecture
Order the steps when a collaborator submits an edit operation.
- 1Session server checks opId in the dedup store to ensure a reconnect retry is not applied twice
- 2Session server acks the sender and broadcasts the transformed op to connected peers
- 3Session server transforms the op against ops since baseRev and appends it as the next serverRev in opstore
- 4Background compactor worker folds accumulated ops into a new object-store snapshot
- 5WebSocket gateway routes the op for docId to the authoritative document session server
Deep dive: Operational Transformation vs Sequence CRDTs
Concurrent edits
- BCD: Alice inserts A at 0; Bob deletes C at 1
OT
- After Alice, transform delete 1 -> 2; both converge to ABD
Sequence CRDT
- Stable character IDs, neighbour links, tombstones; deterministic merge
Evaluate alternatives
docs-no-dedup
- lost-acks causes duplicateEffect
docs-sync-snapshot
- Full synchronous snapshots; goodput <98%
Server-ordered OT
- Small payloads; central session server
Sequence CRDT
- Offline merges; character IDs and tombstones need management
Run the failure scenarios
Inject
gw1 loses responses for 80 ms
Observe
Retry same opId after durable append
Recover
opdedup prevents repeated insertion
Inject
Compactors down 700 ms
Observe
Loads replay longer tails
Recover
Live edits/loads >99% goodput
Inject
30% traffic targets one doc
Observe
Cache serves loads
Recover
Session serialises edits; cap active writers
Wrap up and self-assess
Converge
Jupiter OT or sequence CRDT
Persist
Idempotent WAL; asynchronous snapshots
Route
Lease-owned document sessions; keep presence ephemeral
Answer this step to continue.