Ticket booking (Ticketmaster)
Sell 50,000 concert seats to 2 million fans waiting at 10:00:00 AM without ever selling the same seat twice. The design pairs a virtual waiting room at the edge with two-phase seat reservation (an atomic bounded-TTL hold followed by payment capture) and conditional atomic updates (`decrementIfPositive` / CAS) on seat rows.
Component labels and connections
- Clients (client) · calls: Load balancer
- Load balancer (loadBalancer) · calls: Booking API 1, Booking API 2
- Booking API 1 (service) · calls: Seat-map cache, Seat inventory DB, Hold TTL expiry queue
- Booking API 2 (service) · calls: Seat-map cache, Seat inventory DB, Hold TTL expiry queue
- Seat-map cache (cache) · calls: none
- Seat inventory DB (database) · calls: none
- Hold TTL expiry queue (queue) · calls: none
- Hold expiry workers (worker) · calls: Seat inventory DB
Requirements
Functional
- View an event's available seats by section and price tier.
- Two-phase booking: place a temporary hold (e.g. 5–10 minutes) on specific seats while the user enters payment details, then confirm the ticket or automatically release the hold on expiry.
- Fair admission during massive on-sale drops via a virtual waiting room.
Non-functional
- Hard correctness invariant: never sell or hold the same seat for two users (`oversell` must be zero).
- Seat-map browsing must stay fast (< 100 ms) even at 100:1 read-to-write ratio; stale availability by a couple of seconds on the map is acceptable because the hold step is authoritative.
- Survive 100× flash-crowd traffic at the exact second tickets go on sale without cascading failure.
Worked estimates
Flash-crowd arrival
virtual waiting room shapes an unbounded spike into the exact TPS the booking DB can sustain
2M fans in waiting room admitted at controlled rate of ~10k checkout sessions/s
Seat-map reads
≈ 100k reads/s — absorbed by a short-TTL (2–5 s) seat-map cache + push diffs
500k active browsers refreshing/selecting sections ÷ 5 s
Hold / reserve writes
≈ 500–2,000 atomic seat updates/s per hot event
50,000 seats × ~3 hold attempts per seat (contention + expiries) over 5 minutes
Inventory dataset
≈ 6.4 MB per stadium event — an entire stadium's inventory fits inside CPU L3 cache!
50,000 seats × ~128 B per seat row
API
GET /v1/events/{eventId}/seats
Request: section filter
Response: 200 { seats: [{ seatId, status, priceCents }] } (cached 2–5 s)
POST /v1/seats/{seatId}/hold
Request: { userId, idempotencyKey }
Response: 200 { holdId, holdToken, expiresAt } · 409 seat already held/sold
POST /v1/holds/{holdId}/confirm
Request: { holdToken, paymentToken }
Response: 200 { ticketId } · 410 hold expired (fencing check on holdToken)
POST /v1/seats/{seatId}/release
Request: { holdToken }
Response: 200
Called on user cancel or by the hold-expiry reaper.
Data model
- seats — store: bookingdb
seats: `(seatId, eventId, status: AVAILABLE|HELD|SOLD, remaining: 1|0, holdToken, holdExpiresAt)`. Partitioned by `eventId` (or `(eventId, sectionId)` for stadium-scale events).
Atomic hold transition: `UPDATE seats SET status='HELD', remaining=0, holdToken=:tok, holdExpiresAt=:exp WHERE seatId=:id AND (status='AVAILABLE' OR (status='HELD' AND holdExpiresAt < NOW()))`.
holdToken acts as a fencing token: if a user's 5-minute hold expires and Seat A is re-held by User 2 (`holdToken = 2`), User 1's late payment callback carrying `holdToken = 1` is rejected and refunded.
High-level design
Trace permitted calls in the architecture above, then open the reference canvas to inspect routing.
Deep dives
Two-phase reservation and the late-payment race (fencing tokens)
Charging a credit card takes seconds; you cannot hold a database transaction open across it. Phase 1 atomically flips the seat from `AVAILABLE` to `HELD` with `holdExpiresAt = now + 5m` and increments a monotonic `holdToken`. Phase 2 charges the card and then commits `HELD → SOLD WHERE seatId = :id AND holdToken = :tok`. If User 1's payment finishes at 5m02s after the hold expired and Seat A was given to User 2 (`holdToken` incremented), the `WHERE holdToken = :tok` check fails and triggers an automatic refund for User 1 instead of double-selling the seat.
Expiring holds reliably
Two mechanisms work together so a seat is never stuck in `HELD`: (1) a delayed message in the `holdexpirer` queue (or timing wheel) fires at `t = +5m` to flip `HELD → AVAILABLE` and invalidate the seat-map cache, and (2) lazy expiry in the hold SQL predicate itself (`WHERE status = 'AVAILABLE' OR (status = 'HELD' AND holdExpiresAt < NOW())`), which lets a new buyer claim an expired seat immediately even if the background reaper is lagging.
Virtual waiting room and bot protection
When 2 million fans arrive for 50,000 seats, letting all 2 million hit the seat map and hold API causes 97.5% wasted work and lock thrashing. A virtual waiting room at the CDN/edge assigns each authenticated fan a queue position at on-sale time and issues signed, short-lived admission tokens at the exact rate checkout can drain (~5,000 users/min), turning a thundering herd into steady flow.
Failure scenarios
Everyone clicks the same front-row seat on a Taylor Swift drop
Scenario and expected result
During an on-sale drop, 25% of all traffic targets the same hot seat ID.
Expected: The atomic `decrementIfPositive` grants the hold to exactly one request and returns `409 Conflict` to every concurrent contender. Goodput dips slightly because 409 rejections count as unsuccessful attempts, while the `oversell` invariant stays completely clean (`clean: true`). Compare the read-then-write alternative.
Fans hammer F5 on the seat map (2× traffic spike)
Scenario and expected result
From t = 1.2 s to 2.2 s traffic doubles as waiting fans repeatedly refresh the seat map.
Expected: The short-TTL seat-map cache with request coalescing absorbs the read storm so the booking database remains dedicated to atomic hold and release transactions.
Booking database primary crashes mid-sale
Scenario and expected result
The primary seat inventory database fails at t = 1.5 s and promotes a semi-sync replica by t = 2.0 s.
Expected: Cached seat maps keep rendering during the failover. Seat holds fail closed for ~500 ms until the new leader takes over with fencing; no confirmed hold is lost and no seat is double-sold.
Alternative designs
Reference
- Inspect the architecture above and run the same scenario.
Read seat availability, then write (`SELECT` then `UPDATE`)
- If the API reads `remaining == 1` in one query and then sends `UPDATE seats SET remaining = remaining - 1`, two concurrent requests for the same front-row seat both read `1` and both succeed — double-selling the seat (`oversell`). Run the `hot-event` scenario against this alternative to see the violation.
Reference
- Inspect the architecture above and run the same scenario.
Hold seats until payment completes without a TTL
- Without a strict 5-minute lease/TTL on `HELD` seats, users who close their browser tab or scalper bots that open thousands of checkouts lock up the entire stadium inventory indefinitely without paying.
Reference
- Inspect the architecture above and run the same scenario.
Single-thread actor per event in memory (with WAL)
- Because one stadium is only ~50,000 seats (~6 MB), a single-threaded in-memory state machine per event (backed by a replicated write-ahead log) can process 50,000 seat holds per second with zero lock contention. More custom operational tooling than SQL `UPDATE ... WHERE`, but eliminates row-lock queuing on ultra-hot events.
Clarifying questions
- Do users pick exact individual seats on a map, or 'best available' in a section/tier?
- How long is a seat held during checkout before it expires?
- What is the peak concurrent user count for a major stadium on-sale drop?
- How should we handle adjacent multi-seat requests (e.g. 4 seats together)?
Common mistakes
- Reading seat availability with `SELECT` and then writing `UPDATE` in a separate statement (`oversell`).
- Holding a database lock while waiting for the user to type credit card details or for the PSP to respond.
- Forgetting the race condition where a payment succeeds 1 second after the 5-minute hold expired (missing fencing token).
- Relying only on a background cron job to expire holds without lazy expiry on the `holdExpiresAt` timestamp.
- Not caching the seat map or omitting a virtual waiting room for 100× flash sales.
Self-assessment rubric
requirements / levels 1-4
- 1/4 Book a ticket.
- 2/4 Seat map + hold + pay.
- 3/4 Two-phase hold with TTL, zero oversell invariant, and flash-crowd handling.
- 4/4 Addresses late-payment races after hold expiry and fair queuing.
estimates / levels 1-4
- 1/4 None.
- 2/4 QPS.
- 3/4 Read vs write ratio, stadium dataset size in RAM, and on-sale spike factor.
- 4/4 Proves one stadium's seats are only a few megabytes — contention, not data size, is the bottleneck.
api / levels 1-4
- 1/4 POST /book.
- 2/4 Hold and confirm endpoints.
- 3/4 Returns `409` on contention, `holdToken` + `expiresAt`, and idempotent confirm.
- 4/4 Includes virtual waiting room admission token and explicit release.
data model / levels 1-4
- 1/4 Seats table.
- 2/4 Seat status column.
- 3/4 Seat state machine (`AVAILABLE/HELD/SOLD`) with `holdToken` and `holdExpiresAt`.
- 4/4 Lazy expiry predicate + sharding by `(eventId, sectionId)`.
high level / levels 1-4
- 1/4 API + DB.
- 2/4 Adds cache.
- 3/4 Edge waiting room, coalesced seat-map cache, atomic DB holds, hold-expiry queue.
- 4/4 Explains failover with fencing and why payment stays outside DB locks.
deep dive / levels 1-4
- 1/4 None.
- 2/4 Mentions row locks.
- 3/4 Atomic CAS/`decrementIfPositive` vs read-then-write overselling.
- 4/4 Fencing tokens (`holdToken`) for late payments, lazy+active expiry, and single-writer event actors.
wrap-up / levels 1-4
- 1/4 Stops.
- 2/4 Mentions latency.
- 3/4 Overselling, stuck holds, and bot floods as top risks.
- 4/4 Zero-oversell audit checks, hold-expiry lag alerts, and waiting-room admission pacing.