Latency numbers
Build a request latency budget, distinguish service time from waiting, and calculate sequential and parallel critical paths.
22 min8 steps
By the end you can
- Distinguish service time, queueing, propagation and serialization
- Calculate latency for sequential calls and parallel joins
- Choose an optimization from a measured request trace
Keep the orders of magnitude straight
1 ms = 1,000 µs = 1,000,000 ns. A 5 ms operation is 5,000 µs. State the number you assume.
- CPU cache1 ns
- Main memory100 ns
- SSD read100,000 ns
- Same-datacenter round trip500,000 ns
- Disk seek10,000,000 ns
- Cross-region round trip80,000,000 ns
Log scale: each gridline step is 10×.
Four costs hide inside one duration
One call hides four costs: waiting, service, serialization, propagation. Fix the biggest term.
- Wait for connection18 ms
- Network8 ms
- Database work4 ms
Sequential round trips add up
Dependent calls add. Batching removes round trips; keep batches bounded.
Loading question…
Parallel work follows the slowest branch
Independent calls cost the slowest branch, then whatever follows.
Loading question…
Read the critical path, not the sum of spans
Overlapping spans are counted once on the critical path. Measure at the user's boundary too.
- Parent span100 ms
- Child A70 ms
- Child B (overlaps A)70 ms
Distance remains after faster hardware
Faster hardware cannot remove 80 ms of propagation. Change the path: cache near the reader, move compute, or drop the call.
Averages do not promise a deadline
Same 20 ms average, different tails. A budget finds big terms; only a distribution predicts the deadline.
Steady
- Almost always ~20 ms
- Deadline is safe
Spiky
- Mostly fast
- Occasional 1 s pauses
- Tail breaks deadline
Check your mastery
Recompute sequential and parallel paths with fresh numbers. Score 80% or more.
Mastery check
5 questions on fresh variants. One attempt each, no time limit. Score 80% or more to master the lesson.
Next: throughput, then caching.
Answer this step to continue.