Percentiles and tail latency
Calculate percentile ranks, predict tail amplification in fan-out, and choose measurements and mitigations that preserve the user experience.
24 min8 steps
By the end you can
- Locate a percentile using a stated nearest-rank convention
- Calculate the probability that a join-all request encounters a slow branch
- Choose valid percentile aggregation and a bounded tail-latency intervention
An average can hide the slow users
99 requests at 10 ms and one at 1,010 ms average 20 ms. One user waited over a second. p99 = the value at or below which about 99% of observations fall.
- Mean20 ms
- p99 (nearest rank)10 ms
- Slowest request1,010 ms
Log scale: each gridline step is 10×.
Calculate the rank before reading the value
Nearest rank = ceil(p × N), one-based. The rank is a position, not a time. State sample size, interval and convention.
p99 rank
990
ceil(0.99 × 1000)
p99 from 10 requests
= max
weak evidence
Loading question…
Do not average machine percentiles
Quantiles are not means. Merge histogram counts or a mergeable sketch, then compute the percentile. Check bucket width near your SLO.
A rare branch delay becomes common in fan-out
P(any slow) = 1 − (1 − q)^n, assuming independent branches. It is a probability at a threshold, not an exact p99.
All fast
36.6%
0.99^100
At least one slow
63.4%
Loading question…
Check the independence assumption
Shared bottlenecks make delays coincide, so the independence formula overstates. Perfect correlation: parent slow rate equals the single-branch rate.
Independent
- Slow events scatter
- About 63.4% of parents slow
Perfectly correlated
- Same 1% of requests
- 1% of parents slow
Reduce tails without causing another queue
Remove variance first: bound batches, admit load, fix hot partitions, isolate noisy neighbours. Hedge only with spare capacity.
Admission control
- Cuts queueing
- Rejects excess
Deadline + partial result
- Bounds waiting
- Caller must see it
Hedged request
- Extra copy after delay
- Bound the rate
- Worse if saturated
Measure the wait the user actually experienced
Include time spent waiting to start. A timeout is a failure even with no latency sample. Watch latency and availability together.
Success-only p99
- Improves
Failures
- Triple
- Slow requests moved here
Check your mastery, then stress the tail
Score 80% or more. State the independence assumption each time.
Mastery check
5 questions on fresh variants. One attempt each, no time limit. Score 80% or more to master the lesson.
In the sandbox, join parallel calls on all results, slow one dependency and compare parent and branch latency. That shows a critical path; the fan-out formula is a separate approximation.
Answer this step to continue.