Skip to content

Timeouts and deadlines

Set timeouts from latency data, fit retries inside a caller's deadline, and propagate the remaining budget down the call chain so no one works for a client that left.

25 min7 steps

By the end you can
  • Set a per-try timeout from the dependency's latency distribution and fit all attempts inside the caller's deadline
  • Propagate a deadline down a call chain and compute the budget left for each hop
  • Explain what an overlong timeout holds hostage and what a too-short timeout wastes

Every remote call needs a timeout

CallerPaymentscharge✕response lost
The caller waits forever. A timeout turns unknown into a decision. It does not say whether the call completed.

A call to a payment service times out after 2 seconds. What do you know?

Answer this step to continue.

What this lesson simplifies

  • Hop costs are fixed numbers; real hops vary, so production systems add margin or measure per-hop latency.
  • Deadline propagation assumes the clocks of caller and callee agree closely, or that a relative timeout is passed and adjusted for transit.
  • Thread-holding estimates use Little's law in steady state and ignore queueing before the call starts.

Sources

  1. M. Brooker, Amazon Builders' Library. Timeouts, retries, and backoff with jitter.
  2. gRPC Authors. Deadlines (gRPC guide).
  3. J. Dean, L. Barroso. The Tail at Scale, 2013.
  4. M. Ulrich (B. Beyer et al., eds.). Addressing Cascading Failures, Site Reliability Engineering, 2016.

Reviewed by T2 content review on . Revision 1.