Deployments: blue/green, canary, flags and rollback
Ship changes without spending the error budget: size a canary's blast radius, plan rolling capacity, separate deploy from release with flags, and keep rollback safe.
30 min7 steps
By the end you can
- Choose between rolling, blue/green and canary deployment for a change and its risks
- Estimate how many users a bad canary affects before automated rollback
- Choose a rolling-update batch size that keeps the fleet within its utilisation target
- Use feature flags and expand/contract changes so a rollback never breaks data
Three ways to replace running code
Rolling
- Replace in batches
- Old and new side by side
- Rollback = another rollout
Blue/green
- Full new fleet, switch at router
- Rollback in seconds
- Pays for two fleets
Canary
- Small share of real traffic
- Compare metrics
- Widen in stages with bake time
A canary's blast radius
Loading question…
Rolling updates need headroom
Loading question…
Configuration is a deployment too
Deploy
WAF rule with a backtracking regex
Seconds
Pushed to every machine at once
CPU
100% worldwide
About 27 min
Traffic dropped, then rolled back
Separate deploy from release
1 Add
New column; old code ignores it
2 Dual write
Write both, read old
3 Backfill
Fill existing rows
4 Read new
Behind a flag
5 Drop
After bake: irreversible
Order the steps of renaming a column without ever blocking a rollback.
- 1Backfill the new column for existing rows
- 2Deploy code that reads the new column, behind a flag
- 3Add the new column; the old code ignores it
- 4Deploy code that writes both columns and still reads the old one
- 5After a bake period, stop writing the old column and drop it
Roll back first, debug later
Before
0.1%
After
3%
Action
roll back first
debug later
Check your mastery
Canary
share x time
x severity
Rolling
survivors <= target
at peak
Score 80% or more. Next: capacity planning.
Mastery check
4 questions on fresh variants. One attempt each, no time limit. Score 80% or more to master the lesson.
Answer this step to continue.