Blue-Green Deployments: Zero-Downtime Releases
Run two identical environments and switch traffic between them. Learn the setup, the database problem, and when it is worth it.
By Velloreth
The idea
Two identical environments. One serves the traffic, and the other receives the new version.
+----------+
| Router |
+----+-----+
|
+------+------+
| |
+---+---+ +---+---+
| Blue | | Green |
| v1.0 | | v1.1 |
| LIVE | | idle |
+-------+ +-------+
The switch moves the traffic from the blue to the green.
The rollback moves it back, which is fast because the blue is still running.
The steps
- Deploy the new version to the idle environment.
- Run the smoke tests against it, using its own address.
- Switch the traffic at the router.
- Watch the metrics for a period.
- Keep the old environment as the rollback target, or tear it down.
The switch is the only moment of risk, and it is a single change at one point.
The switch
A load balancer with two target groups is the common implementation. The switch is a change to which group is active.
A DNS change is the simplest and the slowest, because of the TTL. It is not suitable for an instant rollback.
A reverse proxy such as nginx can switch with a reload.
upstream app {
server 10.0.1.10:3000; # blue
# server 10.0.2.10:3000; # green
}
A service mesh or an ingress controller can switch with a weight, which also allows a canary.
The requirement: the switch must be atomic and fast. A switch that takes minutes is a switch that is not a rollback mechanism.
The database problem
The database is shared, so it cannot be switched with the application.
The consequence: during the switch, the old and the new version both run against the same schema.
A migration must be backward compatible.
The expand and contract pattern:
Phase 1 — expand. Add the new column, nullable, and write to both.
ALTER TABLE users ADD COLUMN email_normalized text;
Phase 2 — backfill. Fill the new column for the existing rows.
UPDATE users SET email_normalized = lower(email) WHERE email_normalized IS NULL;
Phase 3 — switch. The new version reads the new column. The old version still reads the old one.
Phase 4 — contract. After the old version is gone, drop the old column.
ALTER TABLE users DROP COLUMN email;
The rule: a migration that is not backward compatible cannot be deployed with a blue-green switch. A rename is a drop and an add, and it must be split.
A destructive change such as a column drop must wait until no version reads it.
The session problem
A session stored in memory is lost on the switch, because the new environment has no session.
The fix: store the session in a shared store such as Redis, or use a signed token.
A sticky session does not help, because the switch moves all the traffic.
A long-lived connection such as a WebSocket is dropped on the switch. The client must reconnect, which is a design requirement rather than a bug.
The cost
Double the infrastructure during the switch, because both environments run.
A warm environment is required for an instant rollback, so the old one cannot be torn down immediately.
The complexity of the router, the two environments and the migration discipline.
The benefit is an instant, reliable rollback and no mixed-version period.
When to use it
A critical service where a failed release is expensive.
A release that must be reversible in seconds.
A system with a single switch point, such as a load balancer.
When not to use it:
- A small application with a fast deploy and a tolerant user base
- A system where the database change cannot be made compatible
- A system with a stateful connection that cannot reconnect
The alternatives
| Technique | Rollback | Mixed versions | Cost |
|---|---|---|---|
| Recreate | slow | no | low |
| Rolling | slow | yes | low |
| Blue-green | instant | no | high |
| Canary | fast | yes | medium |
A canary sends a small percentage of the traffic to the new version, which limits the blast radius. It is a good complement to a blue-green switch.
A rolling deployment is simpler and it is the right default for a service that tolerates a mixed-version period.
The first deployment
The first time a blue-green switch is used, the temptation is to skip the smoke test because the environment is identical. It is not identical: the configuration differs, the secrets differ, and the database connection is the same one the old version is using. A smoke test that checks the health endpoint, one read and one write is enough to catch the common failures, and it takes a minute. The second habit is to watch the error rate for a defined period rather than to declare success at the switch. A problem that appears after five minutes is a problem that a five-minute watch catches. The third is to keep the old environment running until the next release, because a rollback that requires a redeploy is not a rollback.
Common mistakes
A migration that is not backward compatible. The old version breaks during the switch.
A session in memory. Every user is logged out on the switch.
A DNS switch. The TTL makes the rollback slow.
No smoke test before the switch. The new version is live before it is verified.
Tearing down the old environment immediately. There is no rollback target.
A switch that is not atomic. A partial switch runs both versions for a period.
No monitoring after the switch. The problem is found by the users.
A shared cache with a version-specific key. The new version reads a stale value.
A quotable summary
A blue-green deployment runs two identical environments, one live and one idle, and switches the traffic between them, which makes the rollback a fast and reliable switch back. The database is the hard part, because it is shared and cannot be switched, so a migration must be backward compatible with both versions during the switch, using an expand phase and a contract phase. The cost is double the infrastructure, which is why it suits a critical service with a single switch point.
Related reading
For the alternatives, see ci-cd-pipeline-basics and docker-basics-guide. For the broader context, incident-response-solo-developers and error-handling-patterns.