Deployment Strategies Compared: Blue-Green, Canary, Rolling

Introduction

A deployment strategy is the controlled process of releasing new code to production while minimizing downtime and the cost of getting it wrong. A bad deployment can take down your service, corrupt data, or degrade user experience for hours — the strategy you pick decides how much of that risk you absorb and how fast you can revert.

This guide compares the five strategies you’ll actually use in production — recreate, rolling, blue-green, canary, and shadow deployments — plus feature flags as the release-side complement that decouples “code is live” from “users can see it.” For each one: how it works, when it fits, and where it bites in production. Deeper dives live in the dedicated guides for blue-green, canary, and feature flags.

Deployment Strategies Compared

Strategy	Downtime	Risk	Rollback	Infra Cost	Best For
Recreate	Full stop	High	Slow (redeploy)	1×	Dev/test, non-critical
Rolling	None	Medium	Medium	~1×	Stateless, routine changes
Blue-Green	None	Low	Instant	2×	Instant rollback is critical
Canary	None	Very Low	Fast	~1.1×	High-risk, gradual rollouts
Shadow	None	Very Low	N/A	2×	Validating before exposure
Feature Flags	None	Minimal	Instant	1×	Decouple deploy/release

Prerequisites: What Every Zero-Downtime Deploy Needs

Every strategy except recreate assumes a few things are already true. If any of these are missing, fix that first — no strategy will save you:

• Health checks that actually fail. A /health endpoint that returns 200 while the app can’t reach the database is worse than none — the load balancer keeps sending traffic to a zombie. Configure liveness probes (“is it running?”) and readiness probes (“can it take traffic?”) separately.
• Graceful shutdown. Instances must drain in-flight requests before dying, or rolling updates will truncate user requests mid-flight. See the graceful shutdown recipe.
• Observability before the deploy. You need dashboards and alerts for error rate, latency percentiles, and your one or two business metrics — with values recorded before the deploy starts. Comparing “now” against “before” is the only meaningful signal.
• Backward-compatible changes. Any strategy where two versions coexist requires both to work against the same contracts: API schemas, message formats, and database schemas.
• An automated pipeline. If your deploy is a manual ssh, start with the CI/CD pipeline guide and come back.

Recreate Deployment

Shut down the old version completely, then start the new one. Nothing coexists.

Recreate is the default kubectl delete and redeploy — simple, cheap, and it incurs real downtime. Every user request fails between the kill and the boot of the new version. It also guarantees zero version-skew: there’s exactly one version running at any moment, so you never worry about old code reading new data.

That last property makes recreate the right call for a surprising set of services: single-instance stateful systems where running two versions concurrently is actively dangerous (a migration job that must not run twice, a leader-elected service that can’t split-brain), batch workers where a restart is invisible, and tightly versioned pairs where v1 and v2 share nothing compatible. Use it for dev, staging, and anything where a 30-second outage at a scheduled window costs nothing. For user-facing services with an SLA, it’s a red flag, not a strategy.

Rolling Deployment

Replace old instances gradually with new ones, keeping both alive during the transition:

# Kubernetes rolling update — the default Deployment strategy
kubectl set image deployment/api api=myapp:v2.4.1
kubectl rollout status deployment/api

# If it goes wrong
kubectl rollout undo deployment/api

In Kubernetes, maxSurge controls how many extra pods spin up during the rollout and maxUnavailable controls how many may be down. maxSurge: 1, maxUnavailable: 0 guarantees full capacity but needs headroom for the surge pod; maxUnavailable: 1 is faster but temporarily reduces capacity.

Trade-off: during the rollout, old and new versions coexist and share traffic. If v2 breaks a data contract, v1 instances may fail reading v2-written data — the same version-skew rules as blue-green apply, just continuously instead of at one cutover moment. Rollback isn’t instant either: rollout undo replays the same gradual process backwards, so a deploy that was 80% rolled out takes most of its rollout time to undo. For small fleets this is seconds; for a 200-pod service it’s a real number to know before you need it.

Blue-Green Deployment

Maintain two identical environments. One is live (blue), one is idle (green). Deploy to green, test it, then switch traffic at the load balancer — instantly, atomically.

Before:  Users → [Load Balancer] → [Blue: v2.4.0 live]
                                    [Green: v2.4.0 idle]

After:   Users → [Load Balancer] → [Blue: v2.4.0 idle]
                                    [Green: v2.4.1 live]

# Terraform sketch: blue-green behind an AWS ALB
# Switch traffic by re-weighting the listener rule between target groups
resource "aws_lb_listener_rule" "app" {
  action {
    type = "forward"
    forward {
      target_group {
        arn    = aws_lb_target_group.blue.arn
        weight = 100
      }
      target_group {
        arn    = aws_lb_target_group.green.arn
        weight = 0   # flip to 100 on cutover
      }
    }
  }
}

The killer feature is rollback speed: pointing the listener back at blue takes seconds, and it’s a pure routing change — no data migration, no redeploy. That also makes blue-green the only strategy where “rollback” is genuinely a no-op for the user: the old version was already running and already warm.

The cost is paying for double infrastructure — a blue-green setup on a fleet that costs $10k/month runs ~$20k. For large clusters this dominates the decision; teams often keep green only during the deploy window and tear it down after, trading idle cost for setup complexity. A middle ground: autoscale green from zero to full as part of the deploy, and scale blue to zero after the burn-in period — you pay 2× for minutes or hours instead of permanently.

One more caveat that bites stateful services: blue-green works at the application layer, not the data layer. If you maintain two databases (one per color), writes that land in blue during the green period get lost on cutback. Keep a single shared database/schema and manage changes via expand-contract — or don’t use blue-green for that service at all.

Database Considerations

Blue-green only works cleanly when both versions can operate against the same database:

Change Type	Blue-Green Compatible?
Add column (nullable)	Yes — old code ignores it
Add column (non-nullable)	No — old code can’t insert without it
Rename column	No — old code references old name
Drop column	No — old code may still read it
Add index	Yes — both versions benefit

Rule: blue-green requires backward-compatible schema changes. Use the expand-contract pattern: add the new column (expand), deploy code that reads and writes both, then remove the old column (contract) in a later release. See the schema evolution recipe for the full pattern.

Canary Deployment

Route a small percentage of real traffic to the new version, watch the metrics, and ramp up only if it behaves:

Step 1: 1%   → [Canary v2.4.1], 99% → [Stable v2.4.0]
Step 2: 5%   → [Canary v2.4.1], 95% → [Stable v2.4.0]
Step 3: 25%  → [Canary v2.4.1], 75% → [Stable v2.4.0]
Step 4: 100% → [Canary v2.4.1 becomes stable]

The point isn’t the percentage — it’s that a bad release only hurts 1% of users for five minutes instead of 100% of users until you notice. Canary analysis is the hard part: you need thresholds defined before the deploy and automation that acts on them. Manually staring at Grafana while clicking “next stage” is a canary in name only.

Three details that make or break a canary:

• Sample size. 1% of a 100k-rps service is meaningful in seconds; 1% of a 100-rps service takes hours to produce a statistically useful signal. Size the canary stage by traffic, not by habit — the FAQ below covers how long to wait.
• Sticky routing. If sessions are stateful or users hit both versions alternately, you lose the isolation that makes canary meaningful. Use header or cookie-based routing so the same users stay on the same version during the analysis window.
• What you measure. Latency and error rate catch crashes and regressions; they don’t catch “the new algorithm returns worse results.” Include at least one business metric in the analysis — checkout completion, signup conversion, whatever the service exists to do.

# Flagger: automated canary with metric analysis and auto-rollback
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: api
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  service:
    port: 80
  analysis:
    interval: 30s
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
      - name: request-success-rate
        thresholdRange:
          min: 99
      - name: request-duration
        thresholdRange:
          max: 500

If the error rate spikes or latency crosses the threshold, Flagger automatically routes 100% of traffic back to stable — no human involved. Argo Rollouts runs the same loop with its Rollout CRD and AnalysisTemplate (full example in the scenario section below).

You don’t need a service mesh for basic canary: on plain Kubernetes you can approximate it with two Deployments and replica-ratio routing — crude (pod counts are a blunt percentage knob, and it breaks under autoscaling), but it works for low-stakes services. For real percentage and header-based routing you want a mesh (Istio VirtualService, Linkerd) or an ingress with traffic-split support (NGINX ingress annotations, AWS ALB weighted target groups).

Where Traffic Splitting Happens

Canary, blue-green, and shadow all need a layer that can split or switch traffic. Where that layer lives determines what each strategy can express:

• Load balancer. Weighted target groups (AWS ALB, GCP backend services) or DNS switching (Route53 weighted records). Handles percentage splits and instant cutover for blue-green; no per-user targeting, and DNS caching makes switches slow — keep TTLs low during deploy windows.
• Ingress / reverse proxy. NGINX ingress canary-* annotations, HAProxy, Envoy. Percentage and header-based routing (x-canary: true), cheap to operate, no sidecars. This is the sweet spot for most teams.
• Service mesh. Istio VirtualService, Linkerd traffic split, Kuma. The most expressive option: percentages, headers, cookies, retries, mirroring for shadow deploys. Cost is real: a sidecar on every pod and a control plane to maintain.
• Application level. The app decides internally — which is exactly what feature flags are, and why flags pair so well with any infrastructure-level strategy: the LB moves you between versions, the flag moves you between behaviors inside a version.

A practical progression most teams follow: start with rolling (built-in, free), add ingress or LB weighted routing for blue-green and basic canary, and reach for a mesh only when header-based routing, shadow mirroring, or multi-cluster splits justify the operational cost.

Shadow Deployment (Traffic Mirroring)

Send the new version a copy of production traffic while users only ever see responses from the stable version. The shadow instance processes real requests in parallel; its responses are discarded.

Shadow is the only strategy with truly zero user impact, and it’s the best answer for “does it actually work under real load?” — new search backends, pricing engines, or a rewrite you’re not sure about. It’s usually a pre-canary step: validate in shadow for a day, then start the canary.

The catches: it costs a second copy of production infrastructure, it must never write to shared state (a shadow that charges customers twice is a career event — route writes to a stub or a separate schema), and traffic mirroring requires mesh or LB support — Istio’s VirtualService mirroring, Envoy, or an upstream traffic mirroring setup. Note that shadow traffic also doubles the read load on your database, so size for it or replay traffic at a reduced rate for read-heavy services.

Feature Flags (Decoupling Deploy from Release)

Deploy the code to production but keep it hidden behind a flag. Enable it for specific users or percentages when you’re ready — independently of any deployment.

# LaunchDarkly-style feature flag
if client.variation("new-checkout-flow", user_context, False):
    return new_checkout.handle(request)
return old_checkout.handle(request)

Use Feature Flags For	Don’t Use Feature Flags For
New UI capabilities	Security fixes (shouldn’t be toggleable)
A/B tests	Critical bug patches
Gradual capability rollouts	Data migration code
Kill switches for risky capabilities	Long-lived branching as a lifestyle

Flags complement, don’t replace, deployment strategies. A common production combo: canary the deploy for infrastructure safety, then flag the new feature for product-side control — rollback of the feature is a toggle flip, no redeploy needed.

Flag hygiene is a discipline of its own. Give every flag an owner and an expiry date when you create it; the flag that “we’ll remove next sprint” is still there two years later, undocumented, with three other flags nested inside it. A flag inventory that grows forever turns the codebase into a combinatorial test matrix nobody can reason about — every flag left behind is a branch that only ever gets exercised in one of its states.

Choosing a Strategy

The questions in order:

1. Can you tolerate downtime? If yes, recreate — don’t overengineer.
2. Do you need instant rollback? Blue-green. Pay the 2× infra or accept slower rollback.
3. Is the change risky? Refactors, new logic, anything touching critical paths (payments, auth) → canary.
4. Is real-load behavior unproven? Shadow before the canary.
5. Otherwise, rolling — it’s the right default for routine stateless changes.

And by change type, concretely:

Change	Strategy	Duration
Typo fix	Rolling	2 min
Dependency update	Rolling	5 min
New UI feature	Blue-green	10 min
Logic refactor	Canary 5→20→50→100	30 min
DB schema change	Expand-contract	Multi-deploy
Infra migration	Shadow + canary	1-2 hours

Strategies also compose. Mature teams don’t pick one — they stack them: a rolling update carries the code, feature flags hide the new behavior, and the flag’s gradual enablement acts as the functional canary. The question stops being “which strategy” and becomes “which layer carries which kind of risk” — infrastructure risk goes to the deployment strategy, product risk goes to the flag.

Rollback Strategy

Every strategy above is really a rollback strategy wearing a deployment costume — the strategy determines how fast you can undo, not whether things go wrong. A few rules that hold across all of them:

• Rollback must be automated. A human SSHing at 3 AM to revert is unreliable by definition. Flagger and Argo Rollouts revert on metric breach; kubectl rollout undo should be one command, not a procedure.
• Practice it. A rollback you’ve never executed is a hypothesis. Run it quarterly in staging — ideally deliberately break a deploy to prove the loop works.
• You can’t un-write a database write. Code rolls back; writes don’t. If v2 wrote rows that v1 can’t read, rolling back code doesn’t recover them. This is the real reason expand-contract exists: every intermediate schema state is readable by both versions.
• Keep the previous version available. Blue-green keeps the whole old environment warm; canary keeps stable serving traffic. Don’t tear down the previous version until the new one has survived a full traffic cycle.
• Define “success” before deploying. Write the abort criteria (error rate, latency, business metrics) into the pipeline — deciding what counts as failure during an incident guarantees you’ll pick the wrong answer.

Metrics to Watch During Deployment

Metric	Canary Threshold	Action If Breached
Error rate	< 0.1% above baseline	Rollback canary
Latency p99	< baseline + 20%	Rollback canary
Throughput	No drop > 10%	Rollback canary
Business metric (signups, checkouts)	No drop	Rollback canary

Baselines matter more than the numbers. If your error rate already runs at 0.05%, a 0.1% threshold is too tight; set it relative to the last 24h of healthy traffic. Watch the denominator too: a canary at 1% can show “0 errors” while the real failure mode only triggers on traffic patterns the canary sample didn’t include — night-batch jobs, mobile clients, a specific tenant.

One more operational detail: deploys move metrics for reasons unrelated to the new code. Pod restarts reset warm caches (latency jumps), new instance IDs confuse per-instance dashboards, and maxUnavailable briefly drops capacity (throughput dips). Annotate deploys on your dashboards or you’ll chase ghosts during every rollout.

What Works

• Automate rollback — a human pressing a button at 3 AM is unreliable. See CI/CD pipelines.
• Use synthetic traffic — hit the canary with automated tests before real users.
• Keep deployments small — smaller changes are easier to debug and faster to roll back.
• One change at a time — don’t combine a deploy with a database migration and a config change.
• Test rollback — a rollback you’ve never practiced is a gamble.

Common Mistakes and Troubleshooting

• Deploying on Friday afternoon — you’ll be debugging all weekend.
• No automated rollback — manual reverts are slower and error-prone, especially under pressure.
• Bundling changes in one deploy — a deploy plus a schema migration plus a config change, and when it breaks you can’t tell which of the three caused it.
• Ignoring canary metrics because “the tests passed” — production traffic is the only real test.
• Forgetting database compatibility in blue-green — old and new code must coexist during the switch.
• Deployment rolls back repeatedly — check health checks, resource limits, and startup probes; a failing readiness probe is the usual cause of rolling restarts.
• Container crashes on startup — verify env vars, secrets, and mounted config; read the first 50 lines of logs before scaling replicas.

Scenario: Deployment Pipeline for E-commerce

A payment service on Kubernetes using Argo Rollouts (the Rollout CRD from the Argo Rollouts project — distinct from Argo CD’s Application):

# Argo Rollouts: canary deployment with automated analysis
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payment-service
spec:
  replicas: 10
  strategy:
    canary:
      steps:
        - setWeight: 5        # 5% traffic to v2
        - pause: { duration: 5m }
        - setWeight: 20
        - pause: { duration: 10m }
        - setWeight: 50
        - analysis:           # automatic analysis
            templates:
            - templateName: success-rate
            args:
            - name: service-name
              value: payment-service
        - setWeight: 100
  selector:
    matchLabels: { app: payment-service }
  template:
    spec:
      containers:
        - name: payment
          image: registry.example.com/payment:v2.1.0
          resources:
            requests: { cpu: 200m, memory: 256Mi }
            limits: { cpu: 500m, memory: 512Mi }
---
# AnalysisTemplate: success criteria
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  args:
    - name: service-name
  metrics:
    - name: success-rate
      interval: 1m
      successCondition: result[0] >= 0.99
      failureLimit: 3
      provider:
        prometheus:
          address: http://prometheus:9090
          query: |
            sum(rate(http_requests_total{
              service="{{args.service-name}}",
              status!~"5.."}[2m]))
            / sum(rate(http_requests_total{
              service="{{args.service-name}}"}[2m]))

Automatic abort: error rate above 1% for 2 minutes, p99 over 2× baseline, or the analysis failing 3 times → the rollout reverts the traffic weight. Lessons from running this in production: canary with automatic analysis beats manual canary every time; define success criteria before deploying; expand-contract for schema changes isn’t optional.

Further Reading

• Kubernetes: Performing a Rolling Update — the default Deployment strategy mechanics.
• Flagger Progressive Delivery Operator — automated canary with metric analysis for Kubernetes.
• Argo Rollouts documentation — Rollout CRD and AnalysisTemplate reference.
• Martin Fowler: BlueGreenDeployment and CanaryRelease — the pattern definitions.
• Martin Fowler: ParallelChange — the expand-contract pattern for schema changes.
• LaunchDarkly: Feature flag best practices — flag lifecycle and retirement.
• Companion resources — runnable Flagger, Argo Rollouts and ALB blue-green examples from this guide.