Zero-Downtime Blue-Green Deployments with docker-rollout

You can ship to a single Docker Compose host without dropping a request. The recipe is docker-rollout, a reverse proxy that routes on health, an application that knows when it is ready, and migrations that never break the running version.

docker compose up -d web causes downtime by design. Docker stops the old container before the new one exists. For a service that takes 3–8 seconds to boot, that is a burst of 502s on every deploy.

Blue-green deployment fixes this by running the new version (green) next to the old one (blue), moving traffic to green once it is ready, and only then retiring blue. Kubernetes gives you this with Deployments and readiness probes. On single VM setups, you get it with docker-rollout, which is a single-file shell plugin for the Docker CLI.

docker-rollout scales the service to twice its replica count and then removes the old half, so it is a short-lived blue-green cutover rather than two permanent environments. For a single-host Compose setup, that is exactly what you want.

This guide covers four pieces, in the order you set them up: installation and use, application healthchecks, reverse proxying with Traefik or NGINX, and database migrations. The examples use FastAPI, PostgreSQL and Alembic, but every pattern should carry over to any stack.

How docker-rollout works

Running docker rollout web performs five steps:

1. Count the running containers for the web service (say, 1).
2. Scale web to double that count with the new image (project-web-1 keeps running; project-web-2 starts).
3. Wait for the new containers to report healthy, up to --timeout seconds (default 60). Without a healthcheck, it sleeps --wait seconds (default 10) and hopes.
4. Run the optional --pre-stop-hook inside the old containers, then stop and remove them.
5. If the new containers never become healthy, remove them instead and leave the old ones serving. The deployment fails and production does not.

IT is important to note that this tool does not route traffic. Your reverse proxy does, which is why the proxy and the healthcheck carry most of the weight in this setup. Each deploy also increments the container number (web-1 → web-2 → web-3), so never reference containers by name.

Part 1: Database migrations

This is where most zero-downtime setups actually go down. During a rollout, the old and new versions of your code run against the same database at the same time. If a rollout fails its healthcheck, the old version keeps serving against the new schema indefinitely. Every migration must therefore work with both the code before it and the code after it.

Run migrations as a separate step

Migrate with the new image, in a one-shot container, before the rollout starts:

docker compose run --rm api alembic upgrade head
docker rollout api

Never run migrations on application startup. With docker-rollout, two containers boot during every deploy, and both would race to migrate. A migration failure would also turn into a healthcheck failure, which is harder to read than a failed deploy step.

Safe and unsafe schema changes

Expand/contract: a column rename

Renaming tenants.phone to tenants.phone_number takes three deploys. Each one is compatible with the version on either side of it.

1. Expand. Migration adds phone_number as nullable. Code writes to both columns and reads from phone.
2. Migrate. A batched backfill copies phone into phone_number for existing rows. Code switches reads to phone_number and still writes both.
3. Contract. Code stops touching phone. The migration in the following deploy drops it.

Backfill in batches so no single statement holds row locks for long:

UPDATE tenants
SET phone_number = phone
WHERE id IN (
  SELECT id FROM tenants
  WHERE phone_number IS NULL AND phone IS NOT NULL
  LIMIT 5000
);
-- repeat until 0 rows updated

Fail fast on locks

The dangerous migration is not the slow one. It is the one that waits for a lock: an ALTER TABLE queued behind a long-running query blocks every query that arrives after it, and your API stalls. Set a lock_timeout for the migration connection so it gives up after 5 seconds instead:

# alembic/env.py (psycopg)
connectable = create_engine(
    config.get_main_option("sqlalchemy.url"),
    connect_args={"options": "-c lock_timeout=5s"},
)
# asyncpg: connect_args={"server_settings": {"lock_timeout": "5s"}}

A failed migration fails the deploy before docker rollout runs, and production keeps serving the old version. Rerun it when traffic is lower.

Concurrent indexes in Alembic

CREATE INDEX CONCURRENTLY cannot run inside a transaction, and Alembic wraps each migration in one. Use an autocommit block:

def upgrade() -> None:
    with op.get_context().autocommit_block():
        op.create_index(
            "ix_payments_tenant_id",
            "payments",
            ["tenant_id"],
            postgresql_concurrently=True,
            if_not_exists=True,
        )

A concurrent build that fails leaves an INVALID index behind. if_not_exists will not rebuild it, so drop it with DROP INDEX CONCURRENTLY before rerunning.

Roll forward, not back

Do not run alembic downgrade in production. Because every schema change is backward compatible, rolling back the application is just redeploying the previous image tag against the current schema. Fix a bad migration with a new migration.

Part 2: Application healthchecks

The healthcheck is the contract between your app and the deploy. docker-rollout waits on it, and your revers proxy routes on it. A container that reports healthy before it can serve requests turns zero-downtime into a race.

Liveness vs readiness

Expose two endpoints. Liveness answers "is the process alive?" Readiness answers "should this container receive traffic right now?" Docker has a single health status, so wire it to readiness.

# app/health.py
import asyncio
from pathlib import Path

from fastapi import APIRouter, Response
from sqlalchemy import text

from app.db import engine
from app.cache import redis

router = APIRouter(prefix="/health", tags=["health"])
DRAIN_FILE = Path("/tmp/drain")


@router.get("/live")
async def live() -> dict:
    return {"status": "ok"}


@router.get("/ready")
async def ready(response: Response) -> dict:
    if DRAIN_FILE.exists():
        response.status_code = 503
        return {"status": "draining"}
    try:
        async with asyncio.timeout(2):
            async with engine.connect() as conn:
                await conn.execute(text("SELECT 1"))
            await redis.ping()
    except Exception as exc:
        response.status_code = 503
        return {"status": "unavailable", "reason": type(exc).__name__}
    return {"status": "ready"}

Three rules for the readiness check:

Check only hard dependencies. Postgres and Redis, yes. A third-party payment API or SMS gateway, no. An outage there would mark every container unhealthy and take your whole API offline.

• Bound it with a timeout. A hung connection must fail in 2 seconds, not hang the probe.
• Do the drain check first. The drain file flips readiness to 503 on purpose during a rollout (see below), regardless of dependencies.

If startup does real work (warming caches, loading an ML model), set a module-level ready flag at the end of FastAPI's lifespan startup and return 503 until it is set.

The Docker healthcheck

Slim Python images ship without curl, so probe with Python itself:

services:
  api:
    healthcheck:
      test:
        - CMD
        - python
        - -c
        - "import urllib.request; urllib.request.urlopen('http://localhost:8000/health/ready', timeout=3)"
      interval: 5s
      timeout: 4s
      retries: 2
      start_period: 30s
      start_interval: 1s

urlopen raises on any non-2xx response, so a 503 exits non-zero and Docker counts a failure. start_period gives the app 30 seconds to boot without failures counting, and start_interval: 1s probes every second during that window so the container turns healthy as soon as it is ready instead of waiting for the next 5-second tick. start_interval needs Docker Engine 25 or newer.

Graceful shutdown

When docker-rollout removes the old container, Docker sends SIGTERM, waits stop_grace_period, then sends SIGKILL. Your app must receive that SIGTERM and finish in-flight requests within the window.

# Exec form: uvicorn runs as PID 1 and receives SIGTERM directly
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", \
     "--timeout-graceful-shutdown", "25"]

The shell form (CMD uvicorn app.main:app) wraps the process in /bin/sh, which swallows SIGTERM; Docker then kills the app after the grace period with requests still open. Set stop_grace_period: 30s on the service so Docker waits longer than uvicorn's 25-second drain.

Draining before stopping

Graceful shutdown finishes requests already in flight. It does nothing about new requests the proxy is still sending to the old container. Draining closes that gap: mark the old container unhealthy first, wait for the proxy to stop routing to it, then stop it.

docker rollout api --pre-stop-hook "touch /tmp/drain && sleep 15"

The hook creates /tmp/drain in the old container, readiness returns 503, Docker marks it unhealthy after interval × retries (5 s × 2 = 10 s), and Traefik drops it from the pool. Size the sleep as:

t_{sleep} > t_{interval} \times n_{retries} + t_{longest\ request}

With a 5-second interval, 2 retries and requests that finish within 5 seconds, 15 seconds is enough. Put it on the service as a label so every deploy uses it:

    labels:
      docker-rollout.pre-stop-hook: "touch /tmp/drain && sleep 15"

Part 3: Reverse proxying (with Traefik)

The proxy is what makes the cutover invisible. It must discover containers as they appear, route only to healthy ones, and forget containers the moment they turn unhealthy. Traefik's Docker provider does all three from container labels, with no reload.

The proxy service

services:
  traefik:
    image: traefik:v3   # pin a minor version in production
    command:
      - --providers.docker=true
      - --providers.docker.exposedbydefault=false
      - --providers.docker.network=web
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entrypoint.to=websecure
      - --entrypoints.websecure.address=:443
      - --certificatesresolvers.le.acme.tlschallenge=true
      - [email protected]
      - --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - letsencrypt:/letsencrypt
    networks: [web]
    restart: unless-stopped

networks:
  web:
  internal:

volumes:
  letsencrypt:

Traefik is the only service that publishes ports. exposedbydefault=false means nothing is routed unless it opts in with traefik.enable=true, so Postgres and Redis on the internal network stay unreachable from outside.

Routing the application

services:
  api:
    labels:
      - traefik.enable=true
      - traefik.http.routers.api.rule=Host(`api.example.com`)
      - traefik.http.routers.api.entrypoints=websecure
      - traefik.http.routers.api.tls.certresolver=le
      - traefik.http.services.api.loadbalancer.server.port=8000
      # Active health check: a second, faster signal than Docker's
      - traefik.http.services.api.loadbalancer.healthcheck.path=/health/ready
      - traefik.http.services.api.loadbalancer.healthcheck.interval=2s
      - traefik.http.services.api.loadbalancer.healthcheck.timeout=1s
      # Retry on connection errors to a container that just went away
      - traefik.http.routers.api.middlewares=api-retry
      - traefik.http.middlewares.api-retry.retry.attempts=2
      - traefik.http.middlewares.api-retry.retry.initialinterval=100ms
      - docker-rollout.pre-stop-hook=touch /tmp/drain && sleep 15

The router and service names (api) are defined once, on the Compose service, not per container. When docker-rollout starts api-2, Traefik sees a second container carrying the same labels and adds it as another server behind the same service. That is the whole blue-green mechanism: two servers in one load balancer, then one.

Health-aware routing

Traefik's Docker provider skips containers whose Docker health status is starting or unhealthy. The new container therefore receives no traffic until its readiness check passes, and the draining container leaves the pool as soon as Docker marks it unhealthy.

The Traefik-level healthcheck labels add an active probe every 2 seconds from the proxy itself. It reacts faster than Docker's 5-second interval and catches failures Docker misses, such as a container that is healthy from inside but unreachable over the network.

The retry middleware covers the last edge case: a request sent to a container in the instant it stops accepting connections. Traefik retries only when it cannot reach the server at all, not when your app returns an error, so a request your app handled is not replayed. Keep your webhook and payment-callback handlers idempotent regardless.

What about nginx-proxy?

nginx-proxy bundles NGINX and docker-gen in one image and routes by VIRTUAL_HOST environment variables. It is an alternative proxy, with two catches. Its stock template builds upstreams without reading container health, and it mounts the Docker socket into the internet-facing container.

Part 4: Installation and use

docker-rollout installs as a Docker CLI plugin.

mkdir -p ~/.docker/cli-plugins
curl https://raw.githubusercontent.com/wowu/docker-rollout/main/docker-rollout \
  -o ~/.docker/cli-plugins/docker-rollout
chmod +x ~/.docker/cli-plugins/docker-rollout

docker rollout --help   # verify

If your deploy user runs Docker through sudo, install to /usr/local/lib/docker/cli-plugins/ instead so every user sees it. In production, pin a release tag rather than main (replace main in the URL with e.g. v0.14), so a new upstream commit never changes your deploy behaviour unannounced.

Compose requirements

Two things break the moment a second container of the same service starts:

• container_name must go. Docker will not create two containers with the same name.
• ports must go. Two containers cannot bind the same host port. The proxy reaches the app over the Compose network instead.

A rollout-ready service should look something like this:

services:
  api:
    image: ghcr.io/acme/api:${IMAGE_TAG}
    env_file: .env
    networks: [web, internal]
    stop_grace_period: 30s
    # no container_name, no ports

Command-line options

The pre-stop hook can also live on the service as a label, docker-rollout.pre-stop-hook. docker-rollout reads labels from the old container, so a newly added label takes effect on the deploy after the one that introduces it.

What to roll out, and what not to

Roll out stateless HTTP services: your API, your web frontend. Do not roll out Postgres, Redis or anything holding a volume lock. Workers and Kafka consumers can be rolled out only if two copies running side by side for a few seconds is safe, which it is when consumers are idempotent; otherwise deploy them with plain docker compose up -d.

Putting it together

One script runs the whole deploy: pull the image, migrate, roll out, verify. Each step fails the deploy before the next runs, so a broken release never reaches traffic.

#!/usr/bin/env bash
# deploy.sh <image-tag>
set -euo pipefail
cd /opt/app

export IMAGE_TAG="${1:?usage: deploy.sh <image-tag>}"
PREVIOUS_TAG="$(cat .current-tag 2>/dev/null || true)"

echo "==> Pulling ${IMAGE_TAG}"
docker compose pull api

echo "==> Migrating"
docker compose run --rm api alembic upgrade head

echo "==> Rolling out"
docker rollout api --timeout 90 --wait-after-healthy 3

echo "==> Smoke test"
if ! curl -fsS --retry 3 --retry-delay 2 https://api.example.com/health/ready > /dev/null; then
  echo "Smoke test failed" >&2
  if [[ -n "${PREVIOUS_TAG}" ]]; then
    echo "==> Rolling back to ${PREVIOUS_TAG}" >&2
    IMAGE_TAG="${PREVIOUS_TAG}" docker rollout api --timeout 90
  fi
  exit 1
fi

echo "${IMAGE_TAG}" > .current-tag
echo "==> Deployed ${IMAGE_TAG}"

Tag images by git SHA, never latest. The tag is what makes rollback a one-liner (./deploy.sh <previous-sha>), and the backward-compatible migrations are what make that rollback safe. Call the script from CI over SSH after the image build and push succeed:

# .github/workflows/deploy.yml (deploy job)
- name: Deploy
  run: ssh deploy@${{ secrets.DEPLOY_HOST }} "/opt/app/deploy.sh ${{ github.sha }}"

Limitations of this setup

docker-rollout gives you zero-downtime deploys, not high availability. It is important to know wehere this stops:

• One host is still one host. A kernel update, a reboot or a disk failure takes everything down. The next step is a second host behind a load balancer.
• The proxy is not rolled out. Replacing the Traefik or NGINX container drops connections for a second or two. Do it deliberately, off-peak, separate from app deploys.
• Memory doubles during a rollout. Two copies of the service run for the length of the deploy. Size the host for it.
• Long-lived connections are cut. WebSockets and server-sent event streams outlive any reasonable drain sleep. Clients must reconnect.
• Migration discipline is on you. Nothing stops a careless DROP COLUMN. Review migrations against the table above before merging.

Within those limits, this setup ships to a single Compose host as often as you like, with no dropped requests and no Kubernetes.

Sources

• docker-rollout on GitHub: installation, CLI options, caveats and container draining
• docker-rollout v0.14 release
• docker-gen on GitHub: watched events, template context and flags
• nginx-proxy stock template
• NGINX 1.28 changelog: upstream resolve (1.27.3) and non-idempotent retry behaviour (1.9.13)