24.4. Continuous Deployment
Table of Contents
Understanding Continuous Deployment
Continuous Deployment, often abbreviated as CD, is a way of working where every change that passes automated tests is automatically deployed to production without manual approval. It is the most automated and aggressive form of shipping software in the CI/CD family.
To understand Continuous Deployment clearly, it helps to contrast it with related concepts.
| Concept | Main Goal | Automatic deploy to production? | Human approval? |
|---|---|---|---|
| Continuous Integration | Integrate and test code changes frequently | No | Not applicable |
| Continuous Delivery | Keep code always ready to deploy | Optional, usually manual trigger | Yes, before production |
| Continuous Deployment | Deploy every good change to users automatically | Yes, after passing pipeline | No, except for emergencies/pause |
In this chapter, we focus on what is unique to Continuous Deployment, not CI in general or full pipeline design.
Key rule of Continuous Deployment
Every change that is:
- Merged into the main branch, and
- Successfully passes all automated checks in the pipeline,
must be automatically deployed to production, without additional manual steps.
If this is not true, you are not doing Continuous Deployment.
What Makes Continuous Deployment Different
In Continuous Delivery, you have an automated pipeline that builds, tests, and prepares an artifact, for example a Docker image. A human then decides when to deploy that artifact to production.
In Continuous Deployment, that last decision is moved into the pipeline itself. The pipeline decides, based only on automated checks, whether the new version goes live.
This has several important consequences:
- You must trust your tests and automation much more.
- You will deploy much more often, sometimes multiple times per day.
- You need strong safety nets in production, such as monitoring, rollbacks, and feature flags.
- You must keep changes small and incremental, because each one might be live very soon.
Continuous Deployment is not just a configuration change in your CI tool. It is a way of working that affects how you write code, design APIs, and operate your backend in production.
Typical Continuous Deployment Flow
A simple Continuous Deployment pipeline for a backend service often looks like this:
- Developer pushes or merges code into the main branch.
- CI pipeline starts:
- Fetches code.
- Installs dependencies.
- Builds the application or Docker image.
- Automated checks run:
- Static analysis and linting.
- Unit tests.
- Integration tests.
- API tests.
- Security checks, for example dependency scan.
- If all checks pass:
- The built artifact is tagged, for example Docker image
my-api:1.23.0. - The deployment job updates the production environment:
- For example, updates a Kubernetes Deployment.
- Or runs
docker-compose pull && docker-compose up -d. - The application is now live with the new version.
- Monitoring and alerting watch for problems after deployment.
If any step fails, the pipeline stops, and the old version continues to serve users.
Preconditions for Continuous Deployment
Before you enable Continuous Deployment, some conditions must be true, otherwise the risk will be too high.
Strong Automated Test Suite
You need tests that give you high confidence that a change is safe enough for production.
Minimum practical set:
- Unit tests for business logic.
- Integration tests that touch the database and external services (with test doubles when needed).
- API tests for main endpoints, both success and failure cases.
- Smoke tests that quickly check if the service is usable at all.
If your tests are weak, Continuous Deployment will only ship broken code faster.
Fast Pipeline
The pipeline must complete quickly. If a deployment takes 40 minutes, developers stop getting fast feedback and deployments pile up.
As a rough guide:
- Unit tests: seconds.
- Integration tests: a few minutes.
- Full pipeline to production: within 10 to 15 minutes for a small or medium backend.
You may need to:
- Run tests in parallel.
- Cache dependencies.
- Avoid heavy work on every run, for example rebuilding huge Docker layers unnecessarily.
Reliable Infrastructure as Code
Your deployment process should be described in code and version controlled, for example:
- Kubernetes manifests.
- Docker Compose files.
- Terraform scripts.
- CI configuration files, for example
.github/workflows/*.yml.
This reduces manual steps and makes deployment behavior reproducible.
Strong Observability
Since changes go to production automatically, you must be able to see quickly when something goes wrong.
You need at least:
- Application logs with useful context.
- Metrics, for example request rate, latency, error rates.
- Health checks for the service.
- Alerts when error rates or response times cross thresholds.
These topics are covered in more detail in other chapters, but they are essential for Continuous Deployment to be safe.
Example: Simple Continuous Deployment With GitHub Actions
Consider a small FastAPI backend deployed as a Docker container to a server that runs Docker Compose.
A simplified GitHub Actions workflow might look like this:
name: cd-to-production
on:
push:
branches:
- main
jobs:
build-and-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run tests
run: pytest
- name: Build Docker image
run: docker build -t my-api:${{ github.sha }} .
- name: Push Docker image
run: |
echo "$REGISTRY_PASSWORD" | docker login -u "$REGISTRY_USER" --password-stdin my-registry.example.com
docker tag my-api:${{ github.sha }} my-registry.example.com/my-api:${{ github.sha }}
docker push my-registry.example.com/my-api:${{ github.sha }}
deploy:
needs: build-and-test
runs-on: ubuntu-latest
steps:
- name: Deploy to production
uses: appleboy/ssh-action@v1.0.0
with:
host: ${{ secrets.PROD_HOST }}
username: ${{ secrets.PROD_USER }}
key: ${{ secrets.PROD_SSH_KEY }}
script: |
docker login my-registry.example.com -u "$REGISTRY_USER" -p "$REGISTRY_PASSWORD"
cd /srv/my-api
export IMAGE_TAG=${{ github.sha }}
docker-compose pull api
docker-compose up -d apiCharacteristics that make this Continuous Deployment:
- It triggers on every push to
main. - It runs tests automatically.
- If tests pass, it deploys automatically to production.
- There is no manual approval step.
You can later extend this with more checks, for example integration tests or security scans.
Safety Nets for Continuous Deployment
Because production can change frequently, you need ways to reduce the impact of bad changes.
Blue-Green and Rolling Deployments
These deployment strategies reduce downtime and make it easier to roll back.
- Blue-green: You have two environments, blue and green. You deploy to the idle one, run checks, then switch traffic over. If there is a problem you switch back.
- Rolling: You gradually update instances one by one. At any time some instances still run the old version, so the system keeps serving requests.
Most modern orchestrators such as Kubernetes support rolling deployments by default.
Feature Flags
Feature flags let you control which features are active without deploying new code.
For example:
- Code for a new payment method is merged and deployed behind a feature flag.
- By default, the flag is off, so the feature is not visible.
- You can enable the flag for a small percentage of users or internal testers.
- If a problem arises, you simply turn the flag off, without rolling back the entire deployment.
Feature flags make it easier to keep changes small and merge early, even if the feature is not fully ready for all users.
Fast Rollback
You must be able to revert a bad deployment quickly.
Common rollback methods:
- Redeploy a previous Docker image tag.
- Roll back a Kubernetes Deployment to a previous ReplicaSet.
- Use infrastructure as code to apply a previous configuration version.
A rollback should be a simple, documented command, not a manual procedure that you invent during an incident.
Rollback rule
If a production deployment causes significant errors or incidents,
prefer an immediate rollback over trying to debug live in production.
After rollback, you can debug using logs, staging environments, or canary releases.
Staging vs Direct-to-Production
Some teams deploy directly from the main branch to production. Others deploy first to a staging environment, then automatically to production after extra checks.
Options:
- Direct Continuous Deployment:
- Pros: Very fast feedback, simple flow.
- Cons: Higher risk, depends heavily on test quality.
- Staged Continuous Deployment:
- Automatic deploy to staging after tests.
- Additional automated checks in staging, for example smoke tests, synthetic traffic.
- If staging looks healthy, automatically deploy to production.
- Pros: Extra safety, can run heavier tests.
- Cons: More infrastructure to manage, slightly slower.
A pipeline with staging might use jobs like:
build-and-test→deploy-to-staging→smoke-test-staging→deploy-to-production.
The key is that the full chain is still automatic, with no manual approval.
Managing Risk With Small and Frequent Changes
Continuous Deployment works best when changes are:
- Small in scope.
- Well isolated.
- Easy to understand.
For example:
- Add a new field to the API response and make sure clients tolerate unknown fields.
- Add a new optional database column in one deploy.
- Start writing to the new column while still reading from the old one.
- In a later deploy, migrate data and switch reads to the new column.
This multi-step approach is called an expand and contract pattern and is very compatible with Continuous Deployment.
In contrast, big bang changes, for example completely rewriting a core API, are hard to ship safely with Continuous Deployment.
Common Pitfalls in Continuous Deployment
Some problems show up often when teams move to Continuous Deployment.
Flaky Tests
A flaky test sometimes passes and sometimes fails, even when the code did not change.
Effects:
- Random pipeline failures.
- Developers lose trust in the pipeline.
- People start rerunning jobs until they pass, which hides real problems.
Solutions:
- Quarantine flaky tests and fix them as a priority.
- Stabilize integration test environments.
- Avoid relying on external services during tests, use mocks or test doubles instead.
Slow Pipeline
If a pipeline takes too long, developers:
- Push less often.
- Accumulate large changes in one push.
- Disable tests to make it faster, which reduces safety.
You can:
- Run tests in parallel.
- Split tests into tiers, for example fast tests on every push, slow tests on a schedule or special trigger.
- Cache Docker layers and dependencies.
Manual Configuration Drift
If some configuration is changed manually on servers and not tracked in code, then:
- The same deployment behaves differently on different servers.
- It is harder to reproduce problems.
- Rollbacks might not fully restore the previous state.
Continuous Deployment works best when environments are created and configured from code.
When Continuous Deployment Makes Sense
Continuous Deployment is not required for every project, but it is very useful when:
- You are building a software as a service product.
- You want to deliver features and bug fixes quickly.
- You can invest in good automated tests and observability.
- You can structure work into small, incremental changes.
It may not be ideal when:
- Every release requires legal, compliance, or manual review, for example some medical or financial systems.
- You have extremely long manual test cycles that cannot be automated.
- The cost of a mistake is extremely high and you do not yet have strong safety mechanisms.
You can still automate everything up to a certain point and keep a manual approval step in such cases, which is Continuous Delivery rather than Continuous Deployment.
Summary
Continuous Deployment automatically ships every change that passes your pipeline to production. To make this safe, you need:
- Strong automated tests and fast pipelines.
- Reliable infrastructure as code.
- Observability, monitoring, and alerts.
- Safe deployment strategies and quick rollbacks.
- Small, incremental changes and feature flags.
For backend developers, understanding Continuous Deployment helps you design services that are easier to deploy, monitor, and operate at high speed while keeping risk under control.
Views: 8
KAHIBARO