KAHIBARO
Discord Login Register ☰
☰

21.14. Multi-Container Applications

Table of Contents

Why Multi-Container Matters

Most real backend systems are not a single container. A typical production setup might include:

Running each part in its own container gives you:

In this chapter you will learn how to make multiple containers work together as one application.

Important rule: In Docker, a multi-container application is a group of containers that:

  • Share a network so they can talk to each other by name.
  • Use volumes or other storage to keep important data.
  • Are started and managed together, usually with Docker Compose.

Single Container vs Multi-Container

The single-container approach

When you start, you might try to put everything in one container:

For example, a big Dockerfile that installs PostgreSQL, Redis, Nginx and your app, then runs a shell script that starts them all. This works for a demo, but it has serious problems:

The multi-container approach

Instead, you run each piece as its own container. For example:

ServiceImageRole
webmyorg/myapp:latestFastAPI application
dbpostgres:16PostgreSQL database
redisredis:7Cache and background jobs
workermyorg/myapp:latestCelery or RQ background worker
nginxnginx:stableReverse proxy and static files

Each container:

Important rule: A container should run one main process. If you need many different services, use many containers.

Docker Networks in Multi-Container Setups

To let containers communicate, they must share a Docker network.

Default bridge network vs custom network

When you start containers with docker run and do not specify a network, Docker uses the bridge network. Containers on that network can talk to each other, but DNS names can be tricky to manage manually.

For multi-container applications you usually create a user-defined bridge network:

bash 
docker network create myapp-network

Then run containers on that network:

bash 
docker run -d --name db --network myapp-network postgres:16
docker run -d --name web --network myapp-network myorg/myapp:latest

Inside the web container, you can now connect to PostgreSQL with:

text 
host = "db"
port = 5432

because Docker gives each container a hostname equal to its container name.

Networks and Docker Compose

With Docker Compose, you usually do not create networks manually. Compose creates a network automatically and attaches all services to it.

For a project named myapp, the default network might be named myapp_default. Each service is reachable by its service name:

Example environment values inside the app container:

env 
DATABASE_URL=postgresql://postgres:postgres@db:5432/mydb
REDIS_URL=redis://redis:6379/0

You never need to use localhost inside a container to reach another container. localhost always means "this same container".

Important rule: To connect from one container to another, use the service or container name on the shared Docker network, not localhost.

Introducing Docker Compose

Managing many docker run commands is painful. Docker Compose solves this problem.

A simple docker-compose.yml:

yaml 
version: "3.9"
services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: mydb
    volumes:
      - myapp-db:/var/lib/postgresql/data
  web:
    build: .
    environment:
      DATABASE_URL: postgresql://myuser:mypassword@db:5432/mydb
    ports:
      - "8000:8000"
    depends_on:
      - db
volumes:
  myapp-db:

One command to start everything:

bash 
docker compose up

and to stop everything:

bash 
docker compose down

You will use Docker Compose for almost all local multi-container development.

A Typical 3-Container Setup

Let us build a very common setup:

Directory layout example

text 
myapp/
  app/
    main.py
    ...
  Dockerfile
  requirements.txt
  docker-compose.yml

Example `Dockerfile` for FastAPI

dockerfile 
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Example `docker-compose.yml`

yaml 
version: "3.9"
services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: mydb
    volumes:
      - db-data:/var/lib/postgresql/data
  redis:
    image: redis:7
    volumes:
      - redis-data:/data
  web:
    build: .
    environment:
      DATABASE_URL: postgresql://myuser:mypassword@db:5432/mydb
      REDIS_URL: redis://redis:6379/0
    ports:
      - "8000:8000"
    depends_on:
      - db
      - redis
volumes:
  db-data:
  redis-data:

Now:

Run:

bash 
docker compose up --build

Then open http://localhost:8000 in your browser.

`depends_on` and Startup Order

depends_on defines start order, but it does not wait for the service to be "ready". It only guarantees that the container is started.

For example:

yaml 
web:
  depends_on:
    - db

This means Docker will start db before web, but PostgreSQL might still be initializing when the app tries to connect.

To handle this, you can:

Example simple retry in Python (pseudo-code):

python 
import time
import psycopg2
for attempt in range(10):
    try:
        conn = psycopg2.connect("postgresql://...")
        break
    except Exception:
        print("Database not ready, waiting...")
        time.sleep(2)
else:
    raise RuntimeError("Database not available")

Important rule: depends_on does not guarantee that a service is ready, only that the container has been started.

Sharing Data with Volumes

In multi-container applications, you usually need:

Named volumes for persistent data

Named volumes are managed by Docker. They work well for database storage:

yaml 
services:
  db:
    image: postgres:16
    volumes:
      - my-db-data:/var/lib/postgresql/data
volumes:
  my-db-data:

If the container is removed, the volume still exists and you do not lose your data.

Bind mounts for local development

During development, you often want to edit code on your machine and see changes instantly without rebuilding the image every time. Use a bind mount:

yaml 
services:
  web:
    build: .
    volumes:
      - ./app:/app/app

Now, any changes you make in the local ./app directory are visible inside the container at /app/app.

Example: App + Worker + Broker

A common multi-container pattern:

Example docker-compose setup

yaml 
version: "3.9"
services:
  redis:
    image: redis:7
  web:
    build: .
    environment:
      REDIS_URL: redis://redis:6379/0
    ports:
      - "8000:8000"
    depends_on:
      - redis
  worker:
    build: .
    environment:
      REDIS_URL: redis://redis:6379/0
    command: ["python", "-m", "app.worker"]
    depends_on:
      - redis

Inside your code, you can have:

python 
# app/main.py
import redis
r = redis.from_url("redis://redis:6379/0")
python 
# app/worker.py
import redis
r = redis.from_url("redis://redis:6379/0")

Environment Configuration in Multi-Container Apps

Each service can have its own environment variables. You can:

Inline environment example

yaml 
web:
  environment:
    DATABASE_URL: postgresql://myuser:mypassword@db:5432/mydb
    REDIS_URL: redis://redis:6379/0
    APP_ENV: development

Using a `.env` file with Compose

Create a .env file:

env 
POSTGRES_USER=myuser
POSTGRES_PASSWORD=mypassword
POSTGRES_DB=mydb
APP_ENV=development

Then in docker-compose.yml:

yaml 
services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
  web:
    build: .
    environment:
      DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
      APP_ENV: ${APP_ENV}

Compose will read variables from the .env file when you run docker compose up.

Important rule: Do not hardcode secrets directly in images. Use environment variables or secret management tools for sensitive data.

Scaling Services

One of the biggest advantages of multi-container setups is that you can scale different parts independently.

For example, you might want:

With Docker Compose, you can scale a service:

bash 
docker compose up --scale web=3 --scale worker=2

or define replicas in the file (supported in some Compose versions and in Swarm/Kubernetes).

When you scale web, Docker:

If you have a reverse proxy like Nginx or Traefik, it can route traffic across all web containers.

You will see containers with names like:

Inside the network, they are all reachable by the common service name web through load balancing that your proxy handles.

Adding a Reverse Proxy Container

A common 4-container setup:

Example:

yaml 
version: "3.9"
services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: mydb
    volumes:
      - db-data:/var/lib/postgresql/data
  redis:
    image: redis:7
  web:
    build: .
    environment:
      DATABASE_URL: postgresql://myuser:mypassword@db:5432/mydb
      REDIS_URL: redis://redis:6379/0
    expose:
      - "8000"
    depends_on:
      - db
      - redis
  nginx:
    image: nginx:stable
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - web
volumes:
  db-data:

In nginx.conf you can point to web:8000 as the upstream. The user only accesses Nginx on port 80, and Nginx forwards requests to the FastAPI service.

Debugging Multi-Container Issues

Common problems and how to diagnose them:

1. Container cannot reach another container

Symptoms:

Checklist:

2. Database or Redis loses data after restart

Likely cause: missing or misconfigured volume.

Checklist:

3. Environment variables not loaded

Checklist:

4. Service restarts in a loop

Run:

bash 
docker compose logs <service-name>

Look for Python errors, connection issues, or misconfigurations. Fix the underlying problem, then restart.

Summary

In this chapter you learned the core ideas of multi-container applications with Docker:

These patterns are the foundation for more advanced orchestration tools, such as Kubernetes, that you may use later in production.

Views: 6

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3506 / 24h
Impressum / Imprint
Datenschutz / Data Policy