Deploy Django with Celery to Hetzner

A Django app with background jobs isn’t one process, it’s at least three: gunicorn serving requests, a Celery worker draining the queue, and Redis holding that queue in between. Most tutorials stop at runserver and leave the worker, the broker and the HTTPS certificate as an exercise for later. This guide builds the whole thing on a Hetzner server: gunicorn behind Caddy, a Celery worker and an optional beat scheduler, Postgres and Redis, all as Docker Compose services with real backups and a real upgrade path. The shortcut at the end pushes the same app with a git push instead.

This guide uses Postgres 16 and Redis 8.

What you’ll need

• A Hetzner Cloud server. A CPX31 (4 vCPU, 8 GB) runs gunicorn, a Celery worker, Postgres and Redis together without swapping, at €62.99/month excluding VAT. A smaller CX23 (2 vCPU, 4 GB) at €5.99/month excluding VAT works for a low-traffic app with a light queue, it’s just tighter once Postgres’s shared buffers and Celery’s worker processes are both resident.
• A domain name, pointed at the server’s IP before the HTTPS step.
• A Hetzner Cloud account and an API token (Security → API Tokens, Read & Write) if you want to script server creation; the Hetzner Cloud Console works fine too if you’d rather click through it once.

Step 1: Create the server

From the Hetzner Cloud Console, create a server on Ubuntu 24.04, your chosen size, and a region close to your users. If you’d rather script it:

curl -X POST "https://api.hetzner.cloud/v1/servers" \
  -H "Authorization: Bearer $HETZNER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "django-app",
    "server_type": "cpx31",
    "image": "ubuntu-24.04",
    "location": "nbg1",
    "ssh_keys": ["your-ssh-key-name"]
  }'

You should get back a JSON response with a server.id and a server.public_net.ipv4.ip. SSH in with that IP once it finishes booting, usually under a minute:

ssh root@YOUR_SERVER_IP

Step 2: Install Docker and point DNS

curl -fsSL https://get.docker.com | sh

docker compose version

You should see a Compose v2.x version string. While that installs, create the DNS A record for your domain and confirm it resolves:

dig +short app.example.com

It needs to return the server’s IP before Caddy can issue a certificate later, so get this step done early and let it propagate in the background.

Step 3: Write the Django image

Django doesn’t ship an official image the way Postgres or Redis do, since every project’s dependencies differ, so you build your own:

FROM python:3.12-slim

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends \
    libpq-dev gcc \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

RUN python manage.py collectstatic --noinput

CMD ["gunicorn", "myproject.wsgi:application", "--bind", "0.0.0.0:8000"]

libpq-dev and gcc are there for psycopg2 (the non-binary build); drop them if your requirements.txt pins psycopg2-binary instead, which ships its own compiled libpq. Running collectstatic at build time means a broken static file never reaches production traffic, since the build fails before the image is ever pushed.

Step 4: Write the Docker Compose file

Five services: the Django app as web, the same image again as worker and beat with different commands, Postgres, Redis, and Caddy in front.

services:
  web:
    build: .
    restart: unless-stopped
    command: gunicorn myproject.wsgi:application --bind 0.0.0.0:8000 --workers 3
    env_file: .env
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    networks:
      - app_net

  worker:
    build: .
    restart: unless-stopped
    command: celery -A myproject worker --loglevel=info --concurrency=2
    env_file: .env
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    networks:
      - app_net

  beat:
    build: .
    restart: unless-stopped
    command: celery -A myproject beat --loglevel=info
    env_file: .env
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
    networks:
      - app_net

  db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: myproject
      POSTGRES_USER: myproject
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myproject"]
      interval: 5s
      start_period: 10s
      retries: 20
    networks:
      - app_net

  redis:
    image: redis:8-alpine
    restart: unless-stopped
    volumes:
      - redis_data:/data
    networks:
      - app_net

  caddy:
    image: caddy:2
    restart: unless-stopped
    depends_on:
      - web
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
    networks:
      - app_net

networks:
  app_net:

volumes:
  postgres_data:
  redis_data:
  caddy_data:
  caddy_config:

Drop the beat service entirely if you don’t schedule periodic tasks. worker and beat build from the exact same Dockerfile as web, this is the 12-factor point of the process model: one codebase, three roles, started with three different commands instead of three repos to keep in sync.

Write the .env next to it:

POSTGRES_PASSWORD=a-long-random-string
DJANGO_SETTINGS_MODULE=myproject.settings
SECRET_KEY=a-different-long-random-string
DEBUG=False
ALLOWED_HOSTS=app.example.com
DATABASE_URL=postgres://myproject:a-long-random-string@db:5432/myproject
CELERY_BROKER_URL=redis://redis:6379/0
CELERY_RESULT_BACKEND=redis://redis:6379/0

SECRET_KEY and the Postgres password should each come from openssl rand -hex 32, not something typed by hand.

Step 5: Run migrations and bring the stack up

docker compose up -d --build
docker compose run --rm web python manage.py migrate

You should see Django’s migration output ending in a list of applied migrations, not an error about a missing table or a connection refused to db. Then check the worker picked up the broker:

docker compose logs worker --tail=20

You should see a line like celery@worker ready. confirming it connected to Redis and is waiting for tasks.

Step 6: Write the Caddyfile

app.example.com {
	reverse_proxy web:8000

	handle_path /static/* {
		root * /srv/static
		file_server
	}
}

Caddy requests and renews its own certificate through Let’s Encrypt the first time it can answer on port 80 for a domain that already resolves to this server, no separate certbot step required. If you serve static files from Caddy directly rather than through WhiteNoise, mount the same static directory the web container writes to into the caddy service as /srv/static.

Step 7: Backups

Postgres holds the data that actually matters; back it up with pg_dump on a cron schedule rather than relying on the volume alone:

#!/usr/bin/env bash
set -euo pipefail
STAMP=$(date +%Y%m%d-%H%M%S)
mkdir -p backups
docker compose exec -T db pg_dump -U myproject myproject | gzip > "backups/django-db-$STAMP.sql.gz"
find backups -name '*.sql.gz' -mtime +14 -delete

Copy backups/ off the server on the same schedule, to object storage or another machine. A dump that never leaves the server it was taken on doesn’t protect against the server itself failing.

Step 8: Upgrades

Pull your own new image build, apply migrations, then recreate the app containers one at a time:

docker compose build
docker compose run --rm web python manage.py migrate
docker compose up -d web worker beat

Running the migration as a one-off run --rm before touching the long-running containers means a failed migration stops the deploy instead of partially restarting web against a schema it doesn’t expect.

The shortcut: do it with Railyard

Railyard’s repo inspector reads requirements.txt and manage.py to detect Django on its own, no Dockerfile or Procfile required. It sets the release command to python manage.py migrate --no-input, starts web with gunicorn bound to the assigned port, and runs collectstatic in the build step automatically. A psycopg/psycopg2 dependency provisions Postgres and injects DATABASE_URL; a Redis client (redis, django-redis, rq) does the same for REDIS_URL. If Celery is in your dependencies, it’s offered as a worker process on its own, and Celery Beat is added too if django-celery-beat is present or your settings define a beat schedule, so there’s no Procfile to write by hand for either.

railyard create my-app https://github.com/me/my-app --addons postgres,redis

or paste the same Git URL into the dashboard once GitHub is connected. Scale the worker independently once it’s running:

railyard scale my-app worker=2

Hetzner is one of the providers Railyard can create a server on directly from your own account, billed at Hetzner’s price with no markup; see Add a server. Full framework reference at Django, and Deploy from a repository for what else gets read from the repo. See the programmatic walkthrough at Django on Hetzner for size and region specifics, and Databases and backups for how scheduled Postgres backups and point-in-time restore work once it’s attached.

Troubleshooting

Port 80 or 443 already in use. A default web server install often grabs these first. sudo ss -tlnp | grep -E ':80|:443' shows the process; stop or reconfigure it before Caddy starts.

ALLOWED_HOSTS rejects every request. Django returns a 400 with a clear message naming the host it expected, in logs you can read with docker compose logs web. Add the exact domain (and www. variant if you serve both) to ALLOWED_HOSTS in .env, then restart the web service.

Celery worker starts but nothing ever runs. This is almost always CELERY_BROKER_URL pointing somewhere the worker can’t reach, commonly localhost left over from local development instead of the Compose service name redis. Confirm with docker compose exec worker env | grep CELERY.

Certificate never issues. Check dig +short app.example.com resolves to the server first; Caddy can’t complete an HTTP-01 challenge for a domain that doesn’t point here yet. Also check your cloud firewall allows inbound 80 and 443, not just the ports you’re actively testing against.

Out of memory on a small server. Postgres, Redis, gunicorn and a Celery worker add up fast on a 4 GB machine if gunicorn’s worker count is left too high. --workers 3 here assumes a 4 vCPU box; halve it on a CX23, or move Postgres to its own server once traffic actually requires it.

FAQ

Do I need both a worker and a beat process? Only beat, if you schedule periodic tasks with @shared_task and a CELERYBEAT_SCHEDULE or django-celery-beat. An app that only dispatches tasks on demand (from a view, say) needs just the worker.

Can I run Celery and Django on the same container? You can start both processes in one container with a process supervisor, but a crash in one then takes down the other, and you lose the ability to scale the worker independently of web traffic. Separate containers from the same image, as above, avoid both problems.

Why Caddy instead of nginx? Caddy requests and renews Let’s Encrypt certificates on its own from a six-line config; nginx needs certbot wired in separately and a cron job to renew. Either works, Caddy is just less to maintain.

What if my app needs a specific Python version? Change the FROM python:3.12-slim line to match; Django and Celery both support the currently maintained Python versions, check your own dependencies’ constraints if any pin an older one.

Either path ends with the same three processes talking to the same Postgres and Redis, the Compose file if you want to see every line, the push if you’d rather get back to the actual Django code.