2026-07-17

Running a 5-Node Docker Swarm at Home: Why Criticality Tiering Matters

This post was drafted with assistance from Hermes (my automation crew) and reviewed and approved by me before publishing.

The Stack That Grew Without a Plan

Every homelab starts small. A single server. A handful of containers. Docker Compose on one machine, maybe a NAS for storage. Then you add Plex. Then a dashboard. Then Home Assistant, a reverse proxy, some monitoring tools, a photo backup service, and suddenly you’re running twenty containers on one box and wondering why everything feels fragile.

That was my setup eighteen months ago. Today I run a five-node Docker Swarm across Proxmox VMs, and the single biggest operational decision I made wasn’t about hardware — it was about how I organized the stacks.

The Hardware: Five Nodes on Proxmox

My cluster runs on five Proxmox VMs, each with:

Every node runs Ubuntu Server, joined to the same Docker Swarm. Node labels are the linchpin that makes criticality tiering work. My real labels are more service-specific — plex=true, ispy_host=true, and so on — but for the tiering pattern, imagine one tier=X label per node.

# How node labels are set
docker node update --label-add tier=data node-01
docker node update --label-add tier=media node-02
docker node update --label-add tier=surveillance node-03
docker node update --label-add tier=apps node-04
docker node update --label-add tier=agents node-05

The Three Tiers

The core insight is simple: not all containers matter equally. A database going down takes everything with it. Plex being unavailable is annoying but not catastrophic. Organizing stacks by criticality means you can reason about failure domains, maintenance windows, and resource contention cleanly.

Tier 1: Data Layer

This is your foundation. Databases, message queues, logging infrastructure, and anything whose failure cascades. These stacks run exclusively on dedicated data nodes.

• InfluxDB and Redis — core state for apps and metrics
• MariaDB and PostgreSQL — primary SQL databases
• Central log collector — every container forwards logs via GELF (Graylog Extended Log Format — the wire format, not a Graylog server) to one ingestion point on this node
• NFS client mounts — all persistent storage is NFS-backed from a dedicated OpenMediaVault server, mounted on every node

Why isolate this tier? If your database node gets noisy neighbor contention from a media transcoding job, every service suffers. By pinning data stacks to dedicated nodes, you guarantee compute isolation for stateful workloads.

# Example Docker stack constraint for the data tier
services:
  postgres:
    image: postgres:16
    deploy:
      placement:
        constraints:
          - node.labels.tier == data
    volumes:
      - pgdata:/var/lib/postgresql/data

Tier 2: Access Portals

The access tier is your ingress plane — the services that terminate external traffic and route it inward.

• Traefik as the reverse proxy, configured with Cloudflare DNS-01 challenge for wildcard TLS certificates
• Authentik as the identity-aware layer — SSO in front of internal services
• A dynamic-DNS updater keeping Cloudflare records pointed at home as the residential IP changes

Traefik’s DNS-01 challenge is the right choice here because it issues wildcard certificates that cover every subdomain. No per-service cert management, no port exposure beyond what Traefik handles. The access tier nodes sit between the public internet and your data — keeping them lightly loaded and single-purpose reduces attack surface.

services:
  traefik:
    image: traefik:v2.10
    deploy:
      placement:
        constraints:
          - node.labels.tier == access
    command:
      - "--certificatesresolvers.cloudflare.acme.dnschallenge=true"
      - "--certificatesresolvers.cloudflare.acme.dnschallenge.provider=cloudflare"

Tier 3: Media and Applications

Everything user-facing that doesn’t hold critical state goes here. This is where you have headroom for experimentation.

• Plex — media streaming
• Home Assistant — home automation
• Dashboards and monitoring UIs — Grafana, Uptime Kuma, iSpy cameras
• Internal tools and side projects — anything that can go down without waking you up at 2 AM

These stacks span dedicated media nodes. Because no critical state lives here, you can drain a node, update packages, or redeploy services without worrying about data loss. If a media node crashes, services that aren’t pinned to specific hardware can restart on another node; the GPU-pinned ones wait for their node to come back. If the database node crashes, everything stops — but that node runs nothing else.

Observability: Metrics First, Logs Centralized

Container logging in Swarm requires deliberate architecture. docker logs works for single containers but the moment you’re spread across five nodes, you need centralized aggregation.

Metrics are the primary pane of glass here: Prometheus scrapes every node and service, and Grafana dashboards answer “is it healthy” before logs ever enter the picture. For logs, every container ships structured output via the GELF driver to a single collector on the data tier:

logging:
  driver: gelf
  options:
    gelf-address: udp://logs.internal:12201
    tag: "{{.Name}}/{{.ImageName}}"

GELF is lightweight, UDP-based (no backpressure on your app), and carries structured fields (host, container name, image, timestamp) automatically. When a container crashes anywhere in the cluster, its output lands in one place with metadata already attached — the search UI is another matter. I ran a full Elasticsearch/Logstash/Kibana pipeline on top of the collector for a while and retired it: at homelab scale, ELK’s resource appetite outweighed what it returned. Prometheus and Grafana carry the “is this healthy” question now; the collector still ingests structured logs for when I need to grep them, and that turns out to be rare.

What Tiering Buys You Operationally

Maintenance without panic. Need to apply kernel updates to a media node? Place it in maintenance mode, let Swarm reschedule containers to the remaining media nodes, and reboot one at a time. Try that with a single-server setup — everything goes down.

Resource accounting. Plex transcoding maxes out CPU on a media node? The database on the data node doesn’t stutter. Traefik CPU usage spikes during a TLS handshake burst? The access node handles it without stealing cycles from Plex.

Failure isolation. A buggy Compose file that eats memory OOMs a media node. Your databases keep running. Your reverse proxy keeps routing. Your family keeps watching movies (after Swarm reschedules the media containers). The blast radius is contained by design.

Clear decision rules. Adding a new service? The question isn’t “which server does it go on” — it’s “which tier does it belong to.” That’s a faster, more consistent choice.

Observability as a Separate Concern

In this setup, monitoring infrastructure — Prometheus, Grafana, and the log collector — can live on the data node alongside databases. However, if you’re scaling to dozens of services, consider a dedicated observability node. Logging spikes during errors can compete with database query latency if they’re co-resident. Separating them costs one extra node but pays dividends in MTTR and visibility.

The Homelab Advantage

This tiering pattern is particular to homelab infrastructure, where you own the hardware and can make deliberate placement choices. Cloud deployments lean on managed services (Firestore, DynamoDB, Cloudflare Workers) to sidestep these problems. At home, the tier model is your substitute for that automation — simple, code-based, and portable.

See the full homelab infrastructure documentation for details on node provisioning, NFS configuration, and monitoring setup.

The Takeaway

Criticality tiering isn’t a data-center concept scaled down. It’s a homelab pattern that costs nothing to implement — just node labels and placement constraints — and pays dividends the first time a service crashes. Your future self, woken up by an alert at midnight, will thank you for not putting the database on the same node as Plex.

Five nodes, three tiers, one Swarm. That’s the recipe.