Azure VM Scale Sets are great: a pool of machines that restarts failed instances, balances load and rolls out updates for you. I had ten physical machines sitting on a desk and wanted the same behaviour, with nothing but free tools. This post is the research and the decision.
#1. The Hardware and the Constraint
- 10 machines, each a 13th-gen Intel CPU. About 130 cores in total.
- 8 GB RAM per node. About 80 GB across the cluster, which turns out to be the constraint that drives every decision.
- $0 software budget and zero Kubernetes experience.
The goal was VMSS-like behaviour: auto-restart on failure, load balancing, rolling updates and work spread across all ten nodes.
#2. The Foundation Is the Same Either Way
Proxmox VE goes on every node first. It clusters the machines, gives you a web UI for all ten, and lets you carve up VMs if you need them. Two things matter more than anything else here:
- Static IPs for every node (as DHCP reservations). Dynamic addresses make nodes lose each other after a reboot.
- A managed switch. An unmanaged one works for Swarm but blocks the VLAN features you'll want later.
At 8 GB per node, Ceph is out; it wants far more memory than that. One node serving NFS for shared storage is the practical replacement.
#3. The Two Contenders
Docker Swarm K3s (lightweight Kubernetes)
Setup time ~30 minutes ~2-3 hours (with troubleshooting)
RAM per node ~1.5 GB with services ~3.5 GB with services
Autoscaling manual or scripted automatic (HPA)
Learning curve you already know Docker YAML, networking, controllers
Dashboard Portainer CE Rancher / Lens / k9sBoth give you what VMSS gives you: rescheduling when a node dies, a routing mesh or service load balancer, and zero-downtime rolling updates. The differences are overhead, autoscaling and how much there is to learn.
#4. The Verdict: Swarm First, K3s Later
Starting with full Kubernetes with no experience means spending most of your time debugging YAML and cluster networking rather than running workloads. With zero services deployed, Kubernetes adds no value yet. Docker Swarm:
- Is built into Docker, so the only new commands are
docker swarm initanddocker service create. - Leaves more RAM for the apps, which matters most at 8 GB per node.
- Uses the same container images K3s would, so migrating later is a change of commands, not a rewrite.
At 20+ services, when you need CPU/memory-based autoscaling (Swarm can't do this natively), or when you need fine-grained scheduling, secrets or network policies.
#5. The Build Order
# Phase 1 - foundation
static IPs + Wake-on-LAN -> Proxmox VE on all 10 nodes
# Phase 2 - cluster + storage
pvecm create my-cluster # node 1
pvecm add <node-1-ip> # nodes 2-10
NFS export on node 10, mounted on nodes 1-9
# Phase 3 - Swarm
docker swarm init # node 1
3 managers (odd number for quorum), 7 workers
docker service create --name test --replicas 5 nginx
Portainer for the dashboard
# Phase 4 - prove it
unplug a worker -> service recovers
10 replicas -> one per node
rolling image update -> zero downtime#6. Takeaways
- Pick the orchestrator by what you'll run, not by what's on job listings. The simpler tool you understand beats the powerful one you're fighting.
- Memory per node, not total cores, decides the storage and orchestration stack.
- Keep a migration path open. Containers are the portable layer; the orchestrator is swappable.