Azure VM Scale Sets are great: a pool of machines that restarts failed instances, balances load and rolls out updates for you. I had ten physical machines sitting on a desk and wanted the same behaviour, with nothing but free tools. This post is the research and the decision.

#1. The Hardware and the Constraint

  • 10 machines, each a 13th-gen Intel CPU. About 130 cores in total.
  • 8 GB RAM per node. About 80 GB across the cluster, which turns out to be the constraint that drives every decision.
  • $0 software budget and zero Kubernetes experience.

The goal was VMSS-like behaviour: auto-restart on failure, load balancing, rolling updates and work spread across all ten nodes.

#2. The Foundation Is the Same Either Way

Proxmox VE goes on every node first. It clusters the machines, gives you a web UI for all ten, and lets you carve up VMs if you need them. Two things matter more than anything else here:

  • Static IPs for every node (as DHCP reservations). Dynamic addresses make nodes lose each other after a reboot.
  • A managed switch. An unmanaged one works for Swarm but blocks the VLAN features you'll want later.

At 8 GB per node, Ceph is out; it wants far more memory than that. One node serving NFS for shared storage is the practical replacement.

#3. The Two Contenders

Text
                    Docker Swarm             K3s (lightweight Kubernetes)
Setup time          ~30 minutes              ~2-3 hours (with troubleshooting)
RAM per node        ~1.5 GB with services    ~3.5 GB with services
Autoscaling         manual or scripted       automatic (HPA)
Learning curve      you already know Docker  YAML, networking, controllers
Dashboard           Portainer CE             Rancher / Lens / k9s

Both give you what VMSS gives you: rescheduling when a node dies, a routing mesh or service load balancer, and zero-downtime rolling updates. The differences are overhead, autoscaling and how much there is to learn.

#4. The Verdict: Swarm First, K3s Later

Starting with full Kubernetes with no experience means spending most of your time debugging YAML and cluster networking rather than running workloads. With zero services deployed, Kubernetes adds no value yet. Docker Swarm:

  • Is built into Docker, so the only new commands are docker swarm init and docker service create.
  • Leaves more RAM for the apps, which matters most at 8 GB per node.
  • Uses the same container images K3s would, so migrating later is a change of commands, not a rewrite.
When to move to K3s

At 20+ services, when you need CPU/memory-based autoscaling (Swarm can't do this natively), or when you need fine-grained scheduling, secrets or network policies.

#5. The Build Order

Text
# Phase 1 - foundation
static IPs + Wake-on-LAN -> Proxmox VE on all 10 nodes

# Phase 2 - cluster + storage
pvecm create my-cluster        # node 1
pvecm add <node-1-ip>         # nodes 2-10
NFS export on node 10, mounted on nodes 1-9

# Phase 3 - Swarm
docker swarm init              # node 1
3 managers (odd number for quorum), 7 workers
docker service create --name test --replicas 5 nginx
Portainer for the dashboard

# Phase 4 - prove it
unplug a worker -> service recovers
10 replicas -> one per node
rolling image update -> zero downtime

#6. Takeaways

  • Pick the orchestrator by what you'll run, not by what's on job listings. The simpler tool you understand beats the powerful one you're fighting.
  • Memory per node, not total cores, decides the storage and orchestration stack.
  • Keep a migration path open. Containers are the portable layer; the orchestrator is swappable.