Setting up a Proxmox cluster with Claude Code, CLI-only

Proxmox turns one computer into a host for many virtual machines. Its web UI is fine. The buttons are where you’d expect and the wizards work.

I never open it.

Four Bee-Link mini PCs sat on my desk last December. They had to become a 4-node Proxmox 9.1 cluster: twelve VMs, a GPU server, and ZFS mirrors (two disks holding identical copies, so one disk can fail safely). Each VM type gets a cloud-init template; cloud-init is how cloud servers configure themselves on first boot. A Tailscale subnet router lets me reach the whole network from anywhere. I could plug a monitor into each node and click through the same wizards four times, or I could write the setup once as scripts and run those scripts on every node. I picked the scripts and wrote them with Claude Code, an AI agent that runs in the terminal. This post is how that went, bugs included.

Why CLI over UI for cluster work

The web UI is great at single moves. It’s terrible at the same move four times.

When you point and click, your record of what happened is your memory. Scroll past one checkbox during the install, and node 2 ends up subtly different from node 1. You will scroll past one. You find out three months later, when one host can’t replicate a VM and you can’t remember which node got the swappiness tweak (a kernel memory-tuning knob). I have a 99-proxmox.conf on node 1 that’s just vm.swappiness=10 repeated nine times. I don’t know how that happened. Neither does the UI.

On the CLI the same work is text. Text can be diffed, reviewed, and committed to git. The cluster ends up describing itself in a directory of shell scripts, and the next time I add a node, I run the scripts.

I wanted infrastructure as code (the setup written as runnable files) without the upfront tax of Terraform, the heavyweight standard tool for the job. No resource graph, no provider plugins to bisect, no state file. I wanted to write bash and have someone smarter than me catch when I was about to write something dumb.

The bootstrap: an autoinstall ISO that knows which node it is

Proxmox 9 installs itself unattended: you bake an answer file (in TOML, a config file format) into the install ISO. There is one ISO per node, and each ISO pins a hostname (pve-node1, pve-node2, …, pve-gpu). The answer file drops a first-boot.sh onto the new system. That one script drives everything else.

The first thing first-boot.sh does is figure out which node it just booted on:

HOSTNAME=$(hostname -s)
if [[ "$HOSTNAME" =~ ^pve-node([0-9]+)$ ]]; then
    NODE_NUM="${BASH_REMATCH[1]}"
    NODE_TYPE="standard"
elif [[ "$HOSTNAME" == "pve-gpu" ]]; then
    NODE_NUM=99
    NODE_TYPE="gpu"
else
    echo "ERROR: hostname '$HOSTNAME' doesn't match pattern" >&2
    exit 1
fi

Everything downstream follows from that hostname check: the node’s IP address, its ZFS layout, whether it gets GPU drivers, and whether it creates the cluster or joins it. Node 1 creates the cluster. Nodes 2–4 wait for the cluster to be reachable, then call pvecm add to join it. The GPU node skips Proxmox entirely. It runs Ubuntu 24.04 with the NVIDIA Container Toolkit and exports a ZFS pool over NFS (network file sharing) as the cluster’s backup target.

The first version of first-boot.sh was a spaghetti of ifs. I asked Claude Code to break it into numbered phases (00-config.sh, 02-cluster.sh, 02-storage.sh, 06-vms.sh, 07-monitoring.sh, 08-backup.sh), each idempotent: safe to run twice, because a second run changes nothing. It refactored the script. I read the diff. I ran the result on node 4 first, because node 4 had no live VMs. Nothing broke.

The storage gotcha that cost me an evening

Set zfs.hdsize=100 in the autoinstall file and the installer creates partitions 1–3, then leaves the rest of each NVMe drive unpartitioned. (You should set it; 100 GB is plenty for rpool, the pool that holds the operating system.) Unpartitioned space just sits there, invisible and unused. The docs imply you have 900 GB free. You first have to carve that space out yourself. The UI eventually gets there with point-and-click. The CLI gets there with sgdisk, a command-line partitioning tool:

for dev in "${RPOOL_DEVS[@]}"; do
    p4=$(get_part_path "$dev" 4)
    if [[ ! -b "$p4" ]]; then
        sgdisk -n 4:0:0 -t 4:bf01 "$dev"
    fi
done
partprobe; udevadm settle

-n 4:0:0 means partition 4, from the next available sector to the end of the disk. -t 4:bf01 sets its type to Solaris/ZFS. Run that on both NVMe drives, then zpool create vmpool mirror /dev/.../p4 /dev/.../p4, and you have a 900 GB mirrored pool for VM storage.

The first bug I shipped was in the helper that turned a partition path back into a base device. It used sed 's/-part[0-9]*$//', which strips a trailing suffix like -part3. That handles /dev/disk/by-id/...-part3. On /dev/nvme0n1p3 the sed does nothing, because there is no -part to strip. The script worked perfectly on the test path and silently failed on plain /dev/nvme... device names. The fix took five minutes. The search took an hour, because everything looked fine until I read zpool status (ZFS’s health report) very carefully.

The first VM, declaratively

Once vmpool exists, every VM in this cluster is the same five-step recipe in qm, Proxmox’s command-line tool for VMs: create the empty VM, import a stock Ubuntu cloud image as its disk, grow the disk, attach cloud-init, boot. (ovmf means modern UEFI firmware; the other flags are plumbing.)

qm create $VMID \
  --name $VM_NAME \
  --memory $MEM --cores $CORES \
  --net0 virtio,bridge=vmbr0 \
  --ostype l26 --machine q35 --bios ovmf \
  --efidisk0 vmpool:0,format=raw,efitype=4m,pre-enrolled-keys=1 \
  --scsihw virtio-scsi-single

qm importdisk $VMID /var/lib/vz/template/iso/jammy-server-cloudimg-amd64.img vmpool
qm set $VMID --scsi0 vmpool:vm-$VMID-disk-1,iothread=1,ssd=1,discard=on
qm resize $VMID scsi0 ${DISK_GB}G

qm set $VMID --ide2 vmpool:cloudinit
qm set $VMID --boot order=scsi0 --serial0 socket --vga serial0
qm set $VMID --cicustom "user=local:snippets/${VM_TYPE}-user-data.yaml" \
            --ipconfig0 "ip=$VM_IP/24,gw=$GW"

qm start $VMID

Five qm calls, idempotent once I script the create-or-skip logic. The cloud-init user-data (users, SSH keys, packages) lives at local:snippets/<type>-user-data.yaml, one file per VM role. To add a new VM type, I drop in a new YAML file and point one variable at it. No clicks, no wizard, no “I think I left the cloud-init checkbox unticked.”

The Proxmox UI can do all of this. It just can’t do it the same way every time.

The cloud-init / sshd_config trap

The next bug ran silently for three months before I noticed it.

sshd, the program that answers SSH logins, reads its config from a drop-in directory, a folder of numbered files merged in order. Cloud-init seeds a fresh Ubuntu image with /etc/ssh/sshd_config.d/50-cloud-init.conf containing PasswordAuthentication yes; that file is how cloud-init lets you log in on first boot. My hardening config then writes /etc/ssh/sshd_config.d/99-hardening.conf with PasswordAuthentication no. That line matters: bots guess passwords all day, but they cannot guess a key. The 99- prefix came from the usual last-word-wins reflex you learn from sudoers.d or sysctl.d.

The reflex is wrong here. sshd processes drop-ins alphabetically with first-match-wins semantics. 50-cloud-init.conf comes before 99-hardening.conf, so the first file sets PasswordAuthentication yes and the second file is silently ignored. The fix is one character:

 write_files:
-  - path: /etc/ssh/sshd_config.d/99-hardening.conf
+  - path: /etc/ssh/sshd_config.d/10-hardening.conf
     permissions: '0644'
     content: |
       PasswordAuthentication no
       PermitRootLogin no
       KbdInteractiveAuthentication no

You won’t notice from the outside. SSH still works. Public-key auth still works. The login banner says hardening is enabled. The only way to find the bug is to probe what’s not supposed to work:

ssh -o PreferredAuthentications=password \
    -o PubkeyAuthentication=no \
    ubuntu@vm-ip
# password prompt that should not exist

I caught it on a GitHub Actions runner, a VM that executes CI jobs. Its job logs are public on my Gitea (a self-hosted GitHub). Its IP is reachable on the LAN. Its sshd_config.d had carried the dud 99-hardening.conf since November.

Two lessons:

1. When you configure a daemon through a drop-in directory, check the merge semantics. sshd, nginx, and sysctl each handle duplicate keys differently, and “the first obtained value will be used” is buried in the sshd_config man page, not on the line that introduces Include.
2. Test the negative case. “SSH still works” is not the same as “hardening is in effect.” An ssh -o PreferredAuthentications=password probe in your post-provisioning smoke tests catches this in one line.

Where Claude Code actually closes the last mile

“AI agent runs your infra” is a sentence that should make any sensible operator nervous. In practice I use the agent less as an autopilot and more as a strange but helpful pair programmer:

Iterative discovery. I ask “cloud-init isn’t applying, can you look?” and it runs qm cloudinit dump, notices the snippet path is wrong, proposes a fix, and waits for me to OK it. That is faster than the Stack Overflow tab, and the commands are the same.

Fan-out. I say “snapshot every VM on node3 before I reboot it” and it generates the loop, shows the loop, and runs it. One sentence becomes a dozen commands. The same task in the UI is six clicks per VM, exactly the kind of chore where I’d skip the snapshot and regret it.

State auditing. I ask “what’s running on .82 and what’s it consuming?” and it answers with qm list, qm status, and pvesh get /nodes/pve-node3/status, formatted. The UI gives you the same answer across three screens.

Catching bugs. The agent isn’t magic. It still has to read the script. But it reads faster than I do, and without the assumption that “this worked yesterday.” When I asked why RPOOL_DEVS was empty, it was the first to notice that the sed pattern didn’t match the input.

What the agent is not: a thing I let drive long-running destructive operations on its own. Every command that does something irreversible (deleting a VM, destroying a pool, force-pushing) pauses for confirmation. That’s a setting, not an instinct. Don’t run a coding agent against your cluster without it.

Caveats

• The web UI is still useful for one-offs. The VM console, the cloud-init drive menus, anything rare enough that I’d have to Google the command anyway. CLI-first doesn’t mean CLI-only. It means the CLI is where the durable state lives.
• You will write helper functions you regret. Idempotent scripts are harder than non-idempotent ones, and Claude Code happily wrote me a “robust” function that was robust against the wrong thing. Read every diff.
• For a five-person team with real change management, the right answer is Terraform plus Ansible, the grown-up tools. For a one-person consultancy with four nodes, four hundred lines of bash is genuinely simpler.
• The cluster ends up describing itself in your CLAUDE.md, the notes file the agent reads. That’s the point, and also a risk: when those notes drift from reality, the agent becomes confidently wrong. Re-run audit prompts against the live state now and then.

The meta-loop

The same cluster serves my own language models. Two inference engines, vLLM and SGLang, run on a GPU server with two RTX PRO 6000 cards, and LiteLLM sits in front to give them one shared API. The Claude API calls that wrote half the bootstrap scripts could, in principle, go through that local stack.

The loop doesn’t close today. Claude Code still calls Anthropic’s API, and probably should: my single GPU server won’t beat their fleet on latency or model quality. But the local stack is what I show clients when I pitch sovereign LLM deployments, meaning AI that runs on hardware the client owns. This is what running AI on your own hardware looks like. I drove it here from a terminal. Here is the script.

That’s a more honest pitch than “trust me, I’ve used vLLM before”. The client gets the scripts, the bill of materials, and a list of what broke and how I fixed it. For clients weighing self-hosted AI against cloud lock-in, the failures are the useful part.

The cluster doesn’t need a UI. It needs scripts a stranger could read. The agent helps me write those scripts and runs them, and more than once it noticed that sed 's/-part[0-9]*$//' didn’t match what I thought it matched.

@starflinger.eu · Vienna, May 2026