CluPilotCloud/docs/wireguard-hub.md

WireGuard management hub

Every Proxmox host CluPilot manages is reachable only over this tunnel — the hosts have no public SSH. The hub is the one endpoint that accepts peers.

Where it runs

Inside the queue-provisioning container, not on the VM host. That container is the only one that needs it: AdvanceRunJob, PurgeHost and RemoveWireguardPeer all onQueue('provisioning'), and it is the only worker consuming that queue. LocalWireguardHub shells out to wg / wg-quick, so the interface must exist in the same namespace as the PHP process.

Compose already grants what that needs: cap_add: NET_ADMIN, /dev/net/tun, net.ipv4.ip_forward=1, the published 51820/udp, and a named wireguard volume for /etc/wireguard. The container command runs wg-quick up wg0 before the worker starts, so the tunnel returns after every restart.

The host kernel module does not need modprobe: creating the first type wireguard link auto-loads wireguard.ko.

First-time setup

Only needed when the wireguard volume is empty (the config is not in git — it holds the private key).

docker compose exec queue-provisioning sh -c 'umask 077; wg genkey > /etc/wireguard/hub.key; wg pubkey < /etc/wireguard/hub.key'

Write /etc/wireguard/wg0.conf with that private key:

[Interface]
Address = 10.66.0.1/24
ListenPort = 51820
PrivateKey = <hub.key>

Then set in .env — CLUPILOT_WG_ENDPOINT must be the address a Proxmox host can reach, not a container address:

CLUPILOT_WG_ENDPOINT=10.10.90.185:51820
CLUPILOT_WG_HUB_PUBKEY=<the pubkey printed above>

wg-quick save wg0 rewrites wg0.conf on every peer change, so keep manual edits to a minimum — comments in that file are lost.

Peer lifecycle

LocalWireguardHub::allocateIp() hands out the next free address in CLUPILOT_WG_SUBNET, skipping the hub IP, the network and the broadcast address, and skipping every hosts.wg_ip already taken. addPeer()/removePeer() run wg set and then wg-quick save wg0 — without the save, peers are lost on the next restart.

Verifying it actually works

Mocked tests cannot prove a tunnel. Against the running stack:

1. Start a throwaway peer that stands in for a Proxmox host: docker run -d --name wgpeer --cap-add NET_ADMIN --device /dev/net/tun clupilot-app:dev sleep 900
2. Generate its keypair, register it through the app (not by hand) — app(WireguardHub::class)->addPeer('<peer pubkey>', '10.66.0.2').
3. Bring up the peer with Endpoint = 10.10.90.185:51820, AllowedIPs = 10.66.0.0/24, PersistentKeepalive = 25.
4. wg show wg0 on both sides must report a recent handshake.
5. Prove the direction the pipeline needs — hub → peer TCP, which is what the SSH steps use: listen on 10.66.0.2:2222 in the peer and fsockopen to it from queue-provisioning. A handshake alone does not prove routed TCP.
6. Clean up with removePeer() and delete the container.

Verified end-to-end on 2026-07-25: handshake established, hub→peer TCP banner received, peer persisted to wg0.conf by wg-quick save and removed again.

Before production

• The endpoint becomes the production VM's address; regenerate the keypair there (the dev private key must never travel).
• 51820/udp is the only port that has to be open inbound — the console and the Proxmox hosts stay off the public hostnames (see config/admin_access.php).
• Back up /etc/wireguard/ (the wireguard volume): losing the hub private key means re-peering every host.