diff --git a/docs/wireguard-hub.md b/docs/wireguard-hub.md new file mode 100644 index 0000000..781200e --- /dev/null +++ b/docs/wireguard-hub.md @@ -0,0 +1,85 @@ +# WireGuard management hub + +Every Proxmox host CluPilot manages is reachable **only** over this tunnel — the +hosts have no public SSH. The hub is the one endpoint that accepts peers. + +## Where it runs + +Inside the `queue-provisioning` container, not on the VM host. That container is +the only one that needs it: `AdvanceRunJob`, `PurgeHost` and `RemoveWireguardPeer` +all `onQueue('provisioning')`, and it is the only worker consuming that queue. +`LocalWireguardHub` shells out to `wg` / `wg-quick`, so the interface must exist +in the same namespace as the PHP process. + +Compose already grants what that needs: `cap_add: NET_ADMIN`, `/dev/net/tun`, +`net.ipv4.ip_forward=1`, the published `51820/udp`, and a named `wireguard` +volume for `/etc/wireguard`. The container command runs `wg-quick up wg0` before +the worker starts, so the tunnel returns after every restart. + +The host kernel module does not need `modprobe`: creating the first +`type wireguard` link auto-loads `wireguard.ko`. + +## First-time setup + +Only needed when the `wireguard` volume is empty (the config is not in git — it +holds the private key). + +```bash +docker compose exec queue-provisioning sh -c 'umask 077; wg genkey > /etc/wireguard/hub.key; wg pubkey < /etc/wireguard/hub.key' +``` + +Write `/etc/wireguard/wg0.conf` with that private key: + +```ini +[Interface] +Address = 10.66.0.1/24 +ListenPort = 51820 +PrivateKey = +``` + +Then set in `.env` — `CLUPILOT_WG_ENDPOINT` must be the address a **Proxmox host** +can reach, not a container address: + +``` +CLUPILOT_WG_ENDPOINT=10.10.90.185:51820 +CLUPILOT_WG_HUB_PUBKEY= +``` + +`wg-quick save wg0` rewrites `wg0.conf` on every peer change, so keep manual +edits to a minimum — comments in that file are lost. + +## Peer lifecycle + +`LocalWireguardHub::allocateIp()` hands out the next free address in +`CLUPILOT_WG_SUBNET`, skipping the hub IP, the network and the broadcast address, +and skipping every `hosts.wg_ip` already taken. `addPeer()`/`removePeer()` run +`wg set` and then `wg-quick save wg0` — without the save, peers are lost on the +next restart. + +## Verifying it actually works + +Mocked tests cannot prove a tunnel. Against the running stack: + +1. Start a throwaway peer that stands in for a Proxmox host: + `docker run -d --name wgpeer --cap-add NET_ADMIN --device /dev/net/tun clupilot-app:dev sleep 900` +2. Generate its keypair, register it through the app (not by hand) — + `app(WireguardHub::class)->addPeer('', '10.66.0.2')`. +3. Bring up the peer with `Endpoint = 10.10.90.185:51820`, + `AllowedIPs = 10.66.0.0/24`, `PersistentKeepalive = 25`. +4. `wg show wg0` on both sides must report a recent handshake. +5. Prove the direction the pipeline needs — hub → peer TCP, which is what the SSH + steps use: listen on `10.66.0.2:2222` in the peer and `fsockopen` to it from + `queue-provisioning`. A handshake alone does not prove routed TCP. +6. Clean up with `removePeer()` and delete the container. + +Verified end-to-end on 2026-07-25: handshake established, hub→peer TCP banner +received, peer persisted to `wg0.conf` by `wg-quick save` and removed again. + +## Before production + +- The endpoint becomes the production VM's address; regenerate the keypair there + (the dev private key must never travel). +- `51820/udp` is the only port that has to be open inbound — the console and the + Proxmox hosts stay off the public hostnames (see `config/admin_access.php`). +- Back up `/etc/wireguard/` (the `wireguard` volume): losing the hub private key + means re-peering every host.