docs(wireguard): bring the management hub up and prove it end-to-end

The hub lives in queue-provisioning (the only worker on the provisioning queue,
and LocalWireguardHub shells out to wg in-process). Generated the hub keypair,
wrote wg0.conf, and filled CLUPILOT_WG_ENDPOINT/HUB_PUBKEY.

Verified against the running stack rather than mocks: registered a stand-in
Proxmox peer through the app's own addPeer(), got a handshake, opened hub->peer
TCP over the tunnel (what the SSH steps actually need — a handshake alone does
not prove routed TCP), confirmed wg-quick save persisted the peer, then removed
it through removePeer(). wg0 returns after a container restart.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat/portal-design
nexxo 2026-07-25 21:05:11 +02:00
parent 7247edf9d8
commit 8edb2a69e5
1 changed files with 85 additions and 0 deletions

85
docs/wireguard-hub.md Normal file
View File

@ -0,0 +1,85 @@
# WireGuard management hub
Every Proxmox host CluPilot manages is reachable **only** over this tunnel — the
hosts have no public SSH. The hub is the one endpoint that accepts peers.
## Where it runs
Inside the `queue-provisioning` container, not on the VM host. That container is
the only one that needs it: `AdvanceRunJob`, `PurgeHost` and `RemoveWireguardPeer`
all `onQueue('provisioning')`, and it is the only worker consuming that queue.
`LocalWireguardHub` shells out to `wg` / `wg-quick`, so the interface must exist
in the same namespace as the PHP process.
Compose already grants what that needs: `cap_add: NET_ADMIN`, `/dev/net/tun`,
`net.ipv4.ip_forward=1`, the published `51820/udp`, and a named `wireguard`
volume for `/etc/wireguard`. The container command runs `wg-quick up wg0` before
the worker starts, so the tunnel returns after every restart.
The host kernel module does not need `modprobe`: creating the first
`type wireguard` link auto-loads `wireguard.ko`.
## First-time setup
Only needed when the `wireguard` volume is empty (the config is not in git — it
holds the private key).
```bash
docker compose exec queue-provisioning sh -c 'umask 077; wg genkey > /etc/wireguard/hub.key; wg pubkey < /etc/wireguard/hub.key'
```
Write `/etc/wireguard/wg0.conf` with that private key:
```ini
[Interface]
Address = 10.66.0.1/24
ListenPort = 51820
PrivateKey = <hub.key>
```
Then set in `.env``CLUPILOT_WG_ENDPOINT` must be the address a **Proxmox host**
can reach, not a container address:
```
CLUPILOT_WG_ENDPOINT=10.10.90.185:51820
CLUPILOT_WG_HUB_PUBKEY=<the pubkey printed above>
```
`wg-quick save wg0` rewrites `wg0.conf` on every peer change, so keep manual
edits to a minimum — comments in that file are lost.
## Peer lifecycle
`LocalWireguardHub::allocateIp()` hands out the next free address in
`CLUPILOT_WG_SUBNET`, skipping the hub IP, the network and the broadcast address,
and skipping every `hosts.wg_ip` already taken. `addPeer()`/`removePeer()` run
`wg set` and then `wg-quick save wg0` — without the save, peers are lost on the
next restart.
## Verifying it actually works
Mocked tests cannot prove a tunnel. Against the running stack:
1. Start a throwaway peer that stands in for a Proxmox host:
`docker run -d --name wgpeer --cap-add NET_ADMIN --device /dev/net/tun clupilot-app:dev sleep 900`
2. Generate its keypair, register it through the app (not by hand) —
`app(WireguardHub::class)->addPeer('<peer pubkey>', '10.66.0.2')`.
3. Bring up the peer with `Endpoint = 10.10.90.185:51820`,
`AllowedIPs = 10.66.0.0/24`, `PersistentKeepalive = 25`.
4. `wg show wg0` on both sides must report a recent handshake.
5. Prove the direction the pipeline needs — hub → peer TCP, which is what the SSH
steps use: listen on `10.66.0.2:2222` in the peer and `fsockopen` to it from
`queue-provisioning`. A handshake alone does not prove routed TCP.
6. Clean up with `removePeer()` and delete the container.
Verified end-to-end on 2026-07-25: handshake established, hub→peer TCP banner
received, peer persisted to `wg0.conf` by `wg-quick save` and removed again.
## Before production
- The endpoint becomes the production VM's address; regenerate the keypair there
(the dev private key must never travel).
- `51820/udp` is the only port that has to be open inbound — the console and the
Proxmox hosts stay off the public hostnames (see `config/admin_access.php`).
- Back up `/etc/wireguard/` (the `wireguard` volume): losing the hub private key
means re-peering every host.