docs(wireguard): bring the management hub up and prove it end-to-end
The hub lives in queue-provisioning (the only worker on the provisioning queue, and LocalWireguardHub shells out to wg in-process). Generated the hub keypair, wrote wg0.conf, and filled CLUPILOT_WG_ENDPOINT/HUB_PUBKEY. Verified against the running stack rather than mocks: registered a stand-in Proxmox peer through the app's own addPeer(), got a handshake, opened hub->peer TCP over the tunnel (what the SSH steps actually need — a handshake alone does not prove routed TCP), confirmed wg-quick save persisted the peer, then removed it through removePeer(). wg0 returns after a container restart. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>feat/portal-design
parent
7247edf9d8
commit
8edb2a69e5
|
|
@ -0,0 +1,85 @@
|
|||
# WireGuard management hub
|
||||
|
||||
Every Proxmox host CluPilot manages is reachable **only** over this tunnel — the
|
||||
hosts have no public SSH. The hub is the one endpoint that accepts peers.
|
||||
|
||||
## Where it runs
|
||||
|
||||
Inside the `queue-provisioning` container, not on the VM host. That container is
|
||||
the only one that needs it: `AdvanceRunJob`, `PurgeHost` and `RemoveWireguardPeer`
|
||||
all `onQueue('provisioning')`, and it is the only worker consuming that queue.
|
||||
`LocalWireguardHub` shells out to `wg` / `wg-quick`, so the interface must exist
|
||||
in the same namespace as the PHP process.
|
||||
|
||||
Compose already grants what that needs: `cap_add: NET_ADMIN`, `/dev/net/tun`,
|
||||
`net.ipv4.ip_forward=1`, the published `51820/udp`, and a named `wireguard`
|
||||
volume for `/etc/wireguard`. The container command runs `wg-quick up wg0` before
|
||||
the worker starts, so the tunnel returns after every restart.
|
||||
|
||||
The host kernel module does not need `modprobe`: creating the first
|
||||
`type wireguard` link auto-loads `wireguard.ko`.
|
||||
|
||||
## First-time setup
|
||||
|
||||
Only needed when the `wireguard` volume is empty (the config is not in git — it
|
||||
holds the private key).
|
||||
|
||||
```bash
|
||||
docker compose exec queue-provisioning sh -c 'umask 077; wg genkey > /etc/wireguard/hub.key; wg pubkey < /etc/wireguard/hub.key'
|
||||
```
|
||||
|
||||
Write `/etc/wireguard/wg0.conf` with that private key:
|
||||
|
||||
```ini
|
||||
[Interface]
|
||||
Address = 10.66.0.1/24
|
||||
ListenPort = 51820
|
||||
PrivateKey = <hub.key>
|
||||
```
|
||||
|
||||
Then set in `.env` — `CLUPILOT_WG_ENDPOINT` must be the address a **Proxmox host**
|
||||
can reach, not a container address:
|
||||
|
||||
```
|
||||
CLUPILOT_WG_ENDPOINT=10.10.90.185:51820
|
||||
CLUPILOT_WG_HUB_PUBKEY=<the pubkey printed above>
|
||||
```
|
||||
|
||||
`wg-quick save wg0` rewrites `wg0.conf` on every peer change, so keep manual
|
||||
edits to a minimum — comments in that file are lost.
|
||||
|
||||
## Peer lifecycle
|
||||
|
||||
`LocalWireguardHub::allocateIp()` hands out the next free address in
|
||||
`CLUPILOT_WG_SUBNET`, skipping the hub IP, the network and the broadcast address,
|
||||
and skipping every `hosts.wg_ip` already taken. `addPeer()`/`removePeer()` run
|
||||
`wg set` and then `wg-quick save wg0` — without the save, peers are lost on the
|
||||
next restart.
|
||||
|
||||
## Verifying it actually works
|
||||
|
||||
Mocked tests cannot prove a tunnel. Against the running stack:
|
||||
|
||||
1. Start a throwaway peer that stands in for a Proxmox host:
|
||||
`docker run -d --name wgpeer --cap-add NET_ADMIN --device /dev/net/tun clupilot-app:dev sleep 900`
|
||||
2. Generate its keypair, register it through the app (not by hand) —
|
||||
`app(WireguardHub::class)->addPeer('<peer pubkey>', '10.66.0.2')`.
|
||||
3. Bring up the peer with `Endpoint = 10.10.90.185:51820`,
|
||||
`AllowedIPs = 10.66.0.0/24`, `PersistentKeepalive = 25`.
|
||||
4. `wg show wg0` on both sides must report a recent handshake.
|
||||
5. Prove the direction the pipeline needs — hub → peer TCP, which is what the SSH
|
||||
steps use: listen on `10.66.0.2:2222` in the peer and `fsockopen` to it from
|
||||
`queue-provisioning`. A handshake alone does not prove routed TCP.
|
||||
6. Clean up with `removePeer()` and delete the container.
|
||||
|
||||
Verified end-to-end on 2026-07-25: handshake established, hub→peer TCP banner
|
||||
received, peer persisted to `wg0.conf` by `wg-quick save` and removed again.
|
||||
|
||||
## Before production
|
||||
|
||||
- The endpoint becomes the production VM's address; regenerate the keypair there
|
||||
(the dev private key must never travel).
|
||||
- `51820/udp` is the only port that has to be open inbound — the console and the
|
||||
Proxmox hosts stay off the public hostnames (see `config/admin_access.php`).
|
||||
- Back up `/etc/wireguard/` (the `wireguard` volume): losing the hub private key
|
||||
means re-peering every host.
|
||||
Loading…
Reference in New Issue