The capacity park could not survive its own first poll. ReserveResources
polls every 120 s and declared a maxDuration of 60, and RunRunner measures
a step's budget from a started_at it deliberately does not reset on a poll
— so every re-entry was ruled timed out before the body ran, a timeout
consumes an attempt, and five of them failed a paid order in about six
minutes. The fourteen-day promise, the console's capacity queue and the
"go and buy a server" workflow were unreachable code. maxDuration is now
the whole park plus one poll interval, and a test drives a park through
the runner rather than calling execute() directly, which is why this was
invisible.
Every guest occ call ran as root. `docker compose exec` defaults to root
and the Nextcloud image's console.php exits 1 unless the caller owns
config/config.php, so all five call sites were failing on every instance.
There is now one builder, App\Support\NextcloudOcc, and a test refuses a
second: nothing in app/ may spell the invocation out by hand. deploy/
update.sh had learned this for our own container and nobody carried it
across.
RunAcceptanceChecks was terminal on the first no. certReachable() answers
false for a connect timeout as readily as for a missing certificate, so
one bad second ended a finished, certified Nextcloud as a failed order
with the instance released. The probes retry now and the run still fails
for good with the probe's own reason once the budget is spent; the two
facts a retry cannot change stay terminal.
The in-flight guard asked whether anything was running, not whether it
would do the work. App\Provisioning\WorkInFlight asks the second question,
reading the step lists the runner executes: a domain proven during a
restart is routed, and a storage pack booked during an address run is
delivered instead of charged monthly and forgotten.
The address pipeline ran its steps in the wrong order for the direction it
exists for. Nextcloud before the router in `address` and `plan-change`, as
in `customer`, because a run that got the certificate and then failed left
the customer's own domain serving an untrusted-domain error under a valid
certificate while the portal called it live. Safe in both directions; the
reasoning is written above the pipeline.
A stale guest_ip could never be corrected. ConfigureDnsAndTls compared the
hostnames but never the backend, and nothing re-read the address after the
build. It now records and compares `routed_backend`, and ConfigureNetwork
joins the `restart` pipeline — a cold boot is what moves a DHCP lease — and
asks for the address to be re-applied when the guest has actually moved.
Also: HetznerDnsClient::upsertRecord read only the first page of a zone
that pages at 100, so past a hundred records it created a second A record
for the same name and the cloud was up about half the time; the console's
retry left a live customer's order in `provisioning` for ever for any
maintenance pipeline; and a revived run's first pass burned an attempt on a
timeout that had already happened, because the clause written for it used
`??=`.
Seven test sites wrote config('provisioning.plans.*'), a key that has not
existed since the catalogue moved into the database. Every write was a
no-op, so each test proved something other than its name: the missing
template, the two snapshot regressions and the grandfathered-price scenario
are now actually constructed, ValidateOrder's reasons are asserted rather
than only its type, and the end-to-end run checks that the quota step
delivered something.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A lost response, or a retry after a later step failed, made every further
attempt a 404 — and the host impossible to purge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The console can now create, block and remove VPN accesses, and shows who is
actually connected — traffic counters, source endpoint and last handshake.
Design notes:
- The web container has no wg0, so the page never talks to WireGuard. Live
state is copied in by SyncVpnPeers on the provisioning queue (the only worker
whose container owns the interface); the page polls the database and merely
nudges that job, throttled so many open tabs cannot flood the queue.
- enabled (operator intent) and present (observed on the hub) are stored
separately, so a sync can never quietly undo a block and drift stays visible
as "wird angewendet".
- The sync adopts peers the host pipeline registered, naming them after their
host — otherwise "who is on the VPN" would have blind spots.
- Keypairs are generated in PHP via libsodium rather than shelling out to
wg genkey (no interface in this container, and it keeps generation testable).
The private key is shown once and never stored.
- allocateIp() now also considers vpn_peers, which would otherwise be handed a
tunnel address a host already holds.
- vpn.manage is a new capability held by Owner/Admin only, and it hides the nav
entry too — a VPN access reaches the management network.
Verified end to end against the real hub, not just mocks: created an access in
the browser, connected with the generated config, watched the console flip to
"Verbunden" with real counters, then blocked it and confirmed the peer was
gone from wg0 and the client could no longer handshake.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PhpseclibRemoteShell sets a command timeout (default 2000s) so apt
full-upgrade/install don't hit phpseclib's ~10s default and fail.
- Host removal dispatches RemoveWireguardPeer to the provisioning queue, which
runs in the worker that owns wg0 (the web container can't manage WireGuard).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CreateAutomationToken now bootstraps the pveum role/user/token over the
authenticated SSH session (a fresh host has no API token yet); ProxmoxClient
is read-only in A.
- ConfigureWireguard re-verifies the handshake on the idempotent replay path,
never advancing over a dead tunnel.
- PhpseclibRemoteShell treats a missing SSH exit status as failure (255).
- connectWithKey verifies the pinned host key fingerprint on later logins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo