The SSH private key CluPilot deploys to every host is now stored through
SecretVault (ssh.private_key), preferred over CLUPILOT_SSH_PRIVATE_KEY(_PATH)
at all three real consumers — SshTraefikWriter and HostStep::keyLogin.
kuma.password/kuma.totp stay in .env: they authenticate a separate Python
container at boot, and nothing in this app reads them, so the vault would
have nothing to wire them to.
A new /admin/infrastructure page, backed by App\Support\ProvisioningSettings,
makes the non-secret deployment settings that used to force a shell —
DNS zone, WireGuard hub endpoint and public key, Traefik's dynamic-config
path, the SSH public key, and the monitoring bridge URL — visible and
editable from the console, each wired to every real consumer. WG subnet/hub
IP and Kuma's own connection details stay in .env: the compose file and a
separate container read those before this application ever does.
The password no longer travels through third-party mail servers or sits
forever in an inbox: it moves onto the instance (encrypted, same as
nc_admin_ref) and shows on the dashboard until the customer clicks
"Notiert", which deletes it for good.
Three things the owner asked for, and one the review found underneath them.
The console leaves /admin. It has its own route file now, registered before
routes/web.php because once it has a hostname to itself it sits at the ROOT of
that host — where `/` and `/settings` also exist for the portal, and the first
matching route wins. Where it mounts is one decision in one place: `/` on the
console hostname where it has one to itself, `/admin` on any host otherwise.
Names stay admin.* in both modes, so nothing that builds a URL has to know.
Exclusivity is its own switch, not a consequence of having a hostname. The
first attempt made "this host is the console" follow from ADMIN_HOSTS, and a
development machine lists its own IP there so the console works without DNS —
which took the public site and the portal off that machine entirely. The
switch also registers the console on every listed hostname, canonical last, so
the alternates that exist as recovery paths keep working.
The status page moves out of /legal, where it sat beside the imprint and the
terms. Nothing about the current health of the platform is a legal document. It
is a real page now: portal, instances, provisioning and backups, each derived
from records, aggregate only, and a component with no signal reports "unknown"
rather than "operational".
That last rule is what exposed the real bug. monitoring_targets.status was
written once — 'up', at provisioning — and nothing ever updated it. Both the
console's notices and the new public page read it, so an outage would have been
published as healthy indefinitely. There is a sync job now, on a five-minute
schedule, and a checked_at column so a verdict can go stale instead of standing
forever.
The monitoring contract had to grow a third state for that to be honest.
isHealthy() answers true when no monitoring is configured and false when the
monitor is unreachable; recording that boolean would have published either a
fleet-wide all-clear nobody measured or a fleet-wide outage that was really one
broken monitor. health() returns null for both, the recorder leaves the old
verdict to go stale, and isHealthy() keeps its forgiving semantics for the one
caller that wants them — the provisioning acceptance check, which must not fail
a delivery because monitoring is not set up.
Backups are counted from the instances that need protecting rather than from
the backup rows that exist, so an instance with no schedule at all cannot be
missing from the arithmetic that declares the estate protected.
Also: the update panel polls itself, offers the button only when there is
something to install, and opens the log while a run is in progress.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pipeline re-resolved config('provisioning.plans') by order.plan, so the
subscription snapshot protected a customer's price but not their machine:
shrinking a plan resized an existing customer's VM on its next run. Nothing
created a subscription either, so closing this meant opening the contract at
purchase and pointing provisioning at it.
- OpenSubscription freezes the catalogue onto a subscription when a checkout
is paid; StartCustomerProvisioning calls it inside the order transaction.
- CustomerStep::plan() reads the frozen snapshot. ValidateOrder and
ReserveResources fail closed with no_subscription rather than falling back
to the catalogue, which is the bug itself.
- template_vmid joins the snapshot so a re-clone cannot pick up a blueprint
published after the sale. Deliberately outside FROZEN: it is how we build
the machine, not a term the customer is owed, and a dead template must be
replaceable without cancelling a contract.
- TrafficMeter reads the allowance off the contract too — cutting a plan's
traffic was otherwise enough to start throttling someone who bought more.
- The migration backfills contracts for orders that already bought something,
reconstructed from what was actually delivered where an instance exists,
and adopts an existing order-less contract instead of opening a second.
Orders paid in a currency the catalogue cannot price get none, matching the
checkout path.
price_cents stays the catalogue's NET price, which is what PlanChange
prorates against — not Order::amount_cents, which holds Stripe's GROSS total.
Reconciling the two belongs to the proof register and Stripe (phases 4/5).
Also pins STRIPE_WEBHOOK_SECRET blank in phpunit.xml: the operator's real
secret was reaching the suite from .env and rejecting every unsigned test
payload, which is why 7 webhook tests failed before any of this.
Verified in the browser: with team traffic cut from 3000 to 500 GB in the
catalogue, the customer's portal still shows 3 TB.
373 tests green. Codex review clean after three rounds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- MONITORING_ATTEMPTS is capped by the run's max_attempts, so a large value can
no longer burn the retry budget and fail the very provisioning the degradation
exists to protect (covered by a new test at the budget boundary)
- /health is now dependency-free (200 in ~2ms with Kuma absent, verified: the
container healthcheck stays healthy); Kuma state is exclusively in /ready
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- attempt is 0-based, so compare one-based: ATTEMPTS=2 really means two tries
- bridge accepts add_monitor returning {monitorID}|{id}|int (v1.2.1 returns a
dict — re-verified: first create on a fresh Kuma is HTTP 200 with the id)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kuma has no write REST API (monitor CRUD is Socket.IO only), so docker/kuma-bridge
translates the exact generic REST contract HttpMonitoringClient already speaks —
no PHP client change needed. Opt-in compose profile 'monitoring'.
Verified end-to-end against a real Uptime Kuma 1.x:
create -> id, create again -> same id (idempotent), list, status, delete
and PHP MonitoringClient -> bridge -> Kuma round-trip.
The e2e run caught a real bug: add_monitor takes 'maxretries', not 'retries'.
Monitoring is now observability, not a delivery gate:
- RegisterMonitoring retries MONITORING_ATTEMPTS times on an outage, then
continues degraded with a visible 'info' event
- RunAcceptanceChecks no longer fails when monitoring isn't green (a fresh Kuma
monitor legitimately reports 'pending' — confirmed in the e2e run); it records
the gap instead. MONITORING_REQUIRED=true restores strict gating.
- docs/monitoring-uptime-kuma.md documents setup, failure behaviour, alternatives
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Clone recovery: once the lock clears but the task ref is lost, destroy the
possibly-incomplete VM and re-clone instead of advancing onto it.
- applyFirewall clears existing rules before adding, so retries don't accumulate
duplicate Proxmox firewall entries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Nextcloud: parse occ status JSON (installed + not in maintenance).
- Admin: query occ user:info for the account.
- Monitoring: MonitoringClient::isHealthy checks the provider, not the local row.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TraefikWriter now targets the host that serves the traffic (DNS points at it):
SshTraefikWriter SSHes in and writes the file-provider YAML there. Interface
takes the traffic host + guest backend.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CompleteProvisioning scrubs admin_password whenever credentials_sent exists,
so a crash between recording and scrubbing can't leave the password in context.
- HttpMonitoringClient looks up an existing monitor by URL before creating one,
so a retry after a crashed POST doesn't create a duplicate external monitor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Clone recovery polls until the VM lock clears instead of advancing onto a
possibly-incomplete clone.
- RegisterBackup/RegisterMonitoring create the local row BEFORE the run-resource
breadcrumb, so a crash between them can't leave the row permanently missing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- ConfigureCloudInit sets cores/memory from the plan (not the template default).
- RegisterBackup uses a deterministic Proxmox job id (ignore-exists) + firstOrCreate.
- RegisterMonitoring is idempotent by URL + firstOrCreate — no duplicate schedules
or monitors after a crash/retry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- applyFirewall sets policy_in=DROP so only 80/443 are exposed.
- Capacity/placement account for disk_gb (the real VM allocation), not the
smaller Nextcloud user quota — no systematic overcommit.
- CloudReady sent synchronously again; the step retries on mail failure with the
password preserved (no lost credential in a failed queue job).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CloudReady receives the ENCRYPTED password (decrypts only when rendering), so
the queued payload never holds plaintext.
- RegisterBackup uses a valid Proxmox calendar expression (02:00).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CreateCustomerAdmin puts OC_PASS on the docker invocation (survives the &&).
- RegisterBackup creates a real vzdump job via ProxmoxClient::createBackupJob.
- RegisterMonitoring registers via a new MonitoringClient service (interface +
fake + http). Both persist the external id [E] before the breadcrumb, so
acceptance reflects a real registration, not a fabricated row.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Clone: recover via vmExists() when the task ref was lost, instead of
re-cloning the reserved vmid (which Proxmox rejects).
- Deploy: persist the DB password ENCRYPTED before compose up and reuse it on
retry, so a crash can't regenerate a mismatching credential.
- Complete: gate credential delivery on a credentials_sent breadcrumb so a
retry after a crash doesn't re-send the email.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Failed run releases the instance (status=failed) so its quota stops counting
against host capacity.
- CompleteProvisioning activates instance+order only AFTER onboarding tasks +
credential delivery succeed.
- ConfigureNetwork polls until the guest has an address (no wrong Traefik route).
- customers.email unique + race-safe customer resolution in the intake action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RunRunner::onPoll no longer resets started_at, so a poll step's own deadline
(WaitForGuestAgent 270s, ConfigureDnsAndTls cert 840s) actually fires instead
of resetting every poll. maxDuration raised above each own-deadline so the
step's fail wins over the generic timeout. Shared core fix (A + B).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Require a real customer email (no unknown@ fallback merging customers).
- Stripe signature: enforce 5-min replay tolerance + accept any rotating v1.
- DeployApplicationStack no longer persists the DB password in the run context.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Stripe webhook fails closed when the signing secret is missing (outside local/testing).
- ReserveResources places + creates the instance under a per-datacenter lock so
concurrent orders can't overcommit a host.
- CloneVirtualMachine reserves the vmid + breadcrumb BEFORE cloning, so a crash
can't mint a new vmid and orphan the first clone.
- CreateCustomerAdmin checks user existence and resets the password instead of
re-running user:add on retry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Stripe: only checkout.session.completed with payment_status=paid (dedupes the
paired payment_intent event and blocks unpaid async sessions).
- Capture the guest IP (ConfigureNetwork) and point Traefik at the VM, not the
Proxmox host, so ACME/HTTP-01 can reach Nextcloud.
- Resize the disk to an absolute target (not '+…') so a retry can't double it.
(Codex #4 timeout was a false positive — started_at resets per step.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo
Claude