The hub lives in queue-provisioning (the only worker on the provisioning queue,
and LocalWireguardHub shells out to wg in-process). Generated the hub keypair,
wrote wg0.conf, and filled CLUPILOT_WG_ENDPOINT/HUB_PUBKEY.
Verified against the running stack rather than mocks: registered a stand-in
Proxmox peer through the app's own addPeer(), got a handshake, opened hub->peer
TCP over the tunnel (what the SSH steps actually need — a handshake alone does
not prove routed TCP), confirmed wg-quick save persisted the peer, then removed
it through removePeer(). wg0 returns after a container restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- MONITORING_ATTEMPTS is capped by the run's max_attempts, so a large value can
no longer burn the retry budget and fail the very provisioning the degradation
exists to protect (covered by a new test at the budget boundary)
- /health is now dependency-free (200 in ~2ms with Kuma absent, verified: the
container healthcheck stays healthy); Kuma state is exclusively in /ready
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified against a real Kuma by stopping it mid-test:
- get_monitors() reads the library's CACHED monitorList event, so it reported
'up' long after Kuma died. Reachability now does a real HTTP round-trip plus a
live socket check.
- the cached list could yield an id for a monitor that no longer exists; a match
is now confirmed with a live get_monitor before it is returned, so CluPilot can
never record a target that is never checked.
- liveness (/health, always 200 while serving) split from readiness (/ready, 503
when Kuma is unreachable) — the container healthcheck must not restart a
healthy bridge just because a dependency is down.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- attempt is 0-based, so compare one-based: ATTEMPTS=2 really means two tries
- bridge accepts add_monitor returning {monitorID}|{id}|int (v1.2.1 returns a
dict — re-verified: first create on a fresh Kuma is HTTP 200 with the id)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kuma has no write REST API (monitor CRUD is Socket.IO only), so docker/kuma-bridge
translates the exact generic REST contract HttpMonitoringClient already speaks —
no PHP client change needed. Opt-in compose profile 'monitoring'.
Verified end-to-end against a real Uptime Kuma 1.x:
create -> id, create again -> same id (idempotent), list, status, delete
and PHP MonitoringClient -> bridge -> Kuma round-trip.
The e2e run caught a real bug: add_monitor takes 'maxretries', not 'retries'.
Monitoring is now observability, not a delivery gate:
- RegisterMonitoring retries MONITORING_ATTEMPTS times on an outage, then
continues degraded with a visible 'info' event
- RunAcceptanceChecks no longer fails when monitoring isn't green (a fresh Kuma
monitor legitimately reports 'pending' — confirmed in the e2e run); it records
the gap instead. MONITORING_REQUIRED=true restores strict gating.
- docs/monitoring-uptime-kuma.md documents setup, failure behaviour, alternatives
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Detailed build contract for Subsystem B (15 customer-provisioning steps),
written against the real Subsystem-A contracts: ProvisioningStep/StepResult
(advance/retry/poll/fail), RunResource idempotency breadcrumbs, ProvisioningRun
context, the config/provisioning.php pipeline registry, and the read-only
ProxmoxClient it must extend. Covers new models (customers/orders/instances/…),
ProxmoxClient VM-lifecycle additions, Hetzner-DNS + Traefik services, Stripe
idempotent intake, live progress binding to the existing admin/customer views,
build order and DoD. Unblocks B.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- RunRunner catches step-resolution errors (removed/renamed pipeline) and
fails the run terminally instead of looping forever every tick.
- queue-provisioning worker gains NET_ADMIN + tun + persistent wireguard
volume + published UDP port so LocalWireguardHub can manage wg0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Dedicated 'provisioning' queue connection (retry_after 2400) + worker
(--timeout=2100); AdvanceRunJob tries=1, timeout=2100 on that queue.
- Run lock TTL raised to 2100s so a long step can't be run concurrently by a
duplicate tick-dispatched job.
- StartHostOnboarding creates host+run in a transaction, dispatches after commit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Focused TDD build plan for the engine: topology decision baked in (Option 1
standalone fleet as the software model, cluster-per-datacenter as the growth
target — hosts get datacenter + nullable cluster fields, placement filters by
datacenter). Covers data model, DB-state-machine orchestrator, 15 customer steps,
new ProxmoxClient, Stripe idempotent intake, and wiring the existing admin/customer
progress views to real data via Reverb. References the state handoff for workflow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full project state for a fresh session: env facts, stack, docker workflow, what's
built (landing + customer portal + admin console), verification workflow (pest /
R12 puppeteer / R15 codex), hard-won gotchas, open items, and the recommended next
block (provisioning engine v1.0).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo