Re-measuring before writing found the spec's numbers had moved: seventeen
permissions now, not sixteen, because the mailbox work added mail.manage.
It also turned up something the spec did not anticipate — ConfirmsPassword
resolves the password through auth()->user(), the default guard, so all
five components using it break the moment a second guard exists. That is
now its own task.
Self-review caught two references to things that do not exist: users has
no uuid column, so the impersonation link addresses the customer instead,
and there is no layouts.guest — the auth pages use layouts.portal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Task 2 reviewer rewired the model to Laravel's APP_KEY-based Crypt
facade and every test stayed green. Laravel's encrypter is a container
singleton resolved once from app.key, so config()->set('app.key') never
rebuilds it and the rotation the test performed was invisible.
Replaced with a positive proof — rotating SECRETS_KEY must break
decryption — plus the APP_KEY case with forgetInstance(), which is what
makes that direction mean anything.
Seventeen models use App\Models\Concerns\HasUuid, and R11 is the reason:
URLs address records by UUID, not by integer primary key. The plan told
the implementer to hand-roll booted() instead, losing getRouteKeyName()
with it. Authoring error, corrected before the remaining tasks copy it.
The Task 1 implementer copied '// path/to/file.php' into the files
verbatim, which breaks Pint's blank_line_after_opening_tag and matches
no file in the repo. The annotation was mine; the constraint now says so
before the next seven tasks repeat it.
Self-review against the spec turned up four gaps and one latent bug in
what already ships.
The bug: Laravel 13's MailManager reads only 'scheme' when building an
SMTP transport (MailManager.php:196) — the old 'encryption' fallback is
gone — and .env carries MAIL_SCHEME=tls, which Symfony does not accept;
it knows smtp and smtps. Nothing has failed because MAIL_MAILER=log
means no connection has ever been opened. The plan stores a human-facing
tls/ssl choice and translates it where it is used, and never copies
MAIL_SCHEME through.
The gaps: the cancellation mail and the provisioning notification were
edited but not tested, secrets.manage was never proven insufficient for
the new page, and a missing SECRETS_KEY had no task making the page say
so rather than throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console offers a single mail.password pointing at one SMTP mailbox,
which is nowhere near enough for five sending addresses. A mailbox is
address, display name, username and password; five of them as registry
entries would be twenty secrets named mail.support.username, and the
question of which mail sends from which address would still have no home.
Send-only stays usable because every message carries Reply-To on its own
mailbox, so a customer's reply lands in a mailbox someone reads. That is
what makes IMAP unnecessary rather than merely deferred.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decided: whoever is in operators has no business in users. A row left
behind is the same mixing the separation exists to end, only smaller.
The one case that deletes nothing is a row with a customer, a seat or an
order on it. That would mean the same person is operator and paying
customer, and a silent delete would take billing data with it — so the
migration stops and names the address instead. Neither existing account
is in that state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator console and the customer portal share one users table, one
login page and one guard. Three reported faults follow from that single
fact: the console serves the portal's sign-in page, that page offers
"Registrieren", and the link 404s because RestrictAdminHost::SHARED does
not list register.
Measured rather than assumed: driving the host guard directly gives
/login through and /register 404 on the console host, and all sixteen
Spatie permissions turn out to be console permissions — so RBAC moves to
the new guard rather than being duplicated across two.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three templates the design was signed off against lived in a
session-scoped scratchpad and would have disappeared with the session — the one
artefact the whole conversion is measured against. They are in docs/design/ now,
with the measurements that were got wrong at least once written down beside
them, and a note that the way to check is to render and measure rather than to
read the source. That distinction already settled one disagreement: the source
said the metric grid used a 20px gap, rendered it is 14px.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console has been unreachable over the VPN, and the cause was the readiness
probe rather than the tunnel. The gateway binds to the WireGuard hub address
alone; the probe asked 127.0.0.1, where nothing has ever listened. It could only
fail, so VPN_READY stayed false, and on that basis the application withheld the
`DNS = 10.66.0.1` line from every client config it issued. A phone then built
the tunnel, asked its normal resolver for the console hostname, got the public
address and had its connection closed — indistinguishable from "this site does
not exist", which is exactly what it looked like.
Probing over TLS on the bare address would not have worked either: the site
matches on the console's hostname, so a request without SNI is offered no
certificate. The gateway now answers a plain-HTTP health port on the hub
address, which removes TLS, SNI and name resolution from the question and
answers only what is being asked — is this gateway listening, in this network
namespace, right now. Caddy refuses to start when the certificate is unreadable,
so a health port that answers still proves the whole file loaded.
Also here, all found while looking:
- Icons pushed their label onto a second line and rendered a size larger than
asked for. Tailwind's preflight makes an svg display:block, and `.size-4` and
`.size-5` have equal specificity, so stylesheet order decided — and it emits
size-4 first. Every icon written as 16px was silently 20px. Recorded as R18.
- Four error pages printed `errors.404.hint` in place of a sentence: the lang
files give those codes a null hint and Laravel returns the key for a null line.
- The Developer role had no label in either language, so the dropdown showed
`admin_settings.role_developer`.
- The secrets area held one key, which is not worth a password gate. It now
carries the credentials that actually stop the business when they expire —
DNS, monitoring, SMTP — and the test button appears only where a checker
exists, instead of reporting on Stripe whatever was being looked at.
- The update button said nothing about when a queued run would start or where a
running one had got to; both are now shown, and a failure names its step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
VERSION at the repository root is the release number and the only place it is
written down — a file rather than `git describe`, because a source archive, a
shallow CI checkout and a container built without .git have no tags to
describe. Cutting a release is editing that file and merging it; CI creates the
annotated v<VERSION> tag once green, only on the commit that raised the number,
and never moves it. Servers are pinned to these.
Two modes, and the difference is the point. RELEASE=v1.0.0 pins a server to an
immutable tag, checked out detached: it moves when someone decides it moves.
BRANCH=main follows the edge, where CI tags a commit only after it is pushed.
The mode is remembered, so re-running the updater on a pinned box does not
quietly walk it back onto main.
Going backwards is refused, by commit ancestry rather than by comparing version
strings — a tag can be cut from anywhere, and only ancestry says whether history
is going back. The schema has already moved forward by then, and the migrations
needed to reverse it are not in the older checkout at all.
What the console reports comes from a manifest written atomically after every
step succeeded, never from live git: git says what the files are, not whether
the deployment came up. So a failed update keeps reporting the version that is
actually serving — including its version number, not the newer one the checkout
has already moved to. A deployment pinned to the tag reads "1.0.0 (abc1234)";
anything else reads "1.0.0-dev (abc1234) · main", because every commit after the
tag still carries VERSION=1.0.0 and is not that release. Reporting it as one is
how a bug gets filed against the wrong code.
Codex reviewed the design before it was written and the result six times after.
Its findings, all real: the version had to come from the manifest too, not just
the commit; branch names need JSON escaping or the manifest silently vanishes; a
`case` glob does not anchor and accepted 1x.2y.3garbage; pinning to the commit a
server already sits on skipped recording the mode, and then skipped detaching;
a manifest that failed to write would never be repaired; an unparseable
timestamp would 500 every admin page; and an empty tag list read as a
connectivity failure.
466 tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It still named two webhook events; the application handles six, and the four
new ones are the billing cycle itself — renewals, failed payments, status
changes and endings. It also said nothing about stripe:sync-catalogue, without
which Stripe has no prices and nothing can be sold at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Design is decided and Codex-reviewed; none of it is built. Records the live
split-brain bug to fix first, the three-table catalogue, computed availability,
the proof register with frozen add-ons, the Stripe split, what already exists,
and the working rules that cost time this session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
--env-file makes the whole run non-interactive, which is what a bare server
needs: install git, clone, run, done. The file also carries the operational
secrets (Hetzner DNS, Stripe, SMTP, the Proxmox key path); those are optional,
so a first install can happen before the Stripe account exists and the features
stay dark until the values are filled in.
The installer warns when the file is readable by more than its owner, and
clupilot.env is gitignored — it holds every secret an installation will need.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The installer still needs root for apt and the firewall, but it now creates a
clupilot service account that owns the checkout and runs Docker, maps the
container user to it (HOST_UID/GID), and update.sh refuses to run as root —
root-owned files in the checkout are files the application cannot write.
The account has no password login: docker group membership is root-equivalent
on the host, so it is reachable only via sudo -u or an SSH key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
install.sh sets up a bare Debian/Ubuntu server end to end: Docker, git,
WireGuard, clone, generated secrets, stack, migrations, hub keypair, the Owner
account and a closed firewall. Re-runnable: it keeps an existing .env and never
regenerates the hub key, which would disconnect every onboarded host.
update.sh pulls and applies. Migrations run before the new containers take
traffic, the image is rebuilt only when its definition changed, and the queue
workers are restarted — they are long-running processes that otherwise keep the
old classes in memory, which cost us an hour during development.
clupilot:create-admin creates or promotes an Owner, so re-running the installer
fixes a lost role instead of failing on a taken address.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hub lives in queue-provisioning (the only worker on the provisioning queue,
and LocalWireguardHub shells out to wg in-process). Generated the hub keypair,
wrote wg0.conf, and filled CLUPILOT_WG_ENDPOINT/HUB_PUBKEY.
Verified against the running stack rather than mocks: registered a stand-in
Proxmox peer through the app's own addPeer(), got a handshake, opened hub->peer
TCP over the tunnel (what the SSH steps actually need — a handshake alone does
not prove routed TCP), confirmed wg-quick save persisted the peer, then removed
it through removePeer(). wg0 returns after a container restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- MONITORING_ATTEMPTS is capped by the run's max_attempts, so a large value can
no longer burn the retry budget and fail the very provisioning the degradation
exists to protect (covered by a new test at the budget boundary)
- /health is now dependency-free (200 in ~2ms with Kuma absent, verified: the
container healthcheck stays healthy); Kuma state is exclusively in /ready
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified against a real Kuma by stopping it mid-test:
- get_monitors() reads the library's CACHED monitorList event, so it reported
'up' long after Kuma died. Reachability now does a real HTTP round-trip plus a
live socket check.
- the cached list could yield an id for a monitor that no longer exists; a match
is now confirmed with a live get_monitor before it is returned, so CluPilot can
never record a target that is never checked.
- liveness (/health, always 200 while serving) split from readiness (/ready, 503
when Kuma is unreachable) — the container healthcheck must not restart a
healthy bridge just because a dependency is down.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- attempt is 0-based, so compare one-based: ATTEMPTS=2 really means two tries
- bridge accepts add_monitor returning {monitorID}|{id}|int (v1.2.1 returns a
dict — re-verified: first create on a fresh Kuma is HTTP 200 with the id)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kuma has no write REST API (monitor CRUD is Socket.IO only), so docker/kuma-bridge
translates the exact generic REST contract HttpMonitoringClient already speaks —
no PHP client change needed. Opt-in compose profile 'monitoring'.
Verified end-to-end against a real Uptime Kuma 1.x:
create -> id, create again -> same id (idempotent), list, status, delete
and PHP MonitoringClient -> bridge -> Kuma round-trip.
The e2e run caught a real bug: add_monitor takes 'maxretries', not 'retries'.
Monitoring is now observability, not a delivery gate:
- RegisterMonitoring retries MONITORING_ATTEMPTS times on an outage, then
continues degraded with a visible 'info' event
- RunAcceptanceChecks no longer fails when monitoring isn't green (a fresh Kuma
monitor legitimately reports 'pending' — confirmed in the e2e run); it records
the gap instead. MONITORING_REQUIRED=true restores strict gating.
- docs/monitoring-uptime-kuma.md documents setup, failure behaviour, alternatives
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Detailed build contract for Subsystem B (15 customer-provisioning steps),
written against the real Subsystem-A contracts: ProvisioningStep/StepResult
(advance/retry/poll/fail), RunResource idempotency breadcrumbs, ProvisioningRun
context, the config/provisioning.php pipeline registry, and the read-only
ProxmoxClient it must extend. Covers new models (customers/orders/instances/…),
ProxmoxClient VM-lifecycle additions, Hetzner-DNS + Traefik services, Stripe
idempotent intake, live progress binding to the existing admin/customer views,
build order and DoD. Unblocks B.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- RunRunner catches step-resolution errors (removed/renamed pipeline) and
fails the run terminally instead of looping forever every tick.
- queue-provisioning worker gains NET_ADMIN + tun + persistent wireguard
volume + published UDP port so LocalWireguardHub can manage wg0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Dedicated 'provisioning' queue connection (retry_after 2400) + worker
(--timeout=2100); AdvanceRunJob tries=1, timeout=2100 on that queue.
- Run lock TTL raised to 2100s so a long step can't be run concurrently by a
duplicate tick-dispatched job.
- StartHostOnboarding creates host+run in a transaction, dispatches after commit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Focused TDD build plan for the engine: topology decision baked in (Option 1
standalone fleet as the software model, cluster-per-datacenter as the growth
target — hosts get datacenter + nullable cluster fields, placement filters by
datacenter). Covers data model, DB-state-machine orchestrator, 15 customer steps,
new ProxmoxClient, Stripe idempotent intake, and wiring the existing admin/customer
progress views to real data via Reverb. References the state handoff for workflow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full project state for a fresh session: env facts, stack, docker workflow, what's
built (landing + customer portal + admin console), verification workflow (pest /
R12 puppeteer / R15 codex), hard-won gotchas, open items, and the recommended next
block (provisioning engine v1.0).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo