The previous commit moved the nftables lockdown out of Task 5 and explained why
in its message. A commit message is not where the next person looks. Task 5 now
carries it: the bridge is normally already there because the ISO installer
writes it, the lockdown belongs in Task 9 because SecureHostFirewall ran last
for a reason, and "check from outside" cannot mean what it says before the
tunnel exists — so the script checks inside-out and says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PVE arrives with the image now, so this section accepts it rather than installs
it — but it needs every line of knowledge from InstallProxmoxVe.php to do that,
just pointed the other way.
The kernel is the claim, not the package. An installed proxmox-ve running on a
Debian kernel is exactly the half state RebootIntoPveKernel exists because of,
so uname is what gets checked.
The codename table now verifies a pairing instead of creating one. An image
whose PVE major does not match its Debian base was built wrong, and no amount
of fixing package sources heals a PVE compiled against a different libc — that
machine gets reinstalled with the right image. The other half of the table,
codename to PVE major, is written down here because the ISO route needs it and
the original file only had the suite side.
Removing the subscription repositories is the part that everything after this
depends on. A Proxmox image ships pve-enterprise, and without a subscription it
fails every apt-get update — after which no package install in any later
section succeeds. Both spellings are handled, because PVE 8 used one-line .list
files and PVE 9 moved to deb822, and ceph is in the list because the same trap
is set there a second time. The proof is a clean apt-get update at the end,
which is the only thing that tells removal apart from overwriting.
One thing changed after the first draft: when pveversion cannot be parsed, the
pairing check used to be skipped in silence. That is the fake that R19 records
as worse than no check at all, because it stops the next person from looking.
It now says "Paarung UNGEPRÜFT" in the report and in the log.
Verified without hardware: dash-clean; bookworm maps to PVE 8, trixie to PVE 9,
forky is refused; the pveversion parser reads 9 and 8 out of both real formats
and yields nothing for garbage; and against a copy of /etc/apt/sources.list.d
holding all five subscription files plus an unrelated one, exactly the five go
and the unrelated one stays. Step 2 unticked — there is no Proxmox here to
accept.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The official Proxmox ISO, an answer file baked into it, and QEMU running it
against the physical disks from the rescue system. Options were looked up
rather than recalled: prepare-iso takes --fetch-from iso, --answer-file and
--on-first-boot, and it names its own output, so the finished ISO is found by
glob afterwards instead of being handed an --output flag I would have invented.
validate-answer runs BEFORE the ISO is baked. An answer file with a typo drops
the installer into interactive mode, where it then hangs invisibly inside QEMU
until the hour runs out — the failure would arrive as a timeout and say nothing
about the cause.
The codename table comes over verbatim from InstallProxmoxVe.php, which the
platform plan deletes. It is used here to install the assistant into the RESCUE
system, so it keys off the rescue system's own codename. An unknown one aborts;
not "trixie is newest, so trixie", because the next Debian will be unknown too
and by then nobody is watching.
The ISO is checksummed against the mirror's SHA256SUMS. Booting an operating
system that could have become anything in transit is not a risk worth taking,
and a truncated download is enough to cause it — malice is not required.
The reboot carry-across is the gap this plan was amended for, and [first-boot]
takes exactly one executable. So that one file carries everything: the script,
its lib/, the progress file and the arguments, as a base64 archive in its own
body. That also avoids importing the fresh ZFS pool from the rescue system to
copy files into it.
Two bugs found by running it rather than reading it. The staging copy put the
work directory inside the state directory, so cp refused to copy a directory
into itself — and had it not refused, the gigabyte ISO would have been base64'd
into the hook that gets baked into that same ISO. The fix is not a better
exclude: the two directories have opposite lifetimes, so they now live in
different places, and a guard refuses loudly if anyone points them at each other
again. Structure beats a rule someone has to remember.
Verified without hardware: both files pass sh and dash. The generated answer
file parses as TOML with the expected disk-list, zfs.raid, zfs.arc-max,
first-boot and network.filter, and a 64-character root password. The codename
table accepts bookworm and trixie and refuses bullseye and forky. The shim comes
out at 26 KB with a 3 MB dummy ISO sitting in the work directory, proving it
stayed out. Run in a fresh debian:13-slim container it unpacks to
/opt/clupilot/bootstrap and /var/lib/clupilot, restores the progress file with
its original timestamp, has 700 on the directory and 600 on the arguments that
hold the WireGuard key, skips rescue_checked as already done, and fails cleanly
on the missing qemu. Step 2 stays unticked: a container is not a rescue system,
and nothing here has yet written to a disk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit put installimage with a Proxmox image into Task 3. That
image does not exist. Hetzner's installimage carries no Proxmox and lists it as
unsupported, so anyone following that instruction finds only Debian and ends up
back at the intermediate base system the same commit had just removed. Looked
up rather than remembered, which is what should have happened before writing it
down.
The mechanism the owner described does exist and is the official one since PVE
8.2: build the real Proxmox ISO with an embedded answer file via
proxmox-auto-install-assistant, run it under QEMU against the physical disks
from the rescue system, reboot into a finished node. Hetzner publishes a
tutorial for exactly this. No Debian, no VNC, no click — which is what the
owner said in the first place.
Two things fall out of the answer file that improve the plan.
The hand-written systemd resume service is gone. [first-boot] with
source = "from-iso" is the supported way to have the script pick itself up
after the reboot, and it is the only variant that can work before the tunnel
exists — a from-url hook would be pointing at a CluPilot it cannot reach yet.
The carry-across of the progress file stays necessary regardless: the rescue
system's copy still lives in RAM.
Trap 2 was being applied to the wrong machine. "Root partition last, not on
LVM" comes from the template traps, and the reason it exists is
GrowGuestFilesystem, which grows a GUEST filesystem. The template is built in
Task 8 from a Debian cloud image, not from the host's own layout, so the
justification written into Task 3 ("it applies to the host too, because the
template is built from here") does not hold. It remains binding for the
template, where it belongs. The host gets ZFS, and that it happens to involve
no LVM is a side effect rather than the reason — with arc-max pinned, because
the default takes half the RAM a host needs for its guests.
Also recorded: the answer file needs an fqdn and a root password that the
command line does not carry. The password is generated and thrown away, because
the platform no longer logs in over SSH and the plan already says a half
installed machine is reinstalled rather than repaired — keeping a secret for an
emergency the plan does not have only creates another place a secret lives.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The owner contradicted the plan, and the plan was wrong. It had the script
write Debian 13 and then had Task 4 turn that into a Proxmox host. In reality
you boot the provider's rescue system and install Proxmox directly — there is
no intermediate Debian for anyone to convert.
The section keys do NOT change. They are the only agreement with the platform
plan, and they are not worth touching for a rename — especially since they
still fit: Proxmox VE is Debian with the PVE kernel and packages on top. What
changes is what they mean, written down so the console labels them honestly:
debian_installed is "base system written", proxmox_installed is "PVE proven to
run".
Task 3 now uses installimage with a Proxmox image, and says the image name gets
pinned during acceptance instead of guessed here — the list belongs to the
provider and moves. No debootstrap and no hand-rolled partitioner: the provider
knows its own hardware, and a self-built boot path on someone else's metal only
comes back through their console. Hetzner dedicated only; a netcup path gets
written when there is a netcup machine to accept it on, because an unproven
fallback is exactly the assertion this plan avoids everywhere else (R22).
Task 4 keeps every line of knowledge from InstallProxmoxVe.php and needs all of
it — it just applies it to an image instead of a bare base. The codename table
now VERIFIES the pairing rather than creating it, and the enterprise repo that
ships with a Proxmox image has to go, because without a subscription it fails
every apt-get update and therefore every package install in the sections that
follow. Its acceptance now includes a clean apt-get update, which is the only
way to tell removal from overwriting.
Delivery is settled too, and was written down in neither document: a tar.gz
unpacked to /opt/clupilot/bootstrap. A curl | sh cannot carry a lib/, and the
separate files are load-bearing. A static archive is not an endpoint that
answers questions, so spec §5 stands.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four checks, all of them before a single byte is written, because this is the
only moment the script still has the option of not overwriting a disk.
The expensive one is "is this actually a rescue system". It demands POSITIVE
evidence rather than the absence of a counter-argument: a rescue system runs
from RAM, so its root is tmpfs, an overlay or a ramdisk, and an installed
system has a real partition there. A running pveversion is refused outright —
that is a hypervisor with customers on it. Mounted partitions of real disks are
refused too, because at that point the script cannot tell an empty machine from
someone's data.
There is deliberately no flag to skip the check. Such a flag gets used exactly
once, on the evening it should not have been. A rescue system this does not
recognise belongs in the runbook, not in a bypass.
The disk floor is written as arithmetic rather than a number, because the repo
has no threshold to borrow — plans live in the database, not in
config/provisioning.php. Template ~20 GB, Proxmox and its swap and backups
~20 GB, one smallest customer ~50 GB, rounded up to 100. It is explicitly not
capacity planning; HostCapacity and reserve_pct do that after registration.
This only turns away the machine the whole thing cannot fit on.
Network and clock come from one plain-HTTP HEAD, deliberately without TLS —
otherwise a wrong clock would be checking itself and would report a certificate
error instead of the time. Where `date -d` is missing (busybox), the clock is
NOT checked and the report says so: a check that silently waves things through
is worse than none, because it stops the next person from looking. That is the
same lesson R19 records about ->timezone(config('app.timezone')).
Verified here, in both directions where a direction existed. On this VM (root
on /dev/sda1, 80 GB, /dev/kvm present): refuses with three findings at once,
exits 1, writes nothing. In a debian:13-slim container (root on overlay, no
/dev/kvm): the overlay root is accepted as rescue-like and the missing
/dev/kvm is caught, which is the abort case the plan names. Against a receiver
serving a Date header two years off: 63074141 seconds of drift, refused, with
the exact `date -u -s` line to fix it. Step 2 stays unticked — none of these is
a rescue system on a dedicated server.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Argument parsing, the state directory, and the half of the script that knows
the way back to CluPilot. No sections are hooked up yet; main lists the nine it
will get, in the order from spec §7, so what is missing stays visible.
report() writes locally first and only then tries to send, because before
wireguard_joined there is no path at all — the lines queue up and flush_reports
delivers them later WITH THEIR OWN TIMESTAMPS. That last part is the whole
point: a twenty-minute install that arrives in one batch looks like one second
in the console, and nobody can see which section was slow.
Every network operation is explicitly caught. The script runs under `set -e`,
and a failing send is the normal case for the first five sections, not an
error — a report that aborts the run would be the diagnosis that kills the
patient.
What is sent had no agreed shape. The section keys were the only agreement
between the two plans; the envelope around them was not written down anywhere,
so it is written down here, at the top of report.sh, for the platform side to
read once. There is deliberately no "running" state: the console derives "open"
from the absence of a report (spec §7), and a third state would be a second
truth about the same thing.
The sent-marker is a line count in its own file rather than a flag rewritten
into progress.jsonl. That keeps the log append-only, so a power cut mid-write
costs at most a partial last line instead of a rewritten file — and after the
reboot in Task 3 that file is the only thing that remembers anything.
Verified here, without hardware: both syntax checks pass; two reports against
an unreachable CluPilot queue up without aborting and leave progress.sent at 0;
a receiver brought up afterwards gets both in one batch, carrying 17:28:55 and
17:28:57 rather than the 17:29:22 they arrived at; a message containing quotes,
a backslash, a tab and a newline survives as valid JSON; and section_done
counts a `done` as done and a `failed` as not. Step 2 of this task is NOT
ticked — none of that is a rescue system, and the plan is right to insist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Read on review, before a line of shell was written. All three would have been
found on hardware instead — two of them late, and one of them only by noticing
something that was silently absent.
The progress file did not survive the reboot. Task 1 writes it in the rescue
system, whose root lives in RAM; Task 3 installs Debian and reboots. The plan
said the resuming service uses "the same progress file" without saying how it
gets there. Because flush_reports first runs in Task 6, AFTER the reboot,
rescue_checked and debian_installed would simply never have reached the console
— and nothing would have pointed at the reboot as the reason. Task 3 now carries
the file, the script, and the arguments across, the last one root-readable only
because the WireGuard key is in it.
RebootIntoPveKernel.php was on the platform plan's delete list but not on this
plan's read list. Six files were named; seven are deleted. This is the one that
proves the boot path BEFORE issuing the one irreversible command in the whole
run: a -pve kernel image present, its initramfs present (a full /boot leaves
exactly the half-written one), and update-grub checked on its exit status, which
its predecessor discarded. Losing that would have cost a machine that does not
come back, and on a dedicated server that means the provider's console.
Reading it also corrected a rule this plan states. Task 9 rightly refuses
`pveum ... || true`, but that is not a rule against `|| true` — RebootIntoPveKernel
tolerates one deliberately, with the reason written next to it, and a script
that applies the ban literally breaks the kernel removal in Task 3. The rule is
now stated as what it is: no `|| true` over a command whose failure means
something.
The key swap in Task 9 had no proven handshake. Task 6 insists a tunnel counts
only with one, then Task 9 re-keys that same tunnel and discards the old key on
ordering alone. Ordering says when to discard, not whether the new key carries.
Four numbered steps now, with the proof third and the discard fourth, and a
fallback to the old key if the handshake does not come — the one failure nobody
fixes remotely, at the end of a run that did everything else right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Platform: ten tasks, all provable by the suite. Enrolment code and key pair,
the tunnel-only restriction (its test is the reason the design looks the way it
does), register, progress, routes, retiring SshTraefikWriter, the console, the
customer's own subdomain, shrinking the pipeline to six steps that only look,
and pulling the readiness checks along.
Script: ten tasks, none provable by the suite — a script that installs an
operating system cannot be run in SQLite. Each one ends on real hardware
instead, and two of them insist on it: the bridge section must have its
self-revert triggered on purpose, and the template must actually be cloned and
booted before it counts.
The one real coupling is written into both: platform task 9 DELETES the seven
step files the script plan reads its knowledge from. Do not run it before that
knowledge has been lifted — git log keeps them, but nobody reads history they
do not know to look for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The route pull stays inside the tunnel. The operator's criterion was not
availability but "publicly reachable, yes or no" — and an endpoint that survives
a tunnel outage must by definition be reachable outside it. So nothing new opens
from outside, on the host or here.
The subdomain becomes the customer's choice at checkout, with suggestions from
their own name and a sentence saying the address is permanently public: every
Let's Encrypt certificate lands in the Certificate Transparency logs, which are
public, searchable and never deleted.
Also recorded, both decided and both getting their own spec: resetting a
Nextcloud to factory state with a 14-day safety backup, and an outage mail to
the operator after two consecutive failed checks — measured today that neither
an incident nor a mail exists, so a customer VM can fail unnoticed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator was right to challenge POST /enrol. The three values a script needs
before the tunnel exists — hub key, hub address, assigned tunnel IP — are all
known to the console when the host is created, so they travel in the copied
command instead of being fetched.
The one thing that forced an endpoint was the hub needing the host's public key
before the tunnel is up. So the console generates the pair and hands it over,
and the host swaps in a freshly generated key through the tunnel afterwards.
The key that passed through a clipboard lives for minutes.
Net effect: the network boundary is unchanged. No new way in from outside.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2321854 removed it. That commit's own work — discarding an unconfirmed
registration and sweeping abandoned ones — is untouched and correct; what went
with it was every file of the Stripe price adoption merged an hour earlier:
AdoptStripePrice, IdempotencyKey, the unique-index migration, both test files,
the spec, the plan, and the edits in six more.
Nothing was lost. The commits stayed in history and the files stayed on disk,
untracked, which is what a staged deletion from a stale tree leaves behind.
Restored from 4eb90c8, the reviewed head, by explicit path — the fourteen paths
2321854 damaged and not one more, so the ten files that commit legitimately
added or changed keep exactly what it gave them.
This is the failure the repo's own rule exists to prevent: stage by path, never
`git add -A`, because a second session's index does not know what a first one
merged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Abmelden und neu registrieren" was advice that could not work. Signing out
frees nothing: the address is still held by the unique index, so registering
again with the address somebody meant fails — and every abandoned attempt
stayed in `users` for ever. Reported exactly that way, and it was right.
The button discards the account instead. Only an UNCONFIRMED one, and only
one with nothing behind it: the webhook creates a customer's login from a
paid checkout, so a user can sit on this page unconfirmed while already
having a contract, and deleting that is data loss dressed up as a
convenience. It refuses and says where to go. A confirmed account is closed,
not discarded — that is ConfirmCloseAccount, with an invoice history behind
it.
R23: confirmed in a modal. The modal mutates nothing; it dispatches the event
the page listens for, so the checks stay in the one place they already are
rather than being duplicated where they could drift.
And nothing has to be discarded by hand. clupilot:prune-unverified removes
registrations nobody confirmed after five days — long enough for somebody who
signed up on a Friday and found the mail in a spam folder on Monday, short
enough that a typo does not hold the correct address for a month. It skips a
confirmed account whatever its age, and any account with a customer record
behind it, matched on user_id AND on the address, because either link means
somebody is a customer. Every removal is logged with the address: a line that
disappears is what somebody asks about later, and a count answers nothing.
The deadline is said on the page as well as in the mail. Somebody who never
received the mail is looking at the page, and the deadline is the reason they
do not have to do anything about an attempt they abandon.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ten follow-ups, none of them blocking and none of them done. The one that
matters: activePricesFor() recognises an orphaned Price and nothing recognises an
orphaned PRODUCT — and a second Product is worse than a duplicate Price, because
every activePricesFor() then asks about the wrong Product and recognition goes
blind for that whole family. The branch disables its own guard through the gap it
did not close.
Second: AdoptStripePrice promises in its own docblock that no adoption can move
money, but can only compare amount, currency and interval. The four properties
that also decide what a Price charges are filtered in HttpStripeClient, which the
fake cannot express — so no test of the class can reach the promise it makes.
Also recorded: two pre-existing test failures this branch surfaced without
causing, with the diagnosis, so the next person does not derive them again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The money gate compared amount, currency and interval and left the rest to a
filter that excluded two fields. A recurring, licensed, monthly price with
transform_quantity divide_by 10, at our exact figure, on our product, carrying
our addon key passed all five conditions. Modules bill BY quantity — a pack is
one item at quantity n — so a customer holding three would have been charged
one. A tiered price was excluded only by accident: Stripe reports unit_amount
null for it and we read that as 0, which fails the amount match unless the
caller's own figure is 0, and PlanPrices::ensure() has no zero guard.
activePricesFor() now also skips a non-empty transform_quantity and any
billing_scheme other than per_unit. Absent keys stay Stripe's defaults, which
is the direction that matters: read the other way round the filter would refuse
every legitimate price and turn recognition into a permanent no-op. The
contract docblock names the four properties instead of claiming to cover all of
them, and says the list is not everything Stripe can put on a price — only
what is known to change what one charges.
And a comment that says a gap is covered is worse than the gap. The sync said a
crash between Stripe creating a product and us storing its id gives back the
same product next run: true for twenty-four hours, false afterwards, and the
exact claim this branch was written to refute. There is no recognition step for
products at all, so an interrupted run leaves an orphan and the next run past
the expiry mints a second one — worse than a duplicate price, because
activePricesFor() then looks at the wrong product and recognition goes blind
for the whole family. Written down as a known gap at both createProduct call
sites, with the trade this branch chose: folding name and metadata into the key
means a renamed family now mints a second product where it used to answer 400,
and a blockade reaches a paying customer while a duplicate product does not.
The sweep no longer reports "created" for what it adopted — the count is taken
before ensure() runs, and telling those two apart is the whole point.
The test named after 2026-07-29 could not reproduce it: the fake's key ledger
was empty, so createPrice() could not have thrown and the test could not tell
"adoption prevented the 400" from "no 400 was possible". Seeded now with the
key today's call sends and a fingerprint over the metadata the pre-9da1358 call
sent. Remove the adopt() call from AddonPrices::ensure() and the test dies with
Stripe's own sentence instead of an assertion.
Also: the plan document still instructed migrate:fresh --env=testing. There is
no .env.testing here, so it targets the live development database; struck
through with the reason, not deleted, and a Korrekturen section records the
five defects the plan's own reviews found in it. Four comments corrected —
"below" that meant above, a counter claimed unable to tie that ties with a
planted price, the unique index's second meaning for remember()'s catch, and
that MariaDB DDL is not transactional so the dedupe commits before the index.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plan register has had this since 2026_07_30_110000. The module register
never did, so two rows could claim one Price and archiving the one would have
withdrawn the Price the other was still selling — Block D of the real-run
handoff.
It is also the net under the adoption step: at a VAT rate of nought both
treatments charge the same amount, and the check that refuses an already-claimed
Price is then the only thing keeping the two apart.
Duplicates are deleted, not archived. An archived row goes on claiming the id
and a unique index knows nothing about archived_at; the row rebuilds itself on
the next ensure(), and subscription_addons holds the Stripe id as text rather
than a foreign key, so no booking loses its price.
AddonPrices::remember()'s catch comment described the guard as the only thing
standing between two rows and one Price; now that the index exists, rewritten
to describe it as the net under that guard instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six tasks, each ending on a green billing folder and its own commit: the key
fingerprint (and a fake that finally reproduces Stripe's refusal), the two
listing calls, the adoption step itself, then each price service in turn, then
one row per Stripe price.
The spec now names both test files. What goes on the wire is checked through
Http::fake against the real client — asserting against the fake would only prove
the fake agrees with itself, which is the mistake that let the incident through.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A run on 2026-07-29 23:11 created price_1TygdEC7u8NpJ8pOt3nsoyYw at Stripe and
died before the row was written. 9da1358 then added the tax_treatment metadata
field, so the next sweep sent the same idempotency key with different parameters
and Stripe refused it — for twenty-four hours, and not only for the sweep:
AddonPrices::ensure() also runs inside a customer's module booking.
Two defects, one spec: nothing asks Stripe what it already has, and the key
covers less than the call sends. The spec settles what identity means here
(product, amount, currency, interval — never metadata, which Stripe lets you
edit) and where the key's fingerprint may go (the catalogue calls, never a
refund, where a duplicate costs the customer their money twice).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-measuring before writing found the spec's numbers had moved: seventeen
permissions now, not sixteen, because the mailbox work added mail.manage.
It also turned up something the spec did not anticipate — ConfirmsPassword
resolves the password through auth()->user(), the default guard, so all
five components using it break the moment a second guard exists. That is
now its own task.
Self-review caught two references to things that do not exist: users has
no uuid column, so the impersonation link addresses the customer instead,
and there is no layouts.guest — the auth pages use layouts.portal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Task 2 reviewer rewired the model to Laravel's APP_KEY-based Crypt
facade and every test stayed green. Laravel's encrypter is a container
singleton resolved once from app.key, so config()->set('app.key') never
rebuilds it and the rotation the test performed was invisible.
Replaced with a positive proof — rotating SECRETS_KEY must break
decryption — plus the APP_KEY case with forgetInstance(), which is what
makes that direction mean anything.
Seventeen models use App\Models\Concerns\HasUuid, and R11 is the reason:
URLs address records by UUID, not by integer primary key. The plan told
the implementer to hand-roll booted() instead, losing getRouteKeyName()
with it. Authoring error, corrected before the remaining tasks copy it.
The Task 1 implementer copied '// path/to/file.php' into the files
verbatim, which breaks Pint's blank_line_after_opening_tag and matches
no file in the repo. The annotation was mine; the constraint now says so
before the next seven tasks repeat it.
Self-review against the spec turned up four gaps and one latent bug in
what already ships.
The bug: Laravel 13's MailManager reads only 'scheme' when building an
SMTP transport (MailManager.php:196) — the old 'encryption' fallback is
gone — and .env carries MAIL_SCHEME=tls, which Symfony does not accept;
it knows smtp and smtps. Nothing has failed because MAIL_MAILER=log
means no connection has ever been opened. The plan stores a human-facing
tls/ssl choice and translates it where it is used, and never copies
MAIL_SCHEME through.
The gaps: the cancellation mail and the provisioning notification were
edited but not tested, secrets.manage was never proven insufficient for
the new page, and a missing SECRETS_KEY had no task making the page say
so rather than throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console offers a single mail.password pointing at one SMTP mailbox,
which is nowhere near enough for five sending addresses. A mailbox is
address, display name, username and password; five of them as registry
entries would be twenty secrets named mail.support.username, and the
question of which mail sends from which address would still have no home.
Send-only stays usable because every message carries Reply-To on its own
mailbox, so a customer's reply lands in a mailbox someone reads. That is
what makes IMAP unnecessary rather than merely deferred.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decided: whoever is in operators has no business in users. A row left
behind is the same mixing the separation exists to end, only smaller.
The one case that deletes nothing is a row with a customer, a seat or an
order on it. That would mean the same person is operator and paying
customer, and a silent delete would take billing data with it — so the
migration stops and names the address instead. Neither existing account
is in that state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator console and the customer portal share one users table, one
login page and one guard. Three reported faults follow from that single
fact: the console serves the portal's sign-in page, that page offers
"Registrieren", and the link 404s because RestrictAdminHost::SHARED does
not list register.
Measured rather than assumed: driving the host guard directly gives
/login through and /register 404 on the console host, and all sixteen
Spatie permissions turn out to be console permissions — so RBAC moves to
the new guard rather than being duplicated across two.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three templates the design was signed off against lived in a
session-scoped scratchpad and would have disappeared with the session — the one
artefact the whole conversion is measured against. They are in docs/design/ now,
with the measurements that were got wrong at least once written down beside
them, and a note that the way to check is to render and measure rather than to
read the source. That distinction already settled one disagreement: the source
said the metric grid used a 20px gap, rendered it is 14px.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console has been unreachable over the VPN, and the cause was the readiness
probe rather than the tunnel. The gateway binds to the WireGuard hub address
alone; the probe asked 127.0.0.1, where nothing has ever listened. It could only
fail, so VPN_READY stayed false, and on that basis the application withheld the
`DNS = 10.66.0.1` line from every client config it issued. A phone then built
the tunnel, asked its normal resolver for the console hostname, got the public
address and had its connection closed — indistinguishable from "this site does
not exist", which is exactly what it looked like.
Probing over TLS on the bare address would not have worked either: the site
matches on the console's hostname, so a request without SNI is offered no
certificate. The gateway now answers a plain-HTTP health port on the hub
address, which removes TLS, SNI and name resolution from the question and
answers only what is being asked — is this gateway listening, in this network
namespace, right now. Caddy refuses to start when the certificate is unreadable,
so a health port that answers still proves the whole file loaded.
Also here, all found while looking:
- Icons pushed their label onto a second line and rendered a size larger than
asked for. Tailwind's preflight makes an svg display:block, and `.size-4` and
`.size-5` have equal specificity, so stylesheet order decided — and it emits
size-4 first. Every icon written as 16px was silently 20px. Recorded as R18.
- Four error pages printed `errors.404.hint` in place of a sentence: the lang
files give those codes a null hint and Laravel returns the key for a null line.
- The Developer role had no label in either language, so the dropdown showed
`admin_settings.role_developer`.
- The secrets area held one key, which is not worth a password gate. It now
carries the credentials that actually stop the business when they expire —
DNS, monitoring, SMTP — and the test button appears only where a checker
exists, instead of reporting on Stripe whatever was being looked at.
- The update button said nothing about when a queued run would start or where a
running one had got to; both are now shown, and a failure names its step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
VERSION at the repository root is the release number and the only place it is
written down — a file rather than `git describe`, because a source archive, a
shallow CI checkout and a container built without .git have no tags to
describe. Cutting a release is editing that file and merging it; CI creates the
annotated v<VERSION> tag once green, only on the commit that raised the number,
and never moves it. Servers are pinned to these.
Two modes, and the difference is the point. RELEASE=v1.0.0 pins a server to an
immutable tag, checked out detached: it moves when someone decides it moves.
BRANCH=main follows the edge, where CI tags a commit only after it is pushed.
The mode is remembered, so re-running the updater on a pinned box does not
quietly walk it back onto main.
Going backwards is refused, by commit ancestry rather than by comparing version
strings — a tag can be cut from anywhere, and only ancestry says whether history
is going back. The schema has already moved forward by then, and the migrations
needed to reverse it are not in the older checkout at all.
What the console reports comes from a manifest written atomically after every
step succeeded, never from live git: git says what the files are, not whether
the deployment came up. So a failed update keeps reporting the version that is
actually serving — including its version number, not the newer one the checkout
has already moved to. A deployment pinned to the tag reads "1.0.0 (abc1234)";
anything else reads "1.0.0-dev (abc1234) · main", because every commit after the
tag still carries VERSION=1.0.0 and is not that release. Reporting it as one is
how a bug gets filed against the wrong code.
Codex reviewed the design before it was written and the result six times after.
Its findings, all real: the version had to come from the manifest too, not just
the commit; branch names need JSON escaping or the manifest silently vanishes; a
`case` glob does not anchor and accepted 1x.2y.3garbage; pinning to the commit a
server already sits on skipped recording the mode, and then skipped detaching;
a manifest that failed to write would never be repaired; an unparseable
timestamp would 500 every admin page; and an empty tag list read as a
connectivity failure.
466 tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It still named two webhook events; the application handles six, and the four
new ones are the billing cycle itself — renewals, failed payments, status
changes and endings. It also said nothing about stripe:sync-catalogue, without
which Stripe has no prices and nothing can be sold at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Design is decided and Codex-reviewed; none of it is built. Records the live
split-brain bug to fix first, the three-table catalogue, computed availability,
the proof register with frozen add-ons, the Stripe split, what already exists,
and the working rules that cost time this session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
--env-file makes the whole run non-interactive, which is what a bare server
needs: install git, clone, run, done. The file also carries the operational
secrets (Hetzner DNS, Stripe, SMTP, the Proxmox key path); those are optional,
so a first install can happen before the Stripe account exists and the features
stay dark until the values are filled in.
The installer warns when the file is readable by more than its owner, and
clupilot.env is gitignored — it holds every secret an installation will need.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The installer still needs root for apt and the firewall, but it now creates a
clupilot service account that owns the checkout and runs Docker, maps the
container user to it (HOST_UID/GID), and update.sh refuses to run as root —
root-owned files in the checkout are files the application cannot write.
The account has no password login: docker group membership is root-equivalent
on the host, so it is reachable only via sudo -u or an SSH key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
install.sh sets up a bare Debian/Ubuntu server end to end: Docker, git,
WireGuard, clone, generated secrets, stack, migrations, hub keypair, the Owner
account and a closed firewall. Re-runnable: it keeps an existing .env and never
regenerates the hub key, which would disconnect every onboarded host.
update.sh pulls and applies. Migrations run before the new containers take
traffic, the image is rebuilt only when its definition changed, and the queue
workers are restarted — they are long-running processes that otherwise keep the
old classes in memory, which cost us an hour during development.
clupilot:create-admin creates or promotes an Owner, so re-running the installer
fixes a lost role instead of failing on a taken address.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hub lives in queue-provisioning (the only worker on the provisioning queue,
and LocalWireguardHub shells out to wg in-process). Generated the hub keypair,
wrote wg0.conf, and filled CLUPILOT_WG_ENDPOINT/HUB_PUBKEY.
Verified against the running stack rather than mocks: registered a stand-in
Proxmox peer through the app's own addPeer(), got a handshake, opened hub->peer
TCP over the tunnel (what the SSH steps actually need — a handshake alone does
not prove routed TCP), confirmed wg-quick save persisted the peer, then removed
it through removePeer(). wg0 returns after a container restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- MONITORING_ATTEMPTS is capped by the run's max_attempts, so a large value can
no longer burn the retry budget and fail the very provisioning the degradation
exists to protect (covered by a new test at the budget boundary)
- /health is now dependency-free (200 in ~2ms with Kuma absent, verified: the
container healthcheck stays healthy); Kuma state is exclusively in /ready
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified against a real Kuma by stopping it mid-test:
- get_monitors() reads the library's CACHED monitorList event, so it reported
'up' long after Kuma died. Reachability now does a real HTTP round-trip plus a
live socket check.
- the cached list could yield an id for a monitor that no longer exists; a match
is now confirmed with a live get_monitor before it is returned, so CluPilot can
never record a target that is never checked.
- liveness (/health, always 200 while serving) split from readiness (/ready, 503
when Kuma is unreachable) — the container healthcheck must not restart a
healthy bridge just because a dependency is down.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- attempt is 0-based, so compare one-based: ATTEMPTS=2 really means two tries
- bridge accepts add_monitor returning {monitorID}|{id}|int (v1.2.1 returns a
dict — re-verified: first create on a fresh Kuma is HTTP 200 with the id)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kuma has no write REST API (monitor CRUD is Socket.IO only), so docker/kuma-bridge
translates the exact generic REST contract HttpMonitoringClient already speaks —
no PHP client change needed. Opt-in compose profile 'monitoring'.
Verified end-to-end against a real Uptime Kuma 1.x:
create -> id, create again -> same id (idempotent), list, status, delete
and PHP MonitoringClient -> bridge -> Kuma round-trip.
The e2e run caught a real bug: add_monitor takes 'maxretries', not 'retries'.
Monitoring is now observability, not a delivery gate:
- RegisterMonitoring retries MONITORING_ATTEMPTS times on an outage, then
continues degraded with a visible 'info' event
- RunAcceptanceChecks no longer fails when monitoring isn't green (a fresh Kuma
monitor legitimately reports 'pending' — confirmed in the e2e run); it records
the gap instead. MONITORING_REQUIRED=true restores strict gating.
- docs/monitoring-uptime-kuma.md documents setup, failure behaviour, alternatives
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo