One real conflict, in SyncStripeCatalogue::handle(), and both sides were right.
Main's parallel work resolves the adoption singletons and takes "before" counts
so a run can tell created from adopted. The operating-mode work refuses a
catalogue whose stored objects belong to the other mode, and records which mode
the catalogue now belongs to.
Kept both, with the mode guard first. It REFUSES, so it must not run behind
anything that has already built state; the counters only need to be in place
before the create loop, and they still are. Reversing that order would have the
command resolve singletons and take counts on a catalogue it is about to reject.
2017 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The owner caught me putting the archive on clupilot.cloud. That is the customer
zone, and looking into it turned up the same confusion already sitting in the
code — in the one place that matters most.
RegisterHostDns says "fsn-01.node.clupilot.com" in its own docblock. The
validation comment in Datacenters says it. ServicesTest writes it out verbatim.
The step itself built the name from config('provisioning.dns.zone') — the
CUSTOMER zone — so on this installation a host was actually called
fsn-01.node.clupilot.cloud. Three places asserting one thing and the code doing
another.
OfficialDomains explains why that matters and is worth not weakening: two
registrable domains by design, the company's for site, portal and console, the
instance zone for customer workloads. A Nextcloud is third-party software that
strangers sign into, and on the same registrable domain as the portal it shares
cookie scope with it. A host name in that zone does not break the separation,
but it puts it in question, and the next slip is more expensive.
So there is now a platform_zone, derived from APP_URL when unset, and
RegisterHostDns uses it.
My own archiveUrl was broken for a second reason. It fell back to url() when
APP_HOST is empty — and APP_HOST is empty on most installations, because empty
means "the portal answers on any hostname" and that is the default. Called from
the console, url() would have produced the CONSOLE hostname, and the line would
have 404'd on a machine that is not allowed to reach the admin area at all. It
takes the host from APP_URL now.
The script no longer guesses its own name. It used reverse DNS, then the tunnel
address, then a hard-coded clupilot.net — a third domain that appears nowhere
else in this project and was simply invented. PrepareBaseSystem has the same
invention. A guessed name does not stay guessed: it ends up in /etc/hostname, in
/etc/hosts, in every log line and in every certificate request the machine ever
makes. CluPilot knows the name because it just assigned it, so it passes --fqdn
and the script refuses without it. The value now also survives the reboot in the
arguments file, which it would not have.
The ACME contact moved to .com for the same reason it was wrong: the operator
does not live in the customer zone.
Open, and NOT decided here: the owner also wants the host to get a public DNS
record and a certificate on the .com name. RegisterHostDns deliberately writes
host names only into the tunnel's dnsmasq, and says why — publishing them hands
every scanner the internal subnet and roughly how many hosts sit behind it.
Nothing in the current design needs a public certificate for a host's own name;
Traefik serves customer domains, not this one. Reversing that is a security
decision and belongs to the owner, not to this commit.
1992 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One entry per section: how you notice, where to look, what it usually is. Above
all of them the rule the plan already states — a half installed machine is
reinstalled, not repaired — with the reason, which is not convenience. The
script is built so a second run on a freshly imaged machine always comes out the
same; a run on a machine somebody straightened out by hand is no longer that.
Repairing trades a repeatable result for a one-off, and the difference only
shows up on the next host.
rescue_checked is the stated exception, because nothing has been written yet. It
is the only section where fixing in place is right, and the runbook says so
rather than leaving the reader to infer it.
Two things that are easy to get wrong are answered up front. Whether the reboot
has happened is one `findmnt -no FSTYPE /` — tmpfs or overlay means still in the
rescue system, zfs means the first-boot hook already resumed. And when a host is
unreachable, the first question is whether the firewall had even run: it is
applied last, so if `registered` is not done, the cause is the network and not
the rules.
The installer can be watched while it runs. If QEMU is still up, forwarding 5900
over SSH shows what the ISO is doing, and an input mask there means the answer
file was not accepted and the installer has dropped into interactive mode where
it will sit until the hour expires. That failure otherwise arrives as a timeout
saying nothing.
The last heading is the honest one: everything here is written from the script
and the findings behind it, not from incidents. What actually stops on the first
real run belongs in this file afterwards, quoted as it appeared in the console.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last section carries three things, and none of them could sit anywhere
else: the registration and key swap, Traefik's token plus the proof that a route
table actually arrives, and the nftables lockdown that ran last in the old
pipeline for a reason.
`|| true` is gone from the pveum calls, and not by deleting it. `role add ||
true` applied the privilege list only on the run that first created the role, so
a privilege added later reached new hosts and no existing one — which is exactly
how Sys.Modify was missing everywhere, surfacing much later and somewhere else
as a 403 on POST /cluster/backup that failed a paying customer at
register_backup. So: add, and on failure modify, as two separate commands so a
failure can be attributed. For the user and the ACL the tolerated `|| true` is
replaced by looking: try, and if it fails, check whether the desired state holds
anyway. A `|| true` hides "already there" and "went wrong" equally well.
The privilege list is copied verbatim from config/provisioning.php and checked
against it — 18 privileges, byte-identical, Sys.Modify included. It is granted on
/ because both endpoints that need it check / and nothing narrower satisfies
them.
The key swap follows the four steps this plan was amended to require. Register
first and hold the answer, switch wg0 to the new private key, PROVE a fresh
handshake, and only then let the old key go. If the handshake does not come, it
puts the old key back and fails loudly. Ordering alone says when to discard, not
whether the new key carries — and a host that locks itself out on the final step
is the one failure nobody fixes remotely.
Traefik's token is written here because here is where it exists, and the route
table is then fetched directly with that token and that URL. Section 7 could not
give this proof; it is not skipped, it is given where it can be.
The lockdown comes dead last, with the full ruleset from SecureHostFirewall
including the ICMP correction — ICMPv6 neighbour discovery and packet-too-big,
IPv4 fragmentation-needed — because a bare policy drop takes IPv6 down within
minutes and black-holes large transfers instead of failing them. The ruleset is
validated with `nft -c` before it is applied, since a partially loaded ruleset
means policy drop is in place and the exceptions are not. There is no automatic
reopening: a firewall that reopens itself under failure is not a firewall, so the
emergency release is a script for the provider's console.
Verified without hardware: all seven files dash-clean; json_field reads
full-tokenid, value and host_token without jq; the privilege list is identical to
the config's; and the rendered ruleset carries policy drop, both ICMP families,
80/443, the DHCP rebind rule and the tunnel-only 22/8006. Step 2 unticked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All three Block A traps are CHECKED here rather than assumed, because each one
has already cost a paid order and none of them shows a symptom until it is too
late.
Trap 2 is checked before anything is built: virt-filesystems is asked whether
the image uses LVM and whether root is the last partition, and the section
refuses if either is wrong. Debian's cloud image satisfies both by construction,
which is exactly why it is the base — but "satisfies it by construction" is a
claim about an artefact somebody else builds, and it gets verified rather than
trusted.
Traps 1 and 3 are checked AFTER the image is customised, because virt-customize
reports success even when apt quietly did not install something. qemu-guest-agent
has to be in /usr/bin or every provisioning run hangs in WaitForGuestAgent until
it times out, and the compose file inside the image has to carry `user:
www-data` or every occ call fails — including the acceptance check that decides
whether a customer's instance is usable.
The template is verified by its ATTRIBUTE, not its existence. VerifyVmTemplate
checks only that 9000 is there, which a VM that merely happens to be numbered
9000 also passes; `template: 1` is passed only by a template.
Docker comes from Docker's own repository, not Debian's. docker.io ships no
compose plugin and Debian's docker-compose is the old Python one, which does not
read this file at all.
The compose file is deliberately customer-independent: no password, no name, no
domain. Everything variable arrives in /opt/nextcloud/.env, written into the
guest by cloud-init at clone time. Baking a password into an image copies it
onto every host and into every instance, and leaves it there long after the
customer has changed it.
OVERWRITEPROTOCOL, OVERWRITEHOST and TRUSTED_PROXIES are set because TLS ends at
Traefik on the host. Without them Nextcloud builds its own URLs with http://,
the login loops, and WebDAV clients get handed an address that does not exist.
Storage is discovered rather than named: local-zfs is what the PVE installer
creates on ZFS, local-lvm on ext4, and `local` frequently cannot hold disks at
all — an importdisk there fails only after the copy.
Verified without hardware: dash-clean; the compose file parses, carries `user:
www-data` on the app service, publishes 80, and every credential is a ${...}
reference rather than a literal. Step 2 unticked, and it is the one the plan is
most insistent about: the template must actually be cloned, started, and asked
`occ status` as www-data. Without that clone it is not proven.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Binary plus systemd unit, no Docker on the hypervisor: nftables stays the only
owner of the host's firewall rules and a Docker daemon would bring its own
chain.
The names were read out of SshTraefikWriter::render() rather than guessed, which
is what the handoff asks for in as many words. The route endpoint emits
entryPoints ["websecure"] and certResolver "letsencrypt", so the static config
declares exactly those. Name them differently here and the routers point at
nothing while Traefik still reports a clean start — a failure with no symptom
at the place it happens.
An ordering problem this plan had not resolved: the http provider needs the
durable host token, and that token only exists after POST /host/register, two
sections later. Rather than reorder the section keys — they are the contract
with the platform plan — the static config is written twice. Here with an empty
token, so the service stands and holds 80 and 443, and again in Task 9 with the
real one. The proof does not disappear, it moves to where it can be given.
That proof asks the endpoint directly with the same token and URL the config
carries, instead of counting Traefik's routers. A fresh host has no customers,
so its table is legitimately empty, and "zero routers" would mean both "fine"
and "never fetched".
Ports are checked separately from the service. "Running" and "listening" are two
claims, and Traefik starts cleanly even when a typo means an entryPoint was
never created.
The binary is checksummed against the release's own checksums file, for the same
reason the ISO is: what listens on 80 and 443 and holds every customer's
certificate does not get taken off the network unverified. The version is
discovered at runtime, because a pinned number becomes a 404 mid-takeover.
acme.json is created at 600 before Traefik ever runs. It holds the private keys
of every customer certificate, and Traefik refuses wider permissions — rightly.
The unit runs with CAP_NET_BIND_SERVICE and nothing else, ProtectSystem=strict,
NoNewPrivileges: this is the one process on the box reachable from the open
internet.
Verified without hardware: dash-clean, the generated config parses as YAML, and
it carries web/websecure/traefik as entryPoints, letsencrypt as the resolver,
the Bearer header on the http provider, web redirecting to websecure, and 640 on
the config with 600 on acme.json and the token file. Step 2 unticked — it wants a
fetched route set, and that is Task 9's to show.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three interface states come over from ConfigureWireguard.php exactly as
that file works them out, because the one line they replace got two of them
wrong. `systemctl enable --now wg-quick@wg0 || wg-quick up wg0` brought the
interface up via its fallback WITHOUT the systemd enablement, so the tunnel did
not come back after a reboot; and on the next attempt both halves failed with
"wg0 already exists", so it retried forever against a tunnel that was working.
Enable before start, because enable is the half that survives a reboot, and it
is repaired even when the interface is already up by other means.
The configuration is compared before it is written. The file used to be
rewritten unconditionally with nothing reloading it, so correcting a wrong hub
key showed a new file and identical behaviour — the running interface still
held the old peer. Knowing whether it CHANGED is what lets an unchanged, working
tunnel be left alone and a corrected one actually be applied. A change means a
restart rather than `wg syncconf`, because syncconf applies peers only and a
corrected Address or AllowedIPs would silently do nothing.
AllowedIPs is computed, not copied. Writing the host's own address there would
let the tunnel handshake and leave every other participant unreachable — a fault
that looks like a routing problem somewhere else entirely.
The handshake target is derived from --api rather than taking its own argument.
CluPilot's tunnel address is already in there, and a second value meaning the
same thing is a second value that eventually disagrees with the first.
The tunnel counts only when it has been PROVEN, never when the file is on disk.
A stored tunnel address without proof made every later connection attempt
useless and the recovery was hand-editing the database. Once it is proven,
flush_reports runs and the whole history from the rescue system onwards reaches
the console carrying its original timestamps.
Verified without hardware: dash-clean; the subnet arithmetic is right on octet
boundaries and off them (192.168.5.130/25 gives 192.168.5.128/25, 172.16.4.9/12
gives 172.16.0.0/12); the API host parser handles a bare address, a scheme, a
port and a path; and wg0.conf renders with AllowedIPs on the network rather than
the host. Step 2 unticked — it wants all five earlier sections showing up in the
console with real timestamps, and that needs a hub.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit moved the nftables lockdown out of Task 5 and explained why
in its message. A commit message is not where the next person looks. Task 5 now
carries it: the bridge is normally already there because the ISO installer
writes it, the lockdown belongs in Task 9 because SecureHostFirewall ran last
for a reason, and "check from outside" cannot mean what it says before the
tunnel exists — so the script checks inside-out and says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PVE arrives with the image now, so this section accepts it rather than installs
it — but it needs every line of knowledge from InstallProxmoxVe.php to do that,
just pointed the other way.
The kernel is the claim, not the package. An installed proxmox-ve running on a
Debian kernel is exactly the half state RebootIntoPveKernel exists because of,
so uname is what gets checked.
The codename table now verifies a pairing instead of creating one. An image
whose PVE major does not match its Debian base was built wrong, and no amount
of fixing package sources heals a PVE compiled against a different libc — that
machine gets reinstalled with the right image. The other half of the table,
codename to PVE major, is written down here because the ISO route needs it and
the original file only had the suite side.
Removing the subscription repositories is the part that everything after this
depends on. A Proxmox image ships pve-enterprise, and without a subscription it
fails every apt-get update — after which no package install in any later
section succeeds. Both spellings are handled, because PVE 8 used one-line .list
files and PVE 9 moved to deb822, and ceph is in the list because the same trap
is set there a second time. The proof is a clean apt-get update at the end,
which is the only thing that tells removal apart from overwriting.
One thing changed after the first draft: when pveversion cannot be parsed, the
pairing check used to be skipped in silence. That is the fake that R19 records
as worse than no check at all, because it stops the next person from looking.
It now says "Paarung UNGEPRÜFT" in the report and in the log.
Verified without hardware: dash-clean; bookworm maps to PVE 8, trixie to PVE 9,
forky is refused; the pveversion parser reads 9 and 8 out of both real formats
and yields nothing for garbage; and against a copy of /etc/apt/sources.list.d
holding all five subscription files plus an unrelated one, exactly the five go
and the unrelated one stays. Step 2 unticked — there is no Proxmox here to
accept.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The official Proxmox ISO, an answer file baked into it, and QEMU running it
against the physical disks from the rescue system. Options were looked up
rather than recalled: prepare-iso takes --fetch-from iso, --answer-file and
--on-first-boot, and it names its own output, so the finished ISO is found by
glob afterwards instead of being handed an --output flag I would have invented.
validate-answer runs BEFORE the ISO is baked. An answer file with a typo drops
the installer into interactive mode, where it then hangs invisibly inside QEMU
until the hour runs out — the failure would arrive as a timeout and say nothing
about the cause.
The codename table comes over verbatim from InstallProxmoxVe.php, which the
platform plan deletes. It is used here to install the assistant into the RESCUE
system, so it keys off the rescue system's own codename. An unknown one aborts;
not "trixie is newest, so trixie", because the next Debian will be unknown too
and by then nobody is watching.
The ISO is checksummed against the mirror's SHA256SUMS. Booting an operating
system that could have become anything in transit is not a risk worth taking,
and a truncated download is enough to cause it — malice is not required.
The reboot carry-across is the gap this plan was amended for, and [first-boot]
takes exactly one executable. So that one file carries everything: the script,
its lib/, the progress file and the arguments, as a base64 archive in its own
body. That also avoids importing the fresh ZFS pool from the rescue system to
copy files into it.
Two bugs found by running it rather than reading it. The staging copy put the
work directory inside the state directory, so cp refused to copy a directory
into itself — and had it not refused, the gigabyte ISO would have been base64'd
into the hook that gets baked into that same ISO. The fix is not a better
exclude: the two directories have opposite lifetimes, so they now live in
different places, and a guard refuses loudly if anyone points them at each other
again. Structure beats a rule someone has to remember.
Verified without hardware: both files pass sh and dash. The generated answer
file parses as TOML with the expected disk-list, zfs.raid, zfs.arc-max,
first-boot and network.filter, and a 64-character root password. The codename
table accepts bookworm and trixie and refuses bullseye and forky. The shim comes
out at 26 KB with a 3 MB dummy ISO sitting in the work directory, proving it
stayed out. Run in a fresh debian:13-slim container it unpacks to
/opt/clupilot/bootstrap and /var/lib/clupilot, restores the progress file with
its original timestamp, has 700 on the directory and 600 on the arguments that
hold the WireGuard key, skips rescue_checked as already done, and fails cleanly
on the missing qemu. Step 2 stays unticked: a container is not a rescue system,
and nothing here has yet written to a disk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit put installimage with a Proxmox image into Task 3. That
image does not exist. Hetzner's installimage carries no Proxmox and lists it as
unsupported, so anyone following that instruction finds only Debian and ends up
back at the intermediate base system the same commit had just removed. Looked
up rather than remembered, which is what should have happened before writing it
down.
The mechanism the owner described does exist and is the official one since PVE
8.2: build the real Proxmox ISO with an embedded answer file via
proxmox-auto-install-assistant, run it under QEMU against the physical disks
from the rescue system, reboot into a finished node. Hetzner publishes a
tutorial for exactly this. No Debian, no VNC, no click — which is what the
owner said in the first place.
Two things fall out of the answer file that improve the plan.
The hand-written systemd resume service is gone. [first-boot] with
source = "from-iso" is the supported way to have the script pick itself up
after the reboot, and it is the only variant that can work before the tunnel
exists — a from-url hook would be pointing at a CluPilot it cannot reach yet.
The carry-across of the progress file stays necessary regardless: the rescue
system's copy still lives in RAM.
Trap 2 was being applied to the wrong machine. "Root partition last, not on
LVM" comes from the template traps, and the reason it exists is
GrowGuestFilesystem, which grows a GUEST filesystem. The template is built in
Task 8 from a Debian cloud image, not from the host's own layout, so the
justification written into Task 3 ("it applies to the host too, because the
template is built from here") does not hold. It remains binding for the
template, where it belongs. The host gets ZFS, and that it happens to involve
no LVM is a side effect rather than the reason — with arc-max pinned, because
the default takes half the RAM a host needs for its guests.
Also recorded: the answer file needs an fqdn and a root password that the
command line does not carry. The password is generated and thrown away, because
the platform no longer logs in over SSH and the plan already says a half
installed machine is reinstalled rather than repaired — keeping a secret for an
emergency the plan does not have only creates another place a secret lives.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The owner contradicted the plan, and the plan was wrong. It had the script
write Debian 13 and then had Task 4 turn that into a Proxmox host. In reality
you boot the provider's rescue system and install Proxmox directly — there is
no intermediate Debian for anyone to convert.
The section keys do NOT change. They are the only agreement with the platform
plan, and they are not worth touching for a rename — especially since they
still fit: Proxmox VE is Debian with the PVE kernel and packages on top. What
changes is what they mean, written down so the console labels them honestly:
debian_installed is "base system written", proxmox_installed is "PVE proven to
run".
Task 3 now uses installimage with a Proxmox image, and says the image name gets
pinned during acceptance instead of guessed here — the list belongs to the
provider and moves. No debootstrap and no hand-rolled partitioner: the provider
knows its own hardware, and a self-built boot path on someone else's metal only
comes back through their console. Hetzner dedicated only; a netcup path gets
written when there is a netcup machine to accept it on, because an unproven
fallback is exactly the assertion this plan avoids everywhere else (R22).
Task 4 keeps every line of knowledge from InstallProxmoxVe.php and needs all of
it — it just applies it to an image instead of a bare base. The codename table
now VERIFIES the pairing rather than creating it, and the enterprise repo that
ships with a Proxmox image has to go, because without a subscription it fails
every apt-get update and therefore every package install in the sections that
follow. Its acceptance now includes a clean apt-get update, which is the only
way to tell removal from overwriting.
Delivery is settled too, and was written down in neither document: a tar.gz
unpacked to /opt/clupilot/bootstrap. A curl | sh cannot carry a lib/, and the
separate files are load-bearing. A static archive is not an endpoint that
answers questions, so spec §5 stands.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four checks, all of them before a single byte is written, because this is the
only moment the script still has the option of not overwriting a disk.
The expensive one is "is this actually a rescue system". It demands POSITIVE
evidence rather than the absence of a counter-argument: a rescue system runs
from RAM, so its root is tmpfs, an overlay or a ramdisk, and an installed
system has a real partition there. A running pveversion is refused outright —
that is a hypervisor with customers on it. Mounted partitions of real disks are
refused too, because at that point the script cannot tell an empty machine from
someone's data.
There is deliberately no flag to skip the check. Such a flag gets used exactly
once, on the evening it should not have been. A rescue system this does not
recognise belongs in the runbook, not in a bypass.
The disk floor is written as arithmetic rather than a number, because the repo
has no threshold to borrow — plans live in the database, not in
config/provisioning.php. Template ~20 GB, Proxmox and its swap and backups
~20 GB, one smallest customer ~50 GB, rounded up to 100. It is explicitly not
capacity planning; HostCapacity and reserve_pct do that after registration.
This only turns away the machine the whole thing cannot fit on.
Network and clock come from one plain-HTTP HEAD, deliberately without TLS —
otherwise a wrong clock would be checking itself and would report a certificate
error instead of the time. Where `date -d` is missing (busybox), the clock is
NOT checked and the report says so: a check that silently waves things through
is worse than none, because it stops the next person from looking. That is the
same lesson R19 records about ->timezone(config('app.timezone')).
Verified here, in both directions where a direction existed. On this VM (root
on /dev/sda1, 80 GB, /dev/kvm present): refuses with three findings at once,
exits 1, writes nothing. In a debian:13-slim container (root on overlay, no
/dev/kvm): the overlay root is accepted as rescue-like and the missing
/dev/kvm is caught, which is the abort case the plan names. Against a receiver
serving a Date header two years off: 63074141 seconds of drift, refused, with
the exact `date -u -s` line to fix it. Step 2 stays unticked — none of these is
a rescue system on a dedicated server.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Argument parsing, the state directory, and the half of the script that knows
the way back to CluPilot. No sections are hooked up yet; main lists the nine it
will get, in the order from spec §7, so what is missing stays visible.
report() writes locally first and only then tries to send, because before
wireguard_joined there is no path at all — the lines queue up and flush_reports
delivers them later WITH THEIR OWN TIMESTAMPS. That last part is the whole
point: a twenty-minute install that arrives in one batch looks like one second
in the console, and nobody can see which section was slow.
Every network operation is explicitly caught. The script runs under `set -e`,
and a failing send is the normal case for the first five sections, not an
error — a report that aborts the run would be the diagnosis that kills the
patient.
What is sent had no agreed shape. The section keys were the only agreement
between the two plans; the envelope around them was not written down anywhere,
so it is written down here, at the top of report.sh, for the platform side to
read once. There is deliberately no "running" state: the console derives "open"
from the absence of a report (spec §7), and a third state would be a second
truth about the same thing.
The sent-marker is a line count in its own file rather than a flag rewritten
into progress.jsonl. That keeps the log append-only, so a power cut mid-write
costs at most a partial last line instead of a rewritten file — and after the
reboot in Task 3 that file is the only thing that remembers anything.
Verified here, without hardware: both syntax checks pass; two reports against
an unreachable CluPilot queue up without aborting and leave progress.sent at 0;
a receiver brought up afterwards gets both in one batch, carrying 17:28:55 and
17:28:57 rather than the 17:29:22 they arrived at; a message containing quotes,
a backslash, a tab and a newline survives as valid JSON; and section_done
counts a `done` as done and a `failed` as not. Step 2 of this task is NOT
ticked — none of that is a rescue system, and the plan is right to insist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three sentences left over from the wave before, all of the kind this branch
exists to end.
- "Every active Price" was one word short. activePricesFor() sends
`type: recurring` as well as `active: true`, so a one-time Price is never
listed — which made the hand-built-payment-link sentence false for a one-time
link: neither listed nor archived. Both places say `recurring` now, and the
docblock says why the filter is not a gap: createPrice() always sends
`recurring[interval]` and is the only call that mints a Price at one of our
Products, so the filter can only ever hide somebody else's one-time Price.
A filter left out of a sentence is a gap the next reader goes looking for.
- The sweep test file still opened "The orphans nobody can adopt" — the exact
narrowing the command's own docblock had to be corrected away from. It happens
to describe the fixtures, which are all metadata-less, so it now says that
instead of claiming it of the command.
- The design doc called --archive "harmlos" seven lines above the correction
establishing that it is not, for an adoptable orphan. A document that
contradicts itself seven lines apart teaches nobody anything.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Read on review, before a line of shell was written. All three would have been
found on hardware instead — two of them late, and one of them only by noticing
something that was silently absent.
The progress file did not survive the reboot. Task 1 writes it in the rescue
system, whose root lives in RAM; Task 3 installs Debian and reboots. The plan
said the resuming service uses "the same progress file" without saying how it
gets there. Because flush_reports first runs in Task 6, AFTER the reboot,
rescue_checked and debian_installed would simply never have reached the console
— and nothing would have pointed at the reboot as the reason. Task 3 now carries
the file, the script, and the arguments across, the last one root-readable only
because the WireGuard key is in it.
RebootIntoPveKernel.php was on the platform plan's delete list but not on this
plan's read list. Six files were named; seven are deleted. This is the one that
proves the boot path BEFORE issuing the one irreversible command in the whole
run: a -pve kernel image present, its initramfs present (a full /boot leaves
exactly the half-written one), and update-grub checked on its exit status, which
its predecessor discarded. Losing that would have cost a machine that does not
come back, and on a dedicated server that means the provider's console.
Reading it also corrected a rule this plan states. Task 9 rightly refuses
`pveum ... || true`, but that is not a rule against `|| true` — RebootIntoPveKernel
tolerates one deliberately, with the reason written next to it, and a script
that applies the ban literally breaks the kernel removal in Task 3. The rule is
now stated as what it is: no `|| true` over a command whose failure means
something.
The key swap in Task 9 had no proven handshake. Task 6 insists a tunnel counts
only with one, then Task 9 re-keys that same tunnel and discards the old key on
ordering alone. Ordering says when to discard, not whether the new key carries.
Four numbered steps now, with the proof third and the discard fourth, and a
fallback to the old key if the handshake does not come — the one failure nobody
fixes remotely, at the end of a run that did everything else right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Platform: ten tasks, all provable by the suite. Enrolment code and key pair,
the tunnel-only restriction (its test is the reason the design looks the way it
does), register, progress, routes, retiring SshTraefikWriter, the console, the
customer's own subdomain, shrinking the pipeline to six steps that only look,
and pulling the readiness checks along.
Script: ten tasks, none provable by the suite — a script that installs an
operating system cannot be run in SQLite. Each one ends on real hardware
instead, and two of them insist on it: the bridge section must have its
self-revert triggered on purpose, and the template must actually be cloned and
booted before it counts.
The one real coupling is written into both: platform task 9 DELETES the seven
step files the script plan reads its knowledge from. Do not run it before that
knowledge has been lifted — git log keeps them, but nobody reads history they
do not know to look for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
stripe:sweep-orphan-prices described itself as listing "what AdoptStripePrice
will not take over". It computes nothing of the sort: it lists every active
Price at one of our Products that no register row knows, adoptable or not — the
only question it asks is whether a row holds the id. And then it invited the
destructive path, with no word anywhere about running stripe:sync-catalogue
first.
Following that advice can make a plan unsellable. Archive an orphan the next
sync would have adopted and the sync can no longer see it: activePricesFor()
asks Stripe for active prices only, so adoption returns null and createPrice()
fires under the same key the crashed run used. For twenty-four hours Stripe
answers a repeated key by replaying the stored response — the archived Price's
id, in the body it had while it was live. The register writes it down as live,
and the checkout is refused. This is the same class of defect the branch exists
against: a comment that was eloquent and wrong.
- The docblock now says what the command lists, names the twenty-four-hour
replay so nobody has to rediscover it, and the ordering requirement is in
$description and printed on every path that found something, --archive
included. The reassurance that archiving is harmless is now scoped to what
THIS codebase sells; a payment link an operator built by hand against one of
our Products is withdrawn all the same, and says so.
- stripe:sync-catalogue --dry-run says "created or adopted" again. A dry run
counts intents without calling ensure(), so it cannot know which Stripe
already holds — and a dry run is the first thing an operator runs after an
interrupted one. The live line keeps both figures, where they are knowable.
- The sweep's plan side gets a test: an orphan at a family Product is reported
and a known plan Price never is. Deleting either half of the lookup turns it
red; the five tests it joins all stayed green for the products() half.
- The throttle test pinned "once" and neither "per price" nor "per day". A key
mutated to a constant would have silenced every orphan after the first and
kept it green — the silent no-op the branch is built against. It now plants a
second orphan, and travels a day to hear the first one again.
- Two of my own documents corrected. The spec sentence claiming an adoptable
orphan never appears in this list is what the implementation faithfully built;
it holds only if a sync has run since the orphan appeared, which nothing
enforced. And the two adoption classes do not carry the same run log:
$adoptions on the Price side, $duplicates on the Product side.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The route pull stays inside the tunnel. The operator's criterion was not
availability but "publicly reachable, yes or no" — and an endpoint that survives
a tunnel outage must by definition be reachable outside it. So nothing new opens
from outside, on the host or here.
The subdomain becomes the customer's choice at checkout, with suggestions from
their own name and a sentence saying the address is permanently public: every
Let's Encrypt certificate lands in the Certificate Transparency logs, which are
public, searchable and never deleted.
Also recorded, both decided and both getting their own spec: resetting a
Nextcloud to factory state with a 14-day safety backup, and an outage mail to
the operator after two consecutive failed checks — measured today that neither
an incident nor a mail exists, so a customer VM can fail unnoticed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The recognition step refuses a price that proves nothing about itself, and that
is right — adopting a stranger's price is worse than minting a second one. It
left an operator with a log line and no way to act on it.
Reports by default and touches nothing. With --archive the orphans stop being
sold, which is harmless for the reason it always is here: Stripe goes on billing
every subscription already on an archived price, and nothing is sold on a price
no row knows.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator was right to challenge POST /enrol. The three values a script needs
before the tunnel exists — hub key, hub address, assigned tunnel IP — are all
known to the console when the host is created, so they travel in the copied
command instead of being fetched.
The one thing that forced an endpoint was the hub needing the host's public key
before the tunnel is up. So the console generates the pair and hands it over,
and the host swaps in a freshly generated key through the tunnel afterwards.
The key that passed through a clipboard lives for minutes.
Net effect: the network boundary is unchanged. No new way in from outside.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six tasks, each ending on a green billing folder and its own commit: move the
money gate into the class that promises it, four fixes that make the fake behave
like Stripe, ask Stripe for its products, recognise an orphaned one, count
adopted apart from created, and hand the operator a sweep for the orphans nobody
can adopt.
Self-review dropped one item the spec had carried over from the last review: the
zero guard in PlanPrices::ensure(). Its only purpose was to keep a tiered price
unreachable, and the gate move rejects those directly — so it would be a
behaviour change with no remaining benefit, and it would make a free plan
unsellable. Recorded as deliberately not built, with the reason, rather than
quietly dropped.
Also corrected: a test that counted every product where the run mints one per
family, and a step that named the test it replaces only by description.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The one that matters is orphan PRODUCTS. A price orphan is recognised now; a
product orphan is not, and it is worse — a second product makes every
activePricesFor() ask about the wrong one, so price recognition goes blind for
that whole family and the guard disables itself through the gap beside it.
Duplicated products are reported, not deactivated. That is the owner's call and
it is the careful one: an archived PRICE is provably harmless, which is what the
grandfathering rule has always rested on, but deactivating a PRODUCT makes its
prices unsellable and contracts can be running on those. What was proven for one
is not proven for the other.
The money gate moves into the class whose docblock promises it. Today
AdoptStripePrice can compare three properties and the other four are filtered in
one client, which the fake cannot express — so no test can reach the promise the
class makes.
Also: a sweep command for orphans nobody can adopt, a day's throttle on the
warning that currently fires on every customer booking, and six mechanical
fixes.
Self-review caught one of my own: the plan to tell "created" from "adopted" by
comparing the price id before and after the call cannot work — there is no id
before, in either case. AdoptStripePrice counts its own adoptions instead, which
needs it bound as a singleton, and the spec now says why.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2321854 removed it. That commit's own work — discarding an unconfirmed
registration and sweeping abandoned ones — is untouched and correct; what went
with it was every file of the Stripe price adoption merged an hour earlier:
AdoptStripePrice, IdempotencyKey, the unique-index migration, both test files,
the spec, the plan, and the edits in six more.
Nothing was lost. The commits stayed in history and the files stayed on disk,
untracked, which is what a staged deletion from a stale tree leaves behind.
Restored from 4eb90c8, the reviewed head, by explicit path — the fourteen paths
2321854 damaged and not one more, so the ten files that commit legitimately
added or changed keep exactly what it gave them.
This is the failure the repo's own rule exists to prevent: stage by path, never
`git add -A`, because a second session's index does not know what a first one
merged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Abmelden und neu registrieren" was advice that could not work. Signing out
frees nothing: the address is still held by the unique index, so registering
again with the address somebody meant fails — and every abandoned attempt
stayed in `users` for ever. Reported exactly that way, and it was right.
The button discards the account instead. Only an UNCONFIRMED one, and only
one with nothing behind it: the webhook creates a customer's login from a
paid checkout, so a user can sit on this page unconfirmed while already
having a contract, and deleting that is data loss dressed up as a
convenience. It refuses and says where to go. A confirmed account is closed,
not discarded — that is ConfirmCloseAccount, with an invoice history behind
it.
R23: confirmed in a modal. The modal mutates nothing; it dispatches the event
the page listens for, so the checks stay in the one place they already are
rather than being duplicated where they could drift.
And nothing has to be discarded by hand. clupilot:prune-unverified removes
registrations nobody confirmed after five days — long enough for somebody who
signed up on a Friday and found the mail in a spam folder on Monday, short
enough that a typo does not hold the correct address for a month. It skips a
confirmed account whatever its age, and any account with a customer record
behind it, matched on user_id AND on the address, because either link means
somebody is a customer. Every removal is logged with the address: a line that
disappears is what somebody asks about later, and a count answers nothing.
The deadline is said on the page as well as in the mail. Somebody who never
received the mail is looking at the page, and the deadline is the reason they
do not have to do anything about an attempt they abandon.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ten follow-ups, none of them blocking and none of them done. The one that
matters: activePricesFor() recognises an orphaned Price and nothing recognises an
orphaned PRODUCT — and a second Product is worse than a duplicate Price, because
every activePricesFor() then asks about the wrong Product and recognition goes
blind for that whole family. The branch disables its own guard through the gap it
did not close.
Second: AdoptStripePrice promises in its own docblock that no adoption can move
money, but can only compare amount, currency and interval. The four properties
that also decide what a Price charges are filtered in HttpStripeClient, which the
fake cannot express — so no test of the class can reach the promise it makes.
Also recorded: two pre-existing test failures this branch surfaced without
causing, with the diagnosis, so the next person does not derive them again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The money gate compared amount, currency and interval and left the rest to a
filter that excluded two fields. A recurring, licensed, monthly price with
transform_quantity divide_by 10, at our exact figure, on our product, carrying
our addon key passed all five conditions. Modules bill BY quantity — a pack is
one item at quantity n — so a customer holding three would have been charged
one. A tiered price was excluded only by accident: Stripe reports unit_amount
null for it and we read that as 0, which fails the amount match unless the
caller's own figure is 0, and PlanPrices::ensure() has no zero guard.
activePricesFor() now also skips a non-empty transform_quantity and any
billing_scheme other than per_unit. Absent keys stay Stripe's defaults, which
is the direction that matters: read the other way round the filter would refuse
every legitimate price and turn recognition into a permanent no-op. The
contract docblock names the four properties instead of claiming to cover all of
them, and says the list is not everything Stripe can put on a price — only
what is known to change what one charges.
And a comment that says a gap is covered is worse than the gap. The sync said a
crash between Stripe creating a product and us storing its id gives back the
same product next run: true for twenty-four hours, false afterwards, and the
exact claim this branch was written to refute. There is no recognition step for
products at all, so an interrupted run leaves an orphan and the next run past
the expiry mints a second one — worse than a duplicate price, because
activePricesFor() then looks at the wrong product and recognition goes blind
for the whole family. Written down as a known gap at both createProduct call
sites, with the trade this branch chose: folding name and metadata into the key
means a renamed family now mints a second product where it used to answer 400,
and a blockade reaches a paying customer while a duplicate product does not.
The sweep no longer reports "created" for what it adopted — the count is taken
before ensure() runs, and telling those two apart is the whole point.
The test named after 2026-07-29 could not reproduce it: the fake's key ledger
was empty, so createPrice() could not have thrown and the test could not tell
"adoption prevented the 400" from "no 400 was possible". Seeded now with the
key today's call sends and a fingerprint over the metadata the pre-9da1358 call
sent. Remove the adopt() call from AddonPrices::ensure() and the test dies with
Stripe's own sentence instead of an assertion.
Also: the plan document still instructed migrate:fresh --env=testing. There is
no .env.testing here, so it targets the live development database; struck
through with the reason, not deleted, and a Korrekturen section records the
five defects the plan's own reviews found in it. Four comments corrected —
"below" that meant above, a counter claimed unable to tie that ties with a
planted price, the unique index's second meaning for remember()'s catch, and
that MariaDB DDL is not transactional so the dedupe commits before the index.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six tasks, each ending on a green billing folder and its own commit: the key
fingerprint (and a fake that finally reproduces Stripe's refusal), the two
listing calls, the adoption step itself, then each price service in turn, then
one row per Stripe price.
The spec now names both test files. What goes on the wire is checked through
Http::fake against the real client — asserting against the fake would only prove
the fake agrees with itself, which is the mistake that let the incident through.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A run on 2026-07-29 23:11 created price_1TygdEC7u8NpJ8pOt3nsoyYw at Stripe and
died before the row was written. 9da1358 then added the tax_treatment metadata
field, so the next sweep sent the same idempotency key with different parameters
and Stripe refused it — for twenty-four hours, and not only for the sweep:
AddonPrices::ensure() also runs inside a customer's module booking.
Two defects, one spec: nothing asks Stripe what it already has, and the key
covers less than the call sends. The spec settles what identity means here
(product, amount, currency, interval — never metadata, which Stripe lets you
edit) and where the key's fingerprint may go (the catalogue calls, never a
refund, where a duplicate costs the customer their money twice).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-measuring before writing found the spec's numbers had moved: seventeen
permissions now, not sixteen, because the mailbox work added mail.manage.
It also turned up something the spec did not anticipate — ConfirmsPassword
resolves the password through auth()->user(), the default guard, so all
five components using it break the moment a second guard exists. That is
now its own task.
Self-review caught two references to things that do not exist: users has
no uuid column, so the impersonation link addresses the customer instead,
and there is no layouts.guest — the auth pages use layouts.portal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Task 2 reviewer rewired the model to Laravel's APP_KEY-based Crypt
facade and every test stayed green. Laravel's encrypter is a container
singleton resolved once from app.key, so config()->set('app.key') never
rebuilds it and the rotation the test performed was invisible.
Replaced with a positive proof — rotating SECRETS_KEY must break
decryption — plus the APP_KEY case with forgetInstance(), which is what
makes that direction mean anything.
Seventeen models use App\Models\Concerns\HasUuid, and R11 is the reason:
URLs address records by UUID, not by integer primary key. The plan told
the implementer to hand-roll booted() instead, losing getRouteKeyName()
with it. Authoring error, corrected before the remaining tasks copy it.
The Task 1 implementer copied '// path/to/file.php' into the files
verbatim, which breaks Pint's blank_line_after_opening_tag and matches
no file in the repo. The annotation was mine; the constraint now says so
before the next seven tasks repeat it.
Self-review against the spec turned up four gaps and one latent bug in
what already ships.
The bug: Laravel 13's MailManager reads only 'scheme' when building an
SMTP transport (MailManager.php:196) — the old 'encryption' fallback is
gone — and .env carries MAIL_SCHEME=tls, which Symfony does not accept;
it knows smtp and smtps. Nothing has failed because MAIL_MAILER=log
means no connection has ever been opened. The plan stores a human-facing
tls/ssl choice and translates it where it is used, and never copies
MAIL_SCHEME through.
The gaps: the cancellation mail and the provisioning notification were
edited but not tested, secrets.manage was never proven insufficient for
the new page, and a missing SECRETS_KEY had no task making the page say
so rather than throw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console offers a single mail.password pointing at one SMTP mailbox,
which is nowhere near enough for five sending addresses. A mailbox is
address, display name, username and password; five of them as registry
entries would be twenty secrets named mail.support.username, and the
question of which mail sends from which address would still have no home.
Send-only stays usable because every message carries Reply-To on its own
mailbox, so a customer's reply lands in a mailbox someone reads. That is
what makes IMAP unnecessary rather than merely deferred.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decided: whoever is in operators has no business in users. A row left
behind is the same mixing the separation exists to end, only smaller.
The one case that deletes nothing is a row with a customer, a seat or an
order on it. That would mean the same person is operator and paying
customer, and a silent delete would take billing data with it — so the
migration stops and names the address instead. Neither existing account
is in that state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator console and the customer portal share one users table, one
login page and one guard. Three reported faults follow from that single
fact: the console serves the portal's sign-in page, that page offers
"Registrieren", and the link 404s because RestrictAdminHost::SHARED does
not list register.
Measured rather than assumed: driving the host guard directly gives
/login through and /register 404 on the console host, and all sixteen
Spatie permissions turn out to be console permissions — so RBAC moves to
the new guard rather than being duplicated across two.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- RunRunner catches step-resolution errors (removed/renamed pipeline) and
fails the run terminally instead of looping forever every tick.
- queue-provisioning worker gains NET_ADMIN + tun + persistent wireguard
volume + published UDP port so LocalWireguardHub can manage wg0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Dedicated 'provisioning' queue connection (retry_after 2400) + worker
(--timeout=2100); AdvanceRunJob tries=1, timeout=2100 on that queue.
- Run lock TTL raised to 2100s so a long step can't be run concurrently by a
duplicate tick-dispatched job.
- StartHostOnboarding creates host+run in a transaction, dispatches after commit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo