A booked storage pack was a row in `subscription_addons` and nothing else:
priced, frozen, charged every month, and delivered nowhere. Nextcloud's
quota came from `instances.quota_gb`, which is the package alone; the disk
was sized for the package alone; and on the one occasion a disk did grow —
a plan upgrade — the guest never saw it, because nothing stretched the
partition or the filesystem over the new space.
One authority for the allowance. StorageAllowance adds the package and the
booked packs, and everything that needs the figure asks it: the quota step,
the downgrade check, the portal, the console, the repair command. It is
DERIVED rather than stored — a column would have to be rewritten by booking,
cancelling, granting, a grant expiring and a plan change, and the day one of
those paths forgot it would be silently wrong in whichever direction costs
somebody money. `quota_applied_gb` keeps its own meaning: what the guest was
last actually told, which is how a machine with the allowance enforced is
told from one where the figure has only ever been a row in our database.
Buying it delivers it. BookAddon asks ApplyStorageAllowance for a run on
booking AND on cancellation; the new `storage` pipeline grows the virtual
disk to the allowance plus the package's own overhead (read off the package
— 20/40/50/100 GB on the four catalogue packages — never a ratio invented
here), then GrowGuestFilesystem makes the guest see it, then the quota is
applied. No cold boot is involved: the data disk is scsi0 and Proxmox's
resize on a running guest is a qemu block_resize, so the capacity reaches
the guest while it runs. The new step rescans, growparts and grows the
filesystem with the tool its type actually needs — ext2/3/4, xfs, btrfs —
and fails loudly on one it does not know rather than handing it to resize2fs
and hoping. Idempotent end to end: NOCHANGE from growpart is not a failure,
and every tool here exits 0 when there is nothing left to grow.
The same step now runs in the plan-change pipeline, where an upgrade used to
stop one step short of the customer.
A downgrade blocked by data gets a way out. The block stays — it is correct
— but the check now measures against the target package PLUS the packs the
customer already owns, and reports the numbers behind the refusal: what is
stored, what would be allowed, how much has to go, and how many packs would
cover it instead. The portal offers both routes: book exactly those packs
(confirmed in a modal, R23), or delete data and have the fill level measured
on demand rather than waiting for the nightly sampler. That reading is taken
by DiskUsageProbe, which CollectInstanceTraffic now uses too, so there is one
notion of "how full is it" and not two.
A plan change keeps booked packs. They were paid for separately and have
nothing to do with which package the customer is on; the new package's disk
and quota are sized with them included.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three places where the product said one thing and did another.
**Nothing could restart a virtual machine.** ProxmoxClient had startVm and no
way to stop anything, so `restart_required_since` — set by every plan change
that grows a running guest — could only ever be cleared by a resize step that
happened to find the machine stopped, which nothing in the product could
arrange. A paid upgrade's cores and RAM could stay unreached for the life of the
contract, and the cloud page's "Neu starten" was a toast.
There is now a `restart` pipeline: shutdown, start, wait for the agent, confirm.
A shutdown and a start rather than a reboot, because only a cold boot makes qemu
read the VM definition again — a reset would take the machine round and bring it
back exactly as small as it was. The shutdown is a REQUEST, bounded at ten
minutes, and there is no escalation: ProxmoxClient deliberately does not expose
Proxmox's forceStop, because the guest is a Nextcloud and cutting power to a
database mid-write to apply a CPU change trades an inconvenience for a restore
from backup. A guest that ignores ACPI for ten minutes fails the run instead, and
a person decides what happens to it.
`restart_required_since` is cleared by the last step, from what the RUNNING guest
reports (`cpus`/`maxmem`) — not by the button, and not by the run merely
finishing. A machine that comes back smaller than it was sold fails the run.
Started by App\Actions\RestartInstance, which re-checks authorisation itself
rather than trusting a hidden button: an operator holding the new
`instances.restart` may restart anything, a portal user their own machine and
nothing else. Refused while another run is in flight against the same order.
Confirmed in a modal on both sides (R23), addressed by uuid (R11).
**Existing instances had no storage quota.** ApplyStorageQuota reaches new and
changed machines; everything built before it kept the whole disk whatever it had
paid for, and nothing recorded which was which. `instances.quota_applied_gb` is
now what the guest was actually told, written by the step only after the occ call
succeeds — so "sold" and "enforced" are separate facts and can be compared.
`clupilot:apply-quotas` sweeps the estate: one `quota` run per instance through
the same step the pipelines use, skipping what it should and saying why, a no-op
once an allowance is enforced, and inert under --dry-run. Deliberately NOT
scheduled — a nightly sweep would quietly cover for the pipeline step the day it
stopped working, which is how the original hole survived this long.
**A Traefik router was never torn down.** TraefikWriter::remove() had no caller
anywhere. What was missing was the moment: ConfirmCancelPackage wrote a date into
`service_ends_at` and nothing ever went back to it, so every route this platform
ever wrote was written for good — pointing at a guest address the host is free to
reassign. `clupilot:end-due-services` (hourly) keeps that appointment through
App\Actions\EndInstanceService. "Ended" means cancellation_scheduled AND
`service_ends_at` passed, never the day a cancellation is scheduled: that
customer has paid to the end of the term and is working in it. The DNS record
goes with the route — it is in our zone, pointing at a host that serves other
people, which is the shape of every subdomain takeover there has ever been. The
customer's own domain is in their zone and is not ours to touch; the virtual
machine is left alone, because the cancellation flow promises a data export first
and destroying disks is not this change's decision.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two ways of charging a customer for something they did not get.
A downgrade was due when `subscriptions.current_period_end` said so, and
Stripe pushes that column forward on every renewal. So a downgrade booked in
March was deferred to the end of April by April's renewal, and to the end of
May by May's — each time by a whole term, and each time the customer was
billed again for the package they had asked to leave. The due date was being
re-derived from a moving target on every scheduler tick.
It is decided once now, at the moment the customer decides, and stamped onto
the contract: `pending_plan` and `pending_effective_at`, the two columns that
have been in the schema since the first migration and were written by nothing
at all. The shop stamps them, PlanChange reads them instead of the period end
when a move is already booked, and clupilot:apply-due-plan-changes finds
contracts by the stamped date rather than orders by a date it recomputes.
The order stays what it always was — the customer's own record of the request,
what the cart shows them and what they remove to change their mind — so it is
still consumed with the change, and removing it unbooks the change. What it no
longer does is decide when. A second downgrade replaces the first rather than
queueing behind it; an upgrade clears one, or it would come due months later
and undo the bigger package; a contract cancelled before the date takes the
booking with it; and a customer with no contract can no longer place a
downgrade at all, which was accepted before and could never be carried out.
A booked change is now visible where the contract is read: on the plan card in
the portal, because the cart entry disappears when it is paid for and the
booking does not, and on the customer row in the console, because that is
where an operator answers "what is this customer on".
The second defect is the same money in one step: BookAddon guarded duplicates
per ORDER — the unique index on (order_id, addon_key) — so two orders for the
same module on one contract both went through. A stale tab or a double click
bought priority support twice.
But storage is deliberately sold in packs, and AddonCatalogue sums the
quantities for exactly that reason, so "refuse the second" is only right for
some of them. Which is which is declared beside each price in
config/provisioning.php (`sold_as`), because it is a commercial decision about
each module and not something to infer from a key: storage is a quantity;
off-site backups are on or off, support is prioritised or it is not, Collabora
Pro is one licence, and a machine answers to one own domain. An undeclared
module counts as an entitlement — the cheaper of the two mistakes — and a test
refuses to let one ship undeclared.
Enforced in the action, where the portal, the console's grant screen and a
webhook all pass, with the sentence the customer is shown rather than a
developer's. A retried webhook for the SAME order still gets its one booking
back; that is one purchase arriving twice, not two purchases. The portal stops
offering what would be refused — booked, or already waiting in the cart — and
goes on offering the pack.
What is missing is on the Stripe side and is not invented here: nothing moves
the subscription's item onto the new Price, so the renewal after a plan change
still bills the old plan at Stripe. StripeClient only creates and archives
catalogue objects, and swapping a price also needs the subscription ITEM id,
which nothing stores.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PlanChange could say what a move would cost and whether it was allowed, and
that was all it could do. Billing::purchase() wrote an upgrade order and
nothing ever consumed it: same snapshot, same machine, same quota. A customer
could pay for a bigger package and receive nothing.
ApplyPlanChange is now the single place a change lands — it moves the contract
onto the target's current version, writes one register row, settles the custom
domain, and starts a run that resizes the machine. Applying the same order
twice is a no-op, enforced by a unique event key rather than by a check two
callers could both pass.
Two things a plan change must not do, and now does not. A disk is never shrunk
— Proxmox cannot, so the QUOTA shrinks instead, which is what was sold anyway.
And a live machine is never rebooted as a side effect: cores and RAM are
written, and where they need a restart the instance says so where an operator
and the customer can both see it.
The storage allowance also joins the build pipeline. ApplyStorageQuota was
written for plan changes and ran only there, so a brand-new customer still got
no quota at all — quota_gb reached the instance row and stopped, and every
package delivered the whole disk. The test guarding that pipeline only counted
its steps, which is how a list missing the one step that makes a package's
storage real stayed green for its whole life. It now names the step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
── The hole ────────────────────────────────────────────────────────────────
There was no password reset. Fortify's feature was commented out, so there
was no link on the sign-in form, no page and no route — a customer who forgot
their password was locked out of their own cloud until somebody opened a
shell. On a product whose selling point is that you can ring somebody, that
is a support call a week and an embarrassing one.
Two pages of ours under Fortify's route names (R1/R2), because with views off
Fortify registers only the POST endpoints. The mail is a Mailable in this
product's design rather than the framework's MailMessage: somebody who has
just been locked out is exactly the person a phishing mail is aimed at, and a
message that looks nothing like the rest of our post is one they cannot check
— ours carries the footer that names our domains.
Three decisions worth stating, each with a test:
The answer is identical whether the address is known or not. "No account with
that address" turns the form into a way of finding out who is a customer.
The reset kills every other session. Whoever knew the old password may still
be signed in somewhere, and a reset that leaves them there has fixed nothing.
It does NOT sign the visitor in. The link arrived by email, and a mailbox
somebody else can read would otherwise be a session somebody else gets.
── /sicherheit → /security ─────────────────────────────────────────────────
R13: paths are English. Mine was not. The old path stays as a permanent
redirect because it has already gone out in mail footers.
── One wordmark ────────────────────────────────────────────────────────────
"Sometimes it says Cloud and sometimes it does not" — correct. It was on the
footer, the placeholder and the maintenance screen, and absent from the
header, the sidebar and the sign-in plate. The product is called CluPilot
Cloud; that is the name on the invoices and in every mail, so it is now the
name everywhere. The console sidebar takes the small size so the lockup, the
ADMIN badge and the close button still share one line (R18).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A verified custom domain was reported as the customer's address by
Instance::address(), by the portal and by the credentials mail while
nothing on the platform routed it: the Traefik router's rule was
hard-coded to {subdomain}.{zone}. The address a customer was handed
answered nothing, and a withdrawn domain stayed in Nextcloud's
trusted_domains forever, because the only thing that ever wrote either
was the initial provisioning run.
- TraefikWriter::write() takes a LIST of hostnames under one stable
router name — the platform address always, the verified domain as
well when there is one. One file per instance, so a withdrawal is a
rewrite rather than a second thing somebody has to remember.
- ConfigureDnsAndTls records the custom domain's certificate instead of
waiting on it: it depends on an A record in the customer's own zone,
which may never appear, and must never fail a run. The platform
address keeps its 840s deadline. instances.domain_cert_ok tells
"proven" apart from "answering", and the portal now says which.
- ConfigureNextcloud deletes trusted_domains 2 when there is no verified
domain, so a withdrawn one stops being trusted.
- New `address` pipeline (those two steps) plus ReapplyInstanceAddress,
which starts one against the order and refuses to start a second while
any run is in flight. The route is rewritten when the hostname list
differs from what the router carries — not on route_written, which
would short-circuit exactly the case a re-apply exists for.
- Triggered where the address changes: verification flipping either way
in clupilot:verify-domains, the customer's own domain page, and
CustomDomainAccess::deactivate() on a package downgrade.
- A maintenance run no longer condemns its subject: an address run that
failed used to mark the order failed and release the live instance
with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
── The zone ────────────────────────────────────────────────────────────────
A fresh install now addresses customer instances under clupilot.cloud rather
than clupilot.com. Not branding: a Nextcloud is third-party software that
strangers sign in to, and on the same registrable domain as the portal it
shares cookie scope, document.domain and CAA with it. The same reason
googleusercontent.com and vercel.app exist.
Existing installations keep whatever the console says — the zone has been a
setting all along. `clupilot:check-zone <zone>` reports what a change would
touch before anything is touched: the DNS records, the monitoring targets,
the certificates and Nextcloud's trusted_domains all carry the old name and
the setting reaches none of them. With no instances in service it says so and
stops. Changing it later is a migration with an outage, and the command says
that too.
── Which addresses are ours ────────────────────────────────────────────────
A new public page at /sicherheit: the domains that belong to us, the rule for
reading an address bar, what we never do, and — the part that matters — what
to do if you have already typed your password into a copy of our sign-in
form. Change it, end every other session, change it wherever else you use it,
tell us. In that order.
Reachable without an account, deliberately: somebody who has just given their
password away is signed in nowhere, and the page they need cannot be behind
the thing they lost.
Linked from the three places a person is when the question comes up: every
mail footer (the mail is the vector), the sign-in form (this is the page a
phishing kit copies, so the real one says which host you are on), and the
site footer.
The domain list is DERIVED from configuration and keyed by what each domain
is for. A list typed into a page would eventually tell a customer that a
phishing domain is one of ours, or that one of ours is not — the same
two-sources-of-truth failure that had the price sheet promising an address on
a domain the company does not own. The operator console is deliberately
absent from it: naming it on a public page tells an attacker where to aim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
SITE_HOST takes a comma-separated list now, the first name canonical. It serves
the site; every other name in the list redirects there permanently, path and
query intact. An apex domain and its www are two names for one thing, and
answering on both without picking one splits search rankings and cookies
between them.
The redirect is a host-bound catch-all rather than middleware. Middleware that
only some routes carry is the same half-measure this file already made once,
and a catch-all registered without a host would swallow every path in the
application — which is what the third test is there to stop.
The test harness needed fixing too, and the fix is the finding. Router::dispatch
on a standalone router does not rebind the container's request, so a route
reading one — through the helper OR through an injected parameter, both of which
resolve from the container — gets the test's request rather than the one being
answered. The redirect silently lost its query string in the test while working
in production. dispatchOn() binds it where the framework would.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous release bound the landing page and stopped there. The website duly
vanished from the portal — and the portal stayed reachable from the website:
www.clupilot.com/dashboard, /login, /settings all kept working. Half a
separation is no separation, and it was reported back within the hour.
APP_HOST binds every portal route, Fortify's own POST actions included through
fortify.domain — a sign-in form on the website's hostname would otherwise post
to a route that answers there. SITE_HOST now takes the legal pages and
robots.txt with the landing page rather than leaving them everywhere; binding
them is also what makes route('legal.impressum') produce a www URL from inside
a queued mail, where there is no request to take a hostname from.
There is no clever middleware here and there should not be. Route::domain() is
the mechanism: a route registered for one host does not exist on another, and a
request for it gets the 404 it deserves. The only deliberate exception is "/" on
the portal host, which redirects to the dashboard or the sign-in page — that is
what people type from memory, and turning it into an error page to make a point
about hostnames helps nobody. The Stripe webhook and /up stay host-agnostic:
Stripe posts to whichever URL it was given, and a mismatch there loses payments.
Both are opt-in and empty by default, which every development machine reached
by a bare IP depends on.
The tests now check BOTH directions for every path rather than only the one
that was reported. That is the mistake this commit exists to correct.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The landing page was registered host-agnostically, so app.clupilot.com served
it: marketing copy, a pricing table and a "sign up" call to action, at the
address where a customer's servers live. Reported exactly that way.
SITE_HOST binds the website to its own hostname. Every other host — the portal,
a bare IP — answers "/" with the product: the dashboard for somebody signed in,
the sign-in page for everybody else. Never a 404 there: "/" is what people type
from memory, and turning that into an error page to make a point about
hostnames helps nobody. Empty keeps today's behaviour, which every development
machine reached by IP depends on.
The console is unaffected — its routes are registered first and win on their
own host, the ordering the status page above already documents.
The legal pages are deliberately not bound. An imprint has to be reachable from
wherever the reader is standing, and a mail footer links to it from whichever
host sent the mail.
Also drops "just reply to this message" from two mails. A reply lands in the
billing or provisioning mailbox rather than in the support queue somebody
actually works through — so it now says to raise it in the portal, where it
gets a place in that queue instead of getting lost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Registration opened an account that could sign in immediately on an address
nobody had proved was theirs — and anybody can type a stranger's into a
registration form. Both halves were commented out: the Fortify feature and
MustVerifyEmail on the model.
`verified` sits on the whole portal group rather than on the pages that spend
money. An unconfirmed address is not a billing problem to be caught at
checkout; it is an account that may not belong to the person holding it, and
the servers, the users and the backups behind it are worth as much to whoever
typed the address as to whoever owns it.
The mail is a Mailable in this product's design rather than Laravel's
notification, which would arrive with the framework's logo, button and footer
as the first thing anybody ever receives from CluPilot. It also has to leave
through the SYSTEM mailbox like every other account mail, which the
notification path does not do on its own.
The link is signed and expires, and the address is part of what is signed — so
a link issued to a mistyped address stops working the moment the address is
corrected, rather than confirming an address nobody can read. Tests cover the
tampered link, the expired one, the corrected address, and one account trying
to confirm another.
The waiting page is ours (Fortify has views off) and offers the only two things
that move somebody forward: send it again, or sign out and register with the
address they meant. The resend is throttled, because otherwise it is a button
that makes the server send mail to an arbitrary address as fast as it can be
clicked. Somebody already confirmed is redirected away from it — the `verified`
middleware only guards the other direction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five purpose-named mailers (cp_maintenance, cp_provisioning, cp_support,
cp_billing, cp_system) land in config/mail.php as plain static entries —
no query runs to build them. Mail::extend('mailbox', ...) in
AppServiceProvider registers a transport that looks up its mailbox
through MailboxResolver only once a mailer is actually resolved, which
for queued mail is inside the worker at send time, never at boot.
Mutation-tested the brief's own three tests by deleting the MAIL_MAILER
guard from MailboxTransport::delegate(): all three stayed green. They
only exercise __toString()/describe(), a code path separate from the one
that actually sends. Added two tests that reach into delegate() itself
(a bound closure, since it's private) and assert the transport it
actually builds: LogTransport while MAIL_MAILER=log, EsmtpTransport once
it isn't. Re-ran the same mutation against the strengthened suite: it
now fails (EsmtpTransport where LogTransport was expected), then passes
again once reverted.
Verified two things against the installed versions before writing this:
LogTransport takes a Psr\Log\LoggerInterface and Log::channel(null)
resolves to the default channel rather than throwing, and
Mail::extend('mailbox', ...) is read by
MailManager::createSymfonyTransport() via $config['transport'] before it
would fall back to createSmtpTransport() — both matched the brief
exactly, no changes needed there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console announced an update for "spätestens um 15:21" while the clock on
the wall said 17:21. Not a formatting slip in one string: fourteen views and
four components printed database values straight out, so every absolute time in
the product was two hours out all summer and one hour out all winter.
Two of those views looked as though they had handled it. They called
->timezone(config('app.timezone'))
which reads like "convert to local time" and, app.timezone being the STORAGE
zone that must stay UTC, converts to UTC — a no-op wearing the costume of a
fix, which is worse than no call at all because it stops the next person
looking. That exact string is now banned by test.
Worse than the labels were two FORMS. Maintenance windows and plan versions
filled their datetime-local fields from UTC and parsed what came back as UTC. A
datetime-local field carries no offset — it is the digits a person reads off
their own clock — so an operator typing 21:00 scheduled a window for 23:00
their time, and nothing anywhere said so. App\Support\LocalTime now holds
toField() and fromField() side by side, because getting one end right is not
half a fix, it is a fresh bug.
The mechanism is one Carbon macro, ->local(). It copies before converting:
Illuminate\Support\Carbon is mutable, so without that, rendering a timestamp
would rewrite the model attribute as a side effect and anything comparing or
saving it afterwards would be an hour or two out. Both Carbon classes get the
same body — Carbon keeps macros in one global table, so the second registration
replaces the first for every Carbon class, and two different bodies meant the
immutable version, safe for itself, silently became the mutable one's
implementation too. My own test caught that.
The existing tests had not caught any of this because they built their expected
values with the same wrong call the views used — a test that recomputes the
implementation asserts nothing. They now assert the wall clock, and the queued
update additionally asserts that the UTC time is NOT shown.
Recorded as R19 in CLAUDE.md and enforced by tests/Feature/DisplayTimezoneTest:
no Blade or Livewire component may format an absolute time without ->local(),
the UTC no-op is banned, storage stays UTC, both sides of the DST boundary are
checked, ->local() must not mutate, and a form field must round-trip to the
same instant.
APP_DISPLAY_TIMEZONE, default Europe/Vienna. 615 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The owner's phone shows a blank page on the VPN. That page is the reverse proxy
refusing the request, and it refuses because the phone is not on the tunnel: the
client config routes only the management subnet, so a request to the console's
public hostname leaves over the mobile network and arrives from a carrier
address. It is also why the peer reads "last contact: never" — WireGuard only
handshakes when it has traffic to send, and nothing is ever routed in.
The obvious fixes are both wrong. Adding the server's public address to
AllowedIPs routes the WireGuard endpoint into the tunnel it is trying to
establish. A public DNS record pointing at 10.66.0.1 publishes the internal
subnet to anyone enumerating the domain — the owner raised that himself, and he
was right.
So the console is served INSIDE the tunnel instead. A small resolver and a small
gateway share the hub container's network namespace, which is where wg0 lives,
and answer on 10.66.0.1 directly. A client therefore needs no route beyond the
subnet it already has, the host's docker bridge is never exposed, and the
request reaches the application with its real 10.66.0.x source — which is what
the console's own allowlist checks. The gateway reuses the certificate the
public Caddy already renews: a certificate is bound to the name, not to the
address serving it, so nothing new is issued and nothing internal reaches a
public zone.
Most of this commit is the arithmetic of not lying about it. Review found
fourteen ways the arrangement could report itself working while it was not:
enabling the compose profile after the services were started rather than before;
starting a gateway with empty certificate paths, which can only crash; treating
"container created" as "container running"; carrying a stale readiness marker
through a failed restart; reusing the previous hostname's certificate after a
rename; leaving the profile enabled when the hostname was cleared, in a script
that exits early precisely when nothing else would fix it; reading the renewal
timestamp as a user who cannot traverse Caddy's storage; and a `find | head`
that aborts the whole installer under pipefail the moment two issuers hold a
certificate for the same name.
One of them was mine and worth naming: to let the container read the
certificate, I had made the TLS private key world-readable. On a multi-user host
that hands the console's identity to every local account.
The client is told about the resolver only when both services are confirmed
running, because a config naming a resolver that does not exist takes the
device's entire name resolution with it for as long as the tunnel is up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Changing a key meant a shell, a file edit and a cache rebuild — and the person
who owns the Stripe account is not necessarily the person who owns the server.
Two gates, not one. The capability decides who may open the page; every operator
has console.view, and that must not mean "can read the payment key". The
password decides whether this SESSION may see or change anything, because the
realistic threat is not a stranger but an unlocked machine, and a session is
exactly what that hands over. Both are re-checked server-side on every action —
a Livewire action is reachable by anyone who can post to /livewire/update.
The value is stored encrypted under a key of its own, SECRETS_KEY, and the vault
refuses to work without it rather than falling back to APP_KEY: rotating APP_KEY
is ordinary maintenance and would otherwise destroy every stored credential,
discovered when Stripe stops answering. It is read where it is used, not
overlaid onto config at boot — an overlay adds a query to every request
including the public site, and leaves queue workers holding whatever was true
when they started. It is never shown again, only outlined, and never enters a
Livewire property that would carry it to the browser and back in the snapshot.
A registry, not an env editor: a form that can set any environment variable is a
privilege-escalation primitive, and one bad value bricks the installation with
no way back through that same form.
The test button is the part that matters. It reports which Stripe account the
key belongs to, whether it is LIVE or test — the most expensive mistake here is
pasting one where the other belongs, and both look identical in a form — and
which webhook endpoints exist with the events each subscribes to. A key can be
perfectly valid while the endpoint listens for the wrong five events, and
nothing fails until a payment goes unrecorded.
The webhook signing secret deliberately stays in .env. It is read on every
incoming payment event; in the database, a database problem becomes silently
failing signature checks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
STATUS_HOST binds /status to a single hostname. Every other host redirects
there rather than 404ing: a status page is the one address people keep in a
bookmark and reach for when something is already wrong, so breaking old links
to prove a point about separation would be exactly backwards. Unset, nothing
changes.
The /legal/status redirect goes through route() instead of a literal path. A
relative redirect lands on whatever host the visitor is already on — which,
once the page has a hostname of its own, is precisely where it no longer
answers.
The installer told the operator to point Stripe's webhook at the customer
portal. Stripe posts server-to-server and never sees the portal; the endpoint
is the api hostname, which the installer never even asked for. It asks now, and
prints the address that actually receives the events. Getting this wrong does
not fail loudly — payments simply stop being recorded.
It also writes ADMIN_HOST_EXCLUSIVE=false on a fresh install. Switching the
console onto its own hostname before the DNS for it exists makes the console
unreachable under any other name, and that is not a thing to have on by default.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three gaps, all of the "half-built" kind.
Nobody could change a password. Fortify's updatePasswords feature was switched
off, so an account created with a generated password kept it until someone
opened a shell on the server — including the owner's own. Both settings pages
have the form now, sharing one concern, because two copies of a password rule is
how one of them ends up weaker. It goes through Fortify's own action rather than
hashing here, and the current password is required: an unlocked machine should
not be two keystrokes away from locking its owner out.
The datacenter edit form had no active switch. The column and the scope existed,
and the list page has a toggle — but the form you open to change the thing did
not offer the one lifecycle action a location actually needs. Switching one off
now says what it does NOT do: existing hosts keep running. Once, on the actual
transition, and as the only message — the generic "saved" toast would otherwise
replace it, and an already-inactive location would have announced its own
deactivation every time its name was edited.
The staff table's actions cell is empty for your own row, because you cannot
revoke yourself. With a one-person team the column was therefore always blank
and read as broken rather than as a rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things the owner asked for, and one the review found underneath them.
The console leaves /admin. It has its own route file now, registered before
routes/web.php because once it has a hostname to itself it sits at the ROOT of
that host — where `/` and `/settings` also exist for the portal, and the first
matching route wins. Where it mounts is one decision in one place: `/` on the
console hostname where it has one to itself, `/admin` on any host otherwise.
Names stay admin.* in both modes, so nothing that builds a URL has to know.
Exclusivity is its own switch, not a consequence of having a hostname. The
first attempt made "this host is the console" follow from ADMIN_HOSTS, and a
development machine lists its own IP there so the console works without DNS —
which took the public site and the portal off that machine entirely. The
switch also registers the console on every listed hostname, canonical last, so
the alternates that exist as recovery paths keep working.
The status page moves out of /legal, where it sat beside the imprint and the
terms. Nothing about the current health of the platform is a legal document. It
is a real page now: portal, instances, provisioning and backups, each derived
from records, aggregate only, and a component with no signal reports "unknown"
rather than "operational".
That last rule is what exposed the real bug. monitoring_targets.status was
written once — 'up', at provisioning — and nothing ever updated it. Both the
console's notices and the new public page read it, so an outage would have been
published as healthy indefinitely. There is a sync job now, on a five-minute
schedule, and a checked_at column so a verdict can go stale instead of standing
forever.
The monitoring contract had to grow a third state for that to be honest.
isHealthy() answers true when no monitoring is configured and false when the
monitor is unreachable; recording that boolean would have published either a
fleet-wide all-clear nobody measured or a fleet-wide outage that was really one
broken monitor. health() returns null for both, the recorder leaves the old
verdict to go stale, and isHealthy() keeps its forgiving semantics for the one
caller that wants them — the provisioning acceptance check, which must not fail
a delivery because monitoring is not set up.
Backups are counted from the instances that need protecting rather than from
the backup rows that exist, so an instance with no schedule at all cannot be
missing from the arithmetic that declares the estate protected.
Also: the update panel polls itself, offers the button only when there is
something to install, and opens the log while a run is in progress.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
VERSION at the repository root is the release number and the only place it is
written down — a file rather than `git describe`, because a source archive, a
shallow CI checkout and a container built without .git have no tags to
describe. Cutting a release is editing that file and merging it; CI creates the
annotated v<VERSION> tag once green, only on the commit that raised the number,
and never moves it. Servers are pinned to these.
Two modes, and the difference is the point. RELEASE=v1.0.0 pins a server to an
immutable tag, checked out detached: it moves when someone decides it moves.
BRANCH=main follows the edge, where CI tags a commit only after it is pushed.
The mode is remembered, so re-running the updater on a pinned box does not
quietly walk it back onto main.
Going backwards is refused, by commit ancestry rather than by comparing version
strings — a tag can be cut from anywhere, and only ancestry says whether history
is going back. The schema has already moved forward by then, and the migrations
needed to reverse it are not in the older checkout at all.
What the console reports comes from a manifest written atomically after every
step succeeded, never from live git: git says what the files are, not whether
the deployment came up. So a failed update keeps reporting the version that is
actually serving — including its version number, not the newer one the checkout
has already moved to. A deployment pinned to the tag reads "1.0.0 (abc1234)";
anything else reads "1.0.0-dev (abc1234) · main", because every commit after the
tag still carries VERSION=1.0.0 and is not that release. Reporting it as one is
how a bug gets filed against the wrong code.
Codex reviewed the design before it was written and the result six times after.
Its findings, all real: the version had to come from the manifest too, not just
the commit; branch names need JSON escaping or the manifest silently vanishes; a
`case` glob does not anchor and accepted 1x.2y.3garbage; pinning to the commit a
server already sits on skipped recording the mode, and then skipped detaching;
a manifest that failed to write would never be repaired; an unparseable
timestamp would 500 every admin page; and an empty tag list read as a
connectivity failure.
466 tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Product per plan family and a Price per priced row of a published version,
plus the four subscribed webhook events that were arriving and being ignored.
`stripe:sync-catalogue` mirrors the catalogue. Idempotent twice over: a stored
id is skipped, and each call carries an idempotency key derived from our own
row, so a crash between Stripe creating a Price and us recording its id gives
back the same object rather than a second one. That matters more here than
usual — a Stripe Price cannot be edited or deleted, so a duplicate is
permanent. Drafts are not synced at all: a version that has promised nothing
has no business in a price list. --dry-run shows what would be created.
The webhook now handles invoice.paid, invoice.payment_failed,
customer.subscription.updated and customer.subscription.deleted. It never
re-derives an amount: the invoice is the authority for what was charged, and
recomputing it from our catalogue would produce a second, disagreeing answer.
A failed payment is recorded and nothing else — Stripe runs the dunning
schedule, and cutting a customer off on the first failure would punish an
expired card as though it were a refusal to pay.
Only a cycle renewal moves the term on. Stripe also sends paid invoices for
prorations and manual charges, and the checkout's own invoice — which is
already in the register as the purchase, and would otherwise double every
customer's first payment.
Stripe does not guarantee delivery order, which is the source of most of the
care here:
- state changes are judged and written in one transaction with the row held,
so two deliveries cannot both pass a check made on a stale copy;
- an older event never overwrites a newer one, with a rank breaking ties
between events sharing a second — their timestamps have one-second
resolution, and a failed attempt and its successful retry routinely do;
- a term is never shortened, and an ended contract is never revived;
- an event for a contract that does not exist yet is HELD and replayed the
moment it appears, rather than acknowledged and forgotten. A cancellation
dropped that way would leave us serving someone who had already left. Held
rows that never match are pruned after a week.
Register entries are keyed by what they are about — the invoice, the attempt,
the subscription — with a unique index doing the work, because a check two
concurrent deliveries can both pass is not idempotency.
PlanChange is now documented as the preview shown before someone confirms, not
the invoice: Stripe's proration accounts for tax, existing credit and the exact
second the change lands, and ours cannot.
Not run against the live account — `stripe:sync-catalogue` creates objects that
cannot be deleted, so that is the owner's call. The dry run lists 12.
457 tests green. Codex review clean after seven rounds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Plans lived in config/provisioning.php, where the owner cannot reach them and
where a plan has no history. They now live in plan_families / plan_versions /
plan_prices — a name that never moves, what the plan WAS at a point in time,
and a price per version and term with its own Stripe Price id.
- Availability is computed on every read (available_from <= now < until,
half-open, UTC) plus a per-family sales kill switch. Nothing schedules a plan
into or out of sale: a job that fails to run is a plan that silently
misbehaves.
- Overlaps crash rather than resolve. currentVersion() uses sole(), and
scheduling takes a lock on the family and rejects an overlapping window —
two versions on sale at once would decide a customer's price by row order.
- A version is frozen from publication, not from first sale, and so is its
price: a checkout is not instant, and an amount edited between the session
opening and the webhook landing would contract someone at a price they were
never quoted. Repricing publishes a new version, as Stripe requires anyway.
- Neither a published version, its price, nor a family with customers can be
deleted, and a family key cannot be renamed. All of those would null the
provenance off existing contracts.
- Subscriptions and orders record plan_version_id. A historical reference
resolved by plan NAME hands back today's terms, which is the split-brain one
level up. A checkout that carried its version is honoured even after the
window closes — they paid for what they were shown.
Switched atomically and failing closed: config('provisioning.plans') and
plan_features are gone, and nothing falls back to them. A fallback would
resurrect a plan the owner had just switched off. The seed lives in the
migration, and PlanCatalogueTest pins what the config catalogue sold as the
shadow comparison. `php artisan plans:check` reports overlaps, gaps and
missing prices before a customer finds them.
Contract-backed displays, which were reading the live catalogue:
seat limits, the cloud card, the plan card, and admin MRR — the last three now
divide a yearly contract down via Subscription::monthlyPriceCents(), since all
three label the figure per month.
Two recovery gaps closed along the way: the order now commits before the
contract is opened (so a contract failure can never erase the record of a
payment), a Stripe retry repairs a missing contract and restarts a run stranded
with no_subscription, and TickProvisioning sweeps runs left pending by a crash
rather than only running/waiting ones.
402 tests green. Codex review clean after thirteen rounds. Verified in the
browser: portal, billing and console render unchanged off the new catalogue.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pipeline re-resolved config('provisioning.plans') by order.plan, so the
subscription snapshot protected a customer's price but not their machine:
shrinking a plan resized an existing customer's VM on its next run. Nothing
created a subscription either, so closing this meant opening the contract at
purchase and pointing provisioning at it.
- OpenSubscription freezes the catalogue onto a subscription when a checkout
is paid; StartCustomerProvisioning calls it inside the order transaction.
- CustomerStep::plan() reads the frozen snapshot. ValidateOrder and
ReserveResources fail closed with no_subscription rather than falling back
to the catalogue, which is the bug itself.
- template_vmid joins the snapshot so a re-clone cannot pick up a blueprint
published after the sale. Deliberately outside FROZEN: it is how we build
the machine, not a term the customer is owed, and a dead template must be
replaceable without cancelling a contract.
- TrafficMeter reads the allowance off the contract too — cutting a plan's
traffic was otherwise enough to start throttling someone who bought more.
- The migration backfills contracts for orders that already bought something,
reconstructed from what was actually delivered where an instance exists,
and adopts an existing order-less contract instead of opening a second.
Orders paid in a currency the catalogue cannot price get none, matching the
checkout path.
price_cents stays the catalogue's NET price, which is what PlanChange
prorates against — not Order::amount_cents, which holds Stripe's GROSS total.
Reconciling the two belongs to the proof register and Stripe (phases 4/5).
Also pins STRIPE_WEBHOOK_SECRET blank in phpunit.xml: the operator's real
secret was reaching the suite from .env and rejecting every unsigned test
payload, which is why 7 webhook tests failed before any of this.
Verified in the browser: with team traffic cut from 3000 to 500 GB in the
catalogue, the customer's portal still shows 3 TB.
373 tests green. Codex review clean after three rounds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With grandfathered prices the comparison inverts: a business plan bought when
it cost less than today's team plan would have treated a move to team as an
upgrade — charged immediately, while the customer loses resources. The plan's
rank is frozen with the rest of the snapshot and decides the direction; prices
only decide the amount.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex was right that this could misstate real charges: an EU business with a
VAT ID registered in another country is billed under reverse charge, and we
were adding 20 % Austrian VAT to their total anyway. TaxTreatment resolves it
from the customer's VAT ID, and the whole page — cart, plan cards, add-on
cards — now states one treatment instead of contradicting itself.
Explicitly NOT handled: cross-border sales to private individuals, which are
taxed at the buyer's national rate under OSS. That needs a maintained rate
table and a tax adviser, not a guess, so those fall back to the domestic rate —
over-collecting rather than under-collecting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two contradicting plan upgrades sat in the cart at 799 and 399 € — no checkout
could resolve which one the customer meant. Choosing another now replaces the
pending one and says so; add-ons still stack, because buying 200 GB as two packs
is a sensible thing to want.
Every price now states net or gross and how often. The cart shows net per line
with "pro Monat" or "einmalig", then subtotal, VAT and gross — and separates
the monthly recurring amount from a one-off traffic top-up sharing the same
cart, because those are two different commitments. The rate is configurable
(CLUPILOT_TAX_PERCENT, 20 % default) since it follows the seller's country.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fsn-01.node.clupilot.com, numbered per datacenter and never reused — removing a
host must not renumber its neighbours onto its name, so the number is stored
rather than derived.
The record points at the host's WireGuard address, not its public IP: the name
exists so an operator can reach a host by name over the VPN, and publishing a
Proxmox host's public address would hand every scanner a target, which is the
one thing this network design avoids.
DNS is convenience, not a prerequisite: a failure logs an event and lets the
onboarding finish rather than stranding a host that is otherwise ready.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Customers can now see what they have used and what is left, and the service
slows down rather than stopping when the allowance is gone — a slow Nextcloud
gets a top-up, a dead one gets a cancellation.
- instance_traffic keeps one row per instance per month. Proxmox counters are
cumulative since the VM last started, so usage is the difference between two
samples, and a counter that went backwards means a restart, not a refund.
- Outbound is what counts: inbound is free at our providers and egress is what
Hetzner's 20 TB per server applies to.
- CollectInstanceTraffic samples every 15 minutes, warns once per threshold
(80/95 %) rather than on every run, and at 100 % limits the VM's NIC via
Proxmox — released again as soon as the customer tops up or the month rolls
over.
- The dashboard gets a full-width band (it throttles the service, so it is not
a tile among tiles) with the top-up offer right next to the warning.
- Bytes are formatted in SI units now: allowances are computed in SI, so
dividing by 1024 made a 1000 GB plan read as "931 GB".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
While the product is still being built, the marketing site and the customer
portal should not be reachable — but they must stay reachable for us.
- A toggle in the console (site.manage, Owner/Admin) stored in a new
app_settings table, because this has to be flippable without a deploy.
- Outsiders get a placeholder with 503 + noindex, not 200: a 200 invites search
engines to index the placeholder as the site's content, which is far harder
to undo than to prevent.
- Anyone on the management VPN, and any signed-in operator, sees the real site.
The console, Livewire's endpoint, the Stripe webhook and the health check are
always reachable — otherwise the switch could only be flipped once.
- robots.txt is generated by the app and follows the switch. It had to stop
being a static file: nginx short-circuited it and Laravel's stock file said
'Disallow:' with an empty value, so crawlers were never told anything.
- Settings reads fall back to the default when the table is unavailable. The
gate reads one on every request, so a deploy running new code before migrate
would otherwise answer the entire site with a 500.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reworked after a design consultation with Codex, which pushed back on my first
proposal in three useful ways.
Ownership and rights:
- vpn.manage split into vpn.view.all and vpn.manage.all. Seeing an access is
not managing it, and neither is holding its private key.
- Record-level rules live in VpnPeerPolicy, not in permissions: an access
belongs to a person, and ownership is what grants sight of it. Every operator
reaches the page, but sees only their own unless they may see all.
- Issuing an access is not self-service — it reaches the management network, so
it needs vpn.manage.all even for oneself.
- New Developer role: sees everything, manages nothing. Writing code does not
imply authority over other people's access.
- kind (staff|host|system) replaces a null user_id that had to mean two
different things at once.
Config storage, opt-in per access:
- Downloading is owner-only — explicitly NOT for view.all or manage.all. An
admin who needs access revokes this one and issues their own, which keeps the
record of who holds what honest.
- The password is asked on EVERY retrieval, rate-limited. Laravel's
password.confirm keeps a 15-minute session stamp, which would authorise
unlimited later downloads from an unattended browser.
- Stored under VPN_CONFIG_KEY, not APP_KEY: a leaked application key must not
also hand over the management network. Purged on revocation and when a staff
member is revoked — console taken away, tunnel left open was the worst case.
- The plaintext never enters a Livewire snapshot (the page polls every five
seconds); the component carries an opaque handle instead.
Also: copy button works without HTTPS again (navigator.clipboard only exists in
a secure context, so over http it silently did nothing), plus config download
as a file and a QR code for the mobile apps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DNS names are case-insensitive and Symfony's getHost() always returns lowercase,
so ADMIN_HOSTS=Admin.Example.com rejected every request and locked operators
out. Normalised in the config AND at the comparison, so the value is safe
whichever route it took in (env, cached config, runtime set).
operator()/admin() moved from RbacTest/HostManagementTest to tests/Pest.php:
they were only loaded when those files happened to be in the run, so a targeted
single-file run died on 'undefined function operator()'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Proxmox fleet and the operator console must never be publicly reachable. The
primary control is the reverse proxy, but nginx here is a catch-all
(server_name _), so /admin was served on EVERY hostname — a proxy
misconfiguration would expose it. ADMIN_HOSTS pins it; any other host gets 404
(not 403: a public domain must not disclose that a console exists).
Prepended to the group instead of the admin route group on purpose: route
middleware is reordered by Laravel's priority list, which runs first — a
guest would then be redirected to /login and learn the console is there. Covered
by a test for exactly that case. Empty ADMIN_HOSTS = unrestricted, so nobody is
locked out by upgrading.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kuma has no write REST API (monitor CRUD is Socket.IO only), so docker/kuma-bridge
translates the exact generic REST contract HttpMonitoringClient already speaks —
no PHP client change needed. Opt-in compose profile 'monitoring'.
Verified end-to-end against a real Uptime Kuma 1.x:
create -> id, create again -> same id (idempotent), list, status, delete
and PHP MonitoringClient -> bridge -> Kuma round-trip.
The e2e run caught a real bug: add_monitor takes 'maxretries', not 'retries'.
Monitoring is now observability, not a delivery gate:
- RegisterMonitoring retries MONITORING_ATTEMPTS times on an outage, then
continues degraded with a visible 'info' event
- RunAcceptanceChecks no longer fails when monitoring isn't green (a fresh Kuma
monitor legitimately reports 'pending' — confirmed in the e2e run); it records
the gap instead. MONITORING_REQUIRED=true restores strict gating.
- docs/monitoring-uptime-kuma.md documents setup, failure behaviour, alternatives
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- .env.example gains a full, commented CluPilot operations block (Stripe, Hetzner
DNS, Traefik, WireGuard hub, SSH identity, monitoring, prod mail) with a note
that everything is optional for local dev
- config/provisioning.php: CLUPILOT_SSH_{PUBLIC,PRIVATE}_KEY_PATH read the key
from a file (a multi-line PEM cannot live in .env), falling back to inline vars
- documents that Uptime Kuma's REST API is read-only (monitor CRUD is Socket.IO),
so it needs a bridge rather than being drop-in for the generic REST client
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- isOperator()/EnsureAdmin/broadcast fall back to is_admin so a legacy admin is
never locked out by a stale permission cache
- published modal: centered max-w-lg card (dynamic modalWidth class was purged
by Tailwind → full-width)
- datacenter edit moved to a modal (no row-height jump); location is now a
country dropdown (config/countries.php) instead of free text
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 5 operator roles (Owner/Admin/Support/Billing/Read-only) seeded via migration
with a capability catalogue; app checks capabilities via Gate, never role names
- every mutating admin action authorizes server-side (hosts/datacenters/customers/
impersonate/provisioning); is_admin reads migrated to console.view / isOperator()
- admin /settings: own account + Owner-only staff invite/role/revoke with
last-owner, self-role and customer-collision guards (transactional)
- sidebar 'zum Kundenportal' link replaced with Settings
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- customers gain profile + branding + closed_at; instances gain cancellation
fields; branding resolver (NULL -> CluPilot defaults) snapshotted into the
provisioning run context
- cancel package: term-end, irreversible, typed-confirm modal (R5)
- close account: guarded (no active package), typed-confirm modal (R5)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- plans gain seats + performance class; customer views show storage/seats/
performance/features, no raw vCPU/RAM (admin keeps specs)
- billing: scoped wire:target + wire:key so one purchase no longer spins all
- chart island: line-colour->transparent gradient for filled line datasets;
MRR / backups / invoices bar charts converted to gradient line
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CreateCustomerAdmin puts OC_PASS on the docker invocation (survives the &&).
- RegisterBackup creates a real vzdump job via ProxmoxClient::createBackupJob.
- RegisterMonitoring registers via a new MonitoringClient service (interface +
fake + http). Both persist the external id [E] before the breadcrumb, so
acceptance reflects a real registration, not a fabricated row.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed webhook (HMAC verify when secret set), CSRF-exempt route. Paid checkout
creates customer+order+run (stripe_event_id unique => duplicate webhooks start
one run) and dispatches. 5 tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PhpseclibRemoteShell sets a command timeout (default 2000s) so apt
full-upgrade/install don't hit phpseclib's ~10s default and fail.
- Host removal dispatches RemoveWireguardPeer to the provisioning queue, which
runs in the worker that owns wg0 (the web container can't manage WireGuard).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo
Claude