PlanChange could say what a move would cost and whether it was allowed, and
that was all it could do. Billing::purchase() wrote an upgrade order and
nothing ever consumed it: same snapshot, same machine, same quota. A customer
could pay for a bigger package and receive nothing.
ApplyPlanChange is now the single place a change lands — it moves the contract
onto the target's current version, writes one register row, settles the custom
domain, and starts a run that resizes the machine. Applying the same order
twice is a no-op, enforced by a unique event key rather than by a check two
callers could both pass.
Two things a plan change must not do, and now does not. A disk is never shrunk
— Proxmox cannot, so the QUOTA shrinks instead, which is what was sold anyway.
And a live machine is never rebooted as a side effect: cores and RAM are
written, and where they need a restart the instance says so where an operator
and the customer can both see it.
The storage allowance also joins the build pipeline. ApplyStorageQuota was
written for plan changes and ran only there, so a brand-new customer still got
no quota at all — quota_gb reached the instance row and stopped, and every
package delivered the whole disk. The test guarding that pipeline only counted
its steps, which is how a list missing the one step that makes a package's
storage real stayed green for its whole life. It now names the step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A verified custom domain was reported as the customer's address by
Instance::address(), by the portal and by the credentials mail while
nothing on the platform routed it: the Traefik router's rule was
hard-coded to {subdomain}.{zone}. The address a customer was handed
answered nothing, and a withdrawn domain stayed in Nextcloud's
trusted_domains forever, because the only thing that ever wrote either
was the initial provisioning run.
- TraefikWriter::write() takes a LIST of hostnames under one stable
router name — the platform address always, the verified domain as
well when there is one. One file per instance, so a withdrawal is a
rewrite rather than a second thing somebody has to remember.
- ConfigureDnsAndTls records the custom domain's certificate instead of
waiting on it: it depends on an A record in the customer's own zone,
which may never appear, and must never fail a run. The platform
address keeps its 840s deadline. instances.domain_cert_ok tells
"proven" apart from "answering", and the portal now says which.
- ConfigureNextcloud deletes trusted_domains 2 when there is no verified
domain, so a withdrawn one stops being trusted.
- New `address` pipeline (those two steps) plus ReapplyInstanceAddress,
which starts one against the order and refuses to start a second while
any run is in flight. The route is rewritten when the hostname list
differs from what the router carries — not on route_written, which
would short-circuit exactly the case a re-apply exists for.
- Triggered where the address changes: verification flipping either way
in clupilot:verify-domains, the customer's own domain page, and
CustomDomainAccess::deactivate() on a package downgrade.
- A maintenance run no longer condemns its subject: an address run that
failed used to mark the order failed and release the live instance
with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
── The zone ────────────────────────────────────────────────────────────────
A fresh install now addresses customer instances under clupilot.cloud rather
than clupilot.com. Not branding: a Nextcloud is third-party software that
strangers sign in to, and on the same registrable domain as the portal it
shares cookie scope, document.domain and CAA with it. The same reason
googleusercontent.com and vercel.app exist.
Existing installations keep whatever the console says — the zone has been a
setting all along. `clupilot:check-zone <zone>` reports what a change would
touch before anything is touched: the DNS records, the monitoring targets,
the certificates and Nextcloud's trusted_domains all carry the old name and
the setting reaches none of them. With no instances in service it says so and
stops. Changing it later is a migration with an outage, and the command says
that too.
── Which addresses are ours ────────────────────────────────────────────────
A new public page at /sicherheit: the domains that belong to us, the rule for
reading an address bar, what we never do, and — the part that matters — what
to do if you have already typed your password into a copy of our sign-in
form. Change it, end every other session, change it wherever else you use it,
tell us. In that order.
Reachable without an account, deliberately: somebody who has just given their
password away is signed in nowhere, and the page they need cannot be behind
the thing they lost.
Linked from the three places a person is when the question comes up: every
mail footer (the mail is the vector), the sign-in form (this is the page a
phishing kit copies, so the real one says which host you are on), and the
site footer.
The domain list is DERIVED from configuration and keyed by what each domain
is for. A list typed into a page would eventually tell a customer that a
phishing domain is one of ours, or that one of ours is not — the same
two-sources-of-truth failure that had the price sheet promising an address on
a domain the company does not own. The operator console is deliberately
absent from it: naming it on a public page tells an attacker where to aim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The owner's phone shows a blank page on the VPN. That page is the reverse proxy
refusing the request, and it refuses because the phone is not on the tunnel: the
client config routes only the management subnet, so a request to the console's
public hostname leaves over the mobile network and arrives from a carrier
address. It is also why the peer reads "last contact: never" — WireGuard only
handshakes when it has traffic to send, and nothing is ever routed in.
The obvious fixes are both wrong. Adding the server's public address to
AllowedIPs routes the WireGuard endpoint into the tunnel it is trying to
establish. A public DNS record pointing at 10.66.0.1 publishes the internal
subnet to anyone enumerating the domain — the owner raised that himself, and he
was right.
So the console is served INSIDE the tunnel instead. A small resolver and a small
gateway share the hub container's network namespace, which is where wg0 lives,
and answer on 10.66.0.1 directly. A client therefore needs no route beyond the
subnet it already has, the host's docker bridge is never exposed, and the
request reaches the application with its real 10.66.0.x source — which is what
the console's own allowlist checks. The gateway reuses the certificate the
public Caddy already renews: a certificate is bound to the name, not to the
address serving it, so nothing new is issued and nothing internal reaches a
public zone.
Most of this commit is the arithmetic of not lying about it. Review found
fourteen ways the arrangement could report itself working while it was not:
enabling the compose profile after the services were started rather than before;
starting a gateway with empty certificate paths, which can only crash; treating
"container created" as "container running"; carrying a stale readiness marker
through a failed restart; reusing the previous hostname's certificate after a
rename; leaving the profile enabled when the hostname was cleared, in a script
that exits early precisely when nothing else would fix it; reading the renewal
timestamp as a user who cannot traverse Caddy's storage; and a `find | head`
that aborts the whole installer under pipefail the moment two issuers hold a
certificate for the same name.
One of them was mine and worth naming: to let the container read the
certificate, I had made the TLS private key world-readable. On a multi-user host
that hands the console's identity to every local account.
The client is told about the resolver only when both services are confirmed
running, because a config naming a resolver that does not exist takes the
device's entire name resolution with it for as long as the tunnel is up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Plans lived in config/provisioning.php, where the owner cannot reach them and
where a plan has no history. They now live in plan_families / plan_versions /
plan_prices — a name that never moves, what the plan WAS at a point in time,
and a price per version and term with its own Stripe Price id.
- Availability is computed on every read (available_from <= now < until,
half-open, UTC) plus a per-family sales kill switch. Nothing schedules a plan
into or out of sale: a job that fails to run is a plan that silently
misbehaves.
- Overlaps crash rather than resolve. currentVersion() uses sole(), and
scheduling takes a lock on the family and rejects an overlapping window —
two versions on sale at once would decide a customer's price by row order.
- A version is frozen from publication, not from first sale, and so is its
price: a checkout is not instant, and an amount edited between the session
opening and the webhook landing would contract someone at a price they were
never quoted. Repricing publishes a new version, as Stripe requires anyway.
- Neither a published version, its price, nor a family with customers can be
deleted, and a family key cannot be renamed. All of those would null the
provenance off existing contracts.
- Subscriptions and orders record plan_version_id. A historical reference
resolved by plan NAME hands back today's terms, which is the split-brain one
level up. A checkout that carried its version is honoured even after the
window closes — they paid for what they were shown.
Switched atomically and failing closed: config('provisioning.plans') and
plan_features are gone, and nothing falls back to them. A fallback would
resurrect a plan the owner had just switched off. The seed lives in the
migration, and PlanCatalogueTest pins what the config catalogue sold as the
shadow comparison. `php artisan plans:check` reports overlaps, gaps and
missing prices before a customer finds them.
Contract-backed displays, which were reading the live catalogue:
seat limits, the cloud card, the plan card, and admin MRR — the last three now
divide a yearly contract down via Subscription::monthlyPriceCents(), since all
three label the figure per month.
Two recovery gaps closed along the way: the order now commits before the
contract is opened (so a contract failure can never erase the record of a
payment), a Stripe retry repairs a missing contract and restarts a run stranded
with no_subscription, and TickProvisioning sweeps runs left pending by a crash
rather than only running/waiting ones.
402 tests green. Codex review clean after thirteen rounds. Verified in the
browser: portal, billing and console render unchanged off the new catalogue.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pipeline re-resolved config('provisioning.plans') by order.plan, so the
subscription snapshot protected a customer's price but not their machine:
shrinking a plan resized an existing customer's VM on its next run. Nothing
created a subscription either, so closing this meant opening the contract at
purchase and pointing provisioning at it.
- OpenSubscription freezes the catalogue onto a subscription when a checkout
is paid; StartCustomerProvisioning calls it inside the order transaction.
- CustomerStep::plan() reads the frozen snapshot. ValidateOrder and
ReserveResources fail closed with no_subscription rather than falling back
to the catalogue, which is the bug itself.
- template_vmid joins the snapshot so a re-clone cannot pick up a blueprint
published after the sale. Deliberately outside FROZEN: it is how we build
the machine, not a term the customer is owed, and a dead template must be
replaceable without cancelling a contract.
- TrafficMeter reads the allowance off the contract too — cutting a plan's
traffic was otherwise enough to start throttling someone who bought more.
- The migration backfills contracts for orders that already bought something,
reconstructed from what was actually delivered where an instance exists,
and adopts an existing order-less contract instead of opening a second.
Orders paid in a currency the catalogue cannot price get none, matching the
checkout path.
price_cents stays the catalogue's NET price, which is what PlanChange
prorates against — not Order::amount_cents, which holds Stripe's GROSS total.
Reconciling the two belongs to the proof register and Stripe (phases 4/5).
Also pins STRIPE_WEBHOOK_SECRET blank in phpunit.xml: the operator's real
secret was reaching the suite from .env and rejecting every unsigned test
payload, which is why 7 webhook tests failed before any of this.
Verified in the browser: with team traffic cut from 3000 to 500 GB in the
catalogue, the customer's portal still shows 3 TB.
373 tests green. Codex review clean after three rounds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With grandfathered prices the comparison inverts: a business plan bought when
it cost less than today's team plan would have treated a move to team as an
upgrade — charged immediately, while the customer loses resources. The plan's
rank is frozen with the rest of the snapshot and decides the direction; prices
only decide the amount.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex was right that this could misstate real charges: an EU business with a
VAT ID registered in another country is billed under reverse charge, and we
were adding 20 % Austrian VAT to their total anyway. TaxTreatment resolves it
from the customer's VAT ID, and the whole page — cart, plan cards, add-on
cards — now states one treatment instead of contradicting itself.
Explicitly NOT handled: cross-border sales to private individuals, which are
taxed at the buyer's national rate under OSS. That needs a maintained rate
table and a tax adviser, not a guess, so those fall back to the domestic rate —
over-collecting rather than under-collecting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two contradicting plan upgrades sat in the cart at 799 and 399 € — no checkout
could resolve which one the customer meant. Choosing another now replaces the
pending one and says so; add-ons still stack, because buying 200 GB as two packs
is a sensible thing to want.
Every price now states net or gross and how often. The cart shows net per line
with "pro Monat" or "einmalig", then subtotal, VAT and gross — and separates
the monthly recurring amount from a one-off traffic top-up sharing the same
cart, because those are two different commitments. The rate is configurable
(CLUPILOT_TAX_PERCENT, 20 % default) since it follows the seller's country.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
fsn-01.node.clupilot.com, numbered per datacenter and never reused — removing a
host must not renumber its neighbours onto its name, so the number is stored
rather than derived.
The record points at the host's WireGuard address, not its public IP: the name
exists so an operator can reach a host by name over the VPN, and publishing a
Proxmox host's public address would hand every scanner a target, which is the
one thing this network design avoids.
DNS is convenience, not a prerequisite: a failure logs an event and lets the
onboarding finish rather than stranding a host that is otherwise ready.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Customers can now see what they have used and what is left, and the service
slows down rather than stopping when the allowance is gone — a slow Nextcloud
gets a top-up, a dead one gets a cancellation.
- instance_traffic keeps one row per instance per month. Proxmox counters are
cumulative since the VM last started, so usage is the difference between two
samples, and a counter that went backwards means a restart, not a refund.
- Outbound is what counts: inbound is free at our providers and egress is what
Hetzner's 20 TB per server applies to.
- CollectInstanceTraffic samples every 15 minutes, warns once per threshold
(80/95 %) rather than on every run, and at 100 % limits the VM's NIC via
Proxmox — released again as soon as the customer tops up or the month rolls
over.
- The dashboard gets a full-width band (it throttles the service, so it is not
a tile among tiles) with the top-up offer right next to the warning.
- Bytes are formatted in SI units now: allowances are computed in SI, so
dividing by 1024 made a 1000 GB plan read as "931 GB".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kuma has no write REST API (monitor CRUD is Socket.IO only), so docker/kuma-bridge
translates the exact generic REST contract HttpMonitoringClient already speaks —
no PHP client change needed. Opt-in compose profile 'monitoring'.
Verified end-to-end against a real Uptime Kuma 1.x:
create -> id, create again -> same id (idempotent), list, status, delete
and PHP MonitoringClient -> bridge -> Kuma round-trip.
The e2e run caught a real bug: add_monitor takes 'maxretries', not 'retries'.
Monitoring is now observability, not a delivery gate:
- RegisterMonitoring retries MONITORING_ATTEMPTS times on an outage, then
continues degraded with a visible 'info' event
- RunAcceptanceChecks no longer fails when monitoring isn't green (a fresh Kuma
monitor legitimately reports 'pending' — confirmed in the e2e run); it records
the gap instead. MONITORING_REQUIRED=true restores strict gating.
- docs/monitoring-uptime-kuma.md documents setup, failure behaviour, alternatives
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- .env.example gains a full, commented CluPilot operations block (Stripe, Hetzner
DNS, Traefik, WireGuard hub, SSH identity, monitoring, prod mail) with a note
that everything is optional for local dev
- config/provisioning.php: CLUPILOT_SSH_{PUBLIC,PRIVATE}_KEY_PATH read the key
from a file (a multi-line PEM cannot live in .env), falling back to inline vars
- documents that Uptime Kuma's REST API is read-only (monitor CRUD is Socket.IO),
so it needs a bridge rather than being drop-in for the generic REST client
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- customers gain profile + branding + closed_at; instances gain cancellation
fields; branding resolver (NULL -> CluPilot defaults) snapshotted into the
provisioning run context
- cancel package: term-end, irreversible, typed-confirm modal (R5)
- close account: guarded (no active package), typed-confirm modal (R5)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- plans gain seats + performance class; customer views show storage/seats/
performance/features, no raw vCPU/RAM (admin keeps specs)
- billing: scoped wire:target + wire:key so one purchase no longer spins all
- chart island: line-colour->transparent gradient for filled line datasets;
MRR / backups / invoices bar charts converted to gradient line
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PhpseclibRemoteShell sets a command timeout (default 2000s) so apt
full-upgrade/install don't hit phpseclib's ~10s default and fail.
- Host removal dispatches RemoveWireguardPeer to the provisioning queue, which
runs in the worker that owns wg0 (the web container can't manage WireGuard).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
hub_ip is now defined in config (default: subnet .1), so a custom
CLUPILOT_WG_SUBNET no longer pings the wrong handshake target.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo
Claude