Commit Graph

9 Commits (29f13275f89c68414b854fc1f9712e8e8e8d030e)

Author SHA1 Message Date
nexxo 19a770e1eb Kuendigung B1, Fix-Welle: der Platz wird frei, und das Loeschen ist vollzogen
K1 — abgebaute Instanzen zaehlten weiter als belegend. `scopeOccupyingHost`
filtert jetzt `torn_down_at`: eine abgebaute Instanz behaelt `ended`, `vmid`
und `disk_gb` als Nachweis, `status != 'failed'` war fuer sie also wahr. Der
Knoten wurde physisch leer und die Buchhaltung blieb voll — die naechste
bezahlte Bestellung derselben Groesse waere geparkt und verworfen worden.

K2 — `deleteVm()` gibt die UPID zurueck, der Abbau wartet den qmdestroy-Auftrag
ab und schreibt `torn_down_at` erst danach. Ein 200 heisst nur, dass Proxmox
den Auftrag angenommen hat; scheitert die Zerstoerung danach, stand bisher eine
laufende Maschine mit einem Datensatz da, der "abgebaut" sagte — und niemand
sah sie je wieder an. Die Attrappe bildet die Asynchronitaet ab
(`destroyedVmids`, `destroyExitStatus`, `destroyHangs`). Die Fristen des
Auftrags wandern mit: Sichern 900 s, Zerstoeren 300 s, Summe unveraendert 1800 s
unter der Uhr des Arbeiters.

K3 — vor `shutdownVm()` steht derselbe `vmStatus()`-Riegel wie im
Nachbarschritt. Eine wegen offener Zahlung gesperrte Cloud und jeder
Wiederholungslauf nach einem Teilfehlschlag treffen einen gestoppten Gast; der
Wurf haette die Instanz unheilbar gemacht und den echten Grund am Datensatz
ueberschrieben. Die Attrappe weist eine Bitte gegen eine stehende Maschine
jetzt ab, und ein Test faehrt erstmals einen zweiten Lauf nach einem
Fehlschlag durch.

W2 — der rote Kasten "Abbau haengt" filtert `status = 'ended'`. Eine
wiederbelebte Instanz waere sonst fuer immer darin stehengeblieben.

Neun Pruefungen im Mahnwesen lassen ihre Cloud jetzt laufen, bevor sie gesperrt
wird — die geschaerfte Attrappe legt offen, dass `SuspendInstance` denselben
fehlenden Riegel hat (Folgepunkt im Bericht).

Zu jedem der vier Punkte eine Zusicherung, die ohne den Fix rot ist; die
Rotproben stehen im Bericht.
2026-08-04 12:39:06 +02:00
nexxo df73e558e9 Kuendigung B1, Tasks 4+5: der Zeitplan-Griff und die Sichtbarkeit
Gekuendigte Kundenmaschinen liefen bisher fuer immer weiter. Task 3 hat den
Abbau gebaut; hier kommen der Griff, der ihn faehrt, und der Ort, an dem man
sieht, was passiert ist.

Die Wartezeit-Frage, entschieden: ein Auftrag je Instanz auf der
provisioning-Warteschlange. Das ist keine Abwaegung — nur queue-provisioning
steht im Netz-Namensraum des vpn-hub, der scheduler-Container nicht. Ein
Befehl, der selbst mit Proxmox spraeche, haette gar keine Route zu einem Host.

Die Fristen stehen ausdruecklich ineinander: die Aktion bekommt 600+1200 =
1800 s, der Auftrag hat $timeout 2100 s, retry_after der Verbindung ist
2400 s. Nur die unterste Uhr hinterlaesst einen lesbaren Grund am Datensatz;
die mittlere toetet den Arbeiterprozess stumm, die oberste startet einen
ZWEITEN Abbau gegen eine Maschine mitten im vzdump. $tries=1, weil ein
sofortiger zweiter Versuch am Herunterfahren einer gesperrten VM scheitern und
den richtigen Grund ueberschreiben wuerde. Die Staffelung ist als Pruefung
festgenagelt. Der Preis — zwanzig statt sechzig Minuten fuers Sichern — steht
im Kopfkommentar ausgeschrieben.

Dazu zwei Entscheidungen, nach denen niemand gefragt hat: eine Obergrenze je
Lauf, weil ueber dieselbe serielle Warteschlange bezahlte Bestellungen laufen;
und eine Reihenfolge, die einen Dauerfall die uebrigen nicht aushungern laesst.

Zeitplan taeglich um 05:30 — der Abbau hat keinen Moment, auf den es ankommt,
aber er darf nicht ins naechtliche vzdump-Fenster um 02:00 fallen.

Sichtbarkeit: zwei Kaesten in der Konsole. „Abbau haengt" (rot, ganz oben) —
eine Instanz mit gefuelltem teardown_error steht unbegrenzt und belegt weiter
einen Platz. Und „Archiviert und abgebaut" (unter der Liste) mit archive_volid
im Klartext. Beide sortieren absteigend und beide haben ein Ende: der Fehler
raeumt sich beim naechsten erfolgreichen Lauf selbst ab, das Archiv faellt
nach zwoelf Monaten heraus. Der Folgepunkt vom Export-Kasten also nicht noch
einmal.

Suite 2929 gruen. 25 neue Pruefungen, vier Mutationsproben rot gesehen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 12:00:37 +02:00
nexxo c9fef59983 Fix-Welle: die Kuendigung sagt jedem nur das, was fuer ihn stimmt
Vier Befunde aus dem Gesamt-Review, und alle vier hatten dieselbe Wurzel:
`export_wish` fuehrt drei Zustaende, und jede Stelle, die den Kunden
ansprach, kannte nur zwei.

K1 — Der Streifen im Dashboard trug einen Schalter, und `(bool) null` ist
`false`. Wer vor dieser Ausrollung gekuendigt hat, las unter dem Streifen
„Kein Export gewuenscht" — als waere das seine eigene Antwort. Jetzt stehen
dort dieselben zwei Auswahlfelder wie im Kuendigungsdialog: gleiche Frage,
gleiche Form, und ein unbeantworteter Zustand markiert schlicht keines von
beiden. Der Satz daneben fragt dann, statt zu behaupten, und sagt, was
passiert, wenn die Frage offen bleibt.

K2 — Die Vertragsseite versprach jedem den Export, auch dem, den der Dialog
eine Sekunde vorher mit einem bewussten „Nein" genau dorthin umgeleitet
hatte. Drei Fassungen statt einer, an `export_wish` gebunden. Die Ja-Fassung
verspricht dabei nicht mehr den Export selbst, sondern dass der Wunsch
vermerkt ist und sich jemand meldet — den Export gibt es nicht, und ein
gebundenes, aber weiterhin unhaltbares Versprechen haette den Fehler nur
verschoben.

Dazu der Zustand danach: eine `ended`-Instanz holt Dashboard::render() nicht
mehr, und der Kunde fiel in denselben Zweig wie jemand, der noch nie etwas
bestellt hat — „Ihre Cloud wird eingerichtet." samt „Paket buchen", am Tag,
an dem ihm die Adresse eingezogen wurde. Der Fall hat jetzt seinen eigenen
Kasten, mit dem Datum, an dem das Paket endete.

W1 — Die Erinnerungsmail behauptete im Praesens, wir bereiteten bereits einen
Export vor. Der Satz sagt jetzt, was stimmt. Und die Antwort hatte in der
ganzen Konsole keinen einzigen Leser: ein „Ja" landete in einer Spalte, die
niemand je zu Gesicht bekam. Ueber der Instanzliste steht deshalb ein
Abschnitt „Datenexport bestellt" — wer, und bis wann. Nicht als Plakette in
der Zeile, weil die Liste geblaettert ist und ein alter Eintrag auf Seite acht
saesse; nicht auf der Uebersicht, weil ein Hinweis, den nichts je wieder
abraeumen kann, Moebel waere.

W3 — Die einzige Pruefung zur Anzeige der Antwort konnte nicht fehlschlagen:
`x-ui.switch` rendert beide Woerter und ueberlaesst dem CSS die Auswahl, also
war `assertSee('Kein Export gewuenscht')` bei true, bei false UND bei null
gruen. Nachgewiesen mit einer Wegwerf-Pruefung gegen den alten Streifen:
dreimal derselbe Satz, dreimal gruen. Jetzt drei Pruefungen, je eine pro
Zustand, am `checked`-Attribut der Auswahlfelder.

Zu jedem der vier Punkte wurde der Fix kurz zurueckgedreht und die neue
Zusicherung rot gesehen; die Ergebnisse stehen im Bericht.

Ganze Suite: 2819 gruen, 2 rot — beide fremd. ReadinessPageTest scheitert an
`server.private_key` aus der parallel laufenden Terminal-Arbeit im selben
Baum; HostStepTest ist auf main vorbestehend rot (install-agent.sh traegt
CONTRACT=3, update.sh HOST_STEP_NEEDS=2, beide unveraendert).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-04 10:00:36 +02:00
nexxo 5b63fdb86c Deliver the storage a customer actually buys
A booked storage pack was a row in `subscription_addons` and nothing else:
priced, frozen, charged every month, and delivered nowhere. Nextcloud's
quota came from `instances.quota_gb`, which is the package alone; the disk
was sized for the package alone; and on the one occasion a disk did grow —
a plan upgrade — the guest never saw it, because nothing stretched the
partition or the filesystem over the new space.

One authority for the allowance. StorageAllowance adds the package and the
booked packs, and everything that needs the figure asks it: the quota step,
the downgrade check, the portal, the console, the repair command. It is
DERIVED rather than stored — a column would have to be rewritten by booking,
cancelling, granting, a grant expiring and a plan change, and the day one of
those paths forgot it would be silently wrong in whichever direction costs
somebody money. `quota_applied_gb` keeps its own meaning: what the guest was
last actually told, which is how a machine with the allowance enforced is
told from one where the figure has only ever been a row in our database.

Buying it delivers it. BookAddon asks ApplyStorageAllowance for a run on
booking AND on cancellation; the new `storage` pipeline grows the virtual
disk to the allowance plus the package's own overhead (read off the package
— 20/40/50/100 GB on the four catalogue packages — never a ratio invented
here), then GrowGuestFilesystem makes the guest see it, then the quota is
applied. No cold boot is involved: the data disk is scsi0 and Proxmox's
resize on a running guest is a qemu block_resize, so the capacity reaches
the guest while it runs. The new step rescans, growparts and grows the
filesystem with the tool its type actually needs — ext2/3/4, xfs, btrfs —
and fails loudly on one it does not know rather than handing it to resize2fs
and hoping. Idempotent end to end: NOCHANGE from growpart is not a failure,
and every tool here exits 0 when there is nothing left to grow.

The same step now runs in the plan-change pipeline, where an upgrade used to
stop one step short of the customer.

A downgrade blocked by data gets a way out. The block stays — it is correct
— but the check now measures against the target package PLUS the packs the
customer already owns, and reports the numbers behind the refusal: what is
stored, what would be allowed, how much has to go, and how many packs would
cover it instead. The portal offers both routes: book exactly those packs
(confirmed in a modal, R23), or delete data and have the fill level measured
on demand rather than waiting for the nightly sampler. That reading is taken
by DiskUsageProbe, which CollectInstanceTraffic now uses too, so there is one
notion of "how full is it" and not two.

A plan change keeps booked packs. They were paid for separately and have
nothing to do with which package the customer is on; the new package's disk
and quota are sized with them included.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 19:13:10 +02:00
nexxo d76de5ada3 Restart a machine, enforce the quota that was sold, end a route that ended
Three places where the product said one thing and did another.

**Nothing could restart a virtual machine.** ProxmoxClient had startVm and no
way to stop anything, so `restart_required_since` — set by every plan change
that grows a running guest — could only ever be cleared by a resize step that
happened to find the machine stopped, which nothing in the product could
arrange. A paid upgrade's cores and RAM could stay unreached for the life of the
contract, and the cloud page's "Neu starten" was a toast.

There is now a `restart` pipeline: shutdown, start, wait for the agent, confirm.
A shutdown and a start rather than a reboot, because only a cold boot makes qemu
read the VM definition again — a reset would take the machine round and bring it
back exactly as small as it was. The shutdown is a REQUEST, bounded at ten
minutes, and there is no escalation: ProxmoxClient deliberately does not expose
Proxmox's forceStop, because the guest is a Nextcloud and cutting power to a
database mid-write to apply a CPU change trades an inconvenience for a restore
from backup. A guest that ignores ACPI for ten minutes fails the run instead, and
a person decides what happens to it.

`restart_required_since` is cleared by the last step, from what the RUNNING guest
reports (`cpus`/`maxmem`) — not by the button, and not by the run merely
finishing. A machine that comes back smaller than it was sold fails the run.

Started by App\Actions\RestartInstance, which re-checks authorisation itself
rather than trusting a hidden button: an operator holding the new
`instances.restart` may restart anything, a portal user their own machine and
nothing else. Refused while another run is in flight against the same order.
Confirmed in a modal on both sides (R23), addressed by uuid (R11).

**Existing instances had no storage quota.** ApplyStorageQuota reaches new and
changed machines; everything built before it kept the whole disk whatever it had
paid for, and nothing recorded which was which. `instances.quota_applied_gb` is
now what the guest was actually told, written by the step only after the occ call
succeeds — so "sold" and "enforced" are separate facts and can be compared.
`clupilot:apply-quotas` sweeps the estate: one `quota` run per instance through
the same step the pipelines use, skipping what it should and saying why, a no-op
once an allowance is enforced, and inert under --dry-run. Deliberately NOT
scheduled — a nightly sweep would quietly cover for the pipeline step the day it
stopped working, which is how the original hole survived this long.

**A Traefik router was never torn down.** TraefikWriter::remove() had no caller
anywhere. What was missing was the moment: ConfirmCancelPackage wrote a date into
`service_ends_at` and nothing ever went back to it, so every route this platform
ever wrote was written for good — pointing at a guest address the host is free to
reassign. `clupilot:end-due-services` (hourly) keeps that appointment through
App\Actions\EndInstanceService. "Ended" means cancellation_scheduled AND
`service_ends_at` passed, never the day a cancellation is scheduled: that
customer has paid to the end of the term and is working in it. The DNS record
goes with the route — it is in our zone, pointing at a host that serves other
people, which is the shape of every subdomain takeover there has ever been. The
customer's own domain is in their zone and is not ours to touch; the virtual
machine is left alone, because the cancellation flow promises a data export first
and destroying disks is not this change's decision.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 18:28:28 +02:00
nexxo 34874adec3 Apply a bought plan change instead of only pricing it
tests / pest (push) Failing after 8m6s Details
tests / assets (push) Successful in 21s Details
tests / release (push) Has been skipped Details
PlanChange could say what a move would cost and whether it was allowed, and
that was all it could do. Billing::purchase() wrote an upgrade order and
nothing ever consumed it: same snapshot, same machine, same quota. A customer
could pay for a bigger package and receive nothing.

ApplyPlanChange is now the single place a change lands — it moves the contract
onto the target's current version, writes one register row, settles the custom
domain, and starts a run that resizes the machine. Applying the same order
twice is a no-op, enforced by a unique event key rather than by a check two
callers could both pass.

Two things a plan change must not do, and now does not. A disk is never shrunk
— Proxmox cannot, so the QUOTA shrinks instead, which is what was sold anyway.
And a live machine is never rebooted as a side effect: cores and RAM are
written, and where they need a restart the instance says so where an operator
and the customer can both see it.

The storage allowance also joins the build pipeline. ApplyStorageQuota was
written for plan changes and ran only there, so a brand-new customer still got
no quota at all — quota_gb reached the instance row and stopped, and every
package delivered the whole disk. The test guarding that pipeline only counted
its steps, which is how a list missing the one step that makes a package's
storage real stayed green for its whole life. It now names the step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 17:18:55 +02:00
nexxo ea643b5e73 Prove a custom domain before serving it, and keep proving it
tests / pest (push) Failing after 8m17s Details
tests / assets (push) Successful in 23s Details
tests / release (push) Has been skipped Details
Two things.

── The update screen, still opening twice ───────────────────────────────────
Reported again on 1.3.9, and the cause was not the one fixed in 1.3.8. The
agent consumes the request file BEFORE it resolves the release — deliberately,
because update.sh may kill the shell and a request left in place would loop —
and writes `state: running` only once it has decided to go ahead. In between,
the request is gone and the status does not say running yet, so the endpoint
honestly answers "nothing is running". The watcher read that as "the run has
finished" and reloaded the page: overlay on the click, gone a poll later, 503
after it.

The overlay now closes only once the server has BOTH confirmed a run and then
stopped reporting it. Before the confirmation, silence means the agent has not
got there yet. Bounded at twenty polls so a request the agent refuses does not
leave the console covered forever.

── Custom domains, proven and re-proven ─────────────────────────────────────
`custom_domain` was a free-text field and everything downstream believed it:
the proxy served it, the certificate was issued for it, Nextcloud trusted it.
Anyone who pointed any hostname at the platform got somebody else's files
under their own name.

The proof is a TXT record at _clupilot-challenge.<domain> holding a token only
this instance has. Nothing is served until it has been read. Every reader now
goes through Instance::address(), which is the one place the decision is
made — `custom_domain ?: subdomain` was the hole, written out four times.

It is re-read every night at 03:40, because a token checked once can be taken
straight back out and a domain that later lapses keeps resolving here. Three
consecutive misses before a live domain is withdrawn: one failed lookup is a
nameserver having a bad minute, and withdrawing takes a working Nextcloud off
its own address. The domain and its token stay on the row so the customer can
put the record back rather than start over.

Changing the domain mints a NEW token. Reusing it would let somebody who once
verified example.com claim any other domain later without touching its DNS —
the old record is still sitting there and only the value is compared.

The domain is changeable at any time, and removable. Fixing it once set was
considered and rejected: adding or moving an address is a proxy entry, a
certificate and one line in trusted_domains.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 14:47:51 +02:00
Claude 30a80b6c15 Report the estate from the database, and print the traffic that is sold
Three console pages were fiction. The front page claimed 42 customers, 39
instances, four hosts named pve-fsn-1..3 and €7,842 a month, over a twelve-month
growth curve; the instance list held seven invented machines; the revenue page
reported churn and a trend for a business with no recorded history. All of it
was hard-coded. It read like a running company and measured nothing.

They now read the database. Two figures are gone rather than approximated —
the revenue trend, which needs a monthly history nobody records, and churn,
which needs a base the data cannot supply. ARR stays, labelled as the
projection it is. The green "all systems normal" badge is computed from the
notice list instead of asserted, and the notices themselves come from failed
runs, hosts reporting errors or gone quiet, and monitoring that is down.

Host load is the one number that had to agree with something else: placement
counts the VM disk allocation, ignores a failed instance that never got a VM,
and subtracts the host's reserve. A dashboard doing its own arithmetic would
show a host as comfortable while orders were already being refused on it, so it
uses the host's own accounting — with the filter moved into a scope both share,
and the sum preloaded so listing hosts stays one query.

The instance list drops the Nextcloud version column: that version is not
recorded anywhere, and a column filled with a plausible number is worse than no
column. Statuses the lifecycle writes but nobody had translated no longer
render as "admin.status.failed".

The price sheet also gains the included traffic, which the catalogue has always
carried and the page simply never printed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 04:48:51 +02:00
nexxo 1aa7535fc4 feat(admin): dark Tactical-Terminal operator console
Separate admin console at /admin, gated to is_admin users (EnsureAdmin middleware
+ migration/seed; a plain customer user is seeded to prove the gate). Distinct dark
aesthetic achieved purely by token scoping: .theme-admin overrides every CSS design
token to a dark graphite / signal-orange palette, so ALL shared components (button,
card, badge, table, stat, chart) render dark with zero new markup (R3). Chart island
now reads tokens from its own element, so charts are theme-aware.

Sections (each full-page class-based Livewire, English routes R13, localized DE/EN):
- Overview: fleet KPIs, fleet-growth line, host-load bars, MRR bars, active runs, alerts.
- Customers: table + plan doughnut. Instances: fleet table (vmid/host/storage).
- Hosts: capacity cards (storage/CPU bars). Provisioning: runs table + live stepper.
- Revenue: MRR/ARR/ARPU/churn KPIs, MRR line, plan doughnut, recent payments.
- Locale-aware month labels/currency (Carbon/Number).

18 new Pest tests (guest redirect / non-admin 403 / admin render per section) → 44
green. R12 browser: all six admin pages HTTP 200, ZERO console errors (Chart.js dark).
Codex (R15) — clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 08:35:03 +02:00