A booked storage pack was a row in `subscription_addons` and nothing else:
priced, frozen, charged every month, and delivered nowhere. Nextcloud's
quota came from `instances.quota_gb`, which is the package alone; the disk
was sized for the package alone; and on the one occasion a disk did grow —
a plan upgrade — the guest never saw it, because nothing stretched the
partition or the filesystem over the new space.
One authority for the allowance. StorageAllowance adds the package and the
booked packs, and everything that needs the figure asks it: the quota step,
the downgrade check, the portal, the console, the repair command. It is
DERIVED rather than stored — a column would have to be rewritten by booking,
cancelling, granting, a grant expiring and a plan change, and the day one of
those paths forgot it would be silently wrong in whichever direction costs
somebody money. `quota_applied_gb` keeps its own meaning: what the guest was
last actually told, which is how a machine with the allowance enforced is
told from one where the figure has only ever been a row in our database.
Buying it delivers it. BookAddon asks ApplyStorageAllowance for a run on
booking AND on cancellation; the new `storage` pipeline grows the virtual
disk to the allowance plus the package's own overhead (read off the package
— 20/40/50/100 GB on the four catalogue packages — never a ratio invented
here), then GrowGuestFilesystem makes the guest see it, then the quota is
applied. No cold boot is involved: the data disk is scsi0 and Proxmox's
resize on a running guest is a qemu block_resize, so the capacity reaches
the guest while it runs. The new step rescans, growparts and grows the
filesystem with the tool its type actually needs — ext2/3/4, xfs, btrfs —
and fails loudly on one it does not know rather than handing it to resize2fs
and hoping. Idempotent end to end: NOCHANGE from growpart is not a failure,
and every tool here exits 0 when there is nothing left to grow.
The same step now runs in the plan-change pipeline, where an upgrade used to
stop one step short of the customer.
A downgrade blocked by data gets a way out. The block stays — it is correct
— but the check now measures against the target package PLUS the packs the
customer already owns, and reports the numbers behind the refusal: what is
stored, what would be allowed, how much has to go, and how many packs would
cover it instead. The portal offers both routes: book exactly those packs
(confirmed in a modal, R23), or delete data and have the fill level measured
on demand rather than waiting for the nightly sampler. That reading is taken
by DiskUsageProbe, which CollectInstanceTraffic now uses too, so there is one
notion of "how full is it" and not two.
A plan change keeps booked packs. They were paid for separately and have
nothing to do with which package the customer is on; the new package's disk
and quota are sized with them included.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The panel could not show the storage ring or the transfer trend because nothing
measured them. instance_traffic keeps one row per billing period — right for an
allowance, useless as a series — and storage consumption was never sampled at
all, so the card could only ever state what was sold rather than what is used.
instance_metrics holds one row per instance per day, written by the traffic
collector on the visit it already makes. A second scheduler entry against the
same Proxmox API would have doubled the load and the failure modes for no gain.
Disk usage comes from `df` inside the guest rather than from Proxmox's own disk
figure, which reports the allocated image: that would show every customer a full
500 GB from their first day.
The rule the whole thing is built on: a reading that could not be taken is not a
reading of zero. The disk columns are nullable and left untouched when the guest
agent does not answer, so yesterday's figure stands instead of the ring dropping
to empty — which would tell a customer their data had vanished. Missing days
stay missing in the series rather than being filled with zeroes, which would
draw an outage that never happened. Both are covered by tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Defaulting missing netin/netout to zero reset the baseline, so the next real
sample added the entire cumulative counter again — a customer throttled for
traffic that was counted twice. Such a sample is skipped and logged now.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- update.sh restarted the long-lived services before clearing the caches, so
they booted from the old cached config and kept it for the life of the
process — the release's configuration never reached the workers.
- install.sh put the Gitea token in the clone URL, which is visible in the
process list to every local user while the clone runs. It goes through a
short-lived askpass helper now.
- The collector's uniqueness lock expired at exactly the collection interval,
so a delayed queue could run two collectors on the same baseline and count
the same delta twice — throttling a customer for traffic they never used.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tying the release to the creation of the new month's row meant a single failed
call was never retried: the row exists from then on, and the current period
looks unthrottled, so nothing could detect the stale NIC limit. A customer would
have stayed slow for a whole month they had paid for. Checked on every sample
now, with a test that makes the first release fail.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two from Codex:
- A throttled instance kept its NIC limit into the new month: the new period
row starts unthrottled, so neither branch of enforce() fired and nothing ever
took the limit off. A customer would have stayed slow through a month they
had already paid for.
- PublicSiteGate exempted livewire/* wholesale. That endpoint is shared by the
console and the portal, so a signed-in customer could keep driving portal
components — billing included — while the portal was supposed to be offline.
Operators pass the normal check anyway, which is all the console needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Customers can now see what they have used and what is left, and the service
slows down rather than stopping when the allowance is gone — a slow Nextcloud
gets a top-up, a dead one gets a cancellation.
- instance_traffic keeps one row per instance per month. Proxmox counters are
cumulative since the VM last started, so usage is the difference between two
samples, and a counter that went backwards means a restart, not a refund.
- Outbound is what counts: inbound is free at our providers and egress is what
Hetzner's 20 TB per server applies to.
- CollectInstanceTraffic samples every 15 minutes, warns once per threshold
(80/95 %) rather than on every run, and at 100 % limits the VM's NIC via
Proxmox — released again as soon as the customer tops up or the month rolls
over.
- The dashboard gets a full-width band (it throttles the service, so it is not
a tile among tiles) with the top-up offer right next to the warning.
- Bytes are formatted in SI units now: allowances are computed in SI, so
dividing by 1024 made a 1000 GB plan read as "931 GB".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nexxo