Commit Graph

30 Commits (4681b135db74e08bbb60b6925145b3bf9fbfddca)

Author SHA1 Message Date
nexxo 7e17bc74b2 fix(vpn): give the QR code the specified four-module quiet zone
Two modules is below the minimum, and against a bordered card some readers fail
to locate the symbol at all.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:40:54 +02:00
nexxo 0d9b62eb50 feat(vpn): ownership, a Developer role, and password-gated config retrieval
Reworked after a design consultation with Codex, which pushed back on my first
proposal in three useful ways.

Ownership and rights:
- vpn.manage split into vpn.view.all and vpn.manage.all. Seeing an access is
  not managing it, and neither is holding its private key.
- Record-level rules live in VpnPeerPolicy, not in permissions: an access
  belongs to a person, and ownership is what grants sight of it. Every operator
  reaches the page, but sees only their own unless they may see all.
- Issuing an access is not self-service — it reaches the management network, so
  it needs vpn.manage.all even for oneself.
- New Developer role: sees everything, manages nothing. Writing code does not
  imply authority over other people's access.
- kind (staff|host|system) replaces a null user_id that had to mean two
  different things at once.

Config storage, opt-in per access:
- Downloading is owner-only — explicitly NOT for view.all or manage.all. An
  admin who needs access revokes this one and issues their own, which keeps the
  record of who holds what honest.
- The password is asked on EVERY retrieval, rate-limited. Laravel's
  password.confirm keeps a 15-minute session stamp, which would authorise
  unlimited later downloads from an unattended browser.
- Stored under VPN_CONFIG_KEY, not APP_KEY: a leaked application key must not
  also hand over the management network. Purged on revocation and when a staff
  member is revoked — console taken away, tunnel left open was the worst case.
- The plaintext never enters a Livewire snapshot (the page polls every five
  seconds); the component carries an opaque handle instead.

Also: copy button works without HTTPS again (navigator.clipboard only exists in
a secure context, so over http it silently did nothing), plus config download
as a file and a QR code for the mobile apps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:31:54 +02:00
nexxo 08670ecbee fix(vpn): stop revoked accesses coming back and stale jobs undoing intent
Two lifecycle holes Codex found in the new VPN management:

1. Delete removed the row before the hub knew. If the removal job was delayed
   or failed, SyncVpnPeers saw a peer it did not recognise and adopted it back
   as a live access — silently restoring what an operator had just revoked, and
   with no row left to revoke it again. Deletion is now a soft-delete
   tombstone: the row survives until the hub confirms the peer is gone, the
   sync retries the removal instead of adopting, and the tombstone is purged
   only once the peer is really absent. It also keeps the key and the tunnel
   address reserved meanwhile, so neither is handed out twice.

2. ApplyVpnPeer applied the state captured at dispatch. A retried block job
   could therefore undo a later unblock, and nothing would repair it — the sync
   only observes state, it never re-applies intent. The job now resolves the
   current desired state from the row and treats its payload as a fallback for
   the case where the row is already gone.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 21:41:03 +02:00
nexxo 71ca0e1394 feat(admin): VPN access management with live peer state
The console can now create, block and remove VPN accesses, and shows who is
actually connected — traffic counters, source endpoint and last handshake.

Design notes:
- The web container has no wg0, so the page never talks to WireGuard. Live
  state is copied in by SyncVpnPeers on the provisioning queue (the only worker
  whose container owns the interface); the page polls the database and merely
  nudges that job, throttled so many open tabs cannot flood the queue.
- enabled (operator intent) and present (observed on the hub) are stored
  separately, so a sync can never quietly undo a block and drift stays visible
  as "wird angewendet".
- The sync adopts peers the host pipeline registered, naming them after their
  host — otherwise "who is on the VPN" would have blind spots.
- Keypairs are generated in PHP via libsodium rather than shelling out to
  wg genkey (no interface in this container, and it keeps generation testable).
  The private key is shown once and never stored.
- allocateIp() now also considers vpn_peers, which would otherwise be handed a
  tunnel address a host already holds.
- vpn.manage is a new capability held by Owner/Admin only, and it hides the nav
  entry too — a VPN access reaches the management network.

Verified end to end against the real hub, not just mocks: created an access in
the browser, connected with the generated config, watched the console flip to
"Verbunden" with real counters, then blocked it and confirmed the peer was
gone from wg0 and the client could no longer handshake.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 21:38:12 +02:00
nexxo 5301ce4309 feat(monitoring): Uptime Kuma support via a REST bridge + resilient monitoring policy
Kuma has no write REST API (monitor CRUD is Socket.IO only), so docker/kuma-bridge
translates the exact generic REST contract HttpMonitoringClient already speaks —
no PHP client change needed. Opt-in compose profile 'monitoring'.

Verified end-to-end against a real Uptime Kuma 1.x:
  create -> id, create again -> same id (idempotent), list, status, delete
  and PHP MonitoringClient -> bridge -> Kuma round-trip.
The e2e run caught a real bug: add_monitor takes 'maxretries', not 'retries'.

Monitoring is now observability, not a delivery gate:
- RegisterMonitoring retries MONITORING_ATTEMPTS times on an outage, then
  continues degraded with a visible 'info' event
- RunAcceptanceChecks no longer fails when monitoring isn't green (a fresh Kuma
  monitor legitimately reports 'pending' — confirmed in the e2e run); it records
  the gap instead. MONITORING_REQUIRED=true restores strict gating.
- docs/monitoring-uptime-kuma.md documents setup, failure behaviour, alternatives

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:04:58 +02:00
nexxo 5b85806605 fix(admin): catch racing ledger insert and re-fetch (idempotent under concurrency)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 16:31:43 +02:00
nexxo 72731112f4 fix(admin): atomically claim maintenance notification retries under a row lock
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 16:30:17 +02:00
nexxo 8bb93137a3 refactor(admin): extract MaintenanceNotifier; atomic provisioning retry; catch-up cancellation for race-delivered announcements
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 16:28:02 +02:00
nexxo 66e8a4bbf6 fix(engine-b): clean re-clone on lost task ref; idempotent firewall rules
- Clone recovery: once the lock clears but the task ref is lost, destroy the
  possibly-incomplete VM and re-clone instead of advancing onto it.
- applyFirewall clears existing rules before adding, so retries don't accumulate
  duplicate Proxmox firewall entries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 13:04:48 +02:00
nexxo 9162446b1f fix(engine-b): acceptance gate verifies real health, not just breadcrumbs
- Nextcloud: parse occ status JSON (installed + not in maintenance).
- Admin: query occ user:info for the account.
- Monitoring: MonitoringClient::isHealthy checks the provider, not the local row.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 13:00:49 +02:00
nexxo 157496f0c5 fix(engine-b): write Traefik route to the serving host over SSH, not locally
TraefikWriter now targets the host that serves the traffic (DNS points at it):
SshTraefikWriter SSHes in and writes the file-provider YAML there. Interface
takes the traffic host + guest backend.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:55:24 +02:00
nexxo 483a79a822 fix(engine-b): treat unreachable cert endpoint as not-ready (poll, not retry-burn)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:52:37 +02:00
nexxo 03c4b6508f fix(engine-b): Proxmox agent exec repeated command fields; Traefik write throws on failure
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:50:57 +02:00
nexxo 8825f9abfe fix(engine-b): Proxmox config uses PUT; guest-agent exec is form-encoded
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:48:58 +02:00
nexxo 778bb7f117 fix(engine-b): always scrub credentials once delivered; idempotent monitoring lookup
- CompleteProvisioning scrubs admin_password whenever credentials_sent exists,
  so a crash between recording and scrubbing can't leave the password in context.
- HttpMonitoringClient looks up an existing monitor by URL before creating one,
  so a retry after a crashed POST doesn't create a duplicate external monitor.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:46:34 +02:00
nexxo 661b52dbe0 fix(engine-b): clone recovery waits for lock; register rows before breadcrumb
- Clone recovery polls until the VM lock clears instead of advancing onto a
  possibly-incomplete clone.
- RegisterBackup/RegisterMonitoring create the local row BEFORE the run-resource
  breadcrumb, so a crash between them can't leave the row permanently missing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:44:22 +02:00
nexxo 3c5a18b5d7 fix(engine-b): apply plan CPU/RAM; idempotent backup + monitoring registration
- ConfigureCloudInit sets cores/memory from the plan (not the template default).
- RegisterBackup uses a deterministic Proxmox job id (ignore-exists) + firstOrCreate.
- RegisterMonitoring is idempotent by URL + firstOrCreate — no duplicate schedules
  or monitors after a crash/retry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:42:10 +02:00
nexxo 5cf1964d7d fix(engine-b): default-deny firewall, disk-based capacity, synchronous credential mail
- applyFirewall sets policy_in=DROP so only 80/443 are exposed.
- Capacity/placement account for disk_gb (the real VM allocation), not the
  smaller Nextcloud user quota — no systematic overcommit.
- CloudReady sent synchronously again; the step retries on mail failure with the
  password preserved (no lost credential in a failed queue job).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:39:15 +02:00
nexxo b4cf94ee1a fix(engine-b): real backup/monitoring registration + OC_PASS scoping
- CreateCustomerAdmin puts OC_PASS on the docker invocation (survives the &&).
- RegisterBackup creates a real vzdump job via ProxmoxClient::createBackupJob.
- RegisterMonitoring registers via a new MonitoringClient service (interface +
  fake + http). Both persist the external id [E] before the breadcrumb, so
  acceptance reflects a real registration, not a fabricated row.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:33:45 +02:00
nexxo e0e4c4e18d fix(engine-b): run guest-agent commands through /bin/sh -c
The Proxmox guest agent execs a program directly; our compound commands
(cd/&&/pipes/redirects/env) need an explicit shell.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:25:24 +02:00
nexxo f43ade79b0 fix(engine-b): address Codex round 6 (crash-window idempotency)
- Clone: recover via vmExists() when the task ref was lost, instead of
  re-cloning the reserved vmid (which Proxmox rejects).
- Deploy: persist the DB password ENCRYPTED before compose up and reuse it on
  retry, so a crash can't regenerate a mismatching credential.
- Complete: gate credential delivery on a credentials_sent breadcrumb so a
  retry after a crash doesn't re-send the email.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:23:28 +02:00
nexxo 446da58061 feat(engine-b): 15-step customer pipeline + DNS/Traefik services
validate->reserve(placement)->clone->cloudinit->start->guestagent->network->
deploy->nextcloud->admin->dns/tls->backup->monitoring->acceptance->complete.
HetznerDnsClient + TraefikWriter (interface+fake+real), CloudReady notification,
hosts.node (set by RegisterCapacity). Secrets transient/encrypted, never
plaintext. 21 tests incl. mocked end-to-end + crash idempotency.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 11:50:26 +02:00
nexxo e5aeb3989e feat(engine-b): ProxmoxClient VM-lifecycle extension (interface + fake + http)
nextVmid/cloneVm/setCloudInit/resizeDisk/startVm/vmStatus/guestAgentPing/
guestExec/taskStatus/applyFirewall. Fake is deterministic with configurable
task/guest failure hooks; Http polls UPID + guest exec-status.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 11:41:48 +02:00
nexxo a6a41719bb fix(engine): long SSH command timeout; wg peer removal on privileged worker
- PhpseclibRemoteShell sets a command timeout (default 2000s) so apt
  full-upgrade/install don't hit phpseclib's ~10s default and fail.
- Host removal dispatches RemoveWireguardPeer to the provisioning queue, which
  runs in the worker that owns wg0 (the web container can't manage WireGuard).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 11:01:02 +02:00
nexxo 8a392bfd06 fix(engine): parse WireGuard CIDR for address allocation (any prefix length)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:48:16 +02:00
nexxo 48ed5e7d34 fix(engine): fail if the WireGuard peer can't be persisted (wg-quick save)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:41:11 +02:00
nexxo fbe87d99d5 fix(engine): honour backoff timing + don't double-count Proxmox storage
- RunRunner returns early for a waiting run whose next_attempt_at is in the
  future, so stale/duplicate jobs can't bypass backoff.
- RegisterCapacity takes the largest VM-capable datastore instead of summing
  overlapping pools (local + local-lvm), preventing placement overcommit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:32:14 +02:00
nexxo 65717bd3bd fix(engine): address Codex review (auth token bootstrap, tunnel recheck, ssh)
- CreateAutomationToken now bootstraps the pveum role/user/token over the
  authenticated SSH session (a fresh host has no API token yet); ProxmoxClient
  is read-only in A.
- ConfigureWireguard re-verifies the handshake on the idempotent replay path,
  never advancing over a dead tunnel.
- PhpseclibRemoteShell treats a missing SSH exit status as failure (255).
- connectWithKey verifies the pinned host key fingerprint on later logins.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:20:59 +02:00
nexxo ac3d17cd6a feat(engine): 11-step host onboarding pipeline (SSH -> WG -> Proxmox)
Idempotent steps: validate, ssh-trust (deploy key + scrub password),
prepare base, wireguard (hub peer), install proxmox-ve, reboot-into-pve
(retry-poll), configure, automation token, verify api, register capacity,
complete. StartHostOnboarding action. 22 tests incl. mocked end-to-end +
crash idempotency.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:04:39 +02:00
nexxo c06ca0ae5d feat(engine): SSH / WireGuard / Proxmox service layer
RemoteShell (phpseclib + fake), WireguardHub (local wg + fake), ProxmoxClient
(REST + fake). Interfaces bound in AppServiceProvider; tests swap fakes via
fakeServices() helper.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 09:58:02 +02:00