Commit Graph

9 Commits (ec8675861e08103ae3cfe6166e047629185a99cd)

Author SHA1 Message Date
nexxo 712803edd6 Serve the custom domain, not just announce it
A verified custom domain was reported as the customer's address by
Instance::address(), by the portal and by the credentials mail while
nothing on the platform routed it: the Traefik router's rule was
hard-coded to {subdomain}.{zone}. The address a customer was handed
answered nothing, and a withdrawn domain stayed in Nextcloud's
trusted_domains forever, because the only thing that ever wrote either
was the initial provisioning run.

- TraefikWriter::write() takes a LIST of hostnames under one stable
  router name — the platform address always, the verified domain as
  well when there is one. One file per instance, so a withdrawal is a
  rewrite rather than a second thing somebody has to remember.
- ConfigureDnsAndTls records the custom domain's certificate instead of
  waiting on it: it depends on an A record in the customer's own zone,
  which may never appear, and must never fail a run. The platform
  address keeps its 840s deadline. instances.domain_cert_ok tells
  "proven" apart from "answering", and the portal now says which.
- ConfigureNextcloud deletes trusted_domains 2 when there is no verified
  domain, so a withdrawn one stops being trusted.
- New `address` pipeline (those two steps) plus ReapplyInstanceAddress,
  which starts one against the order and refuses to start a second while
  any run is in flight. The route is rewritten when the hostname list
  differs from what the router carries — not on route_written, which
  would short-circuit exactly the case a re-apply exists for.
- Triggered where the address changes: verification flipping either way
  in clupilot:verify-domains, the customer's own domain page, and
  CustomDomainAccess::deactivate() on a package downgrade.
- A maintenance run no longer condemns its subject: an address run that
  failed used to mark the order failed and release the live instance
  with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 16:44:52 +02:00
nexxo c56aa14f9d fix(engine): don't reset started_at on poll so step deadlines accumulate
RunRunner::onPoll no longer resets started_at, so a poll step's own deadline
(WaitForGuestAgent 270s, ConfigureDnsAndTls cert 840s) actually fires instead
of resetting every poll. maxDuration raised above each own-deadline so the
step's fail wins over the generic timeout. Shared core fix (A + B).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 12:15:26 +02:00
nexxo c7fb1ce56d fix(engine): dispatch run continuation after releasing the lock
Prevents a second worker from consuming the follow-up job and bailing on the
still-held run lock, which would stall the run until the next scheduler tick.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 11:09:24 +02:00
nexxo 93191f5de9 fix(engine): fail on unresolvable pipeline step; make worker the WG hub
- RunRunner catches step-resolution errors (removed/renamed pipeline) and
  fails the run terminally instead of looping forever every tick.
- queue-provisioning worker gains NET_ADMIN + tun + persistent wireguard
  volume + published UDP port so LocalWireguardHub can manage wg0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:53:40 +02:00
nexxo ced7e6103d fix(engine): durable queueing for long provisioning steps
- Dedicated 'provisioning' queue connection (retry_after 2400) + worker
  (--timeout=2100); AdvanceRunJob tries=1, timeout=2100 on that queue.
- Run lock TTL raised to 2100s so a long step can't be run concurrently by a
  duplicate tick-dispatched job.
- StartHostOnboarding creates host+run in a transaction, dispatches after commit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:36:49 +02:00
nexxo fbe87d99d5 fix(engine): honour backoff timing + don't double-count Proxmox storage
- RunRunner returns early for a waiting run whose next_attempt_at is in the
  future, so stale/duplicate jobs can't bypass backoff.
- RegisterCapacity takes the largest VM-capable datastore instead of summing
  overlapping pools (local + local-lvm), preventing placement overcommit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:32:14 +02:00
nexxo 8a2a2ee695 fix(engine): reset step timer on retry so timed-out steps recover
A retried step (manual or after a timeout) now gets a fresh started_at, so
RunRunner no longer immediately re-detects the timeout and burns the retry
budget without executing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:29:11 +02:00
nexxo 62c4412623 fix(engine): address Codex round 2 (poll budget, host error state, wg race)
- StepResult::poll — polling steps (reboot) wait without consuming the retry
  budget; the step owns its deadline. Reboot maxDuration > deadline.
- Failed runs move a Host subject to 'error' via ProvisioningSubject hook
  (no host stuck 'onboarding').
- ConfigureWireguard allocates + reserves the wg_ip under a global lock;
  unique index on hosts.wg_ip as a backstop against duplicate addresses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 10:26:37 +02:00
nexxo 4f724d57c9 feat(engine): orchestrator core (state machine + tick + lock)
StepResult, ProvisioningStep contract, PipelineRegistry, RunRunner (per-run
lock, advance/retry/fail, backoff, timeout, append-only events + Reverb
StepAdvanced), AdvanceRunJob (provisioning queue), minutely Tick, admin.runs
channel. 21 tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 09:54:52 +02:00