Found by running it on a fresh Debian 12 with systemd, no docker, no git and no
users — which is the first time it has ever been run anywhere.
`optional_env` ended in `[[ -n "$2" ]] && set_env …`. With an empty value the
function returns 1, and under `set -e` that ends the install, silently, with a
half-written root-owned .env. Absent is the NORMAL case for those variables —
they are the optional ones — so this killed every installation that did not
happen to supply a Hetzner token. An `if` returns 0 and does not.
Then it waited for `php -v` before migrating. PHP answers in seconds; the
entrypoint is still running `composer install` on a fresh checkout, which takes
minutes. So `artisan migrate` ran against a checkout with no vendor/ and died
on a missing autoload.php. It now waits for vendor/autoload.php, exactly as the
dependent containers already do, and says where to look if it never appears.
After both: exit 0, 59 migrations applied, the plan catalogue seeded and
consistent, the Owner account created with its role, and the portal answering
200 while the console redirects to login and the unconfigured Stripe webhook
fails closed with 400.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
VERSION at the repository root is the release number and the only place it is
written down — a file rather than `git describe`, because a source archive, a
shallow CI checkout and a container built without .git have no tags to
describe. Cutting a release is editing that file and merging it; CI creates the
annotated v<VERSION> tag once green, only on the commit that raised the number,
and never moves it. Servers are pinned to these.
Two modes, and the difference is the point. RELEASE=v1.0.0 pins a server to an
immutable tag, checked out detached: it moves when someone decides it moves.
BRANCH=main follows the edge, where CI tags a commit only after it is pushed.
The mode is remembered, so re-running the updater on a pinned box does not
quietly walk it back onto main.
Going backwards is refused, by commit ancestry rather than by comparing version
strings — a tag can be cut from anywhere, and only ancestry says whether history
is going back. The schema has already moved forward by then, and the migrations
needed to reverse it are not in the older checkout at all.
What the console reports comes from a manifest written atomically after every
step succeeded, never from live git: git says what the files are, not whether
the deployment came up. So a failed update keeps reporting the version that is
actually serving — including its version number, not the newer one the checkout
has already moved to. A deployment pinned to the tag reads "1.0.0 (abc1234)";
anything else reads "1.0.0-dev (abc1234) · main", because every commit after the
tag still carries VERSION=1.0.0 and is not that release. Reporting it as one is
how a bug gets filed against the wrong code.
Codex reviewed the design before it was written and the result six times after.
Its findings, all real: the version had to come from the manifest too, not just
the commit; branch names need JSON escaping or the manifest silently vanishes; a
`case` glob does not anchor and accepted 1x.2y.3garbage; pinning to the commit a
server already sits on skipped recording the mode, and then skipped detaching;
a manifest that failed to write would never be repaired; an unparseable
timestamp would 500 every admin page; and an empty tag list read as a
connectivity failure.
466 tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The installer clones BRANCH, which defaults to main — and the repository has
no main. Git's own "Remote branch not found" reads like a bad token or an
unreachable server, and the branch is the one thing here that is routinely
wrong. It now asks first and, when the branch is missing, prints the ones that
do exist and how to pass one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It still named two webhook events; the application handles six, and the four
new ones are the billing cycle itself — renewals, failed payments, status
changes and endings. It also said nothing about stripe:sync-catalogue, without
which Stripe has no prices and nothing can be sold at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things that bit us on every deploy:
- artisan down now renders a branded page that says an update is running and
reloads itself, instead of Laravel's bare 503.
- A page left open across a deploy carries a Livewire snapshot and CSRF token
the new code rejects — the user got "419 Page Expired" and a dead interface.
A 419 from a /livewire/ request now reloads the page; the session is still
valid, so that is all it takes. Hooked at the fetch layer rather than through
Livewire's request hook, whose failure callback is not invoked for this case
in the installed version — verified against a real 419 in the browser, not a
simulated one.
- Vite no longer empties its output directory: wiping it is what left an open
page without its stylesheet mid-deploy, which is the "design is completely
broken" symptom. update.sh also rebuilds the caches with optimize:clear +
optimize instead of leaving them half-warm.
Plus CI: .gitea/workflows/tests.yml (Pest + asset build, and a tested- tag only
on a green main) and an opt-in act_runner service under the ci profile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
--env-file makes the whole run non-interactive, which is what a bare server
needs: install git, clone, run, done. The file also carries the operational
secrets (Hetzner DNS, Stripe, SMTP, the Proxmox key path); those are optional,
so a first install can happen before the Stripe account exists and the features
stay dark until the values are filled in.
The installer warns when the file is readable by more than its owner, and
clupilot.env is gitignored — it holds every secret an installation will need.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- update.sh restarted the long-lived services before clearing the caches, so
they booted from the old cached config and kept it for the life of the
process — the release's configuration never reached the workers.
- install.sh put the Gitea token in the clone URL, which is visible in the
process list to every local user while the clone runs. It goes through a
short-lived askpass helper now.
- The collector's uniqueness lock expired at exactly the collection interval,
so a delayed queue could run two collectors on the same baseline and count
the same delta twice — throttling a customer for traffic they never used.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- The provisioning worker boots before install.sh writes
CLUPILOT_WG_HUB_PUBKEY and holds the empty value for the life of its process,
so every host onboarded afterwards would have got an empty PublicKey in its
wg0.conf.
- A failed update left the checkout at the target, so the rerun compared a
commit with itself and skipped exactly the dependency and image steps that had
not finished. The comparison base is now the last commit actually deployed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The installer still needs root for apt and the firewall, but it now creates a
clupilot service account that owns the checkout and runs Docker, maps the
container user to it (HOST_UID/GID), and update.sh refuses to run as root —
root-owned files in the checkout are files the application cannot write.
The account has no password login: docker group membership is root-equivalent
on the host, so it is reachable only via sudo -u or an SSH key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- update.sh recreated the app container only at the very end, so an update that
changes the PHP runtime would have installed dependencies and migrated inside
the old one.
- install.sh randomised DB_PASSWORD but left DB_ROOT_PASSWORD at the value
committed in .env.example — the same root credential on every installation,
reachable from any container on the compose network.
- Settings cached the fallback after a database failure, so one blip could
leave a hidden site publicly visible until someone cleared the cache. Only a
successful read is cached now, proven by a test that drops the table and
brings it back.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex was right on all three, and the first was a design error on my part: the
checkout is bind-mounted into every container, so 'git merge' swaps the running
code instantly — 'migrations before traffic' was not achievable by ordering.
The updater now goes down → merge → dependencies → migrate → assets → restart →
up, trading a short deliberate outage for never serving code whose schema does
not exist yet. If a step fails the site STAYS down: coming back up with new code
on an old schema is worse than staying dark until someone looks.
It also records the last successfully deployed commit. A run that died halfway
left the checkout ahead, so the next run said 'already up to date' and skipped
the rest forever.
And it installs dependencies when a lockfile moved: vendor/ and node_modules/
live in the bind mount and shadow the image, so rebuilding the image never
updated them — migrations could run against stale packages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
install.sh sets up a bare Debian/Ubuntu server end to end: Docker, git,
WireGuard, clone, generated secrets, stack, migrations, hub keypair, the Owner
account and a closed firewall. Re-runnable: it keeps an existing .env and never
regenerates the hub key, which would disconnect every onboarded host.
update.sh pulls and applies. Migrations run before the new containers take
traffic, the image is rebuilt only when its definition changed, and the queue
workers are restarted — they are long-running processes that otherwise keep the
old classes in memory, which cost us an hour during development.
clupilot:create-admin creates or promotes an Owner, so re-running the installer
fixes a lost role instead of failing on a taken address.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>