12 Commits (feature/host-bootstrap)

Author	SHA1	Message	Date
nexxo	006ce3568b	Manage hostnames and watch their certificates from the console ... files.clupilot.com is why this exists. DNS pointed at the right machine, the env var was set, the release was deployed — and there was still no certificate, because /etc/caddy/Caddyfile is maintained by hand and nobody thought of it as a second, separate step. Nothing in the console would have said so. Three settings looked correct and the address did not answer. So the page holds two things side by side. The WISH — which names should be served — and the REALITY: whether the name has a certificate and for how much longer. The second is measured by opening a TLS connection and reading the expiry, not by reading configuration, because the configuration is exactly what looked right while the address was dead. verify_peer stays on: a certificate that fails validation is not a certificate for this question, and a display that called it valid would be the fake R19 records. Applying goes through the existing agent, not a new channel. The console writes a request, the path unit wakes the agent within a second, and the agent calls one fixed command line of the root-owned helper. What that helper is allowed to do is the careful part. It fetches the list ITSELF rather than being handed one, and the list is HOSTNAMES, never Caddy blocks — `php artisan clupilot:proxy-hosts` prints `<name> <purpose>` and nothing else. Each name is matched against a strict pattern before it is used, and the template around it lives in the helper, which the service account cannot touch. install-agent.sh already states the principle for the sudoers grant: a grant is only worth anything if the holder cannot change what it grants. A service account that could write proxy configuration would have everything the proxy can do — redirects anywhere, files from any directory. Purpose is a column rather than a habit. A console name gets the network lock, a public one does not, and a console name published without it looks exactly like a working page. Removing takes the name out of the list and NOT out of the running proxy. Two decisions in one click, and the second one takes a site off the air. There is deliberately no "renew" button. Caddy renews on its own at two thirds of the lifetime; what an operator actually needs is a second attempt after an issuance has failed, and that is a reload — which is what Apply does. Thirty days is treated as a problem rather than a warning: at ninety days' lifetime, a renewal should long since have run, so anything under it is not a tight certificate but a renewal that is not happening. The ACME contact falls back to the owner's address, because a contact nobody reads is the step before expired customer certificates. CONTRACT and HOST_STEP_NEEDS both move to 2, which is what tells a server carrying the older helper to run the installer again — caught by the guard test that compares the two halves. 2035 tests pass, assets build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-30 22:55:55 +02:00
nexxo	15b536f393	Mint the archive collection key from the console instead of by hand ... Setting a NAS up to collect the invoice archive meant three machines and a dozen commands: generate a keypair there, carry the public half to the server, write it into authorized_keys with a restriction, install rsync, and get every path right on the first try. Every one of those steps was a place to be told "permission denied" with no clue which of the three was wrong. It was, and several times over. One button now. The panel asks, the host does it — because the host is where all of it lives: the home directory, ssh-keygen, rrsync, and the archive itself. The panel is www-data in a container and owns none of that, so it uses the mailbox it already uses for updates. The private half is shown exactly once, alongside the finished rsync command, and is never written to the database: it exists to be copied into a NAS, and storing it "for convenience" would put a working credential in every backup of that database. rrsync does the restricting, not a pinned rsync option string. That string differs between rsync versions and fails silently — a refusal with no reason given, which is the shape of the afternoon this replaces. If rrsync is missing the agent refuses rather than issuing an unrestricted key while the panel says it is restricted. rsync now comes with install-agent.sh, which already runs as root once per machine. It has to be on the HOST: a NAS connects by ssh and sshd starts `rsync --server` here, so without it the pull fails with "command not found" from a NAS whose own setup is perfectly correct. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-29 10:08:32 +02:00
nexxo	33e5b15099	Restart the workers automatically after saving .env, instead of handing the operator back to the shell ... Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-29 02:26:34 +02:00
nexxo	4d4d1dd5d5	Stop the update agent dying on its own tag arithmetic ... Since 1.2.0 the agent has ended with a non-zero status on every tick, before writing anything. The console showed a check that never finished and a last-checked time frozen at the moment of the deployment, and the update button could not be reached at all — the one mechanism that would have delivered the fix. Two instances of the same trap, both mine, both under `set -Eeuo pipefail`: The tag count was `release_version_gt … && echo`. The LAST iteration is almost always a tag that is not newer, so the loop ended non-zero, pipefail carried it out of the pipeline, the command substitution inherited it, and set -e ended the agent. `if` rather than `&&` is the whole fix: an if whose condition is false still returns 0. The newest tag came from `git tag … | head -1`. head exits after one line, git takes SIGPIPE, pipefail does the rest. install-agent.sh has carried a written warning about exactly this since it was written; I read it and wrote it anyway. Both now live in deploy/lib/release.sh, because the reason neither was caught is that no PHP test can reach a bash pipeline. tests/Feature/ReleaseComparisonTest.php runs the real functions in a real throwaway repository under the agent's own set -Eeuo pipefail, and asserts the EXIT CODE as well as the answer — an answer nobody lives to read is not an answer. The SIGPIPE case needs four hundred tags to reproduce, because a short list finishes writing before head leaves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-29 00:00:17 +02:00
nexxo	00b9107ee3	Deploy as the application's user, and take back what root already took ... The 500 on the VPN config download was never in the VPN code. storage/logs/ laravel.log was owned by root, mode 644, since 27.07 04:16 — the moment a failed `artisan optimize` wrote its error there during a deployment. From then on the application could not append to its own log: Monolog threw on every attempt, and a throw while logging is a 500 on any page that logs, with nothing written down to say why. The error page said "Er wurde protokolliert". It was not. It surfaced on the config download because that is one of the few pages writing a log line on its way through — on success and on a wrong password alike. Pages that log nothing were unaffected, which is exactly why it looked like a VPN fault for days. `docker compose exec` is root unless told otherwise, and every deployment script relied on that default — including the agent's allowlist sync, which runs every minute. docker/entrypoint.sh had it right all along and drops to www-data for precisely these commands. Now so does everything else, help text included: telling an operator to run artisan as root is how the file ends up owned by root in the first place. update.sh also repairs what an earlier run left behind, before its first unprivileged step rather than after — a root-owned log file is not self-healing, the page that trips over it is nowhere near the deployment that caused it, and with in_app unprivileged a root-owned vendor/ would break the next composer step outright. tests/Feature/DeploymentRunsAsTheAppUserTest.php holds the line. It was checked against the previous commit and finds all nine places that were wrong. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-28 22:52:09 +02:00
nexxo	21b7aaca42	Update only to a released version, never to whatever landed on main ... Asked for directly: a commit on main must change nothing on a server, and only a tag whose version is higher than the installed one may be installed. The agent asked two different questions before this. A pinned server compared tags; everything else counted commits on its branch — so a console following main offered an update for every push, verified or not, and "Jetzt aktualisieren" meant "take whatever is on the branch right now". Landing on main and being released are different events, and only the second is a decision somebody made. It is one question now, regardless of what the checkout is attached to: is there a tag whose version is higher than the deployed one? Compared with sort -V against the version in the deployment manifest — not the VERSION file, which an update moves before it migrates, so a run that failed in between would leave the checkout claiming a version it never finished installing. A run always targets that tag; the branch path is gone. A request that arrives with nothing to install is answered rather than obeyed, before anything announces a run. The console follows: the button is disabled with nothing released, says so in a line rather than leaving a grey button to be guessed at, names the version it would install rather than a count, and refuses the action server-side as well — a Livewire action is a public endpoint, and one place a rule may not live is only in the disabled attribute of a button. VERSION becomes 1.1.0, to be tagged as the first release this mechanism can see. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-28 22:32:44 +02:00
nexxo	07634c8a1d	Answer a check at once, start an update at once, and count down to neither ... Checking for updates was a request written into the same mailbox as a real update, collected by the same agent on the same systemd timer — so asking what was new was answered on the next tick, and the console, having nothing else to show, counted down to it. An operator who asked a question was told an update would start in 3:44. A systemd path unit now wakes the agent the moment the request file is written, for a check and for a run alike. PathChanged, not PathExists: PathExists re-arms as soon as the service goes inactive, so a request the agent could not consume — it holds a lock for the length of an update — would restart it in a tight loop. The timer stays as the fallback, and a drop-in disables systemd's start rate limit, because two triggers on one oneshot service can otherwise wedge the unit and leave neither of them able to run. The countdown is gone from both the settings card and the maintenance overlay. It could not be made honest: its target was recomputed on every poll, and where the interval the agent reports is shorter than the timer actually installed on that host it had already gone by, so the fallback moved it forward again each time. It ran backwards four seconds and snapped back to a full minute, forever — reported exactly that way. A queued run says it is starting; a queued check says nothing beyond its badge, because a check starts nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-28 22:25:07 +02:00
nexxo	f05547921d	Separate checking for updates from applying them, and show live progress	2026-07-28 20:23:06 +02:00
nexxo	3699c7cdb9	Ask the tunnel gateway where it actually listens ... The console has been unreachable over the VPN, and the cause was the readiness probe rather than the tunnel. The gateway binds to the WireGuard hub address alone; the probe asked 127.0.0.1, where nothing has ever listened. It could only fail, so VPN_READY stayed false, and on that basis the application withheld the `DNS = 10.66.0.1` line from every client config it issued. A phone then built the tunnel, asked its normal resolver for the console hostname, got the public address and had its connection closed — indistinguishable from "this site does not exist", which is exactly what it looked like. Probing over TLS on the bare address would not have worked either: the site matches on the console's hostname, so a request without SNI is offered no certificate. The gateway now answers a plain-HTTP health port on the hub address, which removes TLS, SNI and name resolution from the question and answers only what is being asked — is this gateway listening, in this network namespace, right now. Caddy refuses to start when the certificate is unreadable, so a health port that answers still proves the whole file loaded. Also here, all found while looking: - Icons pushed their label onto a second line and rendered a size larger than asked for. Tailwind's preflight makes an svg display:block, and `.size-4` and `.size-5` have equal specificity, so stylesheet order decided — and it emits size-4 first. Every icon written as 16px was silently 20px. Recorded as R18. - Four error pages printed `errors.404.hint` in place of a sentence: the lang files give those codes a null hint and Laravel returns the key for a null line. - The Developer role had no label in either language, so the dropdown showed `admin_settings.role_developer`. - The secrets area held one key, which is not worth a password gate. It now carries the credentials that actually stop the business when they expire — DNS, monitoring, SMTP — and the test button appears only where a checker exists, instead of reporting on Stripe whatever was being looked at. - The update button said nothing about when a queued run would start or where a running one had got to; both are now shown, and a failure names its step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-27 14:43:23 +02:00
Claude	457eeeeaef	Serve the console inside the tunnel, without publishing the internal network ... The owner's phone shows a blank page on the VPN. That page is the reverse proxy refusing the request, and it refuses because the phone is not on the tunnel: the client config routes only the management subnet, so a request to the console's public hostname leaves over the mobile network and arrives from a carrier address. It is also why the peer reads "last contact: never" — WireGuard only handshakes when it has traffic to send, and nothing is ever routed in. The obvious fixes are both wrong. Adding the server's public address to AllowedIPs routes the WireGuard endpoint into the tunnel it is trying to establish. A public DNS record pointing at 10.66.0.1 publishes the internal subnet to anyone enumerating the domain — the owner raised that himself, and he was right. So the console is served INSIDE the tunnel instead. A small resolver and a small gateway share the hub container's network namespace, which is where wg0 lives, and answer on 10.66.0.1 directly. A client therefore needs no route beyond the subnet it already has, the host's docker bridge is never exposed, and the request reaches the application with its real 10.66.0.x source — which is what the console's own allowlist checks. The gateway reuses the certificate the public Caddy already renews: a certificate is bound to the name, not to the address serving it, so nothing new is issued and nothing internal reaches a public zone. Most of this commit is the arithmetic of not lying about it. Review found fourteen ways the arrangement could report itself working while it was not: enabling the compose profile after the services were started rather than before; starting a gateway with empty certificate paths, which can only crash; treating "container created" as "container running"; carrying a stale readiness marker through a failed restart; reusing the previous hostname's certificate after a rename; leaving the profile enabled when the hostname was cleared, in a script that exits early precisely when nothing else would fix it; reading the renewal timestamp as a user who cannot traverse Caddy's storage; and a `find | head` that aborts the whole installer under pipefail the moment two issuers hold a certificate for the same name. One of them was mine and worth naming: to let the container read the certificate, I had made the TLS private key world-readable. On a multi-user host that hands the console's identity to every local account. The client is told about the resolver only when both services are confirmed running, because a config naming a resolver that does not exist takes the device's entire name resolution with it for as long as the tunnel is up. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-27 10:14:33 +02:00
Claude	bc8bbc56a5	Make the console's access list reach the proxy, price in euros, and answer errors ... The console was unreachable, and not for the reason it looked like. Two causes, both mine. A failed `artisan optimize` — the one that hit duplicate route names — left a broken route cache behind, so every request to the console host answered 500. Rebuilt. Underneath that: the reverse proxy has its own console allowlist, hard-coded, and it runs BEFORE the application. Everything added on the console's own access page was therefore ineffective, and when the owner's address changed they were turned away by the proxy before reaching the page that would have fixed it. The proxy now imports a fragment generated from the same list the console manages, regenerated by the host agent and reloaded when it changes. Getting that safe took most of the review. It refuses to rewrite an ambiguous Caddyfile rather than replacing some other site's matcher; it never falls back to a loopback-only list when the application cannot be reached, because that list validates cleanly and locks out every remote operator; it retries a reload that failed instead of assuming it worked; and an installation that upgrades without rerunning the installer is told, because otherwise the whole mechanism is invisibly absent. Prices are entered in euros. The form asked for cents, so €799 was typed as 79900 and one slipped digit was a factor of ten on an invoice. Conversion happens in one place, on the string rather than through a float — (float) '79.90' × 100 is 7989.999… and casting truncates to 7989, one cent short on exactly the prices people charge — and it refuses an amount the column cannot hold instead of failing at the database. Every error code has a page now, in the site's own language and typeface, self-contained so it still renders when the asset manifest is the thing that broke. There was only a bare white 404. The VPN list stops being a seven-column table nobody could fit: names broke across two lines and so did the headings. These are attributes of one access, not quantities compared down a column, so each access is a row — identity on one line, measurements in mono on the next. The status page says what it measures rather than what it promises. "New orders are delivered without failures" read as a marketing claim on a page whose only job is to be believed, and said nothing about the last 24 hours. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-27 06:51:05 +02:00
Claude	276f926d57	Add the update button, and give it a host-side agent that can actually update ... The panel had no way to update the installation, and could not have one on its own: it runs as www-data inside a container, and the update restarts that container. A button that shelled out would kill its own web server mid-response, and would need the application to hold host-level credentials — a far worse thing to own than an out-of-date checkout. So the button is a mailbox. The panel writes a request into the checkout; a systemd timer on the host, running as the service account, consumes it, runs deploy/update.sh and writes back what happened. The same timer answers the question the panel cannot answer alone — is there anything to update — which needs a git fetch, and therefore credentials the application does not have. The parts that were wrong before review, all of which would have shipped as a button that looks fine and does nothing: - The agent was only installed by install.sh, which the installed base never runs again. Every existing server would have shown the button permanently disabled. It has its own root entry point now, install.sh delegates to it, and update.sh says so when the unit is missing. - On a release-pinned server the checkout is detached, so it followed "origin/HEAD" and then ran an update that deliberately stays on the pinned tag: updates advertised, nothing applied. It now compares TAGS in release mode and passes the target release through. - On a branch other than main it advertised that branch's commits and then deployed main's, because update.sh defaults to main. - A request written while the agent was down was executed whenever the agent next started, days later. - A single status file meant the routine five-minute check overwrote a failed update with "idle" before anyone saw it. - An agent that had been stopped left the button enabled forever, because a status file written once counted as an agent. - The agent's error messages were German strings rendered into an English interface; it reports codes now and the panel translates them. Also: the add-address button in the console-access panel was as tall as the field plus its hint, because the hint was a sibling inside the flex row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-07-27 05:33:17 +02:00