Metric history: the per-minute sampler (clusev:sample-metrics) now reads
cpu/mem/disk from the `servers` DB row — the same source the live gauges read,
shared via MariaDB — instead of a cross-container Redis cache the poller may
not share in prod. It gates on last_seen_at freshness (skip offline/unpolled)
and keeps `load` as a best-effort cache read. History now fills whenever the
gauges have data, independent of the cache driver or container split — the
prod detail page no longer shows "Noch keine Verlaufsdaten" after hours.
Service-action failures are persisted: a failed systemctl start/stop/restart
writes a red service.<op>.failed AuditEvent with the FULL systemctl reason in
meta['output'] (mb_scrub-ed, 4000-cap), shown expandable in the audit log — a
truncated, ephemeral toast is no longer the only trace. The failure toast is a
short pointer to the audit log, level=error, and error toasts now linger 9s.
Alert delivery: a warning banner appears on the alerts page when NO channel
would reach anyone (no SMTP + no safe webhook + no Gotify). A new Gotify push
channel (server URL + encrypted app token) posts title/message/priority with
the token in X-Gotify-Key; a self-hosted LAN Gotify is allowed (scheme-checked,
not private-range-blocked like the SSRF-guarded webhooks) — a real push when a
service goes down, no e-mail needed.
Secret-retarget hardening (Codex-found class): changing a target while leaving
its secret blank could send the OLD secret to the NEW endpoint. Closed for the
Gotify token, the SMTP password (save + test-send + the runtime mailer), and
the host-terminal SSH credential (username/port change) — each drops/forbids
the stale secret before the new target is stored. Setting::uncached() reads the
security-sensitive keys DB-direct so a cache-repopulation race can't re-pair an
old secret with a new target. Three Codex rounds, clean; 778 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Uptime monitoring for the services the fleet hosts: HTTP and TCP probes with a live up/down status
board. Completes the monitoring trio (alerts, certs, uptime).
- HealthService::probe(check): native — Laravel HTTP client (withoutRedirecting; reports only the
status code + latency, never the response body → no internal-content exfil) for http, a PHP TCP
stream for tcp. No SSH, no shell, so the target is never interpolated into a command. 2xx/3xx = up;
a 4xx/5xx means "answered but unhealthy"; a connect error = down.
- Health\Index page (route /health, `operate`-gated: route + mount + per-method). Add/remove checks
(http target validated as a real http(s) URL; tcp target as a hostname/IP + a required port). Lazy
per-check probe, each guarded. Up/total summary. Delete via signed ConfirmToken + R5 modal.
Add/delete audited.
- lang/{de,en}/health.php + audit.php health.check_* + shell.nav_health (de/en parity). No emoji.
11 new tests: service (http 2xx up / 5xx down / connection-error down, tcp refused-port down),
component (route gating, add http + audit, reject non-url http target, tcp requires host+port,
reject missing port, scan probes each check, delete via confirmed token). 742 tests green, Pint,
lang parity, self-reviewed. A PUBLIC (unauthenticated) status page is a documented future.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Monitor the certificate expiry of configured TLS endpoints (the panel's own domain, fleet
services, anything reachable) and flag the ones due soon — before they lapse.
- CertService::check(host, port) reads the peer certificate NATIVELY via a PHP SSL stream +
openssl_x509_parse — no SSH, no shell, so `host` is never interpolated into a command. Trust is
intentionally NOT verified (verify_peer off): the goal is to report expiry even for a self-signed
or already-expired cert. certInfo() (subject/issuer/expiry/daysLeft) is split out for unit testing
against a generated cert; status() maps daysLeft → ok / warning (<30) / critical (<7) / expired.
- Certs\Index page (route /certs, `operate`-gated: route + mount + per-method). Add/remove endpoints
(host validated as a hostname/IP — rejects scheme/path/shell chars; port 1-65535). Lazy per-endpoint
scan, each guarded. Delete via signed ConfirmToken + R5 modal. Add/delete audited. Internal
endpoints are allowed on purpose (monitoring internal service certs is legitimate; the check only
reads a cert, it POSTs nothing).
- lang/{de,en}/certs.php + audit.php cert.endpoint_* + shell.nav_certs (de/en parity). No emoji.
10 new tests: service (certInfo extracts subject/expiry from a generated cert, status thresholds,
check errors on an unreachable endpoint), component (route gating, add + audit, reject scheme/
metachar host + out-of-range port, scan checks each endpoint, delete via confirmed token). 730 tests
green, Pint, lang parity, self-reviewed. LE issuance is a documented future (needs ACME on the host).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fleet-wide package-update visibility: pending + SECURITY update counts per server, one-click patch.
Extends MaintenanceService (single-server pending/apply already existed) with the security subset
and a fleet aggregation.
- MaintenanceService::updateCounts(Server) → { pending, security } in ONE round-trip via a new
PackageManager::countsScript() (apt/dnf/zypper). Security is best-effort per family (apt path solid;
dnf/zypper degrade to 0 when the metadata isn't there) and clamped ≤ pending. Either value is null
when the query can't run (shows "unknown", never a misleading 0).
- Patch\Index page (route /patch, `operate`-gated: route + mount + per-method). Lazy per-server scan,
each guarded (an unreachable host → error row). Fleet KPIs (total pending / total security). Applying
upgrades is a stateful action → signed ConfirmToken + R5 modal (modal audits patch.apply from the
sealed token; handler does not re-audit) resolving a client uuid → the sealed server id; a forged
token is a no-op. Reuses the existing applyUpgrades; the result output is mb_scrub'd before it lands
in a Livewire prop.
- countsScript strings are STATIC per manager (no operator input) — no injection surface.
- lang/{de,en}/patch.php + audit.php patch.apply + shell.nav_patch (de/en parity). No emoji.
11 new tests: MaintenanceService (parse pending+security, clamp security≤pending, missing-sentinel /
unsupported-manager → unknown), component (route gating, scan shows counts + survives an unreachable
server, patch opens confirm, applyPatch runs upgrades over the SEALED server, forged token no-op).
720 tests green, Pint, lang parity, self-reviewed (RBAC, ConfirmToken seal, static scripts, scrub).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The multi-server amplifier: run a command (ad-hoc or a saved runbook) across the whole fleet,
a group, or a selection, with a per-server result. Because it is arbitrary remote execution,
authorization + audit are the whole feature.
- The command is intentionally arbitrary; it runs via FleetService::runPlain (the credential's
login user, base64 transport, NO extra sudo) — the same privilege the web terminal already
grants, batched. No injection to prevent; the concern is who may run + that every run is logged.
- CommandRunner.run: per-server runPlain, one server failing never aborts the batch, output
byte-capped (256 KiB) + mb_scrub'd before it reaches a Livewire property.
- Commands\Index page, fully `operate`-gated: route can:operate + mount() + a per-method gate() on
run/runRunbook/execute/createRunbook/confirmDeleteRunbook/deleteRunbook. Every run goes through a
signed single-use ConfirmToken + R5 confirm modal ("run on N servers?"); the modal audits
command.run from the sealed token so execute() does not re-audit. Targets resolve client UUIDs →
ids server-side (withActiveCredential + inGroup + whereIn), and {command, serverIds} is SEALED in
the token — a client can't swap the command or targets after confirm. A forged token is a no-op.
- Runbooks: saved name+command, operate-gated CRUD, delete via ConfirmToken.
- lang/{de,en}/commands.php + audit.php command.run / runbook.* + shell.nav_commands (de/en). No emoji.
15 new tests: CommandRunner (runs each server, one-failing-doesn't-abort, nonzero=not-ok, byte-cap
+scrub), component (route gating, ad-hoc run opens confirm, empty-command / no-target errors, execute
runs the SEALED targets, forged token runs nothing, group scope targets only members, runbook create
+audit + run + delete-via-token). 700 tests green, Pint, lang parity, Codex-reviewed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Manage the containers running ON a fleet server, agentlessly — the integrated Portainer.
List containers + state, view logs, start/stop/restart, all via `docker` over SSH.
- DockerService (over FleetService::runPrivileged → base64 transport, shell-injection-safe):
containers() parses `docker ps -a --format '{{json .}}'`; containerAction() with an op allow-list
(start|stop|restart|pause|unpause); logs() with a clamped tail; composeStacks() read-only. Every
container ref is validated `^[a-zA-Z0-9][\w.-]*$` before interpolation — blocks shell metacharacters
AND a leading-dash argument injection (mirrors serviceAction). A missing/errored docker → [] (never
throws to the UI).
- Docker\Index page (route /docker, sidebar "Docker" with a new box icon): lazy container list for the
active server, open to any role. Container actions are `operate`-gated (abort_unless per method) and
audited (docker.action). Reading LOGS is also operate-gated (container output can leak secrets —
consistent with gating file CONTENT reads); the list/state stays open. ContainerLogs modal re-gates
the read itself.
- lang/{de,en}/docker.php + audit.php docker.action + shell.nav_docker (de/en parity). No emoji.
Codex hardening applied: logs output is byte-capped (head -c 262144) AND mb_scrub'd before it reaches
a Livewire property (invalid UTF-8 / a huge line can't break the JSON snapshot); the ref regex uses
\A…\z anchors so a trailing newline can't slip past.
16 new tests: service (json parse, empty-on-unavailable, valid-op command, reject unknown-op / shell-
metachars / leading-dash / trailing-newline ref, logs clamp+byte-cap, available probe), component
(viewer browses the list; viewer 403 on action AND on logs open + modal read; operator/admin act +
audited; logs modal tail). 685 tests green, Pint, lang parity, Codex-reviewed (fixes applied).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Turns the existing metrics into an ops tool: rules that fire when a server crosses a
threshold and resolve when it recovers, with e-mail + webhook notifications. The event
sink later features (cert-expiry, uptime-down, patch-available, posture-drop) plug into.
- alert_rules (metric cpu|mem|disk|load|offline, comparator gt|lt, threshold, scope
all|group|server) + alert_incidents (firing|resolved, FK cascade).
- AlertEvaluator — per-server state machine: opens ONE firing incident on a fresh breach
(notifies), resolves on recovery (notifies), dedups a sustained breach (one breach = one
alert). Numeric metrics skip a truly-offline server (stale reading); the `offline` rule
fires only on offline, not a reachable "warning".
- AlertNotifier — best-effort e-mail (queued Mailable to configured recipients, falls back
to admin e-mails) + generic webhook JSON POST (Slack/Discord/Mattermost/custom; only http(s)
URLs). Every channel is wrapped so a broken SMTP/unreachable hook is logged, never thrown.
- PollMetrics evaluates after each poll (online, fresh status) and on a failed poll (offline),
each guarded so an alerting error can never kill the poll loop.
- Alerts\Index page, manage-panel (admin): route can:-mw + mount() + per-method gate() on every
mutation. Rule delete via signed ConfirmToken + R5 modal (no double audit). Scope group/server
resolves a client UUID to an id server-side (never trusts a client id). Channels saved to
Settings. Sidebar "Alarme" nav + firing-incident badge (cached 60s).
- lang/{de,en}/alerts.php + audit.php alert.* + shell.nav_alerts (de/en parity). No emoji.
Codex review raised three MEDIUMs, all fixed here:
- Webhook SSRF: strict http(s) scheme + reject hosts resolving to loopback/private/link-local/
reserved ranges (blocks the 169.254.169.254 cloud-metadata classic), validated at save AND send,
redirects disabled — so an admin-configured hook can't be pointed at an internal service.
- Incident dedup is now a DB invariant: a unique `firing_key` ("{rule}:{server}" while firing,
null on resolve) means two concurrent poll ticks can't both open a duplicate incident (the loser
hits a unique violation, caught → no-op).
- `load` is float-safe: threshold + incident value are decimal, compared as floats, so load 1.5 vs
threshold 1 fires (it used to truncate to 1 > 1 = false).
31 new tests: evaluator (fire/resolve/dedup, offline vs warning, scope all/group/server, disabled,
fractional-load, firing_key unique, key-freed-on-resolve), notifier (email/fallback/webhook/
non-http-ignored/failure-swallowed/SSRF-reject/public-accept), component (RBAC route gating, rule
CRUD, uuid→id scope resolve, offline→threshold 0, toggle, delete via token, channels, unsafe-webhook
rejected, incidents render). 669 tests green, Pint, lang parity, Codex-reviewed (fixes applied).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Foundation for the feature program (groups are the scoping primitive alerts, ad-hoc
commands, bulk actions and patch/posture will target). Lean by design: named groups,
no hierarchy, no per-group RBAC.
- server_groups (uuid route key, unique name, color = a @theme token key coerced to a real
token on save) + server_server_group M2M pivot (cascade on delete; servers survive a group
delete). Server::groups() + a `scopeInGroup`.
- Servers\Groups full-page component, manage-fleet (admin) only — route can:-mw + mount()
abort_unless + a per-method gate() on every mutation (create/rename/setColor/toggleMember/
confirmDelete/deleteGroup), since /livewire/update does not re-run route mw. create/rename/
recolour/membership audit directly; DELETE goes through the signed single-use ConfirmToken +
the R5 confirm modal (which writes group.delete from the sealed descriptor, so the handler
does not double-audit). 'groupConfirmed' added to ConfirmToken::ACTIONS.
- /servers/groups is registered BEFORE /servers/{server} so "groups" isn't captured as a uuid.
- Servers\Index: a #[Url] group filter (uuid; shareable) — an unknown uuid falls back to all,
never errors. Filtering is open to every role; only management is admin-gated. A "Gruppen"
button (admin) on the fleet header; colour pills use a literal @theme token map (R3).
- lang/{de,en}/groups.php + audit.php group.* labels (de/en parity). No emoji.
19 new tests (model + component: uuid/colour-allow-list/relations/cascade, RBAC route gating,
create/rename/recolour/membership, delete-via-confirmed-token, forged-token no-op, filter).
638 tests green, Pint, lang parity, Codex-reviewed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The audit log (readable by every role) rendered "Honeypot-Treffer" / "Honeypot hit" next to
the decoy paths, which advertises the deception layer to anyone with panel/audit access — an
attacker seeing it would know wp-admin/phpmyadmin/etc. are traps. Rename the VISIBLE labels to
neutral WAF/IDS-style threat logging so the mechanism is no longer self-revealing. The DB action
keys stay security.honeypot_* (Threats intel, history, error styling, and tests key off them);
only the human-facing text changes:
security.honeypot_hit -> "Angriffsversuch" / "Attack attempt"
security.honeypot_login -> "Login-Angriffsversuch" / "Login attack attempt"
security.honeytoken_used -> "Kompromittierte Zugangsdaten" / "Compromised credentials"
This covers the /audit page, the dashboard audit panel, and the Threats feed rows (all resolve
the same audit.actions label). A regression test asserts the audit log renders the disguised
labels and never leaks "Honeypot"/"Honeytoken" (nor the raw key). 618 tests, de/en parity, Pint.
NOTE: the admin-only Threats page header (subtitle/feed title/Honeytoken KPI) and the settings
honeypot-status section still name it "Honeypot" — those are manage-panel-gated; disguising them
too is a separate, optional follow-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The audit log loaded the latest 50 entries and filtered them in memory, so it
couldn't grow and search only spanned those 50. Now it is paginated (25/page)
with the page deep-linked in the URL via Livewire WithPagination (?page=), and
the search runs in SQL across the whole history — matching actor / action code /
target / server name AND the readable labels (a term is translated to the action
codes whose localized label contains it, so "Endpoint geändert" still finds
wg.set-endpoint). Pagination control mirrors the changelog's windowed pager.
Tests: 25/page with page 2 overflow, search resets to page 1, search filters in
SQL across all pages (a match on page 2 surfaces when filtered). 365 pass, Pint
clean, /audit loads 200 with no console errors, audit lang parity 62/62.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The audit log showed raw machine codes ("Administrator · wg.set-endpoint").
Now each entry renders a localized human-readable label ("WireGuard endpoint
changed"), via an AuditEvent::action_label accessor backed by an audit.actions
lang map (DE/EN); unmapped/dynamic codes (e.g. harden.*.on|off) fall back to a
tidied form, never a bare code. Search also matches the readable label.
Failures are now visible: AuditEvent::is_error flags failed/security events
(failed sign-in, IP ban, failed 2FA, wg.action-failed) which render in red with
an alert icon. And failed WireGuard dashboard actions are now written to the
audit log with the host error message (not just a transient toast), so the
operator can read later what went wrong.
Tests: label mapping + fallback, is_error, page renders label not code, search
by label, WG failure auditing. 360 pass, Pint clean, /audit loads 200 with no
console errors, audit lang parity 59/59.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
boban