Metric history fix, service-failure audit trail, alert channels + secret-retarget hardening

Metric history: the per-minute sampler (clusev:sample-metrics) now reads
cpu/mem/disk from the `servers` DB row — the same source the live gauges read,
shared via MariaDB — instead of a cross-container Redis cache the poller may
not share in prod. It gates on last_seen_at freshness (skip offline/unpolled)
and keeps `load` as a best-effort cache read. History now fills whenever the
gauges have data, independent of the cache driver or container split — the
prod detail page no longer shows "Noch keine Verlaufsdaten" after hours.

Service-action failures are persisted: a failed systemctl start/stop/restart
writes a red service.<op>.failed AuditEvent with the FULL systemctl reason in
meta['output'] (mb_scrub-ed, 4000-cap), shown expandable in the audit log — a
truncated, ephemeral toast is no longer the only trace. The failure toast is a
short pointer to the audit log, level=error, and error toasts now linger 9s.

Alert delivery: a warning banner appears on the alerts page when NO channel
would reach anyone (no SMTP + no safe webhook + no Gotify). A new Gotify push
channel (server URL + encrypted app token) posts title/message/priority with
the token in X-Gotify-Key; a self-hosted LAN Gotify is allowed (scheme-checked,
not private-range-blocked like the SSRF-guarded webhooks) — a real push when a
service goes down, no e-mail needed.

Secret-retarget hardening (Codex-found class): changing a target while leaving
its secret blank could send the OLD secret to the NEW endpoint. Closed for the
Gotify token, the SMTP password (save + test-send + the runtime mailer), and
the host-terminal SSH credential (username/port change) — each drops/forbids
the stale secret before the new target is stored. Setting::uncached() reads the
security-sensitive keys DB-direct so a cache-repopulation race can't re-pair an
old secret with a new target. Three Codex rounds, clean; 778 tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
feat/v1-foundation
boban 2026-07-08 22:21:54 +02:00
parent 182cb0fedd
commit 265136fc42
26 changed files with 661 additions and 33 deletions

View File

@ -8,10 +8,15 @@ use Illuminate\Console\Command;
use Illuminate\Support\Facades\Cache;
/**
* Persist a once-a-minute resource sample per server from the live (cache-backed) reading the
* 15s poller keeps fresh the long history the Server-Details graph plots. No extra SSH: a server
* with no fresh cached reading (not polled / offline) simply gets no sample (an honest gap). Prunes
* beyond the retention window. Scheduled everyMinute in routes/console.php.
* Persist a once-a-minute resource sample per server the long history the Server-Details graph
* plots. The source of truth is the `servers` DB ROW the 15s poller forceFills every tick (cpu/mem/
* disk + last_seen_at), NOT a cache: the poller and this command run in SEPARATE containers in prod,
* so a cache-backed read only worked when the cache store happened to be shared. Reading the row
* means history is collected whenever the live gauges have data (they read the same row), regardless
* of the cache driver. A server not freshly polled (never polled / offline last_seen_at stale)
* simply gets no sample (an honest gap). `load` is not on the row, so it is taken from the live cache
* best-effort when present (stored but not plotted). Prunes beyond the retention window. Scheduled
* everyMinute in routes/console.php.
*/
class SampleMetrics extends Command
{
@ -22,20 +27,24 @@ class SampleMetrics extends Command
public function handle(): int
{
$now = now();
// Only servers the 15s poller updated recently — a stale row means offline / no credential,
// and sampling it would plot a flatline of its last-known values. 90s tolerates a missed tick.
$fresh = $now->copy()->subSeconds(90);
$rows = [];
foreach (Server::query()->get(['id']) as $server) {
$m = Cache::get("metrics:latest:{$server->id}");
if (! is_array($m)) {
foreach (Server::query()->get(['id', 'cpu', 'mem', 'disk', 'last_seen_at']) as $server) {
if ($server->last_seen_at === null || $server->last_seen_at->lt($fresh)) {
continue;
}
// load is not persisted on the row; best-effort from the live cache when the store is shared.
$cached = Cache::get("metrics:latest:{$server->id}");
$rows[] = [
'server_id' => $server->id,
// Percentages — clamp to 0..100 so a stray reading can never plot outside the chart.
'cpu' => min(100, max(0, (int) ($m['cpu'] ?? 0))),
'mem' => min(100, max(0, (int) ($m['mem'] ?? 0))),
'disk' => min(100, max(0, (int) ($m['disk'] ?? 0))),
'load' => round((float) ($m['load'] ?? 0), 2),
'cpu' => min(100, max(0, (int) $server->cpu)),
'mem' => min(100, max(0, (int) $server->mem)),
'disk' => min(100, max(0, (int) $server->disk)),
'load' => is_array($cached) ? round((float) ($cached['load'] ?? 0), 2) : 0.0,
'sampled_at' => $now,
];
}

View File

@ -11,8 +11,10 @@ use App\Models\Setting;
use App\Services\AlertNotifier;
use App\Support\Confirm\ConfirmToken;
use App\Support\Confirm\InvalidConfirmToken;
use App\Support\MailSettings;
use Illuminate\Contracts\View\View;
use Illuminate\Support\Facades\Auth;
use Illuminate\Support\Facades\Crypt;
use Illuminate\Validation\Rule;
use Livewire\Attributes\Layout;
use Livewire\Attributes\On;
@ -45,11 +47,20 @@ class Index extends Component
public string $webhooks = '';
// Gotify push: base server URL (plaintext) + app token (stored encrypted, write-only in the UI).
public string $gotifyUrl = '';
public string $gotifyToken = '';
public bool $gotifyTokenSet = false;
public function mount(): void
{
abort_unless(Auth::user()?->can('manage-panel'), 403);
$this->emailTo = (string) Setting::get('alert_email_to', '');
$this->webhooks = (string) Setting::get('alert_webhooks', '');
$this->gotifyUrl = (string) Setting::get('alert_gotify_url', '');
$this->gotifyTokenSet = ((string) Setting::get('alert_gotify_token', '')) !== '';
}
public function title(): string
@ -165,15 +176,74 @@ class Index extends Component
}
}
// Gotify: scheme-checked (a self-hosted server on a LAN is allowed — no private-range block).
$gotifyUrl = trim($this->gotifyUrl);
if ($gotifyUrl !== '' && ! AlertNotifier::isValidGotifyUrl($gotifyUrl)) {
$this->addError('gotifyUrl', __('alerts.gotify_invalid'));
return;
}
$newToken = trim($this->gotifyToken);
$urlChanged = $gotifyUrl !== (string) Setting::get('alert_gotify_url', '');
// If the URL changes, DROP the old token BEFORE the new URL is stored — so no concurrent
// send can ever pair the old token with the new (possibly attacker-controlled) endpoint. A
// fresh token (if entered) is written afterwards; otherwise re-entry is required.
if ($urlChanged) {
Setting::forget('alert_gotify_token');
$this->gotifyTokenSet = false;
}
Setting::put('alert_email_to', trim($this->emailTo));
Setting::put('alert_webhooks', trim($this->webhooks));
Setting::put('alert_gotify_url', $gotifyUrl);
// Encrypt a NEWLY entered token; an empty field on an UNCHANGED url leaves the stored one.
if ($newToken !== '') {
Setting::put('alert_gotify_token', Crypt::encryptString($newToken));
$this->gotifyToken = '';
$this->gotifyTokenSet = true;
}
$this->audit('alert.channels_update', __('alerts.channels'));
$this->dispatch('notify', message: __('alerts.channels_saved'));
}
/** Remove the stored Gotify app token (write-only credential). */
public function clearGotifyToken(): void
{
$this->gate();
Setting::forget('alert_gotify_token');
$this->gotifyToken = '';
$this->gotifyTokenSet = false;
$this->audit('alert.channels_update', __('alerts.channels'));
$this->dispatch('notify', message: __('alerts.channels_saved'));
}
/** True when at least one delivery channel could actually reach someone (drives the warning).
* Reads the SAVED config with the SAME validation the sender uses, so the banner never hides
* for a channel that would be silently skipped at send time. */
private function channelsReach(): bool
{
if (MailSettings::isConfigured()) {
return true;
}
$webhooks = array_filter(
array_map('trim', preg_split('/[\r\n,]+/', (string) Setting::get('alert_webhooks', '')) ?: []),
fn (string $u) => $u !== '' && AlertNotifier::isSafeWebhookUrl($u),
);
if ($webhooks !== []) {
return true;
}
return AlertNotifier::gotifyTarget() !== null;
}
public function render(): View
{
return view('livewire.alerts.index', [
'channelsReach' => $this->channelsReach(),
'rules' => AlertRule::orderBy('name')->get(),
'incidents' => AlertIncident::with(['rule', 'server'])->where('state', 'firing')->latest('started_at')->limit(50)->get(),
'groups' => ServerGroup::orderBy('name')->get(['uuid', 'name']),

View File

@ -93,7 +93,15 @@ class HostShell extends ModalComponent
$existing = HostCredential::current();
// Switching the auth method (password ↔ key) requires a fresh secret: keeping the stored one
// would feed e.g. an old password into key auth (or vice versa) and silently break the login.
if ($this->secret === '' && $existing !== null && $this->authType !== $existing->auth_type) {
// A fresh secret is required whenever the auth method OR the target (username/port) changes:
// keeping the stored secret would feed an old password into key auth (or vice versa) and
// silently break the login, or reuse one account's secret against a different account/port.
$targetChanged = $existing !== null && (
$this->authType !== $existing->auth_type
|| trim($this->username) !== $existing->username
|| (int) $data['port'] !== $existing->port
);
if ($this->secret === '' && $targetChanged) {
throw ValidationException::withMessages([
'secret' => __('terminal.host_secret_required_for_auth_change'),
]);

View File

@ -3,10 +3,10 @@
namespace App\Livewire\Services;
use App\Livewire\Concerns\WithFleetContext;
use App\Models\AuditEvent;
use App\Services\FleetService;
use App\Support\Confirm\ConfirmToken;
use App\Support\Confirm\InvalidConfirmToken;
use Illuminate\Support\Str;
use Livewire\Attributes\Layout;
use Livewire\Attributes\On;
use Livewire\Component;
@ -166,9 +166,27 @@ class Index extends Component
return;
}
$this->dispatch('notify', message: $res['ok']
? __('services.action_done', ['name' => $name, 'op' => $op])
: __('services.action_failed', ['name' => $name, 'output' => Str::limit($res['output'] ?: __('services.no_permission'), 90)]));
if ($res['ok']) {
$this->dispatch('notify', message: __('services.action_done', ['name' => $name, 'op' => $op]));
} else {
// A toast is ephemeral and truncated — persist the FULL failure reason so the operator
// can re-read WHY a start failed (e.g. a service that crashes on launch). The confirm
// modal already logged the neutral attempt (service.<op>); this red entry records the
// outcome with the complete systemctl output in meta, shown expandable in the audit log.
AuditEvent::create([
'user_id' => auth()->id(),
'server_id' => $active->id,
'actor' => auth()->user()?->name ?? 'system',
'action' => 'service.'.$op.'.failed',
'target' => $name.' · '.$active->name,
'ip' => request()->ip(),
'meta' => ['output' => mb_substr(mb_scrub(trim((string) ($res['output'] ?: __('services.no_permission')))), 0, 4000)],
]);
$this->dispatch('notify',
message: __('services.action_failed_short', ['name' => $name]),
level: 'error');
}
try {
$this->services = $fleet->systemd($active)['services'];

View File

@ -72,6 +72,20 @@ class Email extends Component
abort_unless(auth()->user()?->can('manage-panel'), 403);
$this->validate();
// Retarget guard: if the SMTP HOST or USERNAME changes and no new password was entered, DROP
// the stored password BEFORE the new host is written — so the old credential can never be
// sent to a new (possibly attacker-controlled) SMTP server; re-entry is then required. Same
// class as the alert Gotify token guard. uncached() reads the committed state (no stale cache).
$targetChanged = $this->mail_host !== (string) Setting::uncached('mail_host', '')
|| $this->mail_username !== (string) Setting::uncached('mail_username', '');
if ($targetChanged) {
// Drop the old password BEFORE any target field is written — regardless of whether a
// fresh one was typed — so no concurrent MailSettings::apply() can ever read the new
// host paired with the old password. A freshly typed password is re-written below.
Setting::forget('mail_password');
$this->passwordSet = false;
}
Setting::put('mail_host', $this->mail_host);
Setting::put('mail_port', (string) $this->mail_port);
Setting::put('mail_username', $this->mail_username);
@ -156,6 +170,17 @@ class Email extends Component
}
RateLimiter::hit($key, 600); // 3 test e-mails / 10 minutes per user
// Retarget guard: never test-send the STORED password against a host/username the operator
// just changed in the form but hasn't saved — that could leak the old credential to a new
// (possibly hostile) server. Require the password to be re-entered for a changed target.
if ($this->mail_password === ''
&& ($this->mail_host !== (string) Setting::uncached('mail_host', '')
|| $this->mail_username !== (string) Setting::uncached('mail_username', ''))) {
$this->dispatch('notify', message: __('mail.notify_test_failed', ['error' => __('mail.test_retarget_password_required')]), level: 'error');
return;
}
$to = Auth::user()->email;
try {
@ -186,13 +211,20 @@ class Email extends Component
*/
private function applyMailerConfig(): void
{
// Prefer a FRESHLY typed password (test-before-save); fall back to the stored one only when
// the field is blank — and the sendTest() retarget guard has already ensured that fallback
// is used only when host+username are unchanged from what the stored password was set for.
$password = '';
$stored = Setting::get('mail_password');
if ($stored !== null && $stored !== '') {
try {
$password = Crypt::decryptString($stored);
} catch (Throwable) {
$password = '';
if ($this->mail_password !== '') {
$password = $this->mail_password;
} else {
$stored = Setting::uncached('mail_password');
if ($stored !== null && $stored !== '') {
try {
$password = Crypt::decryptString($stored);
} catch (Throwable) {
$password = '';
}
}
}

View File

@ -61,6 +61,7 @@ class AuditEvent extends Model
'fail2ban.ban', 'wg.action-failed', 'deploy.staging_release_failed',
'deploy.public_failed', 'deploy.stable_failed', 'deploy.yank_failed',
'security.honeypot_hit', 'security.honeypot_login', 'security.honeytoken_used',
'service.start.failed', 'service.stop.failed', 'service.restart.failed',
];
/**

View File

@ -26,6 +26,17 @@ class Setting extends Model
return $value ?? $default;
}
/**
* Read a setting DIRECTLY from the DB, bypassing the read-through cache. For SECURITY-SENSITIVE
* values (e.g. an alert token) where a concurrent cache-miss could re-populate a STALE value just
* after a change and pair an old secret with a new target the cache-aside race the plain get()
* is exposed to. Costs one query; use only where staleness is a correctness/security bug.
*/
public static function uncached(string $key, ?string $default = null): ?string
{
return static::query()->find($key)?->value ?? $default;
}
public static function put(string $key, ?string $value): void
{
static::query()->updateOrCreate(['key' => $key], ['value' => $value]);

View File

@ -8,6 +8,7 @@ use App\Models\AlertIncident;
use App\Models\Setting;
use App\Models\User;
use App\Support\MailSettings;
use Illuminate\Support\Facades\Crypt;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Mail;
@ -30,6 +31,80 @@ class AlertNotifier
$this->email($incident, $resolved);
$this->webhooks($incident, $resolved);
$this->gotify($incident, $resolved);
}
/**
* Gotify push (self-hostable push server): POST /message with the app token in the X-Gotify-Key
* header + a {title, message, priority} body the shape Gotify expects (the generic webhook's
* bare `text` field would not render). A firing alert uses a high priority (8) so it pushes/rings;
* a resolve is low (3). Best-effort like every channel. Unlike the SSRF-guarded webhooks a Gotify
* server is COMMONLY on a private LAN, so only the scheme is enforced (isValidGotifyUrl), not the
* private-range block that is the operator's deliberate, admin-only choice.
*/
private function gotify(AlertIncident $incident, bool $resolved): void
{
// Read + validate + decrypt ONCE and send exactly that pair — no re-read of the settings
// between the check and the POST (which could otherwise pair a new URL with an old token).
$target = self::gotifyTarget();
if ($target === null) {
return;
}
$url = $target['url'];
$token = $target['token'];
$server = $incident->server?->name ?? '—';
$rule = $incident->rule?->name ?? '—';
$title = ($resolved ? __('alerts.mail_subject_resolved') : __('alerts.mail_subject_firing'))."{$server}";
try {
Http::timeout(5)->withoutRedirecting()
->withHeaders(['X-Gotify-Key' => $token])
->asJson()
->post(rtrim($url, '/').'/message', [
'title' => $title,
'message' => "{$rule}: {$incident->rule?->metric} = {$incident->value} (Limit {$incident->rule?->threshold})",
'priority' => $resolved ? 3 : 8,
]);
} catch (Throwable $e) {
Log::warning('alert gotify failed: '.$e->getMessage());
}
}
/** http(s) + a parseable host. NO private-range block (a self-hosted Gotify is usually on a LAN). */
public static function isValidGotifyUrl(string $url): bool
{
$p = parse_url(trim($url));
return is_array($p) && isset($p['scheme'], $p['host'])
&& in_array(strtolower($p['scheme']), ['http', 'https'], true);
}
/**
* The validated Gotify target the sender will use ['url' => string, 'token' => plaintext] or
* null when Gotify can't actually send (missing/invalid URL, or a missing/undecryptable token).
* ONE place reads + validates + decrypts, so the "no delivery channel" banner (which checks for
* null) and the send path can never diverge, and the POST uses exactly the pair that was
* validated in the same read no check-then-reread TOCTOU.
*
* @return array{url: string, token: string}|null
*/
public static function gotifyTarget(): ?array
{
// Setting::uncached (DB-direct, cache-bypass): a stale cached token read just after a URL change
// could otherwise pair an old token with a new endpoint (token exfil). Reads the committed
// state, which — with saveChannels dropping the token before writing the new URL — can never
// present a new-url + old-token pair.
$url = trim((string) Setting::uncached('alert_gotify_url', ''));
$stored = (string) Setting::uncached('alert_gotify_token', '');
if ($url === '' || $stored === '' || ! self::isValidGotifyUrl($url)) {
return null;
}
try {
return ['url' => $url, 'token' => Crypt::decryptString($stored)];
} catch (Throwable) {
return null; // undecryptable (rotated APP_KEY) — treat as not configured
}
}
private function email(AlertIncident $incident, bool $resolved): void

View File

@ -26,6 +26,18 @@ use Throwable;
*/
class MailSettings
{
/** True when a usable SMTP transport is configured (host + from set) drives the alerts
* "no delivery channel" warning. Mirrors the guard in apply() below. */
public static function isConfigured(): bool
{
try {
return ((string) (Setting::get('mail_host') ?? '')) !== ''
&& ((string) (Setting::get('mail_from_address') ?? '')) !== '';
} catch (Throwable) {
return false; // settings not migrated yet
}
}
public static function apply(): void
{
try {
@ -37,7 +49,9 @@ class MailSettings
}
$encryption = Setting::get('mail_encryption', 'tls');
$storedPassword = Setting::get('mail_password');
// The password is a SECRET whose target is $host: read it uncached so a stale cached
// value can't be paired with a freshly-changed host (credential exfil). See Setting::uncached.
$storedPassword = Setting::uncached('mail_password');
$password = null;
if ($storedPassword !== null && $storedPassword !== '') {
$password = Crypt::decryptString($storedPassword);

View File

@ -64,6 +64,14 @@ return [
'rule_deleted' => 'Regel „:name“ gelöscht.',
'channels_saved' => 'Kanäle gespeichert.',
'webhook_invalid' => 'Ungültige oder unsichere Webhook-URL: :url (nur http/https zu öffentlichen Hosts).',
'no_channel_title' => 'Kein Zustellweg konfiguriert.',
'no_channel_hint' => 'Alarme werden ausgelöst, aber an niemanden gesendet. Richte SMTP (Einstellungen → E-Mail), einen Webhook oder Gotify ein.',
'gotify_url_label' => 'Gotify-Server-URL',
'gotify_token_label' => 'Gotify-App-Token',
'gotify_token_set' => '•••••••• (gesetzt)',
'gotify_token_placeholder' => 'App-Token',
'gotify_hint' => 'Selbst gehosteter Gotify-Push — ein privates Netz ist erlaubt. Der Token wird verschlüsselt gespeichert.',
'gotify_invalid' => 'Ungültige Gotify-URL (nur http/https).',
// Delete confirm
'delete_heading' => 'Regel löschen',

View File

@ -17,6 +17,7 @@ return [
'search_placeholder' => 'Akteur, Aktion, Ziel …',
// Empty state
'show_detail' => 'Details anzeigen',
'empty_title' => 'Keine Ereignisse',
'empty_filtered' => 'Für „:query“ wurden keine Audit-Einträge gefunden.',
'empty_none' => 'Es liegen noch keine Audit-Einträge vor.',
@ -33,6 +34,12 @@ return [
// Human-readable labels for the raw action codes shown in the event list.
'actions' => [
'service.start' => 'Dienst gestartet',
'service.stop' => 'Dienst gestoppt',
'service.restart' => 'Dienst neugestartet',
'service.start.failed' => 'Dienststart fehlgeschlagen',
'service.stop.failed' => 'Dienst-Stopp fehlgeschlagen',
'service.restart.failed' => 'Dienst-Neustart fehlgeschlagen',
'alert.rule_create' => 'Alarmregel angelegt',
'alert.rule_update' => 'Alarmregel geändert',
'alert.rule_delete' => 'Alarmregel gelöscht',

View File

@ -38,5 +38,6 @@ return [
'notify_test_ok' => 'Testmail an :email gesendet.',
'notify_test_failed' => 'Testmail fehlgeschlagen: :error',
'notify_test_throttled' => 'Zu viele Testmails bitte in :seconds Sekunden erneut.',
'test_retarget_password_required' => 'Passwort erneut eingeben, um einen geänderten Server zu testen.',
'not_configured' => 'SMTP ist nicht konfiguriert.',
];

View File

@ -50,6 +50,7 @@ return [
'action_error' => ':name: :error',
'action_done' => ':name: :op ausgeführt.',
'action_failed' => ':name: fehlgeschlagen — :output',
'action_failed_short' => ':name: fehlgeschlagen — vollständige Meldung im Audit-Log.',
'no_permission' => 'keine Rechte',
// Page title

View File

@ -64,6 +64,14 @@ return [
'rule_deleted' => 'Rule “:name” deleted.',
'channels_saved' => 'Channels saved.',
'webhook_invalid' => 'Invalid or unsafe webhook URL: :url (only http/https to public hosts).',
'no_channel_title' => 'No delivery channel configured.',
'no_channel_hint' => 'Alerts will fire but reach no one. Set up SMTP (Settings → Email), a webhook, or Gotify.',
'gotify_url_label' => 'Gotify server URL',
'gotify_token_label' => 'Gotify app token',
'gotify_token_set' => '•••••••• (set)',
'gotify_token_placeholder' => 'App token',
'gotify_hint' => 'Self-hosted Gotify push — a private network is allowed. The token is stored encrypted.',
'gotify_invalid' => 'Invalid Gotify URL (http/https only).',
// Delete confirm
'delete_heading' => 'Delete rule',

View File

@ -17,6 +17,7 @@ return [
'search_placeholder' => 'Actor, action, target …',
// Empty state
'show_detail' => 'Show details',
'empty_title' => 'No events',
'empty_filtered' => 'No audit entries found for “:query”.',
'empty_none' => 'There are no audit entries yet.',
@ -33,6 +34,12 @@ return [
// Human-readable labels for the raw action codes shown in the event list.
'actions' => [
'service.start' => 'Service started',
'service.stop' => 'Service stopped',
'service.restart' => 'Service restarted',
'service.start.failed' => 'Service start failed',
'service.stop.failed' => 'Service stop failed',
'service.restart.failed' => 'Service restart failed',
'alert.rule_create' => 'Alert rule created',
'alert.rule_update' => 'Alert rule changed',
'alert.rule_delete' => 'Alert rule deleted',

View File

@ -38,5 +38,6 @@ return [
'notify_test_ok' => 'Test mail sent to :email.',
'notify_test_failed' => 'Test mail failed: :error',
'notify_test_throttled' => 'Too many test e-mails — try again in :seconds seconds.',
'test_retarget_password_required' => 'Re-enter the password to test a changed server.',
'not_configured' => 'SMTP is not configured.',
];

View File

@ -50,6 +50,7 @@ return [
'action_error' => ':name: :error',
'action_done' => ':name: :op executed.',
'action_failed' => ':name: failed — :output',
'action_failed_short' => ':name failed — full message in the audit log.',
'no_permission' => 'no permission',
// Page title

View File

@ -19,6 +19,17 @@
<p class="mt-1 text-sm text-ink-3">{{ __('alerts.subtitle') }}</p>
</div>
{{-- No delivery channel reaches anyone alerts would fire silently. Warn prominently. --}}
@unless ($channelsReach)
<div class="flex items-start gap-2.5 rounded-lg border border-warning/25 bg-warning/10 px-4 py-3">
<x-icon name="alert" class="mt-0.5 h-4 w-4 shrink-0 text-warning" />
<div class="min-w-0">
<p class="text-sm font-medium text-ink">{{ __('alerts.no_channel_title') }}</p>
<p class="mt-0.5 text-xs text-ink-3">{{ __('alerts.no_channel_hint') }}</p>
</div>
</div>
@endunless
{{-- Active incidents --}}
<x-panel :title="__('alerts.incidents_title')" :subtitle="__('alerts.incidents_subtitle')" :padded="false">
@if ($incidents->isEmpty())
@ -169,6 +180,27 @@
@error('webhooks') <p class="mt-1 flex items-center gap-1.5 font-mono text-[11px] text-offline"><x-icon name="alert" class="h-3.5 w-3.5 shrink-0" />{{ $message }}</p> @enderror
<p class="mt-1 font-mono text-[11px] text-ink-4">{{ __('alerts.webhooks_hint') }}</p>
</div>
{{-- Gotify push (self-hostable) a real push when a service goes down, no e-mail needed. --}}
<div class="grid gap-4 sm:grid-cols-2">
<div>
<label class="{{ $label }}">{{ __('alerts.gotify_url_label') }}</label>
<input wire:model="gotifyUrl" type="url" class="{{ $field }} font-mono" placeholder="https://gotify.example.com" />
@error('gotifyUrl') <p class="mt-1 flex items-center gap-1.5 font-mono text-[11px] text-offline"><x-icon name="alert" class="h-3.5 w-3.5 shrink-0" />{{ $message }}</p> @enderror
</div>
<div>
<label class="{{ $label }}">{{ __('alerts.gotify_token_label') }}</label>
<div class="flex items-center gap-2">
<input wire:model="gotifyToken" type="password" autocomplete="off" class="{{ $field }} font-mono"
placeholder="{{ $gotifyTokenSet ? __('alerts.gotify_token_set') : __('alerts.gotify_token_placeholder') }}" />
@if ($gotifyTokenSet)
<x-btn variant="danger-soft" size="field" wire:click="clearGotifyToken" type="button" class="shrink-0">{{ __('common.remove') }}</x-btn>
@endif
</div>
</div>
</div>
<p class="font-mono text-[11px] text-ink-4">{{ __('alerts.gotify_hint') }}</p>
<div class="flex justify-end">
<x-btn variant="primary" type="submit">{{ __('alerts.save') }}</x-btn>
</div>

View File

@ -88,6 +88,14 @@
@if ($e->target)
<p class="truncate font-mono text-[11px] text-ink-3">{{ $e->target }}</p>
@endif
{{-- Failure detail (e.g. the full systemctl reason a service wouldn't start):
collapsed by default so the list stays scannable, full + wrapped on demand. --}}
@if (! empty($e->meta['output']))
<details class="mt-1.5">
<summary class="cursor-pointer font-mono text-[11px] text-ink-3 hover:text-ink-2">{{ __('audit.show_detail') }}</summary>
<pre class="mt-1.5 max-h-56 overflow-auto whitespace-pre-wrap break-words rounded-md border border-line bg-inset px-2.5 py-2 font-mono text-[11px] leading-relaxed text-ink-2">{{ $e->meta['output'] }}</pre>
</details>
@endif
@if ($e->server)
<div class="mt-1.5">
<x-badge tone="accent">

View File

@ -7,7 +7,7 @@
const level = $event.detail?.level ?? $event.detail?.[0]?.level ?? 'success';
const id = (window.crypto?.randomUUID?.() ?? String(Date.now() + Math.random()));
toasts.push({ id, msg, level });
setTimeout(() => { toasts = toasts.filter(t => t.id !== id) }, 4000);
setTimeout(() => { toasts = toasts.filter(t => t.id !== id) }, level === 'error' ? 9000 : 4000);
"
class="pointer-events-none fixed inset-x-0 bottom-4 z-50 flex flex-col items-center gap-2 px-4 sm:items-end sm:px-6"
aria-live="polite"

View File

@ -10,6 +10,7 @@ use App\Models\Setting;
use App\Models\User;
use App\Services\AlertNotifier;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Support\Facades\Crypt;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Mail;
use Tests\TestCase;
@ -98,6 +99,45 @@ class AlertNotifierTest extends TestCase
$this->assertTrue(AlertNotifier::isSafeWebhookUrl('https://8.8.8.8/hook')); // public IP, no DNS needed
}
public function test_gotify_pushes_title_message_priority_with_the_token_header(): void
{
Mail::fake();
Http::fake();
Setting::put('alert_gotify_url', 'https://gotify.example.com/');
Setting::put('alert_gotify_token', Crypt::encryptString('AppTok123'));
app(AlertNotifier::class)->notify($this->incident(), resolved: false);
Http::assertSent(function ($request) {
return $request->url() === 'https://gotify.example.com/message'
&& $request->hasHeader('X-Gotify-Key', 'AppTok123')
&& $request['priority'] === 8 // firing → high priority
&& str_contains($request['message'], 'cpu');
});
}
public function test_gotify_is_skipped_when_url_or_token_missing(): void
{
Mail::fake();
Http::fake();
Setting::put('alert_gotify_url', 'https://gotify.example.com'); // token missing
app(AlertNotifier::class)->notify($this->incident(), resolved: false);
Http::assertNothingSent();
}
public function test_gotify_allows_a_private_lan_server_but_still_requires_http_scheme(): void
{
// A self-hosted Gotify on a LAN is allowed (unlike SSRF-guarded webhooks) …
$this->assertTrue(AlertNotifier::isValidGotifyUrl('http://192.168.1.50:8080'));
$this->assertTrue(AlertNotifier::isValidGotifyUrl('https://gotify.example.com'));
// … but a non-http scheme or missing host is still refused.
$this->assertFalse(AlertNotifier::isValidGotifyUrl('ftp://192.168.1.50'));
$this->assertFalse(AlertNotifier::isValidGotifyUrl('gotify://x'));
$this->assertFalse(AlertNotifier::isValidGotifyUrl('https:///nohost'));
}
public function test_a_failing_channel_is_swallowed(): void
{
Mail::fake();

View File

@ -11,6 +11,7 @@ use App\Models\Setting;
use App\Models\User;
use App\Support\Confirm\ConfirmToken;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Support\Facades\Crypt;
use Livewire\Livewire;
use Tests\TestCase;
@ -179,4 +180,107 @@ class AlertsComponentTest extends TestCase
->assertSee('CPU high')
->assertSee('web1');
}
// ── delivery-channel warning + Gotify ───────────────────────────────────────
public function test_no_channel_warning_shows_when_nothing_is_configured(): void
{
Livewire::actingAs($this->admin())
->test(Index::class)
->assertSee(__('alerts.no_channel_title'));
}
public function test_no_channel_warning_hidden_once_gotify_is_configured(): void
{
Setting::put('alert_gotify_url', 'https://gotify.example.com');
Setting::put('alert_gotify_token', Crypt::encryptString('tok'));
Livewire::actingAs($this->admin())
->test(Index::class)
->assertDontSee(__('alerts.no_channel_title'));
}
public function test_saving_gotify_stores_url_and_encrypts_the_token(): void
{
Livewire::actingAs($this->admin())
->test(Index::class)
->set('gotifyUrl', 'https://gotify.example.com')
->set('gotifyToken', 'SuperSecretTok')
->call('saveChannels')
->assertHasNoErrors()
->assertSet('gotifyToken', '') // write-only: cleared after save
->assertSet('gotifyTokenSet', true);
$this->assertSame('https://gotify.example.com', Setting::get('alert_gotify_url'));
$stored = (string) Setting::get('alert_gotify_token');
$this->assertNotSame('', $stored);
$this->assertNotSame('SuperSecretTok', $stored); // encrypted, not plaintext
$this->assertSame('SuperSecretTok', Crypt::decryptString($stored));
}
public function test_an_invalid_gotify_url_is_rejected(): void
{
Livewire::actingAs($this->admin())
->test(Index::class)
->set('gotifyUrl', 'ftp://nope')
->call('saveChannels')
->assertHasErrors('gotifyUrl');
$this->assertNull(Setting::get('alert_gotify_url'));
}
public function test_clear_gotify_token_removes_it(): void
{
Setting::put('alert_gotify_token', Crypt::encryptString('tok'));
Livewire::actingAs($this->admin())
->test(Index::class)
->assertSet('gotifyTokenSet', true)
->call('clearGotifyToken')
->assertSet('gotifyTokenSet', false);
$this->assertNull(Setting::get('alert_gotify_token'));
}
public function test_changing_gotify_url_without_a_new_token_clears_the_stored_token(): void
{
Setting::put('alert_gotify_url', 'https://old.example.com');
Setting::put('alert_gotify_token', Crypt::encryptString('oldtok'));
// Admin retargets the URL but leaves the token field blank → the old token must NOT survive
// to be sent to the new endpoint (token-exfil-via-retarget guard).
Livewire::actingAs($this->admin())
->test(Index::class)
->set('gotifyUrl', 'https://new.example.com')
->set('gotifyToken', '')
->call('saveChannels')
->assertSet('gotifyTokenSet', false);
$this->assertSame('https://new.example.com', Setting::get('alert_gotify_url'));
$this->assertNull(Setting::get('alert_gotify_token'));
}
public function test_no_channel_warning_shows_when_gotify_token_is_absent_even_with_a_url(): void
{
// URL set but no token → the sender would skip it, so the banner must still warn.
Setting::put('alert_gotify_url', 'https://gotify.example.com');
Livewire::actingAs($this->admin())
->test(Index::class)
->assertSee(__('alerts.no_channel_title'));
}
public function test_changing_url_with_a_fresh_token_stores_the_new_token_not_the_old(): void
{
Setting::put('alert_gotify_url', 'https://old.example.com');
Setting::put('alert_gotify_token', Crypt::encryptString('OLD'));
Livewire::actingAs($this->admin())
->test(Index::class)
->set('gotifyUrl', 'https://new.example.com')
->set('gotifyToken', 'NEW')
->call('saveChannels')
->assertSet('gotifyTokenSet', true);
$this->assertSame('NEW', Crypt::decryptString((string) Setting::get('alert_gotify_token')));
}
}

View File

@ -19,10 +19,13 @@ class MetricHistoryTest extends TestCase
return Server::create(['name' => 'box', 'ip' => '10.0.0.9', 'ssh_port' => 22, 'status' => 'online']);
}
public function test_sample_metrics_persists_a_row_from_the_cached_reading(): void
/** A freshly-polled server (DB row updated by the poller) is sampled WITHOUT any cache the
* history no longer depends on a cross-container cache, so it works whenever the live gauges do. */
public function test_sample_metrics_persists_a_row_from_the_polled_db_row_without_a_cache(): void
{
$s = $this->server();
Cache::put("metrics:latest:{$s->id}", ['cpu' => 40, 'mem' => 55, 'disk' => 12, 'load' => 0.7], now()->addMinutes(5));
$s->forceFill(['cpu' => 40, 'mem' => 55, 'disk' => 12, 'last_seen_at' => now()])->save();
// deliberately NO Cache::put — the DB row is the source of truth.
$this->artisan('clusev:sample-metrics')->assertSuccessful();
@ -30,16 +33,33 @@ class MetricHistoryTest extends TestCase
$this->assertSame(1, MetricSample::count());
}
public function test_sample_metrics_clamps_percentages_and_skips_unpolled_servers(): void
/** load is not on the server row; it is taken from the live cache when present (best-effort). */
public function test_sample_metrics_reads_load_from_the_cache_when_present(): void
{
$s = $this->server();
$s->forceFill(['cpu' => 10, 'mem' => 20, 'disk' => 30, 'last_seen_at' => now()])->save();
Cache::put("metrics:latest:{$s->id}", ['cpu' => 10, 'mem' => 20, 'disk' => 30, 'load' => 1.25], now()->addMinutes(5));
$this->artisan('clusev:sample-metrics')->assertSuccessful();
$this->assertSame(1.25, (float) MetricSample::first()->load);
}
public function test_sample_metrics_clamps_percentages_and_skips_stale_or_unpolled_servers(): void
{
$polled = $this->server();
$this->server(); // a second server with no cached reading → no sample
Cache::put("metrics:latest:{$polled->id}", ['cpu' => 250, 'mem' => 60, 'disk' => 10, 'load' => 0.2], now()->addMinutes(5));
$polled->forceFill(['cpu' => 250, 'mem' => 60, 'disk' => 10, 'last_seen_at' => now()])->save();
// never polled (last_seen_at null) → no sample
$this->server();
// polled long ago (stale) → no sample (an honest gap while offline)
$this->server()->forceFill(['cpu' => 5, 'mem' => 5, 'disk' => 5, 'last_seen_at' => now()->subMinutes(10)])->save();
$this->artisan('clusev:sample-metrics')->assertSuccessful();
$this->assertSame(1, MetricSample::count());
$this->assertSame(100, MetricSample::first()->cpu); // 250 clamped to 100
$this->assertSame($polled->id, MetricSample::first()->server_id);
}
public function test_sample_metrics_prunes_beyond_the_retention_window(): void

View File

@ -0,0 +1,91 @@
<?php
namespace Tests\Feature;
use App\Livewire\Services\Index as Services;
use App\Models\AuditEvent;
use App\Models\Server;
use App\Models\User;
use App\Services\FleetService;
use App\Support\Confirm\ConfirmToken;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Livewire\Livewire;
use Mockery;
use Tests\TestCase;
/**
* A systemd action that FAILS must leave a persistent, readable trail: the ephemeral toast is
* truncated, so the full systemctl reason is written to the audit log (a red service.<op>.failed
* entry with the complete output in meta) where the operator can re-read it.
*/
class ServiceActionAuditTest extends TestCase
{
use RefreshDatabase;
protected function tearDown(): void
{
Mockery::close();
parent::tearDown();
}
private function activeServer(): Server
{
$server = Server::create(['name' => 'box', 'ip' => '10.0.0.2', 'ssh_port' => 22, 'status' => 'online']);
$server->credential()->create(['username' => 'root', 'auth_type' => 'password', 'secret' => 'x']);
session(['active_server_id' => $server->id]);
return $server;
}
private function confirmed(string $event, array $params, int $serverId): string
{
$token = ConfirmToken::issue($event, $params, '', null, $serverId);
ConfirmToken::confirm($token);
return $token;
}
public function test_failed_start_is_persisted_to_the_audit_log_with_the_full_reason(): void
{
$this->actingAs(User::factory()->create(['must_change_password' => false]));
$server = $this->activeServer();
$reason = "monit.service: Failed with result 'exit-code'.\n"
.'monit.service - Pro-active monitoring utility for unix systems: a very long detailed '
.'reason that would be truncated in a 90-char toast but must survive in full here.';
$fleet = Mockery::mock(FleetService::class);
$fleet->shouldReceive('serviceAction')->once()->andReturn(['ok' => false, 'output' => $reason]);
$fleet->shouldReceive('systemd')->andReturn(['services' => []]);
app()->instance(FleetService::class, $fleet);
$token = $this->confirmed('serviceConfirmed', ['op' => 'start', 'name' => 'monit.service'], $server->id);
Livewire::test(Services::class)->call('applyService', $token);
$event = AuditEvent::where('action', 'service.start.failed')->first();
$this->assertNotNull($event, 'a service.start.failed audit event should be written');
$this->assertTrue($event->is_error);
$this->assertSame($server->id, $event->server_id);
// The FULL reason is retained (not truncated), so it is re-readable later.
$this->assertStringContainsString('must survive in full here.', $event->meta['output']);
$this->assertStringContainsString('Failed with result', $event->meta['output']);
}
public function test_successful_action_writes_no_failure_event(): void
{
$this->actingAs(User::factory()->create(['must_change_password' => false]));
$server = $this->activeServer();
$fleet = Mockery::mock(FleetService::class);
$fleet->shouldReceive('serviceAction')->once()->andReturn(['ok' => true, 'output' => '']);
$fleet->shouldReceive('systemd')->andReturn(['services' => []]);
app()->instance(FleetService::class, $fleet);
$token = $this->confirmed('serviceConfirmed', ['op' => 'restart', 'name' => 'nginx.service'], $server->id);
Livewire::test(Services::class)->call('applyService', $token);
$this->assertSame(0, AuditEvent::where('action', 'like', 'service.%.failed')->count());
}
}

View File

@ -97,6 +97,27 @@ class SmtpConfigTest extends TestCase
$this->assertSame('original-pw', Crypt::decryptString(Setting::get('mail_password')));
}
public function test_changing_host_with_a_blank_password_drops_the_stored_password(): void
{
$this->actingUser();
Setting::put('mail_host', 'smtp.old.example.com');
Setting::put('mail_from_address', 'panel@example.com');
Setting::put('mail_password', Crypt::encryptString('original-pw'));
// Retarget the SMTP host but leave the password blank → the old credential must NOT be kept
// (it could otherwise be sent to the new, possibly hostile, server).
Livewire::test(Email::class)
->set('mail_from_name', 'Clusev')
->set('mail_host', 'smtp.attacker.example.com')
->set('mail_password', '')
->call('save')
->assertHasNoErrors()
->assertSet('passwordSet', false);
$this->assertNull(Setting::get('mail_password'));
$this->assertSame('smtp.attacker.example.com', Setting::get('mail_host'));
}
public function test_configured_reflects_host_and_from_presence(): void
{
$this->actingUser();
@ -184,6 +205,26 @@ class SmtpConfigTest extends TestCase
$this->assertSame([$user->email], $recipients, 'test mail must go only to the current user');
}
public function test_test_send_refuses_a_changed_host_with_a_blank_password(): void
{
config(['mail.mailers.smtp.transport' => 'array']);
$this->actingUser();
Setting::put('mail_host', 'smtp.old.example.com');
Setting::put('mail_from_address', 'panel@example.com');
Setting::put('mail_password', Crypt::encryptString('stored-pw'));
// Point the form at a NEW host, leave the password blank → the stored password must NOT be
// test-sent to the new (possibly hostile) server; the test is refused.
Livewire::test(Email::class)
->set('mail_host', 'smtp.attacker.example.com')
->set('mail_from_address', 'panel@example.com')
->set('mail_password', '')
->call('sendTest')
->assertHasNoErrors();
$this->assertCount(0, app('mailer')->getSymfonyTransport()->messages(), 'no mail may be sent to a changed host with a blank password');
}
public function test_test_send_does_nothing_when_unconfigured(): void
{
Mail::fake();

View File

@ -196,18 +196,38 @@ class TerminalTest extends TestCase
public function test_host_shell_edit_keeps_the_stored_secret_when_left_blank(): void
{
$this->hostCredential(); // secret 'r00t-pw'
$this->hostCredential(); // username 'root', port 2200, secret 'r00t-pw'
// Re-save the SAME target (username/port/auth unchanged) with a blank secret → the stored
// secret is kept (e.g. the operator just re-confirmed the form).
Livewire::test(HostShell::class)
->assertSet('configured', true)
->set('username', 'admin') // change something, leave secret blank
->call('save');
->set('secret', '')
->call('save')
->assertHasNoErrors();
$hc = HostCredential::current();
$this->assertSame('admin', $hc->username);
$this->assertSame('root', $hc->username);
$this->assertSame('r00t-pw', $hc->secret); // unchanged
}
public function test_host_shell_requires_a_new_secret_when_the_username_changes(): void
{
$this->hostCredential(); // username 'root', secret 'r00t-pw'
// Retargeting to a different account with a blank secret is refused — the old account's
// secret must not be silently reused for a new username.
Livewire::test(HostShell::class)
->set('username', 'admin')
->set('secret', '')
->call('save')
->assertHasErrors('secret');
$hc = HostCredential::current();
$this->assertSame('root', $hc->username); // untouched — save was rejected
$this->assertSame('r00t-pw', $hc->secret);
}
public function test_host_shell_requires_a_new_secret_when_switching_auth_method(): void
{
$this->hostCredential(); // auth_type 'password', secret 'r00t-pw'