Copy every invoice to the archive when it is issued, and notice when that fails

The application knows nothing about NFS and should not. It writes to a
directory; whether that is a local disk, a bind mount or a NAS over NFS 4.1
belongs to the mount. No protocol library, the target changes without code
changing with it, and the whole thing is testable against a temp directory.

The finding that shaped it: mkdir -p on a path whose mount is NOT there
succeeds. It creates the empty directory beneath the mountpoint, writes the
invoice onto the container's own disk and reports success — so an unmounted NAS
would quietly collect invoices locally while every check said it worked. An
archive that silently is not the archive is worse than none. The root is
therefore never created, only used: a missing root means a missing mount, and
that is a failure. Only the year subdirectory is created, inside a root already
proven to exist.

Written to a .part and renamed. A rename within one filesystem is atomic, so
whatever watches that folder never sees a truncated PDF and takes it for a
finished invoice. The size is read back afterwards rather than the write being
trusted: over a network mount a short write can report success, and an archive
of truncated files looks exactly like one that worked.

Queued, not done during the request. A network mount can block for its whole
timeout, and an invoice must not fail to be issued because a NAS is rebooting.
Dispatched after the commit, or a worker could look for an invoice that has not
landed yet.

And copy-on-issue is not enough on its own: an invoice issued while the office
connection was down never arrives, the queue eventually gives up, and nobody
opens an archive to check whether last Tuesday is in it. An hourly sweep
re-queues whatever is still missing, the failure is recorded on the invoice
rather than only in a log, and the Finance page says how many are outstanding.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

app/Console/Commands/ArchiveUnexportedInvoices.php

@@ -0,0 +1,51 @@
+<?php
+
+namespace App\Console\Commands;
+
+use App\Jobs\ArchiveInvoice;
+use App\Models\Invoice;
+use App\Services\Billing\InvoiceArchive;
+use Illuminate\Console\Command;
+
+/**
+ * The sweep behind the copy-on-issue.
+ *
+ * Copying at issue is right and it is not enough. An invoice issued while the
+ * office connection was down never reaches the archive, the queue eventually
+ * gives up, and nothing about the invoice tells anybody — nobody opens an
+ * archive to check whether last Tuesday is in it. This picks up whatever is
+ * still missing, however long ago it was issued.
+ *
+ * Deliberately re-queues rather than writing here: the job already knows how to
+ * write one, and two code paths that both write an invoice into an archive is
+ * one more than the number that can be kept correct.
+ */
+class ArchiveUnexportedInvoices extends Command
+{
+    protected $signature = 'clupilot:archive-invoices {--limit=200}';
+
+    protected $description = 'Queue a copy of every invoice that is not on the archive yet';
+
+    public function handle(): int
+    {
+        if (! InvoiceArchive::enabled()) {
+            $this->line('No archive path configured — nothing to do.');
+
+            return self::SUCCESS;
+        }
+
+        $pending = Invoice::query()
+            ->whereNull('exported_at')
+            ->orderBy('id')
+            ->limit((int) $this->option('limit'))
+            ->get();
+
+        foreach ($pending as $invoice) {
+            ArchiveInvoice::dispatch($invoice);
+        }
+
+        $this->line("Queued {$pending->count()} invoice(s) for the archive.");
+
+        return self::SUCCESS;
+    }
+}

app/Jobs/ArchiveInvoice.php

@@ -0,0 +1,62 @@
+<?php
+
+namespace App\Jobs;
+
+use App\Models\Invoice;
+use App\Services\Billing\InvoiceArchive;
+use Illuminate\Contracts\Queue\ShouldQueue;
+use Illuminate\Foundation\Queue\Queueable;
+use Throwable;
+
+/**
+ * Put one invoice on the archive, and keep trying if the archive is not there.
+ *
+ * Queued rather than done during the request that issued the invoice: a network
+ * mount can block for as long as its timeout allows, and an invoice must not
+ * fail to be issued because a NAS in an office is rebooting.
+ *
+ * Retries with a backoff and, when it finally gives up, records why on the
+ * invoice itself. An export that fails silently is the failure mode that
+ * matters here — nobody looks at an archive to check whether last Tuesday is
+ * in it, and the sweep needs something to find.
+ */
+class ArchiveInvoice implements ShouldQueue
+{
+    use Queueable;
+
+    public int $tries = 5;
+
+    /** Minutes, roughly: a mount that is gone is usually gone for a while. */
+    public array $backoff = [60, 300, 900, 3600];
+
+    public function __construct(public Invoice $invoice) {}
+
+    public function handle(InvoiceArchive $archive): void
+    {
+        if (! InvoiceArchive::enabled()) {
+            return;
+        }
+
+        // Already there. A retry after a timeout that actually succeeded must
+        // not write it twice, and the file name is the same either way.
+        if ($this->invoice->exported_at !== null) {
+            return;
+        }
+
+        $archive->store($this->invoice);
+
+        $this->invoice->forceFill([
+            'exported_at' => now(),
+            'export_error' => null,
+        ])->save();
+    }
+
+    public function failed(?Throwable $e): void
+    {
+        // On the invoice, not only in the log. The console shows it there, and
+        // a log line nobody reads is not a report.
+        $this->invoice->forceFill([
+            'export_error' => mb_substr((string) $e?->getMessage(), 0, 250),
+        ])->save();
+    }
+}

app/Livewire/Admin/Finance.php

@@ -2,7 +2,9 @@
 
 namespace App\Livewire\Admin;
 
+use App\Models\Invoice;
 use App\Models\InvoiceSeries;
+use App\Services\Billing\InvoiceArchive;
 use App\Support\CompanyProfile;
 use App\Support\Settings;
 use Illuminate\Support\Facades\Storage;
@@ -32,6 +34,14 @@ class Finance extends Component
 
     public float $taxRate = 20.0;
 
+    /**
+     * Where a copy of every invoice is written, or empty for none.
+     *
+     * A plain directory. Whether it is a local disk, a bind mount or a NAS over
+     * NFS 4.1 is the mount's business — nothing here knows or needs to.
+     */
+    public string $archivePath = '';
+
     /** New logo upload, validated on save. */
     public $logo = null;
 
@@ -44,6 +54,7 @@ class Finance extends Component
         $this->company = CompanyProfile::all();
         $this->logoPath = (string) ($this->company['logo_path'] ?? '');
         $this->taxRate = CompanyProfile::taxRate();
+        $this->archivePath = InvoiceArchive::path();
     }
 
     public function saveCompany(): void
@@ -72,6 +83,7 @@ class Finance extends Component
             // ever rendered from then on.
             'logo' => 'nullable|image|mimes:png,webp|max:1024',
             'taxRate' => 'required|numeric|min:0|max:100',
+            'archivePath' => 'nullable|string|max:255',
         ]);
 
         if ($this->logo !== null) {
@@ -92,6 +104,7 @@ class Finance extends Component
 
         CompanyProfile::put($data['company']);
         Settings::set('company.tax_rate', (float) $data['taxRate']);
+        Settings::set(InvoiceArchive::PATH_KEY, trim((string) ($data['archivePath'] ?? '')));
 
         $this->dispatch('notify', message: __('finance.company_saved'));
     }
@@ -119,6 +132,12 @@ class Finance extends Component
             // invoice without a name, an address or a VAT number is not a valid
             // invoice here, and issuing one is worse than refusing to.
             'missing' => CompanyProfile::missingForInvoicing(),
+            // Written from here rather than checked on every render: a stat()
+            // on a network mount that has gone away blocks for the mount's
+            // timeout, and this is a page an operator opens when something is
+            // already wrong.
+            'unexported' => Invoice::query()->whereNull('exported_at')->count(),
+            'exportFailures' => Invoice::query()->whereNotNull('export_error')->count(),
         ]);
     }
 }

app/Models/Invoice.php

@@ -32,6 +32,7 @@ class Invoice extends Model
             'issued_on' => 'date',
             'due_on' => 'date',
             'sent_at' => 'datetime',
+            'exported_at' => 'datetime',
             'net_cents' => 'integer',
             'tax_cents' => 'integer',
             'gross_cents' => 'integer',

app/Services/Billing/InvoiceArchive.php

@@ -0,0 +1,111 @@
+<?php
+
+namespace App\Services\Billing;
+
+use App\Models\Invoice;
+use App\Support\Settings;
+use RuntimeException;
+
+/**
+ * A copy of every invoice on a filesystem outside this machine.
+ *
+ * The application knows nothing about NFS, and that is deliberate. It writes to
+ * a directory; whether that directory is a local disk, a bind mount or a NAS
+ * mounted over NFS 4.1 is the mount's business. Nothing here needs a protocol
+ * library, and the target can change without a line of code changing with it.
+ *
+ * Written to a temporary name in the same directory and then renamed. A rename
+ * within one filesystem is atomic, so whatever is watching that folder — a
+ * backup job, a person — never sees a half-written PDF and mistakes it for a
+ * finished one. Writing straight to the final name is how a truncated file ends
+ * up in an archive looking exactly like an invoice.
+ */
+final class InvoiceArchive
+{
+    public const PATH_KEY = 'invoices.export_path';
+
+    public function __construct(private readonly InvoiceRenderer $renderer) {}
+
+    /** Where copies go, or empty when archiving is switched off. */
+    public static function path(): string
+    {
+        return trim((string) Settings::get(self::PATH_KEY, ''));
+    }
+
+    public static function enabled(): bool
+    {
+        return self::path() !== '';
+    }
+
+    /**
+     * Write one invoice into the archive.
+     *
+     * Throws rather than returning false: the caller is a queued job, and a
+     * throw is what makes the queue retry it. A silent false would leave the
+     * invoice unarchived and the job marked done.
+     */
+    public function store(Invoice $invoice): string
+    {
+        $root = self::path();
+
+        if ($root === '') {
+            throw new RuntimeException('No archive path is configured.');
+        }
+
+        // The root is never created, only used. This is the whole difference
+        // between an archive and a hole in the ground: mkdir -p on a path whose
+        // mount is not there succeeds, writes into the empty directory
+        // underneath the mountpoint, and reports success — so an unmounted NAS
+        // would silently collect invoices on the container's own disk and every
+        // check would say it worked. A missing root means the mount is missing,
+        // and that is a failure, not something to repair by creating it.
+        if (! is_dir($root)) {
+            throw new RuntimeException("The archive path does not exist — is the mount there? {$root}");
+        }
+
+        // Grouped by year, because seven years of invoices in one directory is
+        // a directory nobody opens twice. This one IS created: it lives inside
+        // a root that has already been proven to exist.
+        $directory = rtrim($root, '/').'/'.($invoice->issued_on?->format('Y') ?? 'unbekannt');
+
+        if (! is_dir($directory) && ! @mkdir($directory, 0o775, true) && ! is_dir($directory)) {
+            throw new RuntimeException("Cannot create the archive directory: {$directory}");
+        }
+
+        if (! is_writable($directory)) {
+            // Said plainly. On a mount that has gone away this is the first
+            // thing that fails, and "permission denied" on a path nobody
+            // recognises is a worse message than the path itself.
+            throw new RuntimeException("The archive directory is not writable: {$directory}");
+        }
+
+        $target = $directory.'/'.$invoice->number.'.pdf';
+        $pdf = $this->renderer->forInvoice($invoice);
+
+        // Same directory, so the rename below stays within one filesystem and
+        // is therefore atomic. A temp file in /tmp would make it a copy.
+        $temporary = $target.'.part';
+
+        if (@file_put_contents($temporary, $pdf) === false) {
+            throw new RuntimeException("Could not write {$temporary}");
+        }
+
+        if (! @rename($temporary, $target)) {
+            @unlink($temporary);
+
+            throw new RuntimeException("Could not move the finished file into place: {$target}");
+        }
+
+        // Read back the size rather than trusting the write. Over a network
+        // mount a short write can report success, and an archive of truncated
+        // files is worse than an empty one: it looks like it worked.
+        clearstatcache(true, $target);
+        $written = @filesize($target);
+
+        if ($written === false || $written < 1000) {
+            throw new RuntimeException("The archived file is too small to be an invoice: {$target}");
+        }
+
+        return $target;
+    }
+}

app/Services/Billing/IssueInvoice.php

@@ -2,6 +2,7 @@
 
 namespace App\Services\Billing;
 
+use App\Jobs\ArchiveInvoice;
 use App\Models\Customer;
 use App\Models\Invoice;
 use App\Models\InvoiceSeries;
@@ -81,7 +82,7 @@ final class IssueInvoice
             'meta' => $this->meta($customer, $treatment),
         ];
 
-        return DB::transaction(function () use ($series, $customer, $orders, $snapshot, $totals) {
+        $invoice = DB::transaction(function () use ($series, $customer, $orders, $snapshot, $totals) {
             [$number, $sequence, $year] = $this->numbers->next($series);
 
             // The number is only true once it is on the document, so both are
@@ -104,6 +105,19 @@ final class IssueInvoice
                 'currency' => strtoupper((string) ($orders->first()->currency ?: 'EUR')),
             ]);
         });
+
+        // After the commit, never inside it: a worker can pick a job up before
+        // the transaction lands, and it would then look for an invoice that
+        // does not exist yet.
+        //
+        // Dispatched rather than done here because the archive is a network
+        // mount: it can block for as long as its timeout allows, and an invoice
+        // must not fail to be issued because a NAS in an office is rebooting.
+        if (InvoiceArchive::enabled()) {
+            ArchiveInvoice::dispatch($invoice);
+        }
+
+        return $invoice;
     }
 
     /**

database/migrations/2026_07_29_140000_add_export_state_to_invoices.php

@@ -0,0 +1,35 @@
+<?php
+
+use Illuminate\Database\Migrations\Migration;
+use Illuminate\Database\Schema\Blueprint;
+use Illuminate\Support\Facades\Schema;
+
+/**
+ * Whether a copy of this invoice reached the archive, and what went wrong.
+ *
+ * Two columns rather than one flag, because "not yet" and "tried and failed"
+ * need different answers from a person. An invoice issued while the office
+ * connection was down is not on the archive and nothing about the invoice says
+ * so — the sweep that retries it needs to know which ones to pick up, and the
+ * console needs something to show for the ones it cannot.
+ */
+return new class extends Migration
+{
+    public function up(): void
+    {
+        Schema::table('invoices', function (Blueprint $table) {
+            $table->timestamp('exported_at')->nullable()->after('sent_at');
+            $table->string('export_error')->nullable()->after('exported_at');
+            // The sweep asks for "issued but not exported" on every tick.
+            $table->index('exported_at');
+        });
+    }
+
+    public function down(): void
+    {
+        Schema::table('invoices', function (Blueprint $table) {
+            $table->dropIndex(['exported_at']);
+            $table->dropColumn(['exported_at', 'export_error']);
+        });
+    }
+};

lang/de/finance.php

@@ -31,13 +31,19 @@ return [
         'bank_name' => 'Bank',
         'iban' => 'IBAN',
         'bic' => 'BIC',
+        'archive_path' => 'Archivpfad',
         'logo' => 'Logo',
         'tax_rate' => 'Umsatzsteuer (%)',
         'payment_days' => 'Zahlungsziel (Tage)',
         'payment_terms' => 'Zahlungsbedingungen',
     ],
 
+    'archive_failed' => ':n Rechnung(en) konnten nicht ins Archiv geschrieben werden.',
+    'archive_pending' => ':n Rechnung(en) warten noch auf das Archiv.',
+    'archive_clear' => 'Jede Rechnung ist im Archiv.',
+
     'hint' => [
+        'archive_path' => 'Verzeichnis, in das eine Kopie jeder Rechnung geschrieben wird — z. B. der Einhängepunkt Ihrer NAS. Leer lassen, um nichts zu kopieren.',
         'vat_id' => 'Pflichtangabe auf jeder Rechnung.',
         'tax_rate' => 'Wird in jede Rechnung eingefroren. Eine Änderung wirkt nur auf neue Rechnungen.',
         'logo' => 'PNG oder WEBP, höchstens 1 MB. Wird in jede erzeugte Rechnung eingebettet.',

lang/en/finance.php

@@ -31,13 +31,19 @@ return [
         'bank_name' => 'Bank',
         'iban' => 'IBAN',
         'bic' => 'BIC',
+        'archive_path' => 'Archive path',
         'logo' => 'Logo',
         'tax_rate' => 'VAT (%)',
         'payment_days' => 'Payment terms (days)',
         'payment_terms' => 'Payment conditions',
     ],
 
+    'archive_failed' => ':n invoice(s) could not be written to the archive.',
+    'archive_pending' => ':n invoice(s) are still waiting for the archive.',
+    'archive_clear' => 'Every invoice is on the archive.',
+
     'hint' => [
+        'archive_path' => 'Directory a copy of every invoice is written to — your NAS mount point, for instance. Leave empty to copy nothing.',
         'vat_id' => 'Required on every invoice.',
         'tax_rate' => 'Frozen into every invoice. A change affects new invoices only.',
         'logo' => 'PNG or WEBP, 1 MB at most. Embedded into every invoice rendered.',

resources/views/livewire/admin/finance.blade.php

@@ -82,6 +82,25 @@
             </div>
         </div>
 
+        {{-- The archive. A directory, nothing more: the mount behind it is the
+             operator's business, and keeping it that way means the target can
+             change without a line of code changing with it. --}}
+        <div class="border-t border-line pt-5">
+            <x-ui.input name="archivePath" wire:model="archivePath" :label="__('finance.field.archive_path')"
+                        :hint="__('finance.hint.archive_path')" placeholder="/mnt/rechnungen" />
+            @if ($archivePath !== '')
+                <p class="mt-2 text-xs {{ $exportFailures > 0 ? 'text-danger' : 'text-muted' }}">
+                    @if ($exportFailures > 0)
+                        {{ __('finance.archive_failed', ['n' => $exportFailures]) }}
+                    @elseif ($unexported > 0)
+                        {{ __('finance.archive_pending', ['n' => $unexported]) }}
+                    @else
+                        {{ __('finance.archive_clear') }}
+                    @endif
+                </p>
+            @endif
+        </div>
+
         <div class="flex justify-end border-t border-line pt-5">
             <x-ui.button variant="primary" type="submit" wire:loading.attr="disabled">{{ __('finance.save') }}</x-ui.button>
         </div>

routes/console.php

@@ -45,3 +45,14 @@ Schedule::call(fn () => \App\Models\StripePendingEvent::query()
     ->delete())
     ->daily()
     ->name('stripe-pending-prune');
+
+// Copies of invoices that never reached the archive — a NAS that was rebooting
+// when the invoice was issued, a mount that had gone away. Copy-on-issue is
+// right and is not enough: nobody opens an archive to check whether last
+// Tuesday is in it, so something has to go and look.
+//
+// Hourly rather than by the minute: the failure it repairs lasts as long as an
+// outage lasts, and a retry storm against a mount that is down helps nobody.
+Schedule::command('clupilot:archive-invoices')
+    ->hourly()
+    ->withoutOverlapping();

tests/Feature/Billing/InvoiceArchiveTest.php

@@ -0,0 +1,146 @@
+<?php
+
+use App\Jobs\ArchiveInvoice;
+use App\Models\Customer;
+use App\Models\Invoice;
+use App\Models\Order;
+use App\Services\Billing\InvoiceArchive;
+use App\Services\Billing\IssueInvoice;
+use App\Support\CompanyProfile;
+use App\Support\Settings;
+use Illuminate\Support\Facades\Queue;
+
+/**
+ * A copy of every invoice on a filesystem outside this machine.
+ *
+ * Nothing here knows about NFS, and that is the point: the application writes
+ * to a directory, and whether that is a local disk, a bind mount or a NAS over
+ * NFS 4.1 belongs to the mount. Which is also what makes it testable — these
+ * run against a temporary directory and prove the same code path.
+ */
+beforeEach(function () {
+    CompanyProfile::put([
+        'name' => 'CluPilot Cloud e.U.', 'address' => 'Dreherstraße 66/1/8',
+        'postcode' => '1110', 'city' => 'Wien', 'vat_id' => 'ATU00000000',
+    ]);
+
+    $this->archive = sys_get_temp_dir().'/clupilot-archive-'.bin2hex(random_bytes(6));
+    mkdir($this->archive, 0o775, true);
+    Settings::set(InvoiceArchive::PATH_KEY, $this->archive);
+});
+
+afterEach(function () {
+    if (isset($this->archive) && is_dir($this->archive)) {
+        exec('rm -rf '.escapeshellarg($this->archive));
+    }
+});
+
+function anInvoice(): Invoice
+{
+    $customer = Customer::factory()->create(['name' => 'Muster GmbH']);
+
+    return app(IssueInvoice::class)->forOrders($customer, collect([
+        Order::factory()->create(['customer_id' => $customer->id, 'amount_cents' => 1900, 'currency' => 'EUR', 'status' => 'paid']),
+    ]));
+}
+
+it('queues a copy the moment an invoice is issued', function () {
+    Queue::fake();
+
+    anInvoice();
+
+    Queue::assertPushed(ArchiveInvoice::class);
+});
+
+it('queues nothing when no archive is configured', function () {
+    Queue::fake();
+    Settings::set(InvoiceArchive::PATH_KEY, '');
+
+    anInvoice();
+
+    Queue::assertNotPushed(ArchiveInvoice::class);
+});
+
+it('writes the invoice under its number, grouped by year', function () {
+    // Seven years of invoices in one directory is a directory nobody opens
+    // twice.
+    $invoice = anInvoice();
+
+    (new ArchiveInvoice($invoice))->handle(app(InvoiceArchive::class));
+
+    $expected = $this->archive.'/'.now()->format('Y').'/'.$invoice->number.'.pdf';
+
+    expect(is_file($expected))->toBeTrue()
+        ->and(substr(file_get_contents($expected), 0, 5))->toBe('%PDF-')
+        ->and($invoice->refresh()->exported_at)->not->toBeNull();
+});
+
+it('leaves no half-written file behind under the final name', function () {
+    // Written to a .part and renamed. A rename inside one filesystem is atomic,
+    // so whatever watches that folder never sees a truncated PDF and mistakes
+    // it for a finished invoice.
+    $invoice = anInvoice();
+
+    (new ArchiveInvoice($invoice))->handle(app(InvoiceArchive::class));
+
+    $year = $this->archive.'/'.now()->format('Y');
+
+    expect(glob($year.'/*.part'))->toBe([]);
+});
+
+it('does not write the same invoice twice', function () {
+    // A retry after a timeout that actually succeeded must not do it again.
+    $invoice = anInvoice();
+
+    (new ArchiveInvoice($invoice))->handle(app(InvoiceArchive::class));
+    $first = $invoice->refresh()->exported_at;
+
+    (new ArchiveInvoice($invoice))->handle(app(InvoiceArchive::class));
+
+    expect($invoice->refresh()->exported_at->toIso8601String())->toBe($first->toIso8601String());
+});
+
+it('refuses to create the archive root, so an unmounted NAS is a failure and not a hole', function () {
+    // The finding that made this test necessary. mkdir -p on a path whose mount
+    // is not there SUCCEEDS: it creates the empty directory underneath the
+    // mountpoint, writes the invoice onto the container's own disk, and reports
+    // success. An archive that silently is not the archive is worse than none,
+    // because every check says it worked.
+    // Faked, or the dispatch-on-issue archives it synchronously into the
+    // working directory from beforeEach() before this test has said anything.
+    Queue::fake();
+
+    $invoice = anInvoice();
+    Settings::set(InvoiceArchive::PATH_KEY, sys_get_temp_dir().'/clupilot-not-mounted-'.bin2hex(random_bytes(4)));
+
+    expect(fn () => app(InvoiceArchive::class)->store($invoice))
+        ->toThrow(RuntimeException::class);
+
+    expect($invoice->refresh()->exported_at)->toBeNull()
+        // And nothing was left on disk pretending to be an archive.
+        ->and(is_dir(InvoiceArchive::path()))->toBeFalse();
+});
+
+it('records why it gave up, on the invoice rather than only in a log', function () {
+    // Nobody opens an archive to check whether last Tuesday is in it, so the
+    // failure has to be somewhere a person will pass.
+    $invoice = anInvoice();
+
+    (new ArchiveInvoice($invoice))->failed(new RuntimeException('mount is gone'));
+
+    expect($invoice->refresh()->export_error)->toContain('mount is gone');
+});
+
+it('sweeps up whatever the queue gave up on', function () {
+    Queue::fake();
+
+    $invoice = anInvoice();
+    Queue::assertPushed(ArchiveInvoice::class);
+
+    // The office connection was down; nothing reached the archive.
+    expect($invoice->exported_at)->toBeNull();
+
+    $this->artisan('clupilot:archive-invoices')->assertExitCode(0);
+
+    Queue::assertPushed(ArchiveInvoice::class, 2);
+});