fox-desktop/docs/superpowers/plans/2026-05-11-fox-desktop.md

Fox Desktop Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Electron-App als native Mac-Hülle, die https://fox.pxo.at/hud in einem transparenten Fullscreen-Overlay lädt und native Mac-Fähigkeiten hinzufügt (Tray, Wakeword, AppleScript, LaunchAgent).

Architecture: Electron lädt fox.pxo.at/hud als remote URL in einem rahmenlosen transparenten BrowserWindow. preload.js injiziert window.foxBridge in die HUD-Seite für native Mac-Befehle via IPC. Ein Python-Subprocess mit OpenWakeWord erkennt "hey jarvis" und löst showHud() aus. Kein eigener WebSocket in Electron — der Reverb-Kanal läuft komplett in der HUD-Seite.

Tech Stack: Electron 28, Node.js (child_process, IPC), Jest 29, Python 3 + openwakeword + pyaudio + onnxruntime

Spec: docs/superpowers/specs/2026-05-11-fox-desktop-design.md

---

Dateien

Datei	Aufgabe
package.json	App-Konfig, Scripts, Dependencies
.gitignore	node_modules, dist etc.
electron/main.js	App-Lifecycle, IPC-Handler, globaler Shortcut
electron/hud.js	BrowserWindow (remote URL, transparent, always-on-top)
electron/tray.js	Menu Bar Icon + Kontextmenü
electron/wakeword.js	Python-Subprocess starten/auto-restarten
electron/mac-agent.js	AppleScript-Dispatch (resolveCommand + executeMacCommand)
electron/preload.js	window.foxBridge in HUD-Seite injizieren
python/wakeword.py	OpenWakeWord Loop, schreibt "WAKE" auf stdout
assets/fox-tray.png	16×16 Platzhalter-Icon
__tests__/mac-agent.test.js	Unit-Tests für mac-agent.js
launchagent/at.pxo.fox.plist	LaunchAgent-Vorlage für Auto-Start beim Login
docs/hud-bridge-integration.md	Anleitung für Laravel HUD Anpassungen

---

Task 1: Projekt-Scaffolding

Files:

• Create: package.json

• Create: .gitignore

• Step 1: package.json erstellen

package.json:

{
  "name": "fox-desktop",
  "version": "1.0.0",
  "description": "Fox — Persönlicher KI-Assistent",
  "main": "electron/main.js",
  "scripts": {
    "start": "electron .",
    "test": "jest"
  },
  "dependencies": {
    "axios": "^1.6.0"
  },
  "devDependencies": {
    "electron": "^28.0.0",
    "jest": "^29.0.0"
  },
  "jest": {
    "testEnvironment": "node",
    "testMatch": ["**/__tests__/**/*.test.js"]
  }
}

• Step 2: .gitignore erstellen

.gitignore:

node_modules/
dist/
*.log
.env

• Step 3: Verzeichnisse anlegen

mkdir -p electron python assets __tests__ launchagent docs

Expected: kein Fehler.

• Step 4: Dependencies installieren

npm install

Expected: node_modules/ wird erstellt, keine Fehler.

• Step 5: Commit

git add package.json package-lock.json .gitignore
git commit -m "feat: project scaffolding"

---

Task 2: Platzhalter-Icon

Files:

• Create: assets/fox-tray.png

• Step 1: 16×16 PNG via Python generieren

python3 -c "
import struct, zlib

def mkpng(w, h, rgb):
    def chunk(t, d):
        c = struct.pack('>I', len(d)) + t + d
        return c + struct.pack('>I', zlib.crc32(t + d) & 0xffffffff)
    ihdr = struct.pack('>IIBBBBB', w, h, 8, 2, 0, 0, 0)
    row = b'\x00' + bytes(rgb) * w
    idat = zlib.compress(row * h)
    return b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', ihdr) + chunk(b'IDAT', idat) + chunk(b'IEND', b'')

open('assets/fox-tray.png', 'wb').write(mkpng(16, 16, [255, 165, 0]))
print('Icon erstellt')
"

Expected: Icon erstellt — assets/fox-tray.png existiert.

• Step 2: Commit

git add assets/fox-tray.png
git commit -m "feat: add placeholder tray icon"

---

Task 3: mac-agent.js — TDD

Files:

• Create: __tests__/mac-agent.test.js

• Create: electron/mac-agent.js

• Step 1: Fehlschlagenden Test schreiben

__tests__/mac-agent.test.js:

const { resolveCommand } = require('../electron/mac-agent');

describe('resolveCommand', () => {
  test('bekannter Shortcut → applescript', () => {
    expect(resolveCommand('spotify_play')).toEqual({
      type: 'applescript',
      script: 'tell application "Spotify" to play',
    });
  });

  test('script: Präfix → applescript mit direktem Script', () => {
    expect(resolveCommand('script:tell application "Finder" to activate')).toEqual({
      type: 'applescript',
      script: 'tell application "Finder" to activate',
    });
  });

  test('open: Präfix → URL öffnen', () => {
    expect(resolveCommand('open:https://example.com')).toEqual({
      type: 'open',
      url: 'https://example.com',
    });
  });

  test('app: Präfix → App öffnen', () => {
    expect(resolveCommand('app:Spotify')).toEqual({
      type: 'app',
      name: 'Spotify',
    });
  });

  test('save: Präfix → Datei auf Desktop speichern', () => {
    expect(resolveCommand('save:{"filename":"test.txt","content":"hallo"}')).toEqual({
      type: 'save',
      filename: 'test.txt',
      content: 'hallo',
    });
  });

  test('unbekannter Befehl → unknown', () => {
    expect(resolveCommand('gibts_nicht')).toEqual({ type: 'unknown' });
  });

  test('volume_up ist bekannter Shortcut', () => {
    const result = resolveCommand('volume_up');
    expect(result.type).toBe('applescript');
    expect(result.script).toContain('output volume');
  });
});

• Step 2: Test ausführen — erwartet FAIL

npm test

Expected: Cannot find module '../electron/mac-agent'

• Step 3: mac-agent.js implementieren

electron/mac-agent.js:

const { exec, execSync } = require('child_process');
const util = require('util');
const fs = require('fs');
const execAsync = util.promisify(exec);

const SCRIPTS = {
  spotify_play:  'tell application "Spotify" to play',
  spotify_pause: 'tell application "Spotify" to pause',
  spotify_next:  'tell application "Spotify" to next track',
  spotify_prev:  'tell application "Spotify" to previous track',
  music_play:    'tell application "Music" to play',
  music_pause:   'tell application "Music" to pause',
  music_next:    'tell application "Music" to next track',
  mail_open:     'tell application "Mail" to activate',
  mail_unread:   'tell application "Mail" to return (count of (messages of inbox whose read status is false))',
  calendar_open: 'tell application "Calendar" to activate',
  notes_open:    'tell application "Notes" to activate',
  volume_up:     'set volume output volume (output volume of (get volume settings) + 10)',
  volume_down:   'set volume output volume (output volume of (get volume settings) - 10)',
  volume_mute:   'set volume output muted true',
  volume_unmute: 'set volume output muted false',
  dark_mode_on:  'tell app "System Events" to tell appearance preferences to set dark mode to true',
  dark_mode_off: 'tell app "System Events" to tell appearance preferences to set dark mode to false',
  screenshot:    'do shell script "screencapture -x ~/Desktop/fox-screenshot-$(date +%Y%m%d-%H%M%S).png"',
};

function resolveCommand(command) {
  if (SCRIPTS[command]) {
    return { type: 'applescript', script: SCRIPTS[command] };
  }
  if (command.startsWith('script:')) {
    return { type: 'applescript', script: command.slice(7) };
  }
  if (command.startsWith('open:')) {
    return { type: 'open', url: command.slice(5) };
  }
  if (command.startsWith('app:')) {
    return { type: 'app', name: command.slice(4) };
  }
  if (command.startsWith('save:')) {
    return { type: 'save', ...JSON.parse(command.slice(5)) };
  }
  return { type: 'unknown' };
}

async function executeMacCommand(command) {
  try {
    const resolved = resolveCommand(command);

    if (resolved.type === 'applescript') {
      const { stdout } = await execAsync(`osascript -e '${resolved.script}'`);
      return { success: true, output: stdout.trim() };
    }
    if (resolved.type === 'open') {
      execSync(`open "${resolved.url}"`);
      return { success: true, output: 'Geöffnet' };
    }
    if (resolved.type === 'app') {
      execSync(`open -a "${resolved.name}"`);
      return { success: true, output: `${resolved.name} geöffnet` };
    }
    if (resolved.type === 'save') {
      const filePath = `${process.env.HOME}/Desktop/${resolved.filename}`;
      fs.writeFileSync(filePath, resolved.content, 'utf8');
      return { success: true, output: `Gespeichert: ${filePath}` };
    }
    return { success: false, output: 'Unbekannter Befehl' };

  } catch (err) {
    return { success: false, output: err.message };
  }
}

module.exports = { executeMacCommand, resolveCommand, SCRIPTS };

• Step 4: Tests ausführen — erwartet PASS

npm test

Expected:

PASS __tests__/mac-agent.test.js
  resolveCommand
    ✓ bekannter Shortcut → applescript
    ✓ script: Präfix → applescript mit direktem Script
    ✓ open: Präfix → URL öffnen
    ✓ app: Präfix → App öffnen
    ✓ save: Präfix → Datei auf Desktop speichern
    ✓ unbekannter Befehl → unknown
    ✓ volume_up ist bekannter Shortcut

Tests: 7 passed, 7 total

• Step 5: Commit

git add electron/mac-agent.js __tests__/mac-agent.test.js
git commit -m "feat: mac-agent with AppleScript dispatch and unit tests"

---

Task 4: preload.js — foxBridge

Files:

• Create: electron/preload.js

• Step 1: preload.js erstellen

electron/preload.js:

const { contextBridge, ipcRenderer } = require('electron');

contextBridge.exposeInMainWorld('foxBridge', {
  sendMessage: (text) => ipcRenderer.send('send-message', text),
  hideHud: () => ipcRenderer.send('hide-hud'),
  macCommand: (cmd) => ipcRenderer.invoke('mac-command', cmd),
  onFoxAction: (cb) => ipcRenderer.on('fox-action', (_, data) => cb(data)),
  onHudShow: (cb) => ipcRenderer.on('hud-show', () => cb()),
  onHudHide: (cb) => ipcRenderer.on('hud-hide', () => cb()),
  onStartListening: (cb) => ipcRenderer.on('start-listening', () => cb()),
  onPlayAudio: (cb) => ipcRenderer.on('play-audio', (_, url) => cb(url)),
});

• Step 2: Commit

git add electron/preload.js
git commit -m "feat: preload — injects window.foxBridge into remote HUD"

---

Task 5: hud.js — BrowserWindow

Files:

• Create: electron/hud.js

• Step 1: hud.js erstellen

electron/hud.js:

const { BrowserWindow, screen } = require('electron');
const path = require('path');

const HUD_URL = process.env.FOX_HUD_URL || 'https://fox.pxo.at/hud';
let hudWindow = null;

function createHud() {
  const { width, height } = screen.getPrimaryDisplay().workAreaSize;

  hudWindow = new BrowserWindow({
    width,
    height,
    x: 0,
    y: 0,
    frame: false,
    transparent: true,
    alwaysOnTop: true,
    show: false,
    skipTaskbar: true,
    hasShadow: false,
    webPreferences: {
      preload: path.join(__dirname, 'preload.js'),
      contextIsolation: true,
      nodeIntegration: false,
      webSecurity: false,
      allowRunningInsecureContent: true,
    },
  });

  hudWindow.setAlwaysOnTop(true, 'screen-saver');
  hudWindow.setVisibleOnAllWorkspaces(true, { visibleOnFullScreen: true });
  hudWindow.loadURL(HUD_URL);
  hudWindow.on('closed', () => { hudWindow = null; });

  return hudWindow;
}

function showHud() {
  if (!hudWindow) createHud();
  hudWindow.show();
  hudWindow.focus();
  hudWindow.webContents.send('hud-show');
}

function hideHud() {
  if (!hudWindow) return;
  hudWindow.webContents.send('hud-hide');
  setTimeout(() => {
    if (hudWindow) hudWindow.hide();
  }, 600);
}

module.exports = { createHud, showHud, hideHud };

• Step 2: Commit

git add electron/hud.js
git commit -m "feat: HUD BrowserWindow — loads fox.pxo.at/hud with preload injection"

---

Task 6: tray.js — Menu Bar Icon

Files:

• Create: electron/tray.js

• Step 1: tray.js erstellen

electron/tray.js:

const { Tray, Menu, nativeImage } = require('electron');
const path = require('path');

let tray = null;

function createTray({ onShow, onHide, onQuit }) {
  const iconPath = path.join(__dirname, '../assets/fox-tray.png');
  const icon = nativeImage.createFromPath(iconPath);
  icon.setTemplateImage(true);

  tray = new Tray(icon);
  tray.setToolTip('Fox');

  tray.setContextMenu(Menu.buildFromTemplate([
    { label: 'Fox öffnen', accelerator: 'Cmd+Shift+F', click: onShow },
    { label: 'Fox verstecken', click: onHide },
    { type: 'separator' },
    { label: 'Beenden', click: onQuit },
  ]));

  tray.on('click', onShow);

  return tray;
}

module.exports = { createTray };

• Step 2: Commit

git add electron/tray.js
git commit -m "feat: tray — menu bar icon with context menu"

---

Task 7: Wakeword — Python + Electron Subprocess

Files:

• Create: python/wakeword.py

• Create: electron/wakeword.js

• Step 1: Python-Abhängigkeiten installieren

pip install openwakeword pyaudio numpy onnxruntime

Expected: alle Pakete installiert ohne Fehler. Falls pyaudio auf macOS fehlschlägt:

brew install portaudio && pip install pyaudio

• Step 2: python/wakeword.py erstellen

python/wakeword.py:

#!/usr/bin/env python3
import sys
import numpy as np
import pyaudio
from openwakeword.model import Model

CHUNK = 1280   # 80ms bei 16kHz — von openwakeword erwartet
RATE = 16000
THRESHOLD = 0.5

def main():
    model = Model(wakeword_models=["hey_jarvis"])

    pa = pyaudio.PyAudio()
    stream = pa.open(
        rate=RATE,
        channels=1,
        format=pyaudio.paInt16,
        input=True,
        frames_per_buffer=CHUNK,
    )

    print("Wakeword aktiv", flush=True)

    try:
        while True:
            data = stream.read(CHUNK, exception_on_overflow=False)
            audio = np.frombuffer(data, dtype=np.int16)
            prediction = model.predict(audio)
            for mdl, score in prediction.items():
                if score > THRESHOLD:
                    print("WAKE", flush=True)
                    model.reset()
                    break
    except KeyboardInterrupt:
        pass
    finally:
        stream.stop_stream()
        stream.close()
        pa.terminate()

if __name__ == "__main__":
    main()

• Step 3: Python-Script manuell testen

python3 python/wakeword.py

Expected: Wakeword aktiv — dann "Hey Jarvis" sagen. Expected: WAKE erscheint in der Ausgabe. Mit Ctrl+C beenden.

• Step 4: electron/wakeword.js erstellen

electron/wakeword.js:

const { spawn } = require('child_process');
const path = require('path');

let wakewordProcess = null;
let running = false;

function startWakeword({ onWake }) {
  if (running) return;
  running = true;

  const scriptPath = path.join(__dirname, '../python/wakeword.py');

  wakewordProcess = spawn('python3', [scriptPath], {
    stdio: ['pipe', 'pipe', 'pipe'],
  });

  wakewordProcess.stdout.on('data', (data) => {
    if (data.toString().trim() === 'WAKE') {
      onWake();
    }
  });

  wakewordProcess.stderr.on('data', (data) => {
    console.error('[Wakeword]', data.toString().trim());
  });

  wakewordProcess.on('close', (code) => {
    running = false;
    wakewordProcess = null;
    if (code !== 0) {
      setTimeout(() => startWakeword({ onWake }), 2000);
    }
  });
}

function stopWakeword() {
  if (wakewordProcess) {
    wakewordProcess.kill();
    wakewordProcess = null;
    running = false;
  }
}

module.exports = { startWakeword, stopWakeword };

• Step 5: Commit

git add python/wakeword.py electron/wakeword.js
git commit -m "feat: wakeword — OpenWakeWord subprocess with hey_jarvis model"

---

Task 8: main.js — Alles verkabeln

Files:

• Create: electron/main.js

• Step 1: main.js erstellen

electron/main.js:

const { app, ipcMain, globalShortcut, nativeTheme } = require('electron');
const { createTray } = require('./tray');
const { createHud, showHud, hideHud } = require('./hud');
const { startWakeword, stopWakeword } = require('./wakeword');
const { executeMacCommand } = require('./mac-agent');
const axios = require('axios');

const FOX_URL = process.env.FOX_URL || 'https://fox.pxo.at';

app.dock?.hide();

if (!app.requestSingleInstanceLock()) {
  app.quit();
}

app.whenReady().then(() => {
  nativeTheme.themeSource = 'dark';

  createHud();

  createTray({
    onShow: showHud,
    onHide: hideHud,
    onQuit: () => app.quit(),
  });

  globalShortcut.register('CommandOrControl+Shift+F', showHud);

  startWakeword({ onWake: showHud });

  ipcMain.on('hide-hud', () => hideHud());

  ipcMain.on('send-message', async (_, text) => {
    try {
      await axios.post(`${FOX_URL}/fox/message`, { text });
    } catch (err) {
      console.error('[Fox] send-message fehlgeschlagen:', err.message);
    }
  });

  ipcMain.handle('mac-command', async (_, command) => {
    return await executeMacCommand(command);
  });
});

app.on('will-quit', () => {
  globalShortcut.unregisterAll();
  stopWakeword();
});

app.on('window-all-closed', (e) => {
  e.preventDefault();
});

• Step 2: App erstmalig starten

npm start

Expected:

• Kein Dock-Icon

• Oranges Icon in der Menu Bar

• Terminal zeigt [Wakeword] Wakeword aktiv

• Step 3: Tray-Klick testen

Auf das Tray-Icon klicken.

Expected: BrowserWindow öffnet sich und lädt https://fox.pxo.at/hud.

• Step 4: Tastaturkürzel testen

Cmd+Shift+F drücken (HUD muss vorher geschlossen sein).

Expected: HUD öffnet sich.

• Step 5: Wakeword testen

"Hey Jarvis" sagen.

Expected: HUD öffnet sich automatisch, Terminal zeigt nichts Ungewöhnliches.

• Step 6: Beenden via Tray-Menü testen

Rechtsklick auf Tray → "Beenden".

Expected: App beendet sich sauber, kein hängender Prozess.

• Step 7: Commit

git add electron/main.js
git commit -m "feat: main.js — wires tray, HUD, wakeword, shortcuts and IPC"

---

Task 9: LaunchAgent

Files:

• Create: launchagent/at.pxo.fox.plist

• Step 1: plist-Vorlage erstellen

launchagent/at.pxo.fox.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>at.pxo.fox</string>
    <key>ProgramArguments</key>
    <array>
        <string>/Applications/Fox.app/Contents/MacOS/Fox</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/tmp/fox-desktop.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/fox-desktop.log</string>
    <key>EnvironmentVariables</key>
    <dict>
        <key>FOX_URL</key>
        <string>https://fox.pxo.at</string>
    </dict>
</dict>
</plist>

Aktivieren (erst nach electron-builder-Build und Kopieren nach /Applications):

cp launchagent/at.pxo.fox.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/at.pxo.fox.plist

• Step 2: Commit

git add launchagent/at.pxo.fox.plist
git commit -m "feat: LaunchAgent plist for auto-start on login"

---

Task 10: Laravel HUD Anpassungen dokumentieren

Files:

• Create: docs/hud-bridge-integration.md

• Step 1: Integrations-Dokument erstellen

docs/hud-bridge-integration.md:

# Fox HUD — Electron Bridge Integration

Änderungen in `~/fox` (Laravel App). Alle Punkte sind **optional** —
ohne `window.foxBridge` (normaler Browser) läuft das HUD unverändert weiter.

## 1. ESC → HUD schließen

In `fox-hud.blade.php` oder dem HUD-JavaScript:

```javascript
document.addEventListener('keydown', (e) => {
  if (e.key === 'Escape') window.foxBridge?.hideHud();
});

2. Mac-Aktionen in onAction()

Im bestehenden onAction() Handler am Anfang ergänzen:

if (payload.action === 'spotify_play')  { await window.foxBridge?.macCommand('spotify_play'); return; }
if (payload.action === 'spotify_pause') { await window.foxBridge?.macCommand('spotify_pause'); return; }
if (payload.action === 'spotify_next')  { await window.foxBridge?.macCommand('spotify_next'); return; }
if (payload.action === 'spotify_prev')  { await window.foxBridge?.macCommand('spotify_prev'); return; }
if (payload.action === 'mail_open')     { await window.foxBridge?.macCommand('mail_open'); return; }
if (payload.action === 'calendar_open') { await window.foxBridge?.macCommand('calendar_open'); return; }
if (payload.action === 'notes_open')    { await window.foxBridge?.macCommand('notes_open'); return; }
if (payload.action === 'volume_up')     { await window.foxBridge?.macCommand('volume_up'); return; }
if (payload.action === 'volume_down')   { await window.foxBridge?.macCommand('volume_down'); return; }
if (payload.action === 'screenshot')    { await window.foxBridge?.macCommand('screenshot'); return; }

if (payload.action === 'open_url' && payload.url) {
  await window.foxBridge?.macCommand('open:' + payload.url);
  return;
}
if (payload.action === 'open_app' && payload.app) {
  await window.foxBridge?.macCommand('app:' + payload.app);
  return;
}
if (payload.action === 'save_desktop' && payload.filename) {
  await window.foxBridge?.macCommand('save:' + JSON.stringify({
    filename: payload.filename,
    content: payload.content || '',
  }));
  return;
}

3. Nachrichten senden (optional)

Wenn Nachrichten direkt über Electron statt Livewire gesendet werden sollen:

window.foxBridge?.sendMessage(text);

4. Laravel Route für /fox/message

In routes/api.php oder routes/web.php:

Route::post('/fox/message', function (\Illuminate\Http\Request $request) {
    $text = $request->input('text', '');
    if (empty(trim($text))) {
        return response()->json(['error' => 'Kein Text'], 400);
    }
    $userMessage = app(\App\Services\FoxAgent::class)
        ->recordUserMessage($text, ['source' => 'electron']);
    dispatch(fn() => app(\App\Services\FoxAgent::class)
        ->processUserMessage($userMessage));
    return response()->json(['status' => 'ok']);
})->middleware('api');

5. Electron erkennen

const isElectron = typeof window.foxBridge !== 'undefined';

- [ ] **Step 2: Commit**

```bash
git add docs/hud-bridge-integration.md
git commit -m "docs: HUD bridge integration guide for Laravel app"