# Fox Desktop Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Electron-App als native Mac-Hülle, die `https://fox.pxo.at/hud` in einem transparenten Fullscreen-Overlay lädt und native Mac-Fähigkeiten hinzufügt (Tray, Wakeword, AppleScript, LaunchAgent). **Architecture:** Electron lädt `fox.pxo.at/hud` als remote URL in einem rahmenlosen transparenten BrowserWindow. `preload.js` injiziert `window.foxBridge` in die HUD-Seite für native Mac-Befehle via IPC. Ein Python-Subprocess mit OpenWakeWord erkennt "hey jarvis" und löst `showHud()` aus. Kein eigener WebSocket in Electron — der Reverb-Kanal läuft komplett in der HUD-Seite. **Tech Stack:** Electron 28, Node.js (child_process, IPC), Jest 29, Python 3 + openwakeword + pyaudio + onnxruntime **Spec:** `docs/superpowers/specs/2026-05-11-fox-desktop-design.md` --- ## Dateien | Datei | Aufgabe | |---|---| | `package.json` | App-Konfig, Scripts, Dependencies | | `.gitignore` | node_modules, dist etc. | | `electron/main.js` | App-Lifecycle, IPC-Handler, globaler Shortcut | | `electron/hud.js` | BrowserWindow (remote URL, transparent, always-on-top) | | `electron/tray.js` | Menu Bar Icon + Kontextmenü | | `electron/wakeword.js` | Python-Subprocess starten/auto-restarten | | `electron/mac-agent.js` | AppleScript-Dispatch (resolveCommand + executeMacCommand) | | `electron/preload.js` | `window.foxBridge` in HUD-Seite injizieren | | `python/wakeword.py` | OpenWakeWord Loop, schreibt "WAKE" auf stdout | | `assets/fox-tray.png` | 16×16 Platzhalter-Icon | | `__tests__/mac-agent.test.js` | Unit-Tests für mac-agent.js | | `launchagent/at.pxo.fox.plist` | LaunchAgent-Vorlage für Auto-Start beim Login | | `docs/hud-bridge-integration.md` | Anleitung für Laravel HUD Anpassungen | --- ### Task 1: Projekt-Scaffolding **Files:** - Create: `package.json` - Create: `.gitignore` - [ ] **Step 1: package.json erstellen** `package.json`: ```json { "name": "fox-desktop", "version": "1.0.0", "description": "Fox — Persönlicher KI-Assistent", "main": "electron/main.js", "scripts": { "start": "electron .", "test": "jest" }, "dependencies": { "axios": "^1.6.0" }, "devDependencies": { "electron": "^28.0.0", "jest": "^29.0.0" }, "jest": { "testEnvironment": "node", "testMatch": ["**/__tests__/**/*.test.js"] } } ``` - [ ] **Step 2: .gitignore erstellen** `.gitignore`: ``` node_modules/ dist/ *.log .env ``` - [ ] **Step 3: Verzeichnisse anlegen** ```bash mkdir -p electron python assets __tests__ launchagent docs ``` Expected: kein Fehler. - [ ] **Step 4: Dependencies installieren** ```bash npm install ``` Expected: `node_modules/` wird erstellt, keine Fehler. - [ ] **Step 5: Commit** ```bash git add package.json package-lock.json .gitignore git commit -m "feat: project scaffolding" ``` --- ### Task 2: Platzhalter-Icon **Files:** - Create: `assets/fox-tray.png` - [ ] **Step 1: 16×16 PNG via Python generieren** ```bash python3 -c " import struct, zlib def mkpng(w, h, rgb): def chunk(t, d): c = struct.pack('>I', len(d)) + t + d return c + struct.pack('>I', zlib.crc32(t + d) & 0xffffffff) ihdr = struct.pack('>IIBBBBB', w, h, 8, 2, 0, 0, 0) row = b'\x00' + bytes(rgb) * w idat = zlib.compress(row * h) return b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', ihdr) + chunk(b'IDAT', idat) + chunk(b'IEND', b'') open('assets/fox-tray.png', 'wb').write(mkpng(16, 16, [255, 165, 0])) print('Icon erstellt') " ``` Expected: `Icon erstellt` — `assets/fox-tray.png` existiert. - [ ] **Step 2: Commit** ```bash git add assets/fox-tray.png git commit -m "feat: add placeholder tray icon" ``` --- ### Task 3: mac-agent.js — TDD **Files:** - Create: `__tests__/mac-agent.test.js` - Create: `electron/mac-agent.js` - [ ] **Step 1: Fehlschlagenden Test schreiben** `__tests__/mac-agent.test.js`: ```javascript const { resolveCommand } = require('../electron/mac-agent'); describe('resolveCommand', () => { test('bekannter Shortcut → applescript', () => { expect(resolveCommand('spotify_play')).toEqual({ type: 'applescript', script: 'tell application "Spotify" to play', }); }); test('script: Präfix → applescript mit direktem Script', () => { expect(resolveCommand('script:tell application "Finder" to activate')).toEqual({ type: 'applescript', script: 'tell application "Finder" to activate', }); }); test('open: Präfix → URL öffnen', () => { expect(resolveCommand('open:https://example.com')).toEqual({ type: 'open', url: 'https://example.com', }); }); test('app: Präfix → App öffnen', () => { expect(resolveCommand('app:Spotify')).toEqual({ type: 'app', name: 'Spotify', }); }); test('save: Präfix → Datei auf Desktop speichern', () => { expect(resolveCommand('save:{"filename":"test.txt","content":"hallo"}')).toEqual({ type: 'save', filename: 'test.txt', content: 'hallo', }); }); test('unbekannter Befehl → unknown', () => { expect(resolveCommand('gibts_nicht')).toEqual({ type: 'unknown' }); }); test('volume_up ist bekannter Shortcut', () => { const result = resolveCommand('volume_up'); expect(result.type).toBe('applescript'); expect(result.script).toContain('output volume'); }); }); ``` - [ ] **Step 2: Test ausführen — erwartet FAIL** ```bash npm test ``` Expected: `Cannot find module '../electron/mac-agent'` - [ ] **Step 3: mac-agent.js implementieren** `electron/mac-agent.js`: ```javascript const { exec, execSync } = require('child_process'); const util = require('util'); const fs = require('fs'); const execAsync = util.promisify(exec); const SCRIPTS = { spotify_play: 'tell application "Spotify" to play', spotify_pause: 'tell application "Spotify" to pause', spotify_next: 'tell application "Spotify" to next track', spotify_prev: 'tell application "Spotify" to previous track', music_play: 'tell application "Music" to play', music_pause: 'tell application "Music" to pause', music_next: 'tell application "Music" to next track', mail_open: 'tell application "Mail" to activate', mail_unread: 'tell application "Mail" to return (count of (messages of inbox whose read status is false))', calendar_open: 'tell application "Calendar" to activate', notes_open: 'tell application "Notes" to activate', volume_up: 'set volume output volume (output volume of (get volume settings) + 10)', volume_down: 'set volume output volume (output volume of (get volume settings) - 10)', volume_mute: 'set volume output muted true', volume_unmute: 'set volume output muted false', dark_mode_on: 'tell app "System Events" to tell appearance preferences to set dark mode to true', dark_mode_off: 'tell app "System Events" to tell appearance preferences to set dark mode to false', screenshot: 'do shell script "screencapture -x ~/Desktop/fox-screenshot-$(date +%Y%m%d-%H%M%S).png"', }; function resolveCommand(command) { if (SCRIPTS[command]) { return { type: 'applescript', script: SCRIPTS[command] }; } if (command.startsWith('script:')) { return { type: 'applescript', script: command.slice(7) }; } if (command.startsWith('open:')) { return { type: 'open', url: command.slice(5) }; } if (command.startsWith('app:')) { return { type: 'app', name: command.slice(4) }; } if (command.startsWith('save:')) { return { type: 'save', ...JSON.parse(command.slice(5)) }; } return { type: 'unknown' }; } async function executeMacCommand(command) { try { const resolved = resolveCommand(command); if (resolved.type === 'applescript') { const { stdout } = await execAsync(`osascript -e '${resolved.script}'`); return { success: true, output: stdout.trim() }; } if (resolved.type === 'open') { execSync(`open "${resolved.url}"`); return { success: true, output: 'Geöffnet' }; } if (resolved.type === 'app') { execSync(`open -a "${resolved.name}"`); return { success: true, output: `${resolved.name} geöffnet` }; } if (resolved.type === 'save') { const filePath = `${process.env.HOME}/Desktop/${resolved.filename}`; fs.writeFileSync(filePath, resolved.content, 'utf8'); return { success: true, output: `Gespeichert: ${filePath}` }; } return { success: false, output: 'Unbekannter Befehl' }; } catch (err) { return { success: false, output: err.message }; } } module.exports = { executeMacCommand, resolveCommand, SCRIPTS }; ``` - [ ] **Step 4: Tests ausführen — erwartet PASS** ```bash npm test ``` Expected: ``` PASS __tests__/mac-agent.test.js resolveCommand ✓ bekannter Shortcut → applescript ✓ script: Präfix → applescript mit direktem Script ✓ open: Präfix → URL öffnen ✓ app: Präfix → App öffnen ✓ save: Präfix → Datei auf Desktop speichern ✓ unbekannter Befehl → unknown ✓ volume_up ist bekannter Shortcut Tests: 7 passed, 7 total ``` - [ ] **Step 5: Commit** ```bash git add electron/mac-agent.js __tests__/mac-agent.test.js git commit -m "feat: mac-agent with AppleScript dispatch and unit tests" ``` --- ### Task 4: preload.js — foxBridge **Files:** - Create: `electron/preload.js` - [ ] **Step 1: preload.js erstellen** `electron/preload.js`: ```javascript const { contextBridge, ipcRenderer } = require('electron'); contextBridge.exposeInMainWorld('foxBridge', { sendMessage: (text) => ipcRenderer.send('send-message', text), hideHud: () => ipcRenderer.send('hide-hud'), macCommand: (cmd) => ipcRenderer.invoke('mac-command', cmd), onFoxAction: (cb) => ipcRenderer.on('fox-action', (_, data) => cb(data)), onHudShow: (cb) => ipcRenderer.on('hud-show', () => cb()), onHudHide: (cb) => ipcRenderer.on('hud-hide', () => cb()), onStartListening: (cb) => ipcRenderer.on('start-listening', () => cb()), onPlayAudio: (cb) => ipcRenderer.on('play-audio', (_, url) => cb(url)), }); ``` - [ ] **Step 2: Commit** ```bash git add electron/preload.js git commit -m "feat: preload — injects window.foxBridge into remote HUD" ``` --- ### Task 5: hud.js — BrowserWindow **Files:** - Create: `electron/hud.js` - [ ] **Step 1: hud.js erstellen** `electron/hud.js`: ```javascript const { BrowserWindow, screen } = require('electron'); const path = require('path'); const HUD_URL = process.env.FOX_HUD_URL || 'https://fox.pxo.at/hud'; let hudWindow = null; function createHud() { const { width, height } = screen.getPrimaryDisplay().workAreaSize; hudWindow = new BrowserWindow({ width, height, x: 0, y: 0, frame: false, transparent: true, alwaysOnTop: true, show: false, skipTaskbar: true, hasShadow: false, webPreferences: { preload: path.join(__dirname, 'preload.js'), contextIsolation: true, nodeIntegration: false, webSecurity: false, allowRunningInsecureContent: true, }, }); hudWindow.setAlwaysOnTop(true, 'screen-saver'); hudWindow.setVisibleOnAllWorkspaces(true, { visibleOnFullScreen: true }); hudWindow.loadURL(HUD_URL); hudWindow.on('closed', () => { hudWindow = null; }); return hudWindow; } function showHud() { if (!hudWindow) createHud(); hudWindow.show(); hudWindow.focus(); hudWindow.webContents.send('hud-show'); } function hideHud() { if (!hudWindow) return; hudWindow.webContents.send('hud-hide'); setTimeout(() => { if (hudWindow) hudWindow.hide(); }, 600); } module.exports = { createHud, showHud, hideHud }; ``` - [ ] **Step 2: Commit** ```bash git add electron/hud.js git commit -m "feat: HUD BrowserWindow — loads fox.pxo.at/hud with preload injection" ``` --- ### Task 6: tray.js — Menu Bar Icon **Files:** - Create: `electron/tray.js` - [ ] **Step 1: tray.js erstellen** `electron/tray.js`: ```javascript const { Tray, Menu, nativeImage } = require('electron'); const path = require('path'); let tray = null; function createTray({ onShow, onHide, onQuit }) { const iconPath = path.join(__dirname, '../assets/fox-tray.png'); const icon = nativeImage.createFromPath(iconPath); icon.setTemplateImage(true); tray = new Tray(icon); tray.setToolTip('Fox'); tray.setContextMenu(Menu.buildFromTemplate([ { label: 'Fox öffnen', accelerator: 'Cmd+Shift+F', click: onShow }, { label: 'Fox verstecken', click: onHide }, { type: 'separator' }, { label: 'Beenden', click: onQuit }, ])); tray.on('click', onShow); return tray; } module.exports = { createTray }; ``` - [ ] **Step 2: Commit** ```bash git add electron/tray.js git commit -m "feat: tray — menu bar icon with context menu" ``` --- ### Task 7: Wakeword — Python + Electron Subprocess **Files:** - Create: `python/wakeword.py` - Create: `electron/wakeword.js` - [ ] **Step 1: Python-Abhängigkeiten installieren** ```bash pip install openwakeword pyaudio numpy onnxruntime ``` Expected: alle Pakete installiert ohne Fehler. Falls `pyaudio` auf macOS fehlschlägt: ```bash brew install portaudio && pip install pyaudio ``` - [ ] **Step 2: python/wakeword.py erstellen** `python/wakeword.py`: ```python #!/usr/bin/env python3 import sys import numpy as np import pyaudio from openwakeword.model import Model CHUNK = 1280 # 80ms bei 16kHz — von openwakeword erwartet RATE = 16000 THRESHOLD = 0.5 def main(): model = Model(wakeword_models=["hey_jarvis"]) pa = pyaudio.PyAudio() stream = pa.open( rate=RATE, channels=1, format=pyaudio.paInt16, input=True, frames_per_buffer=CHUNK, ) print("Wakeword aktiv", flush=True) try: while True: data = stream.read(CHUNK, exception_on_overflow=False) audio = np.frombuffer(data, dtype=np.int16) prediction = model.predict(audio) for mdl, score in prediction.items(): if score > THRESHOLD: print("WAKE", flush=True) model.reset() break except KeyboardInterrupt: pass finally: stream.stop_stream() stream.close() pa.terminate() if __name__ == "__main__": main() ``` - [ ] **Step 3: Python-Script manuell testen** ```bash python3 python/wakeword.py ``` Expected: `Wakeword aktiv` — dann "Hey Jarvis" sagen. Expected: `WAKE` erscheint in der Ausgabe. Mit `Ctrl+C` beenden. - [ ] **Step 4: electron/wakeword.js erstellen** `electron/wakeword.js`: ```javascript const { spawn } = require('child_process'); const path = require('path'); let wakewordProcess = null; let running = false; function startWakeword({ onWake }) { if (running) return; running = true; const scriptPath = path.join(__dirname, '../python/wakeword.py'); wakewordProcess = spawn('python3', [scriptPath], { stdio: ['pipe', 'pipe', 'pipe'], }); wakewordProcess.stdout.on('data', (data) => { if (data.toString().trim() === 'WAKE') { onWake(); } }); wakewordProcess.stderr.on('data', (data) => { console.error('[Wakeword]', data.toString().trim()); }); wakewordProcess.on('close', (code) => { running = false; wakewordProcess = null; if (code !== 0) { setTimeout(() => startWakeword({ onWake }), 2000); } }); } function stopWakeword() { if (wakewordProcess) { wakewordProcess.kill(); wakewordProcess = null; running = false; } } module.exports = { startWakeword, stopWakeword }; ``` - [ ] **Step 5: Commit** ```bash git add python/wakeword.py electron/wakeword.js git commit -m "feat: wakeword — OpenWakeWord subprocess with hey_jarvis model" ``` --- ### Task 8: main.js — Alles verkabeln **Files:** - Create: `electron/main.js` - [ ] **Step 1: main.js erstellen** `electron/main.js`: ```javascript const { app, ipcMain, globalShortcut, nativeTheme } = require('electron'); const { createTray } = require('./tray'); const { createHud, showHud, hideHud } = require('./hud'); const { startWakeword, stopWakeword } = require('./wakeword'); const { executeMacCommand } = require('./mac-agent'); const axios = require('axios'); const FOX_URL = process.env.FOX_URL || 'https://fox.pxo.at'; app.dock?.hide(); if (!app.requestSingleInstanceLock()) { app.quit(); } app.whenReady().then(() => { nativeTheme.themeSource = 'dark'; createHud(); createTray({ onShow: showHud, onHide: hideHud, onQuit: () => app.quit(), }); globalShortcut.register('CommandOrControl+Shift+F', showHud); startWakeword({ onWake: showHud }); ipcMain.on('hide-hud', () => hideHud()); ipcMain.on('send-message', async (_, text) => { try { await axios.post(`${FOX_URL}/fox/message`, { text }); } catch (err) { console.error('[Fox] send-message fehlgeschlagen:', err.message); } }); ipcMain.handle('mac-command', async (_, command) => { return await executeMacCommand(command); }); }); app.on('will-quit', () => { globalShortcut.unregisterAll(); stopWakeword(); }); app.on('window-all-closed', (e) => { e.preventDefault(); }); ``` - [ ] **Step 2: App erstmalig starten** ```bash npm start ``` Expected: - Kein Dock-Icon - Oranges Icon in der Menu Bar - Terminal zeigt `[Wakeword] Wakeword aktiv` - [ ] **Step 3: Tray-Klick testen** Auf das Tray-Icon klicken. Expected: BrowserWindow öffnet sich und lädt `https://fox.pxo.at/hud`. - [ ] **Step 4: Tastaturkürzel testen** `Cmd+Shift+F` drücken (HUD muss vorher geschlossen sein). Expected: HUD öffnet sich. - [ ] **Step 5: Wakeword testen** "Hey Jarvis" sagen. Expected: HUD öffnet sich automatisch, Terminal zeigt nichts Ungewöhnliches. - [ ] **Step 6: Beenden via Tray-Menü testen** Rechtsklick auf Tray → "Beenden". Expected: App beendet sich sauber, kein hängender Prozess. - [ ] **Step 7: Commit** ```bash git add electron/main.js git commit -m "feat: main.js — wires tray, HUD, wakeword, shortcuts and IPC" ``` --- ### Task 9: LaunchAgent **Files:** - Create: `launchagent/at.pxo.fox.plist` - [ ] **Step 1: plist-Vorlage erstellen** `launchagent/at.pxo.fox.plist`: ```xml Label at.pxo.fox ProgramArguments /Applications/Fox.app/Contents/MacOS/Fox RunAtLoad KeepAlive StandardOutPath /tmp/fox-desktop.log StandardErrorPath /tmp/fox-desktop.log EnvironmentVariables FOX_URL https://fox.pxo.at ``` Aktivieren (erst nach `electron-builder`-Build und Kopieren nach `/Applications`): ```bash cp launchagent/at.pxo.fox.plist ~/Library/LaunchAgents/ launchctl load ~/Library/LaunchAgents/at.pxo.fox.plist ``` - [ ] **Step 2: Commit** ```bash git add launchagent/at.pxo.fox.plist git commit -m "feat: LaunchAgent plist for auto-start on login" ``` --- ### Task 10: Laravel HUD Anpassungen dokumentieren **Files:** - Create: `docs/hud-bridge-integration.md` - [ ] **Step 1: Integrations-Dokument erstellen** `docs/hud-bridge-integration.md`: ```markdown # Fox HUD — Electron Bridge Integration Änderungen in `~/fox` (Laravel App). Alle Punkte sind **optional** — ohne `window.foxBridge` (normaler Browser) läuft das HUD unverändert weiter. ## 1. ESC → HUD schließen In `fox-hud.blade.php` oder dem HUD-JavaScript: ```javascript document.addEventListener('keydown', (e) => { if (e.key === 'Escape') window.foxBridge?.hideHud(); }); ``` ## 2. Mac-Aktionen in onAction() Im bestehenden `onAction()` Handler am Anfang ergänzen: ```javascript if (payload.action === 'spotify_play') { await window.foxBridge?.macCommand('spotify_play'); return; } if (payload.action === 'spotify_pause') { await window.foxBridge?.macCommand('spotify_pause'); return; } if (payload.action === 'spotify_next') { await window.foxBridge?.macCommand('spotify_next'); return; } if (payload.action === 'spotify_prev') { await window.foxBridge?.macCommand('spotify_prev'); return; } if (payload.action === 'mail_open') { await window.foxBridge?.macCommand('mail_open'); return; } if (payload.action === 'calendar_open') { await window.foxBridge?.macCommand('calendar_open'); return; } if (payload.action === 'notes_open') { await window.foxBridge?.macCommand('notes_open'); return; } if (payload.action === 'volume_up') { await window.foxBridge?.macCommand('volume_up'); return; } if (payload.action === 'volume_down') { await window.foxBridge?.macCommand('volume_down'); return; } if (payload.action === 'screenshot') { await window.foxBridge?.macCommand('screenshot'); return; } if (payload.action === 'open_url' && payload.url) { await window.foxBridge?.macCommand('open:' + payload.url); return; } if (payload.action === 'open_app' && payload.app) { await window.foxBridge?.macCommand('app:' + payload.app); return; } if (payload.action === 'save_desktop' && payload.filename) { await window.foxBridge?.macCommand('save:' + JSON.stringify({ filename: payload.filename, content: payload.content || '', })); return; } ``` ## 3. Nachrichten senden (optional) Wenn Nachrichten direkt über Electron statt Livewire gesendet werden sollen: ```javascript window.foxBridge?.sendMessage(text); ``` ## 4. Laravel Route für /fox/message In `routes/api.php` oder `routes/web.php`: ```php Route::post('/fox/message', function (\Illuminate\Http\Request $request) { $text = $request->input('text', ''); if (empty(trim($text))) { return response()->json(['error' => 'Kein Text'], 400); } $userMessage = app(\App\Services\FoxAgent::class) ->recordUserMessage($text, ['source' => 'electron']); dispatch(fn() => app(\App\Services\FoxAgent::class) ->processUserMessage($userMessage)); return response()->json(['status' => 'ok']); })->middleware('api'); ``` ## 5. Electron erkennen ```javascript const isElectron = typeof window.foxBridge !== 'undefined'; ``` ``` - [ ] **Step 2: Commit** ```bash git add docs/hud-bridge-integration.md git commit -m "docs: HUD bridge integration guide for Laravel app" ```