Add Fox Desktop design spec
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>main
commit
c6ddbc3dc9
|
|
@ -0,0 +1,182 @@
|
||||||
|
# Fox Desktop — Design Spec
|
||||||
|
**Datum:** 2026-05-11
|
||||||
|
**Status:** Genehmigt
|
||||||
|
|
||||||
|
## Überblick
|
||||||
|
|
||||||
|
Fox Desktop ist eine native Mac-App (Electron), die als dünne Hülle um die bestehende Laravel-HUD-App (`fox.pxo.at/hud`) fungiert. Electron fügt native Mac-Fähigkeiten hinzu, ohne das HUD neu zu schreiben.
|
||||||
|
|
||||||
|
## Architektur
|
||||||
|
|
||||||
|
```
|
||||||
|
Mac OS
|
||||||
|
├── Tray Icon (Menu Bar)
|
||||||
|
├── BrowserWindow (Overlay, fullscreen, always-on-top)
|
||||||
|
│ └── lädt https://fox.pxo.at/hud
|
||||||
|
│ └── preload.js injiziert window.foxBridge
|
||||||
|
├── Global Shortcut: Cmd+Shift+F
|
||||||
|
└── Python Subprocess: openwakeword ("hey_jarvis")
|
||||||
|
└── stdout "WAKE" → IPC → showHud()
|
||||||
|
```
|
||||||
|
|
||||||
|
Der Electron-Main-Process hat **keine** eigene WebSocket-Verbindung zu Reverb. Der WebSocket-Kanal gehört der HUD-Seite selbst. Electron ist reiner Plattform-Layer.
|
||||||
|
|
||||||
|
## Prinzipien
|
||||||
|
|
||||||
|
- **Pure Shell:** Electron kapselt, erweitert nicht die App-Logik
|
||||||
|
- **Bridge ist optional:** `window.foxBridge` wird nur in Electron injiziert; das HUD funktioniert im normalen Browser unverändert weiter
|
||||||
|
- **Kein Vite Build:** Es gibt kein eigenes Frontend — nur Electron-Module und Python-Script
|
||||||
|
- **Kein electron-builder** vorerst — erst nach stabilem Dev-Setup
|
||||||
|
|
||||||
|
## Dateien
|
||||||
|
|
||||||
|
| Datei | Aufgabe |
|
||||||
|
|---|---|
|
||||||
|
| `package.json` | Electron-App-Konfiguration, npm-Scripts |
|
||||||
|
| `electron/main.js` | App-Lifecycle, IPC-Handler, globaler Shortcut |
|
||||||
|
| `electron/hud.js` | BrowserWindow erstellen, show/hide mit Transition |
|
||||||
|
| `electron/tray.js` | Menu Bar Icon (Template Image) + Kontextmenü |
|
||||||
|
| `electron/wakeword.js` | Python-Subprocess starten, Auto-Restart bei Absturz |
|
||||||
|
| `electron/mac-agent.js` | AppleScript-Ausführung via osascript |
|
||||||
|
| `electron/preload.js` | `window.foxBridge` in HUD-Seite injizieren |
|
||||||
|
| `python/wakeword.py` | OpenWakeWord Loop, schreibt "WAKE" auf stdout |
|
||||||
|
| `assets/fox-tray.png` | 16×16 Menu Bar Icon (Template Image) |
|
||||||
|
|
||||||
|
## Datenflüsse
|
||||||
|
|
||||||
|
### HUD öffnen (Wakeword)
|
||||||
|
```
|
||||||
|
"hey jarvis" gesprochen
|
||||||
|
→ python/wakeword.py erkennt Wakeword
|
||||||
|
→ stdout: "WAKE\n"
|
||||||
|
→ electron/wakeword.js liest stdout
|
||||||
|
→ IPC intern → showHud()
|
||||||
|
→ BrowserWindow.show() + focus()
|
||||||
|
→ webContents.send('hud-show')
|
||||||
|
→ HUD CSS-Transition startet
|
||||||
|
```
|
||||||
|
|
||||||
|
### HUD öffnen (Tastaturkürzel / Tray)
|
||||||
|
```
|
||||||
|
Cmd+Shift+F gedrückt ODER Tray-Icon geklickt
|
||||||
|
→ main.js globalShortcut / tray.on('click')
|
||||||
|
→ showHud()
|
||||||
|
```
|
||||||
|
|
||||||
|
### Mac-Befehl aus HUD
|
||||||
|
```
|
||||||
|
HUD JavaScript: await window.foxBridge.macCommand('spotify_play')
|
||||||
|
→ preload.js: ipcRenderer.invoke('mac-command', 'spotify_play')
|
||||||
|
→ main.js IPC Handler
|
||||||
|
→ electron/mac-agent.js: executeMacCommand('spotify_play')
|
||||||
|
→ osascript: tell application "Spotify" to play
|
||||||
|
→ Rückgabewert { success: true, output: '' } → HUD
|
||||||
|
```
|
||||||
|
|
||||||
|
### HUD schließen
|
||||||
|
```
|
||||||
|
ESC im HUD ODER foxBridge.hideHud() ODER Tray-Menü "Verstecken"
|
||||||
|
→ ipcRenderer.send('hide-hud')
|
||||||
|
→ main.js → hideHud()
|
||||||
|
→ webContents.send('hud-hide') ← CSS-Transition startet
|
||||||
|
→ setTimeout 600ms → BrowserWindow.hide()
|
||||||
|
```
|
||||||
|
|
||||||
|
## BrowserWindow Konfiguration
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
{
|
||||||
|
width: screenWidth,
|
||||||
|
height: screenHeight,
|
||||||
|
x: 0, y: 0,
|
||||||
|
frame: false,
|
||||||
|
transparent: true,
|
||||||
|
alwaysOnTop: true,
|
||||||
|
show: false,
|
||||||
|
skipTaskbar: true,
|
||||||
|
hasShadow: false,
|
||||||
|
webPreferences: {
|
||||||
|
preload: path.join(__dirname, 'preload.js'),
|
||||||
|
contextIsolation: true,
|
||||||
|
nodeIntegration: false,
|
||||||
|
webSecurity: false, // Remote URL + preload injection
|
||||||
|
allowRunningInsecureContent: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`setAlwaysOnTop(true, 'screen-saver')` und `setVisibleOnAllWorkspaces(true)` nach dem Erstellen.
|
||||||
|
|
||||||
|
## window.foxBridge API
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
window.foxBridge = {
|
||||||
|
sendMessage(text), // Text an Fox Laravel via IPC
|
||||||
|
hideHud(), // HUD verstecken
|
||||||
|
macCommand(cmd), // AppleScript-Befehl ausführen (async, gibt Result zurück)
|
||||||
|
onFoxAction(callback), // Fox-Aktionen empfangen
|
||||||
|
onHudShow(callback), // HUD-Show-Event
|
||||||
|
onHudHide(callback), // HUD-Hide-Event
|
||||||
|
onStartListening(callback), // Mikrofon-Start-Signal
|
||||||
|
onPlayAudio(callback), // Audio-URL zum Abspielen
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Mac-Agent Befehle
|
||||||
|
|
||||||
|
Vordefinierte Shortcuts (z.B. `spotify_play`, `mail_open`, `volume_up`) sowie generische Formen:
|
||||||
|
- `script:<applescript>` — direktes AppleScript
|
||||||
|
- `open:<url>` — URL im Standardbrowser öffnen
|
||||||
|
- `app:<name>` — Mac-App öffnen
|
||||||
|
- `save:<json>` — Datei auf Desktop schreiben
|
||||||
|
|
||||||
|
## Wakeword (OpenWakeWord)
|
||||||
|
|
||||||
|
- Python 3, `openwakeword`, `pyaudio`, `numpy`, `tflite-runtime`
|
||||||
|
- Eingebautes Modell: `hey_jarvis` (kein API-Key, kein Training nötig)
|
||||||
|
- Electron spawnt `python3 python/wakeword.py` als Child-Process
|
||||||
|
- Auto-Restart nach Absturz (2s Delay)
|
||||||
|
- Später: eigenes "hey fox" Modell trainierbar
|
||||||
|
|
||||||
|
## Laravel HUD — Anpassungsstellen
|
||||||
|
|
||||||
|
Diese 3 optionalen Punkte müssen in `fox-hud.blade.php` / HUD-JavaScript hinzugefügt werden:
|
||||||
|
|
||||||
|
1. **ESC schließt HUD:**
|
||||||
|
```javascript
|
||||||
|
document.addEventListener('keydown', (e) => {
|
||||||
|
if (e.key === 'Escape') window.foxBridge?.hideHud();
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Mac-Aktionen in `onAction()`:**
|
||||||
|
```javascript
|
||||||
|
if (payload.action === 'spotify_play') {
|
||||||
|
await window.foxBridge?.macCommand('spotify_play');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// ... weitere Mac-Aktionen
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Nachricht senden (optional, wenn direkt ohne Livewire):**
|
||||||
|
```javascript
|
||||||
|
window.foxBridge?.sendMessage(text);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
- Dev: `npm start` (Electron lädt fox.pxo.at/hud direkt)
|
||||||
|
- LaunchAgent `~/Library/LaunchAgents/at.pxo.fox.plist` für Auto-Start beim Login
|
||||||
|
- `electron-builder` für DMG-Build — später
|
||||||
|
|
||||||
|
## Abhängigkeiten
|
||||||
|
|
||||||
|
**npm:**
|
||||||
|
- `electron` ^28
|
||||||
|
- `axios` ^1.6 (für sendMessage POST)
|
||||||
|
|
||||||
|
**Python:**
|
||||||
|
- `openwakeword`
|
||||||
|
- `pyaudio`
|
||||||
|
- `numpy`
|
||||||
|
- `tflite-runtime`
|
||||||
Loading…
Reference in New Issue