02 Architecture and Core Engines

Architecture & Core Engines

This document outlines the technical architecture, data flows, and low-level engineering systems powering WishPilot.

---

1. High-Level Process Model

WishPilot is engineered on Electron 44, using a secure multi-process model separating node-level OS operations from the user interface:

┌────────────────────────────────────────────────────────────────────────┐
│                        ELECTRON MAIN PROCESS                           │
│                      (electron/main.cjs)                               │
│                                                                        │
│  - Window Management (Studio, HUD, Stealth Notch modes)                │
│  - Global Keyboard Shortcut Listeners (Register / Unregister)          │
│  - Windows Native Display Affinity (SetWindowDisplayAffinity)          │
│  - Screen Capture Engine (desktopCapturer thumbnail stream)            │
│  - Process List Scanner (tasklist platform inspection)                 │
└──────────────────┬──────────────────────────────────┬──────────────────┘
                   │                                  │
      IPC Send /   │                                  │  IPC Handlers &
      Event Bridge │                                  │  Safe API Calls
                   ▼                                  ▼
┌────────────────────────────────────────────────────────────────────────┐
│                        ELECTRON PRELOAD BRIDGE                         │
│                     (electron/preload.cjs)                             │
│                                                                        │
│  - contextBridge.exposeInMainWorld('wishpilot', { ... })               │
│  - Context Isolation: True | Node Integration: False                   │
└──────────────────┬──────────────────────────────────┬──────────────────┘
                   │                                  │
                   ▼                                  ▼
┌────────────────────────────────────────────────────────────────────────┐
│                       REACT 19 RENDERER PROCESS                        │
│                         (src/App.jsx)                                  │
│                                                                        │
│  ┌─────────────────────────┐      ┌─────────────────────────────────┐  │
│  │ Web Audio DSP Engine    │      │ Unified AI Streaming Client     │  │
│  │ (audioTranscriber.js)   │      │ (aiService.js)                  │  │
│  └───────────┬─────────────┘      └────────────────┬────────────────┘  │
│              │                                     │                   │
│              ▼                                     ▼                   │
│  ┌─────────────────────────┐      ┌─────────────────────────────────┐  │
│  │ Groq Whisper STT API    │      │ 9 AI Inference Endpoints        │  │
│  │ (Direct HTTPS)          │      │ (Direct HTTPS / BYOK)           │  │
│  └─────────────────────────┘      └─────────────────────────────────┘  │
└────────────────────────────────────────────────────────────────────────┘

---

2. Multi-Mode Window Architecture

WishPilot operates as a single dynamic window that transitions seamlessly between three operational modes without destroying state:

1. Studio Dashboard Mode (960 × 650 px)

• The main configuration, testing, and debriefing cockpit.
• Provides access to all 6 tabs: Sessions & Roles, Models & Audio, Live Test Lab, Mock Simulator, Debrief & Export, Resume & JD, and Display & Environment.
• Non-draggable content area with intuitive navigation.

2. Floating Stealth HUD Mode (Resizable, Default 460 × 380 px)

• Minimalist, always-on-top translucent heads-up display.
• Drag handle across the header bar (-webkit-app-region: drag).
• Renders streaming answers, 10-second TL;DR punchlines, and instant refinement pills.
• Adjustable opacity slider (from 12% to 100%) and optional click-through mode (SetIgnoreMouseEvents).

3. Stealth Notch Mode (230 × 34 px)

• Compact pill docked at the top center of the screen.
• Displays a real-time mini voice waveform bar, microphone status, and a single-click button to trigger an answer or expand back to full mode.

---

3. Web Audio DSP & Speech-to-Text Pipeline

The speech intelligence engine in src/services/audioTranscriber.js converts raw microphone signals into structured text with ultra-low latency:

[Microphone Hardware] 
         │ 48kHz / 44.1kHz Stereo Float32 Stream
         ▼
[Web Audio API Context] 
         │ AudioNode Graph
         ▼
[AudioWorklet / ScriptProcessorNode]
         │ Real-time chunk capture (4096 samples)
         ▼
[Voice Activity Detection (VAD) & Energy Gate]
         │ Energy Threshold > 0.008 RMS
         ▼
[Linear Interpolation Resampler]
         │ Downsample to 16,000 Hz Mono Float32
         ▼
[Binary RIFF WAV Encoder]
         │ Construct 44-byte standard RIFF header + 16-bit PCM buffer
         ▼
[Multipart Form-Data Dispatch]
         │ Direct HTTPS POST to Groq Whisper Endpoint
         ▼
[Groq Whisper Large v3 Turbo]
         │ ~200ms Processing Latency
         ▼
[Real-Time Spoken Transcript]

Downsampling Algorithm

Microphones typically record at 44,100 Hz or 48,000 Hz. WishPilot downsamples in-memory to 16,000 Hz (the optimal sample rate for Whisper) using weighted accumulator averages, avoiding high CPU overhead:

function downsampleTo16k(inputSamples, inputSampleRate, outputSampleRate = 16000) {
  if (inputSampleRate === outputSampleRate) return inputSamples;
  const ratio = inputSampleRate / outputSampleRate;
  const newLength = Math.round(inputSamples.length / ratio);
  const result = new Float32Array(newLength);
  let offsetResult = 0;
  let offsetBuffer = 0;

  while (offsetResult < result.length) {
    const nextOffsetBuffer = Math.round((offsetResult + 1) * ratio);
    let accum = 0;
    let count = 0;
    for (let i = offsetBuffer; i < nextOffsetBuffer && i < inputSamples.length; i++) {
      accum += inputSamples[i];
      count++;
    }
    result[offsetResult] = count > 0 ? accum / count : (inputSamples[offsetBuffer] || 0);
    offsetResult++;
    offsetBuffer = nextOffsetBuffer;
  }
  return result;
}

---

4. Windows Display Affinity & Window Protection

When presenting your screen during a technical interview or team review, WishPilot utilizes native Windows API display protection:

SetWindowDisplayAffinity(hwnd, WDA_EXCLUDEFROMCAPTURE);

In Electron, this is implemented via:

mainWindow.setContentProtection(true);

How it Works with Windows Desktop Window Manager (DWM):

• The window continues to render normally to your primary physical monitor.
• When any process attempts to invoke the Windows Graphics Capture API (DirectX, BitBlt, Windows Media Capture, Zoom/Teams screen sharing, or Discord streaming), the DWM treats the WishPilot window bounds as transparent.
• As a result, your practice notes, HUD overlay, and transcripts remain exclusively visible to your physical eyes without ever bleeding onto the shared presentation stream.

---

5. Security & BYOK (Bring Your Own Key) Architecture

WishPilot enforces strict local-first security:

1. Zero External Middlemen: There is no proxy server, authentication backend, or logging cluster between WishPilot and the LLM APIs.
2. Key Storage: API keys are stored in window.localStorage under the key wishpilot_api_keys. They never leave your local machine except as Authorization: Bearer <KEY> headers sent directly to api.groq.com, api.openai.com, etc.
3. Screen Capture Memory Buffer: When Ctrl + Shift + S is triggered, the screenshot thumbnail is processed entirely in memory as a base64 Data URL, passed to the vision LLM prompt, and immediately garbage-collected upon answer completion.