🗣️ Dropped Transcript Monitor

Real-time transcription app with network drop awareness. See exactly which parts of your speech were delivered or missed due to network issues during calls.

Features

Native Offline Transcription - Uses Whisper.cpp (whisper-rs) for fast, accurate speech recognition
Dual Network Monitoring Modes:
- Simulated Mode - For testing without a peer connection
- Real WebRTC Mode - P2P connection with heartbeat monitoring (100ms intervals)
Auto-Connect CLI Peer - Lightweight peer server for one-click connection
Real-time RTT Tracking - Monitor network latency with sub-second precision
Visual Feedback - Color-coded transcript showing delivered (normal) vs missed (red) segments
Cross-Platform - Windows, macOS, Linux via Tauri
Fully Offline - No internet required for transcription
STUN-based NAT Traversal - Connects peers behind firewalls

Tech Stack

Frontend: Solid.js + TypeScript + Vite
Backend: Rust + Tauri + whisper-rs
Transcription: Whisper.cpp (native Rust bindings)
Network Monitoring: WebRTC data channel with heartbeat

Project Structure

dropped-transcript/
├── src/              # Frontend (Solid.js UI)
│   ├── App.tsx       # Main UI with mode toggle & WebRTC setup
│   ├── App.css       # Styling
│   ├── services/     # Audio capture
│   └── stores/       # State management
├── src-tauri/        # Backend (Rust)
│   ├── src/
│   │   ├── lib.rs         # Tauri commands
│   │   ├── whisper.rs     # Transcription engine
│   │   ├── network.rs     # Network monitor (dual mode)
│   │   └── webrtc_peer.rs # WebRTC P2P connection
│   └── models/       # Whisper model files
└── PRD.md           # Product requirements

Quick Start

Prerequisites

Node.js & pnpm
Rust & Cargo
Whisper model file

1. Download Whisper Model

cd src-tauri/models
curl -L -o ggml-tiny-q5_1.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-tiny-q5_1.bin

See MODEL_SETUP.md for more model options.

2. Install Dependencies

pnpm install

3. Run Development Server

pnpm tauri dev

4. Build for Production

pnpm tauri build

The final app will bundle everything (Rust binary, Whisper model, frontend) into a single installable file.

Usage

Basic Setup

Start the app
Choose Network Monitoring Mode:
- Simulated - For quick testing without peer setup
- Real - For actual WebRTC connection monitoring
Enter model path (in dev: full path to src-tauri/models/ggml-tiny-q5_1.bin)
Click "Start Recording"
Speak into your microphone
Watch the transcript appear with color coding:
- Normal text = delivered (network OK)
- Red text = missed (network drop)

WebRTC Connection Setup (Real Mode)

The app automatically connects to a peer server when "Real WebRTC" mode is selected.

Terminal 1 - Start the CLI Peer:

cd src-tauri
cargo run --bin peer

# Or with custom settings:
cargo run --bin peer -- --port 9000
cargo run --bin peer -- --host 0.0.0.0 --port 8080

Terminal 2 - Run the GUI:

pnpm tauri dev

In the GUI:

Select "Real WebRTC" mode
✅ Automatically connects to peer server at http://localhost:8080
Status shows "🟢 Connected" with RTT

Auto-Reconnect:

Shut down the peer server → Status shows "🔴 Disconnected"
Start the peer server again → Automatically reconnects within 3 seconds

Simulated Mode (For Testing)

No peer setup required. The network status will be simulated - you can use the browser console to call:

await invoke("mark_network_connected", { rtt: 50 });
await invoke("mark_network_disconnected");

CLI Peer Server

The CLI Peer Server is a lightweight, barebones WebRTC peer that runs without any GUI, audio capture, or transcription functionality. It serves a single purpose: maintaining a WebRTC connection with heartbeat monitoring so the main GUI app can detect network drops.

Features

✅ WebRTC P2P Connection - Full WebRTC peer with data channel
✅ Heartbeat Monitoring - 100ms heartbeat intervals with RTT tracking
✅ WebSocket Signaling - Real-time bidirectional signaling with auto-reconnect
✅ Auto-Connect - GUI can connect with one click
✅ ICE Restart Support - Automatic reconnection on network changes
✅ Minimal Resources - No audio, no transcription, no GUI (~5-10 MB memory, <1% CPU)
✅ Cross-Platform - Works on Windows, macOS, Linux
❌ No Audio Capture
❌ No Transcription Engine
❌ No Tauri/GUI Components

Starting the Peer Server

# Default (localhost:8080)
cargo run --bin peer

# Custom port
cargo run --bin peer -- --port 9000

# Custom host (expose to network)
cargo run --bin peer -- --host 0.0.0.0 --port 8080

Expected Output:

🚀 Dropped Transcript - WebRTC Peer Server
==========================================

📝 Creating WebRTC offer...
✅ Offer created successfully
🌐 Starting server on http://127.0.0.1:8080

📋 Available endpoints:
  WS   ws://127.0.0.1:8080/ws - WebSocket signaling

⏳ Waiting for clients to connect...
   (Press Ctrl+C to exit)

WebSocket API Reference

WebSocket /ws

Bidirectional WebSocket connection for real-time signaling.

Connection URL:

ws://localhost:8080/ws

Messages from Server:

{
  "type": "offer",
  "data": {
    "sdp": "{\"type\":\"offer\",\"sdp\":\"v=0\\r\\no=...\"}",
    "offer_id": "uuid-string"
  }
}

Messages to Server:

{
  "type": "answer",
  "data": {
    "sdp": "{\"type\":\"answer\",\"sdp\":\"v=0\\r\\no=...\"}",
    "offer_id": "uuid-string"
  }
}

Features:

Auto-reconnect: Server automatically sends new offers when WebSocket reconnects
ICE Restart: Automatic ICE restart when network changes detected
Push-based: Server pushes offers immediately when available

Command-Line Options

USAGE:
    peer [OPTIONS]

OPTIONS:
    -p, --port <PORT>      Port to listen on [default: 8080]
        --host <HOST>      Host to bind to [default: 127.0.0.1]
    -h, --help             Print help
    -V, --version          Print version

Use Cases

Local Testing

Run peer and GUI on the same machine for development/testing:

# Terminal 1
cd src-tauri
cargo run --bin peer

# Terminal 2
pnpm tauri dev
# Select "Real WebRTC" mode - Automatically connects!

Remote Testing

Run peer on a server, connect GUI from different machine:

Option 1: SSH Port Forwarding (Easiest)

# Server (192.168.1.100)
cd src-tauri
cargo run --bin peer -- --host 0.0.0.0 --port 8080

# Client - Use SSH port forwarding:
ssh -L 8080:localhost:8080 user@192.168.1.100
# Then connect GUI normally to localhost:8080 (no .env needed)

Option 2: Direct Connection (Requires .env configuration)

# Server (192.168.1.100)
cd src-tauri
cargo run --bin peer -- --host 0.0.0.0 --port 8080

# Client - Configure peer URL:
# 1. Copy .env.example to .env
cp .env.example .env

# 2. Edit .env and set remote peer URL:
# VITE_PEER_SERVER_URL=http://192.168.1.100:8080

# 3. Run the GUI:
pnpm tauri dev
# OR for production build:
pnpm tauri build

Important Notes:

Development builds: .env file is read at build time (restart pnpm tauri dev after changing)
Production builds: Set VITE_PEER_SERVER_URL before building
Default: If not set, defaults to http://localhost:8080
Security: Add .env to .gitignore (already done) to prevent committing secrets

Headless Server

Run peer on a headless server for continuous monitoring:

# Build release version
cargo build --bin peer --release

# Run with nohup
nohup ./target/release/peer --host 0.0.0.0 &

CI/CD Testing

Use peer server in automated tests:

# Start peer in background
cargo run --bin peer &
PEER_PID=$!

# Run tests
cargo test

# Stop peer
kill $PEER_PID

Peer Architecture

┌─────────────────────────┐
│   CLI Peer Server       │
│                         │
│  ┌──────────────────┐   │
│  │ WebSocket Server │   │◄──── WS /ws
│  │ (Axum)           │   │      (Signaling)
│  │ Port 8080        │   │◄────► Offers/Answers
│  └────────┬─────────┘   │
│           │             │
│  ┌────────▼─────────┐   │
│  │ WebRTC Peer      │   │◄────┐
│  │ - Data Channel   │   │     │ Heartbeat
│  │ - Heartbeat      │   │─────┘ (100ms)
│  │ - RTT Tracking   │   │       over WebRTC
│  │ - ICE Restart    │   │
│  └──────────────────┘   │
│                         │
│  ┌──────────────────┐   │
│  │ Network Monitor  │   │
│  │ - Status         │   │
│  │ - Timeout (500ms)│   │
│  └──────────────────┘   │
└─────────────────────────┘

Technical Details

Dependencies:

axum - Web framework with WebSocket support
tokio - Async runtime
webrtc - WebRTC implementation
clap - Command-line argument parsing
tower-http - CORS middleware
serde/serde_json - JSON serialization
futures - Async stream utilities

Module Reuse: The peer server reuses existing modules from the main app:

network.rs - Network monitoring logic
webrtc_peer.rs - WebRTC peer connection logic

Only the Tauri and Whisper modules are excluded.

Connection Flow:

Startup:
- Peer creates WebRTC offer
- WebSocket server starts on specified port
- Offer is ready for clients
GUI Connection:
- GUI connects to WebSocket /ws
- Server pushes offer via WebSocket
- GUI creates answer using Tauri command
- GUI sends answer back via WebSocket
- Peer sets answer and completes connection
Heartbeat Monitoring:
- Heartbeats sent every 100ms via WebRTC data channel
- RTT calculated from timestamps
- Auto-disconnect after 500ms timeout
Reconnection:
- WebSocket reconnects automatically on network recovery
- Server detects reconnection and triggers ICE restart
- New offer pushed to client via WebSocket
- Connection re-established automatically

CORS Configuration: The peer server allows connections from:

http://localhost:1420 (Vite dev server)
tauri://localhost (Tauri production)
http://tauri.localhost (Tauri dev)
All origins for flexibility

Note: WebSocket connections use a simpler CORS model than HTTP requests.

Troubleshooting

Peer Server Won't Start

Error: Address already in use

Solution: Port 8080 is already taken. Use a different port:

cargo run --bin peer -- --port 9000

GUI Can't Connect

Error: WebSocket connection failed or Failed to connect to peer server

Solutions:

Ensure peer server is running
Check the WebSocket URL in console (should be ws://localhost:8080/ws)
Verify firewall isn't blocking WebSocket connections
Check if the port is accessible:
```
# On macOS/Linux
lsof -i :8080
```

Connection Established But Shows Disconnected

Problem: Connection shows as disconnected immediately after connecting.

Solutions:

Check firewall settings
Verify STUN server is reachable
Check network stability
Try increasing timeout in network.rs:137

RTT Too High

Problem: RTT shows >200ms consistently

Solutions:

Check network quality
Ensure both peer and GUI are on same network
Try different STUN server (edit webrtc_peer.rs)

Peer Crashes on Connection

Error: Panic or crash when GUI connects

Solution: Run with debug logging:

RUST_LOG=debug cargo run --bin peer

Performance & Security

Resource Usage:

Memory: ~5-10 MB
CPU: <1% (idle), ~2-5% (active heartbeat)
Network: ~100 bytes/second (heartbeat only)

Scalability:

Single Connection: Optimized for one GUI connection
Multiple Connections: Not currently supported

Network Exposure:

Default: Binds to 127.0.0.1 (localhost only)
Public: Use --host 0.0.0.0 to expose to network
Risk: WebRTC offers contain network information

Authentication:

Current: No authentication
Recommendation: Use firewall rules or VPN for remote access

Data Privacy:

Heartbeat Only: Only timestamps exchanged
No Encryption: Data channel is unencrypted (P2P nature)
STUN Visibility: Public STUN servers can see your IP

Peer vs Full App Comparison

Feature	CLI Peer	Full GUI App
WebRTC Connection	✅	✅
Heartbeat Monitoring	✅	✅
Network Status	✅	✅
Audio Capture	❌	✅
Transcription	❌	✅
GUI	❌	✅
Auto-Connect Server	✅	❌
Binary Size	~10 MB	~100 MB
Memory Usage	~5-10 MB	~50-200 MB
Startup Time	<1s	~2-5s

Configuration

Environment Variables

The app supports environment variables for runtime configuration:

Variable	Description	Default	Example
`VITE_PEER_SERVER_URL`	WebRTC peer server URL	`http://localhost:8080`	`http://192.168.1.100:8080`

Setup:

# 1. Copy the example file
cp .env.example .env

# 2. Edit .env with your values
nano .env

# 3. Restart dev server (required for changes to take effect)
pnpm tauri dev

For production builds:

# Set environment variable before building
export VITE_PEER_SERVER_URL=http://your-server:8080
pnpm tauri build

Model Selection

Edit tauri.conf.json to bundle different models:

{
  "bundle": {
    "resources": [
      "models/ggml-tiny-q5_1.bin"  // Change this
    ]
  }
}

Performance Tuning

Edit src-tauri/src/whisper.rs:

params.set_n_threads(4);  // Adjust thread count

Architecture

┌─────────────────────────┐
│   Frontend (Solid.js)   │
│   - Audio Capture       │
│   - UI Display          │
│   - Mode Toggle         │
└──────────┬──────────────┘
           │ Tauri Commands
           │ - transcribe_audio_chunk
           │ - set_network_mode
           │ - create_webrtc_offer/answer
           ↓
┌─────────────────────────┐      ┌──────────────────┐
│   Backend (Rust)        │◄────►│  Remote Peer     │
│   - whisper-rs          │      │  (WebRTC)        │
│   - NetworkMonitor      │      └──────────────────┘
│     • Simulated mode    │             ▲
│     • Real WebRTC mode  │             │
│   - WebRTCPeer          │      STUN Server
│     • Data channel      │      (NAT Traversal)
│     • Heartbeat (100ms) │
│     • RTT calculation   │
└──────────┬──────────────┘
           │ Returns: transcript + network status
           ↓
┌─────────────────────────┐
│  UI (Color-coded)       │
│  - Delivered (normal)   │
│  - Missed (red)         │
│  - RTT display          │
└─────────────────────────┘

Development

Type Checking

npx tsc --noEmit

Rust Linting

cargo clippy --manifest-path=src-tauri/Cargo.toml

Format Code

pnpm format  # Frontend
cargo fmt --manifest-path=src-tauri/Cargo.toml  # Backend

Name		Name	Last commit message	Last commit date
Latest commit History 9 Commits
.husky		.husky
public		public
src-tauri		src-tauri
src		src
.env.example		.env.example
.gitignore		.gitignore
MODEL_SETUP.md		MODEL_SETUP.md
PRD.md		PRD.md
README.md		README.md
biome.json		biome.json
index.html		index.html
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.json		tsconfig.json
tsconfig.node.json		tsconfig.node.json
vite.config.ts		vite.config.ts

Folders and files

Latest commit

History

Repository files navigation

🗣️ Dropped Transcript Monitor

Features

Tech Stack

Project Structure

Quick Start

Prerequisites

1. Download Whisper Model

2. Install Dependencies

3. Run Development Server

4. Build for Production

Usage

Basic Setup

WebRTC Connection Setup (Real Mode)

Simulated Mode (For Testing)

CLI Peer Server

Features

Starting the Peer Server

WebSocket API Reference

WebSocket /ws

Command-Line Options

Use Cases

Local Testing

Remote Testing

Headless Server

CI/CD Testing

Peer Architecture

Technical Details

Troubleshooting

Peer Server Won't Start

GUI Can't Connect

Connection Established But Shows Disconnected

RTT Too High

Peer Crashes on Connection

Performance & Security

Peer vs Full App Comparison

Configuration

Environment Variables

Model Selection

Performance Tuning

Architecture

Development

Type Checking

Rust Linting

Format Code

Documentation

License

Recommended IDE Setup

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages