Voice Fraud Detection Demo

Real-time AI voice and scam detection for phone calls. Built for Varsity Hack 2026.

What It Does

Two layers of protection:

1. AI Voice Detection — Detects deepfakes and TTS using spectral flatness analysis with logistic regression. Runs every 2 seconds.

2. Content Fraud Detection — Transcribes speech with ElevenLabs, then GPT-4 analyzes for scam patterns (urgency, impersonation, suspicious payment requests). Runs every 10 seconds.

How It Works

Phone 1 (caller) streams audio to the server. Phone 2 (listener) receives the audio and sees real-time AI/Human labels. The laptop dashboard shows detailed classification breakdowns.

All devices connect over WebSocket on the same WiFi network.

Setup

Install dependencies: pip install aiohttp librosa numpy scipy scikit-learn python-dotenv elevenlabs openai joblib

Add your API keys to .env (ELEVEN_LABS_API_KEY and OPENAI_API_KEY).

Generate SSL certs: openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes

Update LAPTOP_IP in demo_server.py to your WiFi IP (find with ipconfig getifaddr en0).

Run: python3 demo_server.py

Endpoints

• /stream — Caller streams microphone
• /listen — Listener hears audio + sees labels
• /dashboard — Detailed breakdown

Key Files

• demo_server.py — Main WebSocket server
• detection_model.py — AI voice classifier (spectral flatness + logistic regression)
• fraud_detector.py — ElevenLabs STT + GPT-4 scam analysis
• ai_voice_detector.joblib — Trained model weights