Your data is exposed. Your identity is mapped. Take it back.
Features • Architecture • Tech Stack • Getting Started • API Docs • Screenshots
DataReaper is a full-stack autonomous AI platform that hunts down your personal data across the web and forces its deletion — without you lifting a finger.
Data brokers silently collect, package, and sell your personal information. DataReaper fights back by combining advanced OSINT reconnaissance, AI-powered identity resolution, and automated multi-jurisdiction legal compliance to systematically eliminate your digital footprint.
Seed Input → OSINT Discovery → Identity Graph → Broker Detection → Legal Dispatch → Autonomous Follow-up
🔍 Intelligent OSINT Discovery
- Multi-platform account discovery seeded from email, phone, or username
- Username enumeration via Maigret across 3,000+ sites
- Anti-detection profile scraping with Playwright headless browser
- Web content extraction via Trafilatura and BeautifulSoup4
- Configurable probe depth: platform candidates, Maigret top-sites, max connections
- DuckDuckGo fallback, paste-site search, and search probe layers (feature-flagged)
🧠 AI-Powered Identity Resolution
- LLM-driven cross-platform data correlation (Groq / llama-3.3-70b-versatile)
- Force-directed interactive identity graph with node-edge visualization
- Nodes: seeds, discovered accounts, usernames, aliases, resolved attributes
- Edges:
pivoted_to,discovered_username,found_on_broker, and more - Real-time graph updates as new data surfaces during scanning
🎯 Broker Detection & Verification
- Automated scanning against a catalog of 100+ data brokers
- Confidence-scored listing verification per broker
- Opt-out rule engine with broker-specific workflows (email / form / phone)
- Contact point discovery and validation
- YAML-driven broker catalog — easily extensible
⚖️ Legal Automation Engine
- Multi-jurisdiction compliance: GDPR, CCPA, DPDP Act (India)
- Automated legal notice generation via AI Legal Agent
- Escalation workflows for non-compliant brokers
- Full audit trail and compliance tracking
- Configurable default jurisdiction per deployment
📧 Autonomous Email Warfare
- Gmail OAuth 2.0 integration for sending and receiving
- AI intent classification for incoming broker emails
- Context-aware reply generation with objection handling
- Thread continuity and conversation memory
- Attachment handling for ID verification requests
- Periodic inbox sync (every 5 minutes) via background scheduler
🛡️ Tripwire Chrome Extension
- Downloadable Chrome extension (
datareaper-tripwire.zip) served directly from the API - Real-time malicious site detection and threat logging
- Password field interception monitoring (block / allow tracking)
- Heartbeat-based session linking via short-lived Redis tokens
- Shield Logs dashboard: per-hostname threat events and password attempt history
👻 Shadow Browser
- Decoy persona engine — AI-generated fake identities browse in the background
- Randomizable personas with age, occupation, and interests
- Decoy session simulation to pollute data broker profiles
- Per-persona browsing history viewer with search and date grouping
- Toggle on/off from the dashboard; communicates with the Tripwire extension via
postMessage
🪞 Access Mirror
- Google OAuth 2.0 connect flow with PKCE
- Live scope analysis: maps granted scopes to risk levels (LOW / MEDIUM / HIGH)
- Per-app grant revocation with audit log
- Data export parser: upload Google Takeout (and Instagram, LinkedIn, Amazon, Spotify, Uber) archives up to 200 MB
- Extracts authorized OAuth apps from Google Takeout exports
- Persists reports to PostgreSQL with in-memory fallback
📊 Real-Time Command Center
- Exposure metrics dashboard with live scan progress
- Activity timeline with chronological event feed
- Threat level assessment and prioritization
- WebSocket-powered live updates
- TanStack Query for optimistic UI and background refetching
🎖️ Privacy War Room
- Centralized broker case management
- One-click deletion request dispatch
- Email thread viewer with AI-generated legal notices
- Escalation management for non-compliant brokers
- Batch operations across multiple broker cases
- Compliance deadline tracking and response time analytics
┌──────────────────────────────────────────────────────────────────────┐
│ Frontend (React 18 + Vite) │
│ │
│ Landing │ Onboarding │ Command Center │ Identity Graph │
│ War Room │ Inbox │ Shield Logs │ Shadow Browser │
│ Access Mirror │ Google Auth Callback │
│ │
│ TanStack Query · React Router 7 · Radix UI · Motion │
└────────────────────────────┬─────────────────────────────────────────┘
│ REST API + WebSocket
┌────────────────────────────▼─────────────────────────────────────────┐
│ Backend (FastAPI 0.116+) │
│ │
│ /api/onboarding /api/scans /api/dashboard /api/recon │
│ /api/targets /api/war-room /api/inbox /api/reports │
│ /api/events /api/shield /api/access-mirror │
│ /v1/content /ws/* │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Agent Orchestration │ │
│ │ Sleuth Agent · Legal Agent · Communications Agent │ │
│ │ Prompt Manager · Agent Registry · Base Agent │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Core Services │ │
│ │ OSINT Engine · Broker Discovery · Email Sync │ │
│ │ Legal Compliance · Identity Resolution · Scraper │ │
│ │ Access Mirror Parser · Report Builder │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────┐ │
│ │ Background Workers (ARQ + Redis) │ │
│ │ run_osint_pipeline · discover_targets │ │
│ │ send_legal_requests · sync_inbox │ │
│ │ continue_battles · build_report_snapshot (cron) │ │
│ │ cleanup_old_events (cron) · sync_active_scan_inboxes │ │
│ └─────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────┘
│
┌──────────────────┼──────────────────┐
▼ ▼ ▼
PostgreSQL Redis Playwright
(SQLAlchemy) (ARQ Queue) (Headless Browser)
1. Validate Seed → email / phone / username
2. Discover Accounts → Maigret + platform probes
3. Extract Usernames → cross-platform pivoting
4. Scrape Profiles → Playwright + Trafilatura
5. Resolve Identity → LLM correlation
6. Discover Targets → broker catalog matching
7. Generate Notices → Legal Agent (GDPR/CCPA/DPDP)
8. Send Requests → Gmail OAuth dispatch
9. Monitor Responses → inbox sync + AI triage
10. Escalate / Follow-up → Communications Agent
| Technology | Version | Purpose | |
|---|---|---|---|
| ⚛️ | React | 18.3.1 | UI framework |
| 🔷 | TypeScript | 5.0+ | Type safety |
| ⚡ | Vite | 6.4+ | Build tool |
| 🔄 | TanStack Query | 5.99+ | Server state |
| 🛣️ | React Router | 7.13+ | Routing |
| 🎨 | Radix UI | Latest | Accessible primitives (30+ components) |
| 💨 | Tailwind CSS | 4.1+ | Utility-first styling |
| 🎬 | Motion | 12.23+ | Animations |
| 📊 | Recharts | 2.15+ | Data visualization |
| 🌐 | Axios | 1.15+ | HTTP client |
| 🔔 | Sonner | 2.0+ | Toast notifications |
| 📋 | React Hook Form | 7.55+ | Form management |
| 🖱️ | React DnD | 16.0.1 | Drag and drop |
| Technology | Version | Purpose | |
|---|---|---|---|
| 🐍 | Python | 3.11+ | Core language |
| 🚀 | FastAPI | 0.116+ | Web framework |
| 🗄️ | SQLAlchemy | 2.0+ | Async ORM |
| ✅ | Pydantic | 2.11+ | Data validation |
| 🔄 | Alembic | 1.14+ | DB migrations |
| 🌐 | Uvicorn | 0.34+ | ASGI server |
| 🐘 | asyncpg | 0.30+ | Async PostgreSQL |
| 📬 | ARQ | 0.26+ | Async task queue |
| ⏰ | APScheduler | 3.11+ | Job scheduling |
| 📝 | Structlog | 25.3+ | Structured logging |
| Technology | Purpose | |
|---|---|---|
| 🤖 | Groq (llama-3.3-70b) | LLM inference for agents |
| 🎭 | Playwright | Anti-detection browser automation |
| 🔍 | Maigret | Username OSINT (3,000+ sites) |
| 🌿 | BeautifulSoup4 | HTML parsing |
| 📄 | Trafilatura | Web content extraction |
| 🔒 | curl-cffi | Anti-detection HTTP client |
| Technology | Purpose | |
|---|---|---|
| 🐘 | PostgreSQL | Primary database |
| 🔴 | Redis | Cache + task queue backend |
| 📧 | Gmail API | OAuth email send/receive |
| 🔑 | Google OAuth 2.0 | User auth + Access Mirror |
| 🐳 | Docker | Containerization |
7 Alembic migrations covering 20+ tables:
users scan_jobs seeds
discovered_accounts graph_nodes graph_edges
brokers broker_listings broker_cases
email_threads email_messages attachments
legal_requests audit_logs consent
activity_events report_snapshots agent_runs
scan_stages access_mirror_reports google_oauth_connections
- Python 3.11+
- Node.js 18+ and pnpm
- PostgreSQL database
- Redis server
- Groq API key
- Gmail API credentials (for email features)
# 1. Clone and enter backend
git clone https://github.com/yourusername/datareaper.git
cd datareaper/backend
# 2. Install dependencies (uv recommended)
pip install uv
uv sync
# 3. Configure environment
cp .env.example .envEdit .env:
# Database
DATABASE_URL=postgresql+asyncpg://user:pass@localhost/datareaper
SYNC_DATABASE_URL=postgresql+psycopg://user:pass@localhost/datareaper
# Redis
REDIS_URL=redis://127.0.0.1:6379/0
# AI
GROQ_API_KEY=your_groq_api_key
GROQ_MODEL=llama-3.3-70b-versatile
# Google Sign-In + Access Mirror
GOOGLE_CLIENT_ID=your_client_id.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET=your_client_secret
# Gmail API (inbox/send features)
GMAIL_CLIENT_ID=your_gmail_client_id
GMAIL_CLIENT_SECRET=your_gmail_client_secret
GMAIL_SENDER_EMAIL=your_sender@gmail.com
GMAIL_SENDER_CLIENT_ID=your_sender_client_id
GMAIL_SENDER_CLIENT_SECRET=your_sender_client_secret
GMAIL_SENDER_REFRESH_TOKEN=your_sender_refresh_token
# App
APP_DEBUG=true
FRONTEND_URL=http://localhost:5173
DEFAULT_JURISDICTION=DPDP# 4. Run migrations
alembic upgrade head
# 5. Seed data (optional)
python scripts/import_broker_catalog.py
python scripts/import_platform_catalog.py
python scripts/seed_demo_data.py
# 6. Start API + worker (Windows)
.\scripts\start_stack.ps1
# Or manually
uvicorn datareaper.main:app --reload --app-dir src --port 8000
arq datareaper.workers.scheduler.WorkerSettingsAPI available at http://localhost:8000
cd frontend
pnpm install
pnpm devFrontend available at http://localhost:5173
cd backend
python scripts/seed_demo_data.pyPopulates sample scans, identity graph, broker cases, and email threads for exploration.
With the backend running:
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
- Health: http://localhost:8000/api/health
POST /api/onboarding/start Start a new privacy scan
GET /api/scans/{scan_id} Scan status and results
GET /api/dashboard/overview Exposure metrics
GET /api/recon/graph/{scan_id} Identity graph data
GET /api/targets/{scan_id} Discovered broker targets
GET /api/war-room/cases All broker cases
POST /api/war-room/cases/{id}/dispatch Send deletion request
GET /api/inbox/threads Email threads
POST /api/inbox/sync Sync Gmail inbox
GET /api/shield/status Tripwire extension status
POST /api/shield/token Issue shield token
GET /api/shield/download Download Tripwire extension
GET /api/access-mirror/google/config Google OAuth config
POST /api/access-mirror/google/connect Connect Google account
GET /api/access-mirror/google/grants View OAuth grants
POST /api/access-mirror/google/revoke Revoke app access
POST /api/access-mirror/parse Parse data export archive
GET /api/reports/{scan_id} Privacy report
WS /ws/scans/{scan_id} Real-time scan events
Hero section, problem statement, three-pillar feature showcase (Scan → Identify → Terminate), process flow visualization, and CTA.
Seed input (email / phone / username), jurisdiction selection (GDPR / CCPA / DPDP), privacy preferences, and scan initialization.
Live exposure metrics, active scan monitoring, threat assessment, quick actions, and chronological activity timeline.
Interactive force-directed graph showing seeds, discovered accounts, usernames, aliases, resolved attributes, and broker exposures — with real-time updates.
Broker target inventory, deletion campaign status, AI-generated legal notice viewer, escalation management, and batch operations.
Decoy persona engine running in the background — AI-generated fake identities browse the web so data brokers see someone else. Shows the currently active persona (Mabel Thornton, 82 · Bridge Player), their interests, a randomize button, the full decoy visit history with timestamps and favicons, simulated account sessions, and a searchable per-persona history panel.
Your data footprint, laid bare. The Google Hub connects your Google account and surfaces every OAuth grant with risk levels (HIGH / LOW) and a per-app Revoke button. The Universal Data Drop accepts Takeout exports from Google, Instagram, LinkedIn, Amazon, Spotify, Uber, and more — DataReaper parses the archive and shows exactly what each platform has built on you.
Tripwire threat log pulled live from the Chrome extension. Shows all malicious hostnames detected during browsing — select one to see Tripwire / malicious URL event counts, password interception stats (blocked vs allowed), the full malicious URL log with timestamps, and a per-attempt password field breakdown.
datareaper/
├── backend/
│ ├── src/datareaper/
│ │ ├── agents/ # Sleuth, Legal, Communications agents
│ │ ├── api/ # FastAPI routes (15 route modules)
│ │ ├── brokers/ # Broker catalog, discovery, opt-out rules
│ │ ├── comms/ # Gmail client, OAuth, intent classifier
│ │ ├── compliance/ # GDPR / CCPA / DPDP legal engine
│ │ ├── core/ # Config, logging, constants, IDs
│ │ ├── db/ # Models, repositories, session
│ │ ├── identity/ # Identity resolution
│ │ ├── integrations/ # Groq LLM, Playwright browser
│ │ ├── osint/ # Discovery pipeline
│ │ ├── scraper/ # Web scraping orchestration
│ │ ├── services/ # Business logic layer
│ │ └── workers/ # ARQ jobs + scheduler
│ ├── data/
│ │ ├── brokers/ # broker_catalog.yaml, opt_out_rules.yaml
│ │ ├── legal/ # gdpr_rules.yaml, ccpa_rules.yaml, dpdp_rules.yaml
│ │ ├── platforms/ # platform_selectors.yaml, probe catalogs
│ │ └── prompts/ # LLM prompt templates (10 prompts)
│ ├── migrations/ # 7 Alembic migrations
│ └── scripts/ # Import, seed, smoke test, replay utilities
├── frontend/
│ ├── src/
│ │ ├── components/ # 18 components including AnimatedDataReaperLogo
│ │ ├── pages/ # 9 pages
│ │ ├── lib/ # API client, hooks, WebSocket, session manager
│ │ └── styles/ # Tailwind, theme, fonts, cursor
│ └── public/
└── .codex_tmp_accessmirror/ # Access Mirror component staging blocks
- Google OAuth 2.0 for user authentication and Gmail access
- JWT-based session management with realtime tokens for WebSocket auth
- CORS configured for frontend origin + Chrome extension ID pattern
- Pydantic v2 input validation on all API endpoints
- SQLAlchemy ORM for SQL injection protection
- Audit trail for all compliance operations
- Consent tracking per user
- No third-party analytics — your data stays in your database
- Responsible use: scan only your own personal information
# Backend
cd backend
pytest
pytest --cov=datareaper --cov-report=html
# Frontend
cd frontend
pnpm test- Fork the repository
- Create a feature branch:
git checkout -b feature/your-feature - Commit:
git commit -m 'Add your feature' - Push:
git push origin feature/your-feature - Open a Pull Request
Before committing:
# Backend
ruff check .
mypy src/
# Frontend
pnpm lintMIT License — see LICENSE for details.
Maigret · Playwright · FastAPI · React · Radix UI · Tailwind CSS · Groq