-
-
Notifications
You must be signed in to change notification settings - Fork 5
Home
🚀 MailGrab v2 is here — more firepower packed: concurrent crawling, an MCP server for AI agents (Claude, Copilot, Cursor, Codex), robots.txt-aware crawling, email de-obfuscation, MX validation, resume/append modes, and a lot more. See the release notes for the full rundown.
MailGrab is a concurrent, robots.txt-aware email harvester: give it a URL (or a file full of them), it crawls the site's links, pulls out email addresses (including ones hidden behind obfuscation tricks), and saves everything in plain text, CSV, and JSON.
This wiki is the deep-dive reference. The README covers installation and a quick start; here you'll find every feature explained, how to use it, and what's actually happening behind the scenes.
Getting oriented
- Installation — getting MailGrab running on Windows, Linux, or macOS
- Quick Start — your first crawl, interactive and batch mode
- MCP Server — use MailGrab as a tool from Claude Code, Claude Desktop, Cursor, Copilot, or Codex CLI
Reference
- CLI Reference — every flag, what it does, its default
-
Environment Variables — the
MAILGRAB_*variables and how they relate to the flags -
Configuration File —
--config, for saving a setup you reuse -
Output Files —
_emails.txt,_results.json, and friends, with exact formats
Features, explained
- Performance & Concurrency — concurrent fetching, connection reuse, timeouts, rate limiting
-
Crawl Correctness — same-domain scoping, URL normalization,
mailto:parsing, de-obfuscation,robots.txt -
Smarter Discovery — sitemap seeding,
Crawl-delay, hop limits, per-domain rate limiting - Email Quality — case folding, placeholder filtering, MX validation
- Robustness & Networking — User-Agent, retries, proxies, and a subtle timeout-logging bug we had to fix twice
-
Ops & Resilience —
--append,--resume,--quiet
Under the hood
- Architecture — how the crawl engine is actually built, the concurrency model, and the bugs adversarial testing found along the way
- FAQ & Troubleshooting — common questions and known limitations
# Interactive
python MailGrab.py
# Fully scripted, respecting the seed's own domain, with a custom User-Agent
python MailGrab.py --url https://example.com --depth 30 --same-domain --user-agent "MyBot/1.0"
# Resume a previous run, keeping what it already found
python MailGrab.py --url https://example.com --depth 30 --resumeEvery crawl writes _emails.txt, _scrappedUrls.txt, _emails.csv, _results.json, and _MailGrabLog.txt to the current directory. See Output Files for what's in each one.
Prefer to let an AI agent drive it instead? See MCP Server — python mailgrab_mcp_server.py exposes the same crawl as a tool call for Claude Code, Claude Desktop, Cursor, Copilot, and Codex CLI.