CLI Reference

CLI Reference

Run python MailGrab.py --help any time to see this list from the source of truth. Every flag has a matching environment variable — see Environment Variables for the mapping and precedence rules (short version: CLI flag > environment variable > built-in default, and --config just pre-fills environment variables, so it sits below both).

Target & depth

Flag	Default	What it does
--url <url>	(prompt)	URL to scan. Skips the interactive URL prompt. http:// is prepended if you don't include a scheme. Always wins over an existing _inputUrls.txt — if you pass --url, batch mode is skipped even if a seed file is sitting in the directory.
--depth <n>	(prompt)	Maximum total pages to fetch — not link-hops from the seed (that's --max-hops). Interactive mode allows 1–500; batch mode (via _inputUrls.txt/--input) allows 1–200. Out-of-range values exit with an error rather than looping forever.
--input <file>	_inputUrls.txt	Path to a file of seed URLs (one per line) for batch mode.

--url and --depth

Prompts exist so MailGrab can be used casually with zero setup — but a script or CI job can't answer a prompt. So:

• Give both --url and --depth (or --depth alone with a ready seed file) → MailGrab runs fully non-interactively, including skipping the final "Press Any Key To Exit" prompt.
• Give only one with no way to satisfy the other (e.g. --depth with no --url and no seed file) → MailGrab exits immediately with a clear error instead of silently falling through to a prompt that would hang forever with no terminal attached.

Performance

Flag	Default	What it does
--concurrency <n>	10	Max pages fetched at once, per BFS round. See Performance & Concurrency.
--delay <seconds>	0	Minimum time between requests to the same domain (not a global pause — see Smarter Discovery). robots.txt's own Crawl-delay, if larger, always wins.
--timeout <seconds>	10	Per-request timeout, so one hung server can't stall the whole crawl.

Discovery & filtering

Flag	Default	What it does
--same-domain	off	Only follow links whose domain matches the seed's. See Crawl Correctness.
--max-hops <n>	unlimited	Cap link-distance from the seed, independent of --depth's total-page cap. --max-hops 0 means "only the seed itself." See Smarter Discovery.
--use-sitemap	off	Also seed the crawl from sitemap.xml (handles sitemap-index files too). Opt-in: adds 1–2 extra requests every run. See Smarter Discovery.
--ignore-robots	off	Don't check robots.txt before fetching a URL.
--verify-mx	off	Drop emails whose domain has no MX record. Opt-in: adds a DNS lookup per unique domain and needs DNS access. See Email Quality.

Output

Flag	Default	What it does
--append	off	Merge with _results.json from a previous run instead of overwriting it. See Ops & Resilience.
--resume	off	Like --append, but also skips re-fetching URLs already visited in a prior run (loaded before crawling starts, not just merged afterward). See Ops & Resilience.
--quiet	off	Suppress all normal console/banner output; print one JSON summary line at the end instead. See Ops & Resilience.

Networking

Flag	Default	What it does
--user-agent <string>	Mozilla/5.0 (compatible; MailGrab/2.0; ...)	Custom User-Agent header for every request.
--proxy <url>	none	Route requests through a proxy. Accepts http://, https://, socks4://, socks4a://, socks5://, socks5h:// (SOCKS support comes from the PySocks dependency). Validated at startup — a malformed value fails immediately with a clear message, not silently per-request. See Robustness & Networking.

Configuration

Flag	Default	What it does
--config <file>	none	JSON file that pre-fills the MAILGRAB_* environment variables for any settings you don't pass as flags or set as real environment variables. See Configuration File.

Examples

Fully headless crawl, staying on-domain, polite about it:

python MailGrab.py --url https://example.com --depth 50 --same-domain --delay 1

Batch mode from a custom file, quiet output for piping into another tool:

python MailGrab.py --input my-seeds.txt --depth 100 --quiet

Behind a Tor SOCKS proxy, verifying deliverability:

python MailGrab.py --url https://example.com --depth 20 --proxy socks5h://127.0.0.1:9050 --verify-mx