Sanitize

Sanitize — clean IP lists

Since v0.1.5

The sanitize command cleans and sorts your input IP list (dnsir.txt) in one step.
Replaces the old Python scripts (remove_ips.py, sort_ip_*.py).

What it does

1. Remove blank lines and comments (# / //)
2. Remove invalid format (not IP:port or IP)
3. Deduplicate — by IP:port (default) or by IP only (--dedup-by-ip)
4. Numeric sort — IP octets then port (not alphabetical)
5. Exclude blacklisted IPs (--exclude)

Usage

dns-cli sanitize <INPUT> [OPTIONS]

Examples

# Clean + sort + overwrite original file
dns-cli sanitize dnsir.txt --inplace

# Clean + sort → separate output file (default: <input>.clean.txt)
dns-cli sanitize dnsir.txt

# Stats only (no file written)
dns-cli sanitize dnsir.txt --dry-run

# Dedup by IP only (ignore port)
dns-cli sanitize dnsir.txt --inplace --dedup-by-ip

# Exclude blacklisted IPs
dns-cli sanitize dnsir.txt --inplace --exclude bad_ips.txt

Options

Flag	Description
<INPUT>	Input file (required)
--output	Output file (default: <input>.clean.txt)
--inplace	Overwrite the input file in place
--dedup-by-ip	Dedup by IP only (ignore port)
--exclude	File with IPs to remove (one per line)
--dry-run	Print stats only, don't write file

Sample output

📊 sanitize results:
   raw lines       : 455120
   blank removed   : 0
   comments removed: 0
   invalid removed : 0
   excluded removed: 0
   before dedup    : 455120
   duplicates      : 334704
   final unique    : 120416
   output          : dnsir.txt

vs Python scripts

Feature	remove_ips.py / sort_ip.py	dns-cli sanitize
Dedup	Basic	IP:port or IP-only
Sort	Alphabetical or incomplete	Numeric (real IPv4 octets)
Invalid lines	No	Yes, removes them
Comments	No	Yes (# / //)
Exclude list	No	Yes (--exclude)
In-place	No	Yes (--inplace)
Speed	Slow (Python)	Fast (Rust)

Suggested workflow

# 1. Clean
dns-cli sanitize dnsir.txt --inplace

# 2. Scan
dns-cli pipeline run --input dnsir.txt --profile mytunnel --preset low