Add per-IP and per-name Rate Limits

[phillip-stephens]

Adds both a --per-ip-ns-rate-limit and --per-name-ns-rate-limit

      --per-ip-ns-rate-limit=      rate limit for queries sent to a specific name server by IP, per second. Use 0 for no rate limit' (default: 0)
      --per-name-ns-rate-limit=    rate limit for queries sent to a specific name server by domain name, per second. Covers both IPv4 and IPv6 addresses the name may resolve to. Use 0 for no rate limit' (default: 0)

Care needs to be taken to not set these rate limits too low, or queries that wouldn't be able to complete due to being rate limited so much they'd exceed one of --timeout, --network-timeout, or --iteration-timeout will return:

{"name":"go.search-start.org","results":{"A":{"data":{"protocol":"","resolver":""},"duration":0.000147374,"error":"cycli
ng lookup failed - rate limit exceeded and out of retries","status":"ERROR","timestamp":"2026-05-21T04:34:16Z"}}}

Reviewer Notes/Caveats

Setting these rate-limits to something meaningful and achieving a benefit in either runtime or accuracy is difficult, at least in --iterative mode. It's the case that a few orgs are responsible for many of the top 100k domain resolutions. These are also the DNS servers that don't seem to meaningfully rate-limit us (and therefore it doesn't really benefit us to rate-limit ourselves). Therefore, you have to set a rate-limit high enough that we don't start erroring out on domains hosted by these large providers, and then the rate is set so high it provides no benefit for small providers that perhaps would do better to have a lower rate of scanning.

TLDR - With at least the top lists, it seems the nameservers responsible for the most domains are the most equipped to handle them, therefore the rate limit doesn't provide much utility that I could determine in this setting.

Because of this, I decided to leave this feature disabled (with a rate set to rate.Infinite) by default and leave it as opt-in for people that have a use case that requires it.

5005 192.35.51.30,f.gtld-servers.net
   4931 192.41.162.30,l.gtld-servers.net
   4925 192.54.112.30,h.gtld-servers.net
   4899 192.43.172.30,i.gtld-servers.net
   4899 192.26.92.30,c.gtld-servers.net
   4875 192.42.93.30,g.gtld-servers.net
   4861 192.48.79.30,j.gtld-servers.net
   4852 192.33.14.30,b.gtld-servers.net
   4850 192.52.178.30,k.gtld-servers.net
   4845 192.55.83.30,m.gtld-servers.net
   4797 192.31.80.30,d.gtld-servers.net
   4785 192.5.6.30,a.gtld-servers.net
   4782 192.12.94.30,e.gtld-servers.net
   2564 162.159.1.33,ns4.cloudflare.com
   2540 162.159.0.33,ns3.cloudflare.com
   2464 162.159.3.11,ns6.cloudflare.com
   2454 162.159.2.9,ns5.cloudflare.com
   2379 162.159.4.8,ns7.cloudflare.com
   2020 162.159.44.18,alberto.ns.cloudflare.com
   1793 108.162.192.209,nelly.ns.cloudflare.com
   1767 172.64.32.209,nelly.ns.cloudflare.com
   1745 172.64.34.252,fish.ns.cloudflare.com
   1725 108.162.195.81,malcolm.ns.cloudflare.com
   1679 64.96.1.1,ns01.trs-dns.com
   1658 64.78.204.1,ns10.trs-dns.info
   1649 64.96.2.1,ns01.trs-dns.net
   1622 64.78.205.1,ns10.trs-dns.org
   1597 108.162.195.18,alberto.ns.cloudflare.com
    907 173.245.58.142,roxy.ns.cloudflare.com
    863 172.64.35.103,roman.ns.cloudflare.com
    838 205.251.193.162,ns-418.awsdns-52.com
    836 205.251.197.26,ns-1306.awsdns-35.org
    829 199.19.57.1,d0.org.afilias-nst.org
    816 205.251.194.154,ns-666.awsdns-19.net
    810 199.249.120.1,b2.org.afilias-nst.org
    806 199.249.112.1,a2.org.afilias-nst.info
    802 205.251.198.61,ns-1597.awsdns-07.co.uk
    773 199.19.56.1,a0.org.afilias-nst.info
    765 213.248.220.1,dns3.nic.uk
    765 199.19.54.1,b0.org.afilias-nst.org
    759 199.19.53.1,c0.org.afilias-nst.info
    756 156.154.100.3,nsa.nic.uk
    756 103.49.80.1,dns2.nic.uk
    753 156.154.101.3,nsb.nic.uk
    749 156.154.102.3,nsc.nic.uk
    734 156.154.103.3,nsd.nic.uk
    730 43.230.48.1,dns4.nic.uk
    725 213.248.216.1,dns1.nic.uk
    645 150.100.6.8,f.dns.jp
    616 202.12.30.131,b.dns.jp
    614 203.119.40.1,g.dns.jp

Related Issues

Closes #594