[Bug] crt.sh search enumeration fails on read timeout

[exigopro]

Bug
Bug

Describe the feature request or bug or other
When running a crt.sh search enumeration, i keep getting a read operation timeout. It seems to only have started being an issue today since I ran multiple times in the past and had no issues then. I think it may be due to the size of the scan being performed, scanning a domain with small amounts of records to check does not give me this read error.

Error:

[*] crt: Performing Crt.sh Search Enumeration against REDACTED.XYZ...
Traceback (most recent call last):
  File "/usr/local/bin/dnsrecon", line 4, in <module>
    __import__('pkg_resources').run_script('dnsrecon==1.0.0', 'dnsrecon')
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 658, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1445, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/local/lib/python3.6/dist-packages/dnsrecon-1.0.0-py3.6.egg/EGG-INFO/scripts/dnsrecon", line 1780, in <module>
  File "/usr/local/lib/python3.6/dist-packages/dnsrecon-1.0.0-py3.6.egg/EGG-INFO/scripts/dnsrecon", line 1718, in main
  File "/usr/local/lib/python3.6/dist-packages/dnsrecon-1.0.0-py3.6.egg/lib/crtenum.py", line 37, in scrape_crtsh
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1368, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.6/urllib/request.py", line 1328, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 281, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.6/ssl.py", line 874, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib/python3.6/ssl.py", line 631, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

System Information (System that tool is running on):

• OS: ubuntu, 18.04.6 LTS (Bionic Beaver)

[exigopro]

crt.sh is doing rate limiting:
The crt.sh web service has been struggling to cope with peak traffic levels for some time. Whenever this happens, many users see HTTP 50x responses. To ensure that everyone has a fair chance of using the service, today we have added some rate limiting.

Requests to the crt.sh web service are now being throttled at 60 requests per IP address per minute. Is there a way to throttle requests with dnsrecon?

[L1ghtn1ng]

@exigopro That would be a no