Failed: Internal Error Encountered: Unable to open option URL

[JArmandoG]

Please provide a description of the issue and any relevant error messages.

After I try to run a new scan, I get the error "failed: Internal Error Encountered: Unable to open option URL"

If you can provide an application stack trace that is even better.

The full error I get:

Unhandled exception (BaseException) encountered during scan. Please report this as a bug: ['Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/socks.py", line 787, in connect\n super(socksocket, self).connect(proxy_addr)\n', 'ConnectionRefusedError: [Errno 111] Connection refused\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/urllib3/contrib/socks.py", line 96, in _new_conn\n conn = socks.create_connection(\n', ' File "/usr/lib/python3/dist-packages/socks.py", line 209, in create_connection\n raise err\n', ' File "/usr/lib/python3/dist-packages/socks.py", line 199, in create_connection\n sock.connect((remote_host, remote_port))\n', ' File "/usr/lib/python3/dist-packages/socks.py", line 47, in wrapper\n return function(*args, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/socks.py", line 800, in connect\n raise ProxyConnectionError(msg, error)\n', 'socks.ProxyConnectionError: Error connecting to SOCKS5 proxy 127.0.0.1:9050: [Errno 111] Connection refused\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen\n httplib_response = self._make_request(\n', ' File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request\n self._validate_conn(conn)\n', ' File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in _validate_conn\n conn.connect()\n', ' File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect\n conn = self._new_conn()\n', ' File "/usr/lib/python3/dist-packages/urllib3/contrib/socks.py", line 127, in _new_conn\n raise NewConnectionError(\n', 'urllib3.exceptions.NewConnectionError: <urllib3.contrib.socks.SOCKSHTTPSConnection object at 0xffffac0f1f90>: Failed to establish a new connection: [Errno 111] Connection refused\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send\n resp = conn.urlopen(\n', ' File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen\n retries = retries.increment(\n', ' File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 574, in increment\n raise MaxRetryError(_pool, url, error or ResponseError(cause))\n', "urllib3.exceptions.MaxRetryError: SOCKSHTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/effective_tld_names.dat (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0xffffac0f1f90>: Failed to establish a new connection: [Errno 111] Connection refused'))\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/usr/share/spiderfoot/sflib.py", line 129, in optValueToData\n res = session.get(val)\n', ' File "/usr/lib/python3/dist-packages/requests/sessions.py", line 555, in get\n return self.request(\'GET\', url, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request\n resp = self.send(prep, **send_kwargs)\n', ' File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send\n r = adapter.send(request, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send\n raise ConnectionError(e, request=request)\n', "requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/effective_tld_names.dat (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0xffffac0f1f90>: Failed to establish a new connection: [Errno 111] Connection refused'))\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/usr/share/spiderfoot/sfscan.py", line 149, in startScan\n self.ts.config[\'_internettlds\'] = self.ts.sf.optValueToData(\n', ' File "/usr/share/spiderfoot/sflib.py", line 136, in optValueToData\n self.error("Unable to open option URL, " + val + ": " + str(e))\n', ' File "/usr/share/spiderfoot/sflib.py", line 318, in error\n raise BaseException("Internal Error Encountered: " + message)\n', "BaseException: Internal Error Encountered: Unable to open option URL, https://publicsuffix.org/list/effective_tld_names.dat: SOCKSHTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/effective_tld_names.dat (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0xffffac0f1f90>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"]

What version of Python are you using?

Python 3.10.5

What version of SpiderFoot are you using (stable release or Git master branch)?

Spiderfoot 3.0 - Master branch (I installed it a while ago but I didn't use it until now)

[bcoles]

Internal Error Encountered: Unable to open option URL, https://publicsuffix.org/list/effective_tld_names.dat
Error connecting to SOCKS5 proxy 127.0.0.1:9050

Those errors imply the configured proxy is unavailable. Ensure the proxy is running; or, check your scan settings and remove the configured proxy.

[JArmandoG]

Thanks I noticed that I needed to upgrade my Spiderfoot instance. Everything is working now