make Scrapy testing suite more robust in environments where non-existing hosts are resolvable

[kmike]

This is an attempt to fix https://github.com/scrapy/scrapy/runs/7377071488?check_suite_focus=true failure.

================================== FAILURES ===================================
_________ Http11ProxyTestCase.test_download_with_proxy_https_timeout __________
self = <tests.test_downloader_handlers.Http11ProxyTestCase testMethod=test_download_with_proxy_https_timeout>
    @defer.inlineCallbacks
    def test_download_with_proxy_https_timeout(self):
        """ Test TunnelingTCP4ClientEndpoint """
        http_proxy = self.getURL('')
        domain = 'https://no-such-domain.nosuch/'
        request = Request(
            domain, meta={'proxy': http_proxy, 'download_timeout': 0.2})
        d = self.download_request(request, Spider('foo'))
>       timeout = yield self.assertFailure(d, error.TimeoutError)
D:\a\scrapy\scrapy\tests\test_downloader_handlers.py:799: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
D:\a\scrapy\scrapy\.tox\py\lib\site-packages\twisted\internet\defer.py:857: in _runCallbacks
    current.result = callback(  # type: ignore[misc]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
failure = <twisted.python.failure.Failure twisted.web._newclient.ResponseNeverReceived: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'wrong version number')]>]>
    def _eb(failure):
        if failure.check(*expectedFailures):
            return failure.value
        else:
            output = "\nExpected: {!r}\nGot:\n{}".format(
                expectedFailures, str(failure)
            )
>           raise self.failureException(output)
E           twisted.trial.unittest.FailTest: 
E           Expected: (<class 'twisted.internet.error.TimeoutError'>,)
E           Got:
E           [Failure instance: Traceback (failure with no frames): <class 'twisted.web._newclient.ResponseNeverReceived'>: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'wrong version number')]>]
E           ]
D:\a\scrapy\scrapy\.tox\py\lib\site-packages\twisted\trial\_asynctest.py:76: FailTest
---------------------------- Captured stderr call -----------------------------
2022-07-17 11:16:25 [scrapy.crawler] INFO: Overridden settings:
{'REQUEST_FINGERPRINTER_IMPLEMENTATION': 'VERSION'}
2022-07-17 11:16:25 [scrapy.extensions.telnet] INFO: Telnet Password: 4d4fc6ef73c68a39
2022-07-17 11:16:25 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.logstats.LogStats']
------------------------------ Captured log call ------------------------------
INFO     scrapy.crawler:crawler.py:61 Overridden settings:
{'REQUEST_FINGERPRINTER_IMPLEMENTATION': 'VERSION'}
INFO     scrapy.extensions.telnet:telnet.py:55 Telnet Password: 4d4fc6ef73c68a39
INFO     scrapy.middleware:middleware.py:51 Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.logstats.LogStats']

I've found a few places where it can be an issue, not sure if that's all of them.

Implementation is ported from Splash testing suite, where there was a similar issue - on some CI environments DNS resolution was successful for non-existing hosts.

The issue with this PR is that in the run above it only failed at one spot, not at all of them. It seems to be related to SSL. So, I'm not sure the fix in this PR is valid :)

[codecov]

Codecov Report

Merging #5560 (26c7031) into master (ebc6f9c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #5560   +/-   ##
=======================================
  Coverage   88.75%   88.75%           
=======================================
  Files         162      162           
  Lines       10774    10774           
  Branches     1843     1843           
=======================================
  Hits         9562     9562           
  Misses        939      939           
  Partials      273      273