CachingHostnameResolver

[elacuesta]

Fixes #4802

Based on #4802 (comment).

tests/CrawlerProcess/caching_hostname_resolver.py times out on the current master branch (git checkout master -- scrapy/resolver && tox -e py38 -- tests/test_crawler.py)

[codecov]

Codecov Report

Merging #4803 into master will increase coverage by 0.00%.
The diff coverage is 93.33%.

@@           Coverage Diff           @@
##           master    #4803   +/-   ##
=======================================
  Coverage   87.85%   87.86%           
=======================================
  Files         160      160           
  Lines        9735     9749   +14     
  Branches     1437     1439    +2     
=======================================
+ Hits         8553     8566   +13     
- Misses        925      926    +1     
  Partials      257      257           

Impacted Files	Coverage Δ
scrapy/resolver.py	92.20% <93.33%> (+0.14%)	⬆️

[elacuesta]

(Replying to #4802 (comment))

@michael-lazar Great work, thanks. Indeed, I got confused with the return types. I observed that in practice, the return type for resolveHostName is a HostResolution object, even though the docs say it returns a IResolutionReceiver. However, storing the actual CachingResolutionReceiver object in the cache does not seem to work either 🤔

I updated this PR with the approach from the linked comment, but I'm still a bit confused. Specially, I'm not sure about importing a class from an underscored module (twisted.internet._resolver.HostResolution)

(tests are still missing!)

@Gallaecio @kmike @wRAR any thoughts?

[Gallaecio]

I'm not sure about importing a class from an underscored module (twisted.internet._resolver.HostResolution)

What about just defining our own? The current upstream implementation seems rather simple.

[elacuesta]

What about just defining our own? The current upstream implementation seems rather simple.

I like that!