[get_retry_request] Adding ability to set log level

[further-reading]

Fixes #5297, fixes #4622

[codecov]

Codecov Report

Merging #5625 (dda9ce4) into master (3cf97e4) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #5625      +/-   ##
==========================================
+ Coverage   88.66%   88.69%   +0.03%     
==========================================
  Files         162      162              
  Lines       10962    10969       +7     
  Branches     1894     1796      -98     
==========================================
+ Hits         9719     9729      +10     
+ Misses        963      960       -3     
  Partials      280      280              

Impacted Files	Coverage Δ
scrapy/downloadermiddlewares/retry.py	100.00% <100.00%> (ø)
scrapy/http/response/text.py	100.00% <100.00%> (ø)
scrapy/settings/default_settings.py	98.77% <100.00%> (+<0.01%)	⬆️
scrapy/utils/python.py	88.46% <0.00%> (-0.13%)	⬇️
scrapy/core/downloader/tls.py	97.05% <0.00%> (-0.09%)	⬇️
scrapy/spiders/crawl.py	94.31% <0.00%> (+0.34%)	⬆️
scrapy/core/downloader/handlers/http11.py	94.09% <0.00%> (+0.96%)	⬆️

[wRAR]

How would one use this? By subclassing the middleware?

[further-reading]

How would one use this? By subclassing the middleware?

The get_retry_request function is was added last year so one could trigger retries in a callback. https://docs.scrapy.org/en/latest/news.html#scrapy-2-5-0-2021-04-06

The purpose of this issue is to add the ability to set log level when using it in a callback. There's a seperate issue #4622 for extending the middleware to then use this feature of get_retry_request, which I'm also going to work on.

[further-reading]

I've updated the middleware so this will fix #4622 as well.

[Gallaecio]

The code looks good to me, beyond my minor feedback.

Next we would need tests to cover the default function value, the ability to override through the function, and same for the setting. And a short documentation entry for the new setting.

[wRAR]

@Gallaecio I see you previously wanted tests and docs but you've approved this now, so what should I do? :)

[Gallaecio]

I forgot about that 🤦