[MRG+1] New signal for reqeuests reached downloader

[whalebot-helmsman]

New signal added with documentation to be emitted for requests reached downloader. It is done to provide better integration between scheduler and downloader without adding much complexity, based on previous discussions: #2435 #2474

Signal name is discussable. Documentation can be improved.

[codecov]

Codecov Report

Merging #3393 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3393      +/-   ##
==========================================
+ Coverage   84.46%   84.47%   +<.01%     
==========================================
  Files         167      167              
  Lines        9362     9366       +4     
  Branches     1390     1390              
==========================================
+ Hits         7908     7912       +4     
  Misses       1199     1199              
  Partials      255      255

Impacted Files	Coverage Δ
scrapy/signals.py	100% <100%> (ø)	⬆️
scrapy/core/downloader/__init__.py	88.97% <100%> (+0.08%)	⬆️
scrapy/spidermiddlewares/offsite.py	100% <0%> (ø)	⬆️

[whalebot-helmsman]

Made a performance comparison with new signal and without one. Full log available https://gist.github.com/whalebot-helmsman/427908453e28e4d91ba24062a3a1aa05

==> /home/hotdox/exc/master <==
2018-08-28 09:11:37 [scrapy.core.engine] INFO: Spider closed (closespider_itemcount)

The results of the benchmark are (all speeds in items/sec) : 


Test = 'Book Spider' Iterations = '10'


Mean : 86.57660125097868 Median : 86.4017169447081 Std Dev : 0.8022010534209844


==> /home/hotdox/exc/branch <==
2018-08-28 09:09:11 [scrapy.core.engine] INFO: Spider closed (closespider_itemcount)

The results of the benchmark are (all speeds in items/sec) : 


Test = 'Book Spider' Iterations = '10'


Mean : 86.94572398605807 Median : 86.71087801245771 Std Dev : 0.9639990239084966

No substantial impact on performance

[kmike]

typo: comma is not needed here

[kmike]

👍 I think that's a good feature which may be used not only for custom schedulers; overhead doesn't seem significant. Signal name and the implementation look fine.