Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make lazy loading Download Handlers optional #3394

Merged
merged 2 commits into from Dec 26, 2018

Conversation

@hcoura
Copy link
Contributor

@hcoura hcoura commented Aug 17, 2018

Download Handlers load only when first request for a scheme is needed, this was put in place because s3 download handler used to be very slow to create.

This makes it optional for Download Handlers to be lazy loaded which is relevant when writing custom handlers that have required settings and wants to raise NotConfiguredException during instantiation.

@codecov
Copy link

@codecov codecov bot commented Aug 17, 2018

Codecov Report

Merging #3394 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3394      +/-   ##
==========================================
+ Coverage   84.46%   84.49%   +0.02%     
==========================================
  Files         167      167              
  Lines        9362     9372      +10     
  Branches     1390     1391       +1     
==========================================
+ Hits         7908     7919      +11     
  Misses       1199     1199              
+ Partials      255      254       -1
Impacted Files Coverage Δ
scrapy/core/downloader/handlers/http10.py 100% <100%> (ø) ⬆️
scrapy/core/downloader/handlers/file.py 100% <100%> (ø) ⬆️
scrapy/core/downloader/handlers/datauri.py 93.75% <100%> (+0.41%) ⬆️
scrapy/core/downloader/handlers/__init__.py 83.63% <100%> (+3.63%) ⬆️
scrapy/core/downloader/handlers/ftp.py 98.24% <100%> (+0.03%) ⬆️
scrapy/core/downloader/handlers/http11.py 92.53% <100%> (+0.02%) ⬆️
@kmike kmike added this to the v1.6 milestone Aug 24, 2018
@kmike
kmike approved these changes Dec 25, 2018
Copy link
Member

@kmike kmike left a comment

Looks good, thanks @hcoura! //cc @dangra

@dangra dangra merged commit 93cf383 into scrapy:master Dec 26, 2018
3 checks passed
3 checks passed
codecov/patch 100% of diff hit (target 84.46%)
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants