Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] Add template for a downloader middleware #2755

Merged
merged 1 commit into from Jul 31, 2017

Conversation

@redapple
Copy link
Contributor

@redapple redapple commented May 24, 2017

While testing downloader middleware with media pipeline (for #2753 (comment)), I looked for a sample downloader middleware and found only a spider middleware template.

I personally never remember what methods to implement, nor what they should return, hence the verbose comments in the template.

Related/alternative #2633

@codecov-io
Copy link

@codecov-io codecov-io commented May 24, 2017

Codecov Report

Merging #2755 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2755   +/-   ##
=======================================
  Coverage   84.69%   84.69%           
=======================================
  Files         164      164           
  Lines        9189     9189           
  Branches     1369     1369           
=======================================
  Hits         7783     7783           
  Misses       1154     1154           
  Partials      252      252
return None

def process_response(self, request, response, spider):
# Called with the respsonse returned from the downloader.

This comment has been minimized.

@aperezalbela

aperezalbela Jun 1, 2017

s/respsonse/response


def process_exception(self, request, exception, spider):
# Called when a download handler or a process_request()
# (from other downloader middelware) raises and exception.

This comment has been minimized.

@aperezalbela

aperezalbela Jun 1, 2017

s/middelware/middleware
s/and/an

@redapple redapple force-pushed the redapple:downloader-mdw-template branch from 24f52d8 to aaaa4da Jul 28, 2017
@redapple
Copy link
Contributor Author

@redapple redapple commented Jul 28, 2017

Thanks for spotting those typos @aperezalbela !

@dangra
Copy link
Member

@dangra dangra commented Jul 28, 2017

LGTM.

@dangra dangra changed the title Add template for a downloader middleware [MRG+1] Add template for a downloader middleware Jul 28, 2017
@redapple redapple added this to the v1.4.1 milestone Jul 31, 2017
@dangra dangra merged commit 19382c5 into scrapy:master Jul 31, 2017
2 checks passed
2 checks passed
codecov/patch Coverage not affected when comparing 9d34b2f...aaaa4da
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kmike kmike modified the milestones: v1.4.1, v1.5 Dec 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.