Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] Make RedirectMiddleware respect Spider.handle_httpstatus_list #1364

Merged

Conversation

@jdemaeyer
Copy link
Contributor

@jdemaeyer jdemaeyer commented Jul 16, 2015

Inspired by #1334

I have decided not to include the HTTPERROR_ALLOWED_CODES setting, as, unlike the handle_httpstatus_list Spider attribute, the name of that setting implies that it is directly tied to the HttpErrorMiddleware.

@nramirezuy
Copy link
Contributor

@nramirezuy nramirezuy commented Jul 17, 2015

is it really necessary to use network for this test ? You can create a Response object with status=301 and a location header.

@jdemaeyer
Copy link
Contributor Author

@jdemaeyer jdemaeyer commented Jul 17, 2015

is it really necessary to use network for this test ? You can create a Response object with status=301 and a location header.

Hm, but that is what I'm doing?

@jdemaeyer
Copy link
Contributor Author

@jdemaeyer jdemaeyer commented Jul 17, 2015

Whoops but I just realized that I kept the dont_redirect meta in when I copied the request from another test

@kmike kmike changed the title Make RedirectMiddleware respect Spider.handle_httpstatus_list [MRG+1] Make RedirectMiddleware respect Spider.handle_httpstatus_list Jul 28, 2015
dangra added a commit that referenced this pull request Aug 3, 2015
…directs

[MRG+1] Make RedirectMiddleware respect Spider.handle_httpstatus_list
@dangra dangra merged commit 5e6c492 into scrapy:master Aug 3, 2015
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@nramirezuy
Copy link
Contributor

@nramirezuy nramirezuy commented Aug 20, 2015

Sorry I'm late to the party. But we missed request.meta [handle_httpstatus_list]; which is basically the same as the spider argument. http://doc.scrapy.org/en/master/topics/spider-middleware.html#module-scrapy.spidermiddlewares.httperror

@jdemaeyer
Copy link
Contributor Author

@jdemaeyer jdemaeyer commented Aug 21, 2015

Sorry I'm late to the party. But we missed request.meta [handle_httpstatus_list]; which is basically the same as the spider argument. http://doc.scrapy.org/en/master/topics/spider-middleware.html#module-scrapy.spidermiddlewares.httperror

Damnit. Proposed fix in #1447

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants