Skip to content

HttpErrorMiddleware not honoring handle_httpstatus_all meta as documented #3851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ddebernardy opened this issue Jul 3, 2019 · 2 comments · Fixed by #4694
Closed

HttpErrorMiddleware not honoring handle_httpstatus_all meta as documented #3851

ddebernardy opened this issue Jul 3, 2019 · 2 comments · Fixed by #4694
Labels

Comments

@ddebernardy
Copy link

    def process_spider_input(self, response, spider):
        if 200 <= response.status < 300:  # common case
            return
        meta = response.meta
        if 'handle_httpstatus_all' in meta:
            return

Shouldn't that be more like:

        if 'handle_httpstatus_all' in meta and meta['handle_httpstatus_all']:
            return

As I read the code, setting meta['handle_httpstatus_all'] = False would likely be treated as if it was set to True.

@kmike
Copy link
Member

kmike commented Jul 4, 2019

+1 to fix it, though I think current docs are technically correct, and match the current behavior:

The handle_httpstatus_list key of Request.meta can also be used to specify which response codes to allow on a per-request basis. You can also set the meta key handle_httpstatus_all to True if you want to allow any response code for a request.

@Gallaecio Gallaecio added the bug label Jul 4, 2019
@Jgaldos
Copy link

Jgaldos commented Jul 20, 2020

@Gallaecio Can I try this bug?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants