Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request doesn't set callback to self.parse automatically when errback is set #3586

Closed
joaquingx opened this issue Jan 16, 2019 · 2 comments · Fixed by #4008
Closed

Request doesn't set callback to self.parse automatically when errback is set #3586

joaquingx opened this issue Jan 16, 2019 · 2 comments · Fixed by #4008

Comments

@joaquingx
Copy link
Contributor

joaquingx commented Jan 16, 2019

Hi, I'm facing an AssertionError when set errback and not callback in Request, is that expected? :

...
    def start_requests(self):
        yield Request(
            self.start_url,
            # callback=self.parse,
            errback=self.parse_error,
            meta={
                'dont_redirect': True,
            },
        )
...

trigger this :

2019-01-16 15:47:22 [scrapy.core.engine] ERROR: Error while obtaining start requests
Traceback (most recent call last):
  File "/home/joaquin/Repos/example/env/lib/python3.6/site-packages/scrapy/core/engine.py", line 127, in _next_request
    request = next(slot.start_requests)
  File "/home/joaquin/Repos/example/example/spiders/br/rj/example.py", line 128, in start_requests
    'dont_redirect': True,
  File "/home/joaquin/Repos/example/env/lib/python3.6/site-packages/scrapy/http/request/__init__.py", line 34, in __init__
    assert callback or not errback, "Cannot use errback without a callback"
AssertionError: Cannot use errback without a callback
2019-01-16 15:47:22 [scrapy.core.engine] INFO: Closing spider (finished)
2019-01-16 15:47:22 [scrapy.core.engine] INFO: Spider closed (finished)
@RomiVu
Copy link

RomiVu commented Jan 17, 2019

when errback is set, callback must be set explicitly.

@joaquingx joaquingx changed the title start_requests doesn't set callback to self.parse automatically when errback is set Request doesn't set callback to self.parse automatically when errback is set Jan 17, 2019
@joaquingx
Copy link
Contributor Author

Hi @RomiVu, thanks for help 🙏. I cannot found a clarification about that in documentation, what I found so far:

callback (callable) – the function that will be called with the response of this request (once its downloaded) as its first parameter. For more information see Passing additional data to callback functions below. If a Request doesn’t specify a callback, the spider’s parse() method will be used. Note that if exceptions are raised during processing, errback is called instead
From https://doc.scrapy.org/en/latest/topics/request-response.html.

If it's expected that ever errback is set also callback must be set, I think that it should be documented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants