You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a scrapy.Request is created with a callback that is a string (and not a callable),
callback (callable) – the function that will be called with the response of this request (once its downloaded) as its first parameter.
Twisted chokes with a confusing twisted.internet.defer._DefGen_Return exception traceback.
The error of using a string for a callback comes from allowing a string in CrawlSpider rules.
callback is a callable or a string (in which case a method from the spider object with that name will be used) to be called for each link extracted with the specified link_extractor.
Suggestion
either allow callback to be string and matched with a spider method, like in CrawlSpider rules
or fail earlier in scrapy.Request.__init__() if a non-None callback is not callable
Originally from https://stackoverflow.com/questions/44259172/scrapy-twisted-internet-defer-defgen-return-exception
When a
scrapy.Request
is created with a callback that is a string (and not a callable),Twisted chokes with a confusing
twisted.internet.defer._DefGen_Return
exception traceback.The error of using a string for a callback comes from allowing a string in
CrawlSpider
rules.Suggestion
CrawlSpider
rulesscrapy.Request.__init__()
if a non-None
callback is not callableHow to reproduce
The text was updated successfully, but these errors were encountered: