-
Notifications
You must be signed in to change notification settings - Fork 10.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Small improvement to the docs for setting ITEM_PIPELINES #3775
Comments
The thing is, I understand your frustration, but I’m not sure how we can improve things. Users are expected to have read https://docs.scrapy.org/en/latest/topics/settings.html#populating-the-settings by the time they look up specific settings in the documentation. |
Out of curiosity, why does an Item need to be declared in ITEM_PIPELINES in order to be processed? I'm learning scrapy and this just bit me -- I was This was counterintuitive to me as a learner -- is there a reason someone would yield an Item without wanting its process method to be called? Related issue for the pipeline docs: #2350 I kind of agree with 2350 -- I'm an experienced python programmer, but it took me a while to figure out the item pipeline from docs. I couldn't find a complete example -- the entire 'item pipelines' docs page, for example, doesn't have the Happy to submit a (small) docs PR if helpful, but fair warning I'm not a scrapy expert. |
This is the first time I hear of someone defining a |
ahhh that makes sense -- I misunderstood the API here fwiw it would really help to add an end-to-end example in the 'item pipelines' docs page one that included yielding from a spider |
In the docs
https://github.com/scrapy/scrapy/blob/65d631329a1434ec013f24341e4b8520241aec70/scrapy/templates/project/module/pipelines.py.tmpl
It says, in the comments:
Please change the instruction to:
I added the setting to my spider's init, and it was hard to find out what was going wrong.
Mentioning settings.py would help others who make the same mistake.
The text was updated successfully, but these errors were encountered: