Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outdated item pipeline example about Deferred #4514

Closed
elacuesta opened this issue Apr 27, 2020 · 4 comments
Closed

Outdated item pipeline example about Deferred #4514

elacuesta opened this issue Apr 27, 2020 · 4 comments

Comments

@elacuesta
Copy link
Member

elacuesta commented Apr 27, 2020

The Take screenshot of item example says "This example demonstrates how to return a Deferred from the process_item() method", but that is no longer the case (async/await syntax is currently used).
I think we should remove that sentence and add a reminder about enabling the asyncio reactor.

Also, I wonder if it'd be good to also restore the previous Deferred example and keep both of them, since not all projects might be able/willing to switch to the asyncio reactor.

@Gallaecio
Copy link
Member

I believe that the example now uses coroutines, but not asyncio, so it should work with a regular Twisted reactor.

PS: I used to mix the two of them as well until @kmike and @wRAR illuminated me.

@elacuesta
Copy link
Member Author

I've read this article a few times, which shows how to use async/await independently from asyncio, but I still get confused sometimes. Might be time to read it again!
In any case, there is no Deferred in play anymore, that needs to be addressed.

@incognitozen
Copy link

The docs need an example of a spider with asynio enabled and used.

@Gallaecio
Copy link
Member

A simple spider does not require asyncio, and a complex spider would not make for a good example.

On the other hand, it may be useful to cover, for example, an example item pipeline that uses an asyncio-based SQL client to record items in a database.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants