Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document async def interfaces for signals #4852

Closed
kmike opened this issue Oct 19, 2020 · 6 comments · Fixed by #5358
Closed

Document async def interfaces for signals #4852

kmike opened this issue Oct 19, 2020 · 6 comments · Fixed by #5358

Comments

@kmike
Copy link
Member

kmike commented Oct 19, 2020

It'd be nice to mention async def approach in https://docs.scrapy.org/en/latest/topics/signals.html#deferred-signal-handlers, and maybe even rewrite the example to it.

@codeslash21
Copy link

@Gallaecio I want to work on this issue. Can you please tell how to start with?

@Gallaecio
Copy link
Member

@codeslash21 Sure.

Scrapy uses Twisted, which has a class called Deferred, which is used to implement asynchronous code. You may find more information about them in the Twisted documentation, but it’s a complex topic, and full understanding is not needed for this task, I think.

However, Twisted also allows to write code using Python coroutines, also known as async def syntax. The code still uses deferreds underneath, but its use is transparent to users, and the resulting code is more readable.

We actually have this covered in the documentation of Scrapy, at https://docs.scrapy.org/en/latest/topics/coroutines.html, with a good example of how a piece of code looks with Deferreds, and how the same code can be rewritten with coroutine syntax.

If you manage to understand that example, I think you should be able to handle this task, which would consist of:

  1. Adding a second paragraph to https://docs.scrapy.org/en/latest/topics/signals.html#deferred-signal-handlers that mentions that you can also use coroutines, with a link to the coroutine topic.
  2. Changing the example to use coroutines instead of deferreds, for better readability.

@codeslash21
Copy link

@Gallaecio Thanks, for your guidance.

@rox117
Copy link

rox117 commented May 3, 2021

is this issue still up for grabs? I didnt see a pull request linked to it

@Gallaecio
Copy link
Member

@rox117 I think so.

@burakcank
Copy link
Contributor

Can I take this issue please ? I have been working on this one for almost a day now, trying to learn my way around coroutines and deferred objects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants