This repository has been archived by the owner. It is now read-only.

Question: How to relinquishing control to the event loop in Python 3.5 #284

Closed
jashandeep-sohi opened this Issue Nov 4, 2015 · 20 comments

Comments

Projects
None yet
5 participants
@jashandeep-sohi

jashandeep-sohi commented Nov 4, 2015

In Python < 3.5, you could do a yield or yield None in a coroutine to give control to the event loop.
In Python 3.5, it is invalid to yield in an async def coroutine. So, what's the proper way of relinquishing control to the event loop?

One pattern is:

class AsyncNone(object):
    __await__(self):
        yield

async def cor():
    for _ in range(100):
        await AsyncNone()

But this seems very hacky.

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Nov 4, 2015

Oh, that's unfortunate indeed. @1st1 Any thoughts? I guess the most straightforward pattern would be

await asyncio.sleep(0)
@1st1

This comment has been minimized.

Member

1st1 commented Nov 5, 2015

I guess we can add asyncio.nop or something alike. I, personally, never needed that.

@jashandeep-sohi

This comment has been minimized.

jashandeep-sohi commented Nov 5, 2015

Could asyncio implement a AsyncNone like sentinel object, and one could just do:
await asyncio.AsyncNone?

@jashandeep-sohi

This comment has been minimized.

jashandeep-sohi commented Nov 5, 2015

or a noop @1st1 beat me to it

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Nov 5, 2015

Well, AsyncNone is not less typing than sleep(0), so why would you want
that? Also, how common is this use case really? (My intuition is that it's
most commonly used by people who don't yet have a lot of asyncio
experience.)

On Wed, Nov 4, 2015 at 4:02 PM, Jashandeep Sohi notifications@github.com
wrote:

Could asyncio implement a AsyncNone like sentinel object, and one could
just do:
await asyncio.AsyncNone?


Reply to this email directly or view it on GitHub
#284 (comment).

--Guido van Rossum (python.org/~guido)

@jashandeep-sohi

This comment has been minimized.

jashandeep-sohi commented Nov 5, 2015

Or when asyncio is being buggy #263 ;)
It can also be useful when you are doing some long computation in the same thread and don't want to block the loop.

AsyncNone is probably not the best name though. Something that makes it clear, you are giving up control would probably be be best. sleep(0) seems cryptic.

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Nov 5, 2015

Hm, for a long computation you should probably put it in a thread.

@vstinner

This comment has been minimized.

Member

vstinner commented Nov 5, 2015

Hi,

It's unclear to me if it's a bad practice or not to try to help the
scheduler.

In eventlet code, it's "common" to put eventlet.sleep(0) to yield the
control to the next pending task.

In asyncio, you can write "yield from asyncio.sleep(0)" or "await
asyncio.sleep(0)".sleep() is quite complex, a simpler version is:

def relinquish_cpu():
fut = asyncio.Future()
fut.set_result(None)
yield from fut # or: await fut

Example of usage:

yield from relinquish_cpu()

Other suggestions fot the function name: give_cpu(), give_control(),
yield_cpu(), etc. Since I'm not a native english speaker, it's not easy for
me to find a short and well defined name :-/

"noop" (no operation) is not a good name since calling it has a real
impact: it yields the control the next pending task.

Note: the pthread library has a pthread_yield() function. There is also the
sched_yield() function from <sched.h>.

Victor

2015-11-05 1:02 GMT+01:00 Yury Selivanov notifications@github.com:

I guess we can add asyncio.nop or something like that. I, personally,
never needed that.


Reply to this email directly or view it on GitHub
#284 (comment).

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Nov 5, 2015

Maybe we can optimize sleep(0). I don't like the idea of introducing a new
function just to save typing the '0'.

On Thu, Nov 5, 2015 at 12:18 AM, Victor Stinner notifications@github.com
wrote:

Hi,

It's unclear to me if it's a bad practice or not to try to help the
scheduler.

In eventlet code, it's "common" to put eventlet.sleep(0) to yield the
control to the next pending task.

In asyncio, you can write "yield from asyncio.sleep(0)" or "await
asyncio.sleep(0)".sleep() is quite complex, a simpler version is:

def relinquish_cpu():
fut = asyncio.Future()
fut.set_result(None)
yield from fut # or: await fut

Example of usage:

yield from relinquish_cpu()

Other suggestions fot the function name: give_cpu(), give_control(),
yield_cpu(), etc. Since I'm not a native english speaker, it's not easy for
me to find a short and well defined name :-/

"noop" (no operation) is not a good name since calling it has a real
impact: it yields the control the next pending task.

Note: the pthread library has a pthread_yield() function. There is also the
sched_yield() function from <sched.h>.

Victor

2015-11-05 1:02 GMT+01:00 Yury Selivanov notifications@github.com:

I guess we can add asyncio.nop or something like that. I, personally,
never needed that.


Reply to this email directly or view it on GitHub
#284 (comment).


Reply to this email directly or view it on GitHub
#284 (comment).

--Guido van Rossum (python.org/~guido)

@1st1

This comment has been minimized.

Member

1st1 commented Nov 5, 2015

Maybe we can optimize sleep(0). I don't like the idea of introducing a new
function just to save typing the '0'.

Agree. Please see PR #285

@1st1

This comment has been minimized.

Member

1st1 commented Nov 5, 2015

@jashandeep-sohi @haypo With PR #285 merged, sleep(0) should be pretty fast! Closing the issue.

@1st1 1st1 closed this Nov 5, 2015

@vstinner

This comment has been minimized.

Member

vstinner commented Nov 5, 2015

@1st1

This comment has been minimized.

Member

1st1 commented Nov 5, 2015

Hum, does it mean that "yield from asyncio.sleep(0[, loop=loop])" is
the official way to "yield"? If yes, it's fine for me.

I think yes. If more people complain and ask for a dedicated function, we can add it later, right?

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Nov 5, 2015

IMO sleep(0) is the perfect spelling for this, since everybody already
knows what it means. :-)

On Thu, Nov 5, 2015 at 12:26 PM, Yury Selivanov notifications@github.com
wrote:

Hum, does it mean that "yield from asyncio.sleep(0[, loop=loop])" is
the official way to "yield"? If yes, it's fine for me.

I think yes. If more people complain and ask for a dedicated function, we
can add it later, right?


Reply to this email directly or view it on GitHub
#284 (comment).

--Guido van Rossum (python.org/~guido)

@jashandeep-sohi

This comment has been minimized.

jashandeep-sohi commented Nov 5, 2015

@1st1 Yeah, sleep(0) should be fine. I don't think too many people need this, but a quick mention in the docs might help the odd soul.

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Nov 5, 2015

Good reminder. The docs currently have yield for this purpose but that
doesn't work inside a async def (which is how this got started).

On Thu, Nov 5, 2015 at 12:33 PM, Jashandeep Sohi notifications@github.com
wrote:

@1st1 https://github.com/1st1 Yeah, sleep(0) should be fine. I don't
think too many people need this, but a quick mention in the docs might help
the odd soul.


Reply to this email directly or view it on GitHub
#284 (comment).

--Guido van Rossum (python.org/~guido)

@regnarg

This comment has been minimized.

regnarg commented Jul 5, 2016

Perhaps this should be documented better. I just spend about 20 minutes looking for how to do a cooperative-multitasking "yield" before finally landing here. It's not a common use case, sure, but there are situations when it might be helpful.

For example I have a function that is usually fast so that it's ok to run it in one go. But under some circumstances, it can take longer to complete and should give opportunity for background event processing once in a while (e.g. every 10000 processed items). This is basically cooperative multitasking. It may be preferrable to threads because the function can be interrupted only in well-controlled places where you don't have to worry about synchronization issues.

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Jul 5, 2016

Sure. Perhaps you can submit a patch to bugs.python.org with an example?

@gvanrossum

This comment has been minimized.

Member

gvanrossum commented Jul 5, 2016

(Resend because IIRC closed issues don't notify unless at-mentioned.) @regnarg Perhaps you can submit a patch to bugs.python.org with an example?

@regnarg

This comment has been minimized.

regnarg commented Jul 5, 2016

Ok, I'll try to do that in the near future. NB: I got notified about the first comment, too.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.