Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debugging+Logging: find out when sync function run in asynq context #28

Closed
LiraNuna opened this issue Apr 4, 2017 · 6 comments
Closed

Comments

@LiraNuna
Copy link

LiraNuna commented Apr 4, 2017

As we migrate our code to use asynq, we would love a way to detect usages of sync calls within asynq contexts.

For example, imagine:

@asynq.async()
def data_fetch_1():
    return 5

@asynq.async()
def data_fetch_times_2():
    data = data_fetch_1()
    return data + data

This will incur a sync fetch within an already asynq call which should be fixed since it incurs performance penalties especially when data_fetch_2 is batched.

I have not seen a way to log/debug/add a hook to figure out when this happens

JelleZijlstra added a commit to JelleZijlstra/asynq that referenced this issue Apr 15, 2017
Mostly the same changes as quora#28. Also removed some missing imports
and added mock as an explicit dependency.
@JelleZijlstra JelleZijlstra mentioned this issue Apr 15, 2017
manannayak pushed a commit that referenced this issue Apr 18, 2017
* cleanup

Mostly the same changes as #28. Also removed some missing imports
and added mock as an explicit dependency.

* fix test failure

Not sure why this started failing, maybe it's due to the Cython version.
@JelleZijlstra
Copy link
Collaborator

Is Manan's addition in #31 sufficient for your use case?

@LiraNuna
Copy link
Author

How can this be turned on?

@manannayak
Copy link
Contributor

if you set options.DUMP_SYNC_CALLS to True in debug.py, you should see a message in stdout every time an async function is called synchronously inside another async function.

@LiraNuna
Copy link
Author

LiraNuna commented May 26, 2017

I'm guessing like this? If so, this is great, thank you!

from asynq.debug import options

options.DUMP_SYNC_CALLS = True

@manannayak
Copy link
Contributor

Yep

@JelleZijlstra
Copy link
Collaborator

We should really document that options mechanism.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants