Error connecting to dask client - giving error There is no current event loop in thread when not using asyncio #3156
Comments
|
I recommend providing a minimal reproducible example if that is easy for
you. It is hard to help otherwise.
…On Fri, Oct 18, 2019 at 7:49 PM ameetshah1983 ***@***.***> wrote:
We are trying to connect to dask client using Client(scheduler_url) syntax
using concurrent.futures threadpool implementation and receiving the error
below.
Traceback (most recent call last):
File "/home/app/dask.py", line 848, in get_dask_client
return Client(scheduler)
File "/base/anaconda/lib/python3.6/site-packages/distributed/client.py", line 668, in __init__
self._loop_runner = LoopRunner(loop=loop, asynchronous=asynchronous)
File "/base/anaconda/lib/python3.6/site-packages/distributed/utils.py", line 364, in __init__
current = IOLoop.current()
File "/base/andaconda/pithon3/lib/python3.6/site-packages/tornado/ioloop.py", line 265, in current
loop = asyncio.get_event_loop()
File "/base/anaconda/pithon3/lib/python3.6/asyncio/events.py", line 694, in get_event_loop
return get_event_loop_policy().get_event_loop()
File "/base/anaconda/pithon3/lib/python3.6/asyncio/events.py", line 602, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.
The function submitting the concurrent future is a async function but the
function being executed is not async and not using asyncio in any way. Dont
need an event loop but for some reason tornado requires it looks like.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#3156?email_source=notifications&email_token=AACKZTAWKMENBOSBXTVVDKLQPJDPNA5CNFSM4JCNIZ5KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HS4FKMQ>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACKZTGQJOR5V5MKAAXWU6DQPJDPNANCNFSM4JCNIZ5A>
.
|
|
Sorry for some delay in getting back. Looks like the issue occurs when running Sanic service in container on linux. Testing it locally on windows does not have any issues. Code to reproduce the issue - Fixed the issue for now based on thread tornadoweb/tornado#2308 by adding - |
|
Looking briefly at the code above it doesn't appear to be runnable without some effort, like setting up scheduler, indenting code correctly, defining |
|
Hi, I'm experiencing the exact same issue, I've put together a self contained script that will reproduce the issue. requirements:script:import time
import asyncio
from sanic import Sanic
from sanic.response import json
from dask import delayed
from dask.distributed import Client, get_client
@delayed(pure=True)
def sleep(wait):
time.sleep(wait)
return wait
@delayed(pure=True)
def task(wait):
# Grab the client and build a
# nested graph to execute
client = get_client()
sleeps = client.compute([sleep(w) for w in wait])
result = client.gather(sleeps)
return result
async def create_server():
app = Sanic(__name__)
@app.listener("before_server_start")
async def _(app_, loop_):
# for every app process that will run
# attach their own client
app_.dask_client = await Client(asynchronous=True, processes=False)
@app.listener("after_server_stop")
async def after_server_stop(app_, loop_):
# clean up dask clients on shutdown
await app_.dask_client.close()
@app.route("/")
async def go(request):
# handle GET request at http://0.0.0.0:8080/
client = request.app.dask_client
result = await client.compute(task([1, 2, 3, 4, 5]))
return json(result)
return app
if __name__ == "__main__":
app = asyncio.run(create_server())
app.run(host="0.0.0.0", port=8080, workers=1)If you run this file and open the browser at If you toggle @mrocklin I hope this makes this issue a bit more approachable now. Thanks for your work. |
Thanks @jamesstidard that does help, but it's evidently still not something that I'm personally going to jump into given current time constraints. I don't have any real experience with Sanic. However, if other people have to investigate this that would be great. My guess is that Sanic is doing something odd with processes or the event loop that clashes with what Dask does. A couple things to try:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We are trying to connect to dask client using Client(scheduler_url) syntax using concurrent.futures threadpool implementation and receiving the error below.
The function submitting the concurrent future is a async function but the function being executed is not async and not using asyncio in any way. Dont need an event loop but for some reason tornado requires it looks like.
The text was updated successfully, but these errors were encountered: