Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raising trio.WouldBlock from within task creation #130

Closed
DRMacIver opened this issue Dec 20, 2023 · 1 comment · Fixed by #141
Closed

Raising trio.WouldBlock from within task creation #130

DRMacIver opened this issue Dec 20, 2023 · 1 comment · Fixed by #141

Comments

@DRMacIver
Copy link

When running redispy's asyncio support on top of trio-asyncio, we occasionally see the following error:

future: <Task finished name='Task-84858' coro=<ClusterNode.disconnect() done, defined at site-packages/redis/asyncio/cluster.py:992> exception=WouldBlock()>Traceback (most recent call last):
 site-packages/redis/asyncio/cluster.py", line 993, in disconnect
    ret = await asyncio.gather(
                ^^^^^^^^^^^^^^^
 site-packages/redis/asyncio/cluster.py", line 995, in <genexpr>
    asyncio.create_task(connection.disconnect())
  File "/usr/lib64/python3.11/asyncio/tasks.py", line 374, in create_task
    task = loop.create_task(coro)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 436, in create_task
    task = tasks.Task(coro, loop=self, name=name, context=context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 site-packages/trio_asyncio/_base.py", line 320, in call_soon
    return self._queue_handle(asyncio.Handle(callback, args, self, **context))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 site-packages/trio_asyncio/_async.py", line 14, in _queue_handle
    self._q_send.send_nowait(handle)
 site-packages/trio/_core/_ki.py", line 181, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
 site-packages/trio/_channel.py", line 183, in send_nowait
    raise trio.WouldBlock
trio.WouldBlock
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "application.py", line 256, in run
    trio_asyncio.run(self.main)
 site-packages/trio_asyncio/_loop.py", line 530, in run
    return trio.run(_run_task, proc, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 site-packages/trio/_core/_run.py", line 2251, in run
    timeout = gen.send(next_send)
              ^^^^^^^^^^^^^^^^^^^
 site-packages/trio/_core/_run.py", line 2644, in unrolled_run
    raise TrioInternalError("internal error in Trio - please file a bug!") from exc
trio.TrioInternalError: internal error in Trio - please file a bug!

I'm afraid I don't have anything resembling a reproducible test case for this - sorry. It's a thing that we hit occasionally in a production run, and I have no idea how to produce it reliably on our systems, let alone produce a simple test case. I suspect it's something to do with the scale we're running at.

@DRMacIver DRMacIver changed the title Raising trio.WouldBlock when used with Raising trio.WouldBlock from within task creation Dec 20, 2023
@oremanj
Copy link
Member

oremanj commented Dec 20, 2023

Wow, I have no idea how that was ever supposed to work. The best kind of backpressure: it just crashes your program if the queue gets too long. The default max queue length is 10000 so I'm not surprised it only shows up under load.

You can specify a different length as an argument to open_loop(). I bet open_loop(math.inf) will solve your problem. It's kind of silly to accept a limit at all; the asyncio methods to enqueue callbacks are synchronous so there's no way to enforce backpressure if the queue fills up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants