You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, scan operations that supply use_threads=false will use 1 CPU thread, the calling thread, and some # of I/O threads. We should remove that CPU thread from being used.
It is being used at the moment because we need something to transfer work onto when asynchronous I/O tasks complete. We can fix this by allowing a serial executor to run an async generator (creating an iterator). This will also allow us to change the execution plan so that it always expects an executor to be present which should simplify the logic.
Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.
Currently, scan operations that supply use_threads=false will use 1 CPU thread, the calling thread, and some # of I/O threads. We should remove that CPU thread from being used.
It is being used at the moment because we need something to transfer work onto when asynchronous I/O tasks complete. We can fix this by allowing a serial executor to run an async generator (creating an iterator). This will also allow us to change the execution plan so that it always expects an executor to be present which should simplify the logic.
Reporter: Weston Pace / @westonpace
Related issues:
PRs and other links:
Note: This issue was originally created as ARROW-15732. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: