Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Build Comm for ucx-py #2344
There appears to be an asyncio compatible implemenation in ucx-py in development branches. Here is an example. It would be interesting to implement the Dask Comm API on top of this interface, both as a possible future high-performance networking solution for Dask, and to help drive development upstream in ucx-py.
On the Dask side this is motivated by a few things:
For someone looking to solve this problem I recommend looking at the following resources:
From an API perspective we would want to be able to do the following:
ucx-py is experimental and undergoing pretty heavy churn. My guess is that the work proposed in this issue would live in a development branch for a while until ucx-py stabilized a bit. Starting this early though will probably be important in order to get ucx-py to stable point.