Lazily register more cudf functions and move to backends file #4396
Conversation
This is a bit nicer than `isinstance(obj, parallel_types())` because it works also with lazily registered types.
This collects lazily registered functions for cupy, sparse, scipy, and cudf.
mrocklin
changed the title
|
I've now moved these (and other lazily registered backend functions) to a separate |
|
@jcrist do you have thoughts on this? Pinging you because I think you're more likely to have a critical eye here. |
|
The challenge I ran into is that I would put these in a project like Another approach to this would be to register worker-plugins (I started dask/distributed#2453 for this) but I'm somewhat inclined to continue with the "just import things in dask" approach for a while longer while we figure out something cleaner. |
mrocklin
force-pushed
the
more-register-cudf
branch
from
9f842b0
to
996bab8
Compare
|
Merging this later today if there are no further comments. |
) * Add has_parallel_type This is a bit nicer than `isinstance(obj, parallel_types())` because it works also with lazily registered types. * Add more cudf lazy registrations * Add explicit backends.py files in dask.array/dataframe This collects lazily registered functions for cupy, sparse, scipy, and cudf.
This also builds on #4395