Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
DOC: DataFrame→Array conversion and unknown chunks #4516
What does this PR implement?
Thanks @jrbourbeau! I've added the note. I mention that this enables downstream computations, but don't point to any examples in case they're fixed (e.g., slicing an array raises a
I can see another use case with arrays:
>>> x = np.random.choice([-1, 0, 1], size=100) >>> y = da.from_array(x, chunks=50) >>> y[y != -1] # dask.array<getitem, shape=(nan,), dtype=int64, chunksize=(nan,)>
I think computing the chunk size could be useful (e.g., with the slicing example above). Looks like #3293 (comment) is the relevant work.
After looking at this again, converting a Dask DataFrame to a Dask array (and the issue of