Skip to content

Provide & use a few more host functions#6330

Draft
jakirkham wants to merge 11 commits intodask:mainfrom
jakirkham:add_host_concat
Draft

Provide & use a few more host functions#6330
jakirkham wants to merge 11 commits intodask:mainfrom
jakirkham:add_host_concat

Conversation

@jakirkham
Copy link
Copy Markdown
Member

Moves host_array to distributed.utils to make it easier to use in other places outside of comms. Also include a couple other utility functions in the same vein to allow copy and concatenating frames. Update ensure_memoryview to use these. Also make use of them in comms and `protocols.


  • Tests added / passed
  • Passes pre-commit run --all-files

@jakirkham jakirkham force-pushed the add_host_concat branch 16 times, most recently from aa0606a to 6ca4c82 Compare May 12, 2022 04:53
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 12, 2022

Unit Test Results

       15 files  ±  0         15 suites  ±0   6h 15m 52s ⏱️ - 11m 1s
  2 833 tests +14    2 752 ✔️ +  18    80 💤  - 1  1  - 3 
20 994 runs  +98  20 054 ✔️ +101  939 💤 ±0  1  - 3 

For more details on these failures, see this check.

Results for commit 292bbe3. ± Comparison against base commit c2b28cf.

♻️ This comment has been updated with latest results.

@jakirkham jakirkham force-pushed the add_host_concat branch 8 times, most recently from da1fcf7 to 9259c13 Compare May 16, 2022 09:21
jakirkham added 5 commits May 31, 2022 16:58
Allows one to write `host_array()` instead of `host_array(0)` for
simplicity.
Provide a way to copy an existing `memoryview` to a new one.
jakirkham added 6 commits May 31, 2022 16:58
Provide a utility function that handles concatenating frames together
using either NumPy backed memory or Python backed memory. Also performs
flattening of the data.
Make sure if `ensure_memoryview` allocates memory that it use memory
backed by the preferred host allocator. Which is either NumPy if
available or `bytearray` if not.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant