Optimize server conversions for client-server array transfers #880
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Improve the performance of `ak.array()` and `pdarray.to_ndarray()` by optimizing how the server converts between bytes and pdarrays. Previously, the server would write to and read from a big-endian memory mapped file to convert between bytes and arrays but this is fairly slow. Optimize this conversion by directly interpreting the underlying memory as the type we're converting to. In the `ak.array()` case we create the local array with `makeArrayFromPtr`, which will create an array from the existing bytes without any copies. For the `to_ndarray()` case `makeArrayFromPtr` is also used on some local memory, which is then reinterpreted as bytes with `createBytesWithOwnedBuffer`. Here's the performance improvement for 16-node-xc: | config | to_ndarray | ak.array | | ------ | ---------: | ---------: | | before | 33 MiB/s | 50 MiB/s | | after | 410 MiB/s | 175 MiB/s | Where `ak.array()` is slower because there are more copies compared to `to_ndarray()` on both the server and client side. Optimizing those copies out is future work.
glitch
approved these changes
Jul 14, 2021
This was referenced Aug 5, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improve the performance of
ak.array()andpdarray.to_ndarray()byoptimizing how the server converts between bytes and pdarrays.
Previously, the server would write to and read from a big-endian memory
mapped file to convert between bytes and arrays but this is fairly
slow. Optimize this conversion by directly interpreting the underlying
memory as the type we're converting to. In the
ak.array()case wecreate the local array with
makeArrayFromPtr, which will create anarray from the existing bytes without any copies. For the
to_ndarray()case
makeArrayFromPtris also used on some local memory, which is thenreinterpreted as bytes with
createBytesWithOwnedBuffer.Here's the performance improvement for 16-node-xc:
Where
ak.array()is slower because there are more copies compared toto_ndarray()on both the server and client side. Optimizing thosecopies out is future work.
Part of #794