-
Notifications
You must be signed in to change notification settings - Fork 224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi-dimensional memoryview breaks packing #526
Comments
The packer actually requests a buffer object with the |
@jfolz, I don't fully understand your comment. When packing a NumPy array (in my code not shown here), I explicilty ask for the memoryview using But even without NumPy: msgpack packs multi-dimensional memoryviews incorrectly as shown in the example, wouldn't you agree? |
I can not reproduce.
|
I tested Python 3.10 and 3.11 but output is same. |
You're right, I misunderstood what you were trying to do. Numpy is not to blame, as it's job is done after you get the |
@methane , that's weird. I reproduced your steps above in a fresh venv with only msgpack installed:
As you can see, the bin array has a correct size of 6 in the first case and an incorrect size 3 (the size of the first dimension) in the second case. |
@jfolz The memoryview is contiguous, but it's multi-dimensional. The question is what msgpack should do with multi-dimensional memoryviews:
Could be another configuration parameter. |
Now I got it. Fallback module packs it wrong.
|
I released 1.0.5rc1. Please try it. |
Issue is fixed with 1.0.5rc1. Thanks a lot! |
But since you now provide wheels for 3.11, I'm no longer using the fallback code, I guess. |
NumPy 1.24 seems to use multi-dimensional memoryviews when getting array data with
array.data
. msgpack does not play nicely with such multi-dimensional views.Example
Here's a crashing example without NumPy, but creating a multi-dimensional memoryview by hand:
Output
Mitigations
data.case(data.format)
bytes
either bydata.tobytes()
orbytes(data)
. I'm not sure if that duplicates the memory buffer.Of course, it would be great if msgpack could do that internally when packing a memoryview.
Thanks for this great package! ❤️
The text was updated successfully, but these errors were encountered: