Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArrowArray::try_from_raw should not assume the pointers are from Arc #1333

Closed
viirya opened this issue Feb 18, 2022 · 0 comments · Fixed by #1334
Closed

ArrowArray::try_from_raw should not assume the pointers are from Arc #1333

viirya opened this issue Feb 18, 2022 · 0 comments · Fixed by #1334
Labels
arrow Changes to the arrow crate bug

Comments

@viirya
Copy link
Member

viirya commented Feb 18, 2022

Describe the bug

ArrowArray::try_from_raw now uses Arc::from_raw to construct the two pointers to the C Data Interface (ArrowArray and ArrowSchema) back to the structs.

But the two pointers could be from other language runtime (e.g. JVM). We've been using the ffi API to pass Arrow Arrays from JVM to Rust. Sometimes we see fatal error reported at JVM (SIGILL). After digging it deeper, we found that sometimes Arc::from_raw on the pointers gives a Arc pointer with strong reference count reaching the maximum limit.

As the pointer is from JVM, it is not produced by Arc::into_raw, the offset calculation in Arc::from_raw simply doesn't work and can get into wrong values for the strong value in ArcInner.

I think that ArrowArray::try_from_raw should not assume the pointers are from Arc. They should be considered as just raw pointers to the C Data Interface.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants