Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read dictionary from nested struct in ipc stream reader panics #1549

Closed
dispanser opened this issue Apr 12, 2022 · 0 comments · Fixed by #1550
Closed

Read dictionary from nested struct in ipc stream reader panics #1549

dispanser opened this issue Apr 12, 2022 · 0 comments · Fixed by #1550
Labels
arrow Changes to the arrow crate bug

Comments

@dispanser
Copy link
Contributor

Describe the bug
We have an application that reads ipc stream data, and it panicked while reading a record batch that contains a dictionary in a nested struct:

   0: rust_begin_unwind
             at /rustc/9d1b2106e23b1abd32fce1f17267604a5102f57a/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /rustc/9d1b2106e23b1abd32fce1f17267604a5102f57a/library/core/src/panicking.rs:116:14
   2: core::panicking::panic_bounds_check
             at /rustc/9d1b2106e23b1abd32fce1f17267604a5102f57a/library/core/src/panicking.rs:84:5
   3: arrow::ipc::reader::read_dictionary
             at /home/pi/src/github/dispanser/arrow-rs/arrow/src/ipc/reader.rs:573:13
   4: arrow::ipc::reader::StreamReader<R>::maybe_next
             at /home/pi/src/github/dispanser/arrow-rs/arrow/src/ipc/reader.rs:983:17
   5: <arrow::ipc::reader::StreamReader<R> as core::iter::traits::iterator::Iterator>::next
             at /home/pi/src/github/dispanser/arrow-rs/arrow/src/ipc/reader.rs:1004:9

This is caused by an incorrectly sized dictionaries_by_field in the stream reader. PR is ready, opening this for tracking purposes.

To Reproduce

Create a record batch containing a nested struct with a dictionary in last position, write to IPC and attempt to read back using ipc::reader::StreamReader.

Expected behavior

A record batch instead of a panic :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants