Skip to content

concat_elements_fixed_size_binary panics / over-allocates on untrusted FixedSizeBinary widths #10972

Description

@alamb

Describe the bug

concat_elements_fixed_size_binary will panic with some invalid parameters

In theory, these invalid values can come from the FixedSizeBinary data type, which may originate in user controlled schema.

To Reproduce

Full reproducer:

issue1_repo.zip

With arrow-string 59.2.0:

[dependencies]
arrow-array = "59.2.0"
arrow-buffer = "59.2.0"
arrow-string = "59.2.0"
use arrow_array::FixedSizeBinaryArray;
use arrow_buffer::Buffer;
use arrow_string::concat_elements::concat_elements_fixed_size_binary;

fn main() {
    let w = 0x7000_0000_i32;
    let a = FixedSizeBinaryArray::try_new(w, Buffer::from(Vec::<u8>::new()), None).unwrap();
    let b = FixedSizeBinaryArray::try_new(w, Buffer::from(Vec::<u8>::new()), None).unwrap();
    let _ = concat_elements_fixed_size_binary(&a, &b);
}

This panics on both release and debug builds

andrewlamb@Andrews-MacBook-Pro-3:/tmp/issue1_repo$ cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/issue1_repo`
calling concat_elements_fixed_size_binary with widths 1879048192 and 1879048192

thread 'main' (44570022) panicked at /Users/andrewlamb/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-array-59.2.0/src/builder/fixed_size_binary_builder.rs:60:9:
value length (-536870912) of the array must >= 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
andrewlamb@Andrews-MacBook-Pro-3:/tmp/issue1_repo$ cargo run --release
    Blocking waiting for file lock on build directory
    Finished `release` profile [optimized] target(s) in 5.88s
     Running `target/release/issue1_repo`
calling concat_elements_fixed_size_binary with widths 1879048192 and 1879048192

thread 'main' (44570754) panicked at /Users/andrewlamb/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/arrow-array-59.2.0/src/builder/fixed_size_binary_builder.rs:60:9:
value length (-536870912) of the array must >= 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Expected behavior

concat_elements_fixed_size_binary should return an ArrowError when the combined width exceeds i32::MAX, rather than panicking, and should not attempt the allocation first.

Additional context

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions