Drafted by Claude Code on Connor's behalf, and not yet edited by him.
VarBinViewArray::validate_views only validates the views of valid rows, which is intentional. So a legal array can hold a view behind a null row naming a buffer that does not exist.
ScalarFnVTable::is_strict consequence 2 reads as an unconditional licence to ignore that: "Values behind null slots are irrelevant, so kernels may compute g densely over all lanes (including garbage) and apply validity afterwards." That holds for a flat fixed-width payload, where a null row is unused bytes and reading it can't fault. It doesn't hold for a VarBinViewArray row, where reading the value means following a buffer index and offset into a data buffer.
A kernel that resolves every row densely panics on a legal array:
index out of bounds: the len is 1 but the index is 9
Repro:
let views = buffer[
BinaryView::make_view(b"a longer string here", 0, 0),
// Null row: buffer 9 does not exist and the offset is past the end of the data.
BinaryView::new_ref(64, *b"junk", 9, 4096),
];
let array = VarBinViewArray::try_new(
views,
Arc::from([ByteBuffer::copy_from(b"a longer string here")]),
DType::Utf8(Nullability::Nullable),
Validity::from_iter([true, false]),
)?;
try_new succeeds, which is correct. Any kernel that then resolves row 1's bytes panics.
Note that no kernel on develop is broken by this, since byte_length reads view.len() out of the view rather than resolving the row. So this is a foot-gun plus a missing test rather than a live bug.
Two steps:
- Narrow consequence 2. Dense evaluation is sound when reading a row can't fault, which excludes offset-following elements.
- Add a regression test pinning
byte_length over the array above to [Some(20), None], guarding against a later "optimization" into resolving rows.
Both exist on claude/strict-scalar-fn-abstraction-ah88x3 as a DENSE_SAFE const on the element type, but neither the doc fix nor the test needs any of that.
Drafted by Claude Code on Connor's behalf, and not yet edited by him.
VarBinViewArray::validate_viewsonly validates the views of valid rows, which is intentional. So a legal array can hold a view behind a null row naming a buffer that does not exist.ScalarFnVTable::is_strictconsequence 2 reads as an unconditional licence to ignore that: "Values behind null slots are irrelevant, so kernels may computegdensely over all lanes (including garbage) and apply validity afterwards." That holds for a flat fixed-width payload, where a null row is unused bytes and reading it can't fault. It doesn't hold for aVarBinViewArrayrow, where reading the value means following a buffer index and offset into a data buffer.A kernel that resolves every row densely panics on a legal array:
Repro:
try_newsucceeds, which is correct. Any kernel that then resolves row 1's bytes panics.Note that no kernel on
developis broken by this, sincebyte_lengthreadsview.len()out of the view rather than resolving the row. So this is a foot-gun plus a missing test rather than a live bug.Two steps:
byte_lengthover the array above to[Some(20), None], guarding against a later "optimization" into resolving rows.Both exist on
claude/strict-scalar-fn-abstraction-ah88x3as aDENSE_SAFEconst on the element type, but neither the doc fix nor the test needs any of that.