fix(arrow-array): make ArrowArrayStreamReader::try_new unsafe#10419
Closed
bit2swaz wants to merge 1 commit into
Closed
fix(arrow-array): make ArrowArrayStreamReader::try_new unsafe#10419bit2swaz wants to merge 1 commit into
bit2swaz wants to merge 1 commit into
Conversation
bit2swaz
force-pushed
the
fix/ffi-stream-try-new-unsafe
branch
from
July 23, 2026 19:03
57a5821 to
26daa74
Compare
Contributor
Author
|
closing this in favor of a broader fix. @/Jefffrey pointed out that the Drop impl has the same soundness hole and theres no way to slap tracking in: #10429 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #10253
Rationale for this change
ArrowArrayStreamReader::try_newis a safe fn, but it callsget_stream_schema, which derefs and calls the caller-suppliedget_schemafn pointer. every field ofFFI_ArrowArrayStreamis public, so safe code can construct a stream with garbage callbacks and hit UB without writing a singleunsafeblock. therelease.is_none()check only catches an already-released stream, not a garbage-populated oneboth
from_rawconstructors in the same file are alreadyunsafefor this exact reason.try_newwas the one missing it.What changes are included in this PR?
ArrowArrayStreamReader::try_newis nowunsafe fnwith a# Safetysection documenting that the stream's callbacks andprivate_datamust be valid per the C stream interfacefrom_rawbody and the two in-module tests wrap theirtry_newcalls inunsafe {}.try_newcall sites inarrow-pyarrowwrap inunsafe {}with a// SAFETY:note#[allow(dead_code)]ontry_newthe other option from #10253 (private fields on
FFI_ArrowArrayStream) would break the#[repr(C)]ABI that external FFI producers fill field by field, without gaining anything; the callbacks still have to be trusted at call time regardless of visibility.Are these changes tested?
yes. a
compile_faildoctest ontry_newverifies that calling it without anunsafeblock is a compile error. the existing round-trip and error-import tests cover the happy path and still passAre there any user-facing changes?
yes, this is a breaking change.
ArrowArrayStreamReader::try_newis nowunsafe. callers that obtain a stream from a legitimate FFI source (e.g.from_raw,_export_to_c) just need to wrap the call inunsafe {}