Skip to content

GH-10613: Fix IPC FileReader projected schema - #10627

Merged
Jefffrey merged 1 commit into
apache:mainfrom
dhruvxvaishnav:fix/ipc-file-reader-projected-schema
Aug 12, 2026
Merged

GH-10613: Fix IPC FileReader projected schema#10627
Jefffrey merged 1 commit into
apache:mainfrom
dhruvxvaishnav:fix/ipc-file-reader-projected-schema

Conversation

@dhruvxvaishnav

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

When FileReader uses a projection, the batches contain the projected schema but RecordBatchReader::schema() returns the complete file schema. This makes the schema reported by the reader inconsistent with the batches it produces.

What changes are included in this PR?

  • Store the projected output schema on FileReader while keeping the complete schema in FileDecoder for decoding.
  • Return the projected schema from FileReader::schema().
  • Validate projection indices when constructing the reader.
  • Add regression tests for projected schemas and invalid projections.

Are these changes tested?

Yes. I ran:

  • cargo test -p arrow-ipc --all-features
  • cargo test -p arrow --all-features
  • cargo clippy -p arrow-ipc --all-targets --all-features -- -D warnings
  • cargo +stable fmt --all -- --check

Are there any user-facing changes?

Yes. Projected IPC file readers now report the same schema as the record batches they produce. There are no public API signature changes.

AI assistance: OpenAI Codex assisted with codebase exploration, implementation support, test preparation, and validation. I reviewed and validated the changes and remain responsible for understanding and maintaining them.

@github-actions github-actions Bot added arrow Changes to the arrow crate arrow-ipc labels Aug 11, 2026
@Jefffrey Jefffrey added the bug label Aug 12, 2026
@Jefffrey
Jefffrey merged commit b6a12bb into apache:main Aug 12, 2026
34 checks passed
@Jefffrey

Copy link
Copy Markdown
Contributor

thanks @dhruvxvaishnav

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate arrow-ipc bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ipc file reader RecordBatchReader::schema ignores column projection

2 participants