Skip to content

fix(parquet): read null fixed-size-list parents - #1099

Open
fallintoplace wants to merge 4 commits into
apache:mainfrom
fallintoplace:fix/parquet-nullable-fixed-size-list
Open

fix(parquet): read null fixed-size-list parents#1099
fallintoplace wants to merge 4 commits into
apache:mainfrom
fallintoplace:fix/parquet-nullable-fixed-size-list

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

Parquet does not store child values for a null fixed-size-list parent, while Arrow requires list_size child slots for every parent. The existing reader assumes every parent consumes exactly list_size values and does not validate the final span.

What changes are included in this PR?

Reconstruct children from valid and null parent runs:

  • validate that each valid parent consumes exactly list_size values
  • validate that null parents consume no stored child values
  • materialize list_size null children for each null parent
  • validate the final parent span

When there are no null parents, validate the spans and reuse the decoded child array directly.

Are these changes tested?

  • go test ./parquet/pqarrow -run 'TestParquetArrowIO/TestFixedSizeList' -count=1

Are there any user-facing changes?

No API changes. This corrects the reported behavior while preserving the existing ownership and compatibility contracts.

@fallintoplace fallintoplace changed the title fix(parquet): read nullable fixed-size lists fix(parquet): read null fixed-size-list parents Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant