Skip to content

[iceberg] Fix contains_nan in partition summary for float/double#7788

Merged
JingsongLi merged 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/fix-nan
May 8, 2026
Merged

[iceberg] Fix contains_nan in partition summary for float/double#7788
JingsongLi merged 1 commit into
apache:masterfrom
ArnavBalyan:arnavb/fix-nan

Conversation

@ArnavBalyan
Copy link
Copy Markdown
Member

Purpose

  • IcebergManifestFile hardcodes contains_nan = false in IcebergPartitionSummary.
  • This value is used by readers for predicate pruning, hardcoding to false causes the files to get skipped that contain NaN matches.
  • Detect NaN via the existing min/max and pass it correctly. Issue only emerges for float/double values.

Tests

  • UT

Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit 40c46f9 into apache:master May 8, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants