Skip to content

[Bug] Committing only null partition values fails with NPE when Iceberg compatibility is enabled #9151

Description

@thswlsqls

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version
master @ 158d83c (2.1-SNAPSHOT)

Compute Engine
Engine-agnostic (core)

Minimal reproduce step

  1. Create pt VARCHAR(10), v INT partitioned by pt, bucket -1, with metadata.iceberg.storage = table-location.
  2. Commit a row with pt = 'a'.
  3. Commit a batch in which every row has pt = NULL — this fails:
java.lang.NullPointerException
  at IcebergConversions.toByteBuffer(IcebergConversions.java:83)
  at IcebergManifestFile$IcebergManifestEntryWriter.result(IcebergManifestFile.java:273)
  at IcebergCommitCallback.call(IcebergCommitCallback.java:256)

What doesn't meet your expectations?
The commit should succeed. Paimon supports null partition values, and the Iceberg spec declares field_summary.lower_bound (510) and upper_bound (511) optional — "or null if all values are null or NaN". IcebergPartitionSummary.schema() already declares both nullable.

Anything else?
When every entry of a manifest holds a null partition value, the collected SimpleColStats min/max stay null and reach toByteBuffer unguarded. The sibling IcebergDataFileMeta.create() has omitted unknown column bounds since #4090; the partition summary path never got the equivalent guard. Reading is affected too: IcebergCommitCallback.createWithDeleteManifestFileMetas hits the same null in toPaimonObject on the next non add-only commit.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions