Clustering with spark records does not work for bootstrap tables with partition columns not written to files

[hudi-bot]

Spark readers will read as UnsafeRow. When we try to add in the partition column values, an UnsupportedOperationException will be thrown because updateValue is not supported for UnsafeRow.

JIRA info

• Link: https://issues.apache.org/jira/browse/HUDI-8318
• Type: Bug
• Fix version(s):
  • 1.1.0

[rangareddy]

This issue was reviewed as part of the JIRA-migrated backlog triage.

Confirmed as a valid, still-open bug (HUDI-8318). Root cause: for bootstrap tables whose partition columns are not written into the base files, clustering with the Spark record merger reads rows as UnsafeRow, and injecting the partition-column values calls updateValue, which UnsafeRow does not support, throwing UnsupportedOperationException. This is a child of HUDI-8317 (Spark record / merger gaps).

No fix has merged yet (the target fixVersion 1.1.0 is not yet released). Fix scope touches the row-copy / partition-value injection path in the file group reader, which is larger than a local patch. Tracking as stream:fix (clustering + bootstrap). Keeping open.