BinaryRow::new(0) emits truncated bytes in native DataSplit serialization

[yanbinyang]

Search before asking

• I searched in the issues and found nothing similar.

Please describe the bug 🐞

BinaryRow::new(0).to_serialized_bytes() currently emits only the 4-byte big-endian arity:

00 00 00 00

This is a truncated Paimon BinaryRow. The Java reference implementation initializes BinaryRow.EMPTY_ROW with its 8-byte fixed part, so SerializationUtils.serializeBinaryRow(BinaryRow.EMPTY_ROW) emits 12 bytes:

00 00 00 00 00 00 00 00 00 00 00 00
|-- arity --| |-------- fixed part --------|

The problem is exposed by the native DataSplit codec added in #565. DataSplit::serialize() calls partition.to_serialized_bytes(), while DataSplit::deserialize() calls the strict BinaryRow::from_serialized_bytes() decoder. A non-partitioned split built with the commonly used BinaryRow::new(0) therefore cannot deserialize its own output:

let split = DataSplit::builder()
    .with_snapshot(1)
    .with_partition(BinaryRow::new(0))
    .with_bucket(0)
    .with_bucket_path("file:/tmp/bucket-0".to_string())
    .with_total_buckets(1)
    .with_data_files(vec![])
    .build()
    .unwrap();

let bytes = split.serialize().unwrap();
DataSplit::deserialize(&bytes).unwrap();

The last line fails with:

BinaryRow: serialized body too short for arity 0: 0 bytes, need at least 8

The existing history confirms that the serializer, rather than the decoder, is wrong:

• fix: write valid empty BinaryRow bytes for empty table stats #349 established that a Java-compatible empty BinaryRow contains the 4-byte arity and the 8-byte null bitmap/fixed part, and introduced EMPTY_SERIALIZED_ROW.
• fix: reject truncated BinaryRow serialized bytes instead of panicking #364 correctly made BinaryRow::from_serialized_bytes() reject a truncated body. That validation should not be weakened.
• feat: deserialize DataSplit from Paimon-native bytes (core + C binding) #565 added native DataSplit deserialization and non-empty Java goldens, but does not cover a split whose partition is BinaryRow::new(0).

I also verified the Java side directly with the Maven Central org.apache.paimon:paimon-bundle:1.4.2 artifact:

• SerializationUtils.serializeBinaryRow(BinaryRow.EMPTY_ROW) returns 12 bytes.
• Java DataSplit#serialize / DataSplit#deserialize round-trips an empty-partition split.
• The Java-generated empty-partition DataSplit is 77 bytes.

Expected behavior: serializing the canonical Rust empty row should produce the same 12-byte representation as Java, and an empty-partition native DataSplit should deserialize and reserialize successfully.

Solution

Normalize only the canonical in-memory stub (arity == 0 && data.is_empty()) in BinaryRow::to_serialized_bytes() by returning the existing EMPTY_SERIALIZED_ROW.

This keeps the strict decoder from #364 unchanged, preserves normal populated-row serialization, and restores compatibility with Java without changing the DataSplit wire version.

Add regression tests for:

• the exact 12-byte Java wire representation of BinaryRow::new(0);
• an empty-partition DataSplit against bytes generated by Java Paimon 1.4.2;
• deserialize and byte-stable reserialize of that split.

Are you willing to submit a PR?

• I'm willing to submit a PR!