Parent Issue
Part of #124 (support partitioned table)
Depends on #126 (BinaryRow deserialization)
Background
Paimon stores data files under partition directories like {table_path}/dt=2024-01-01/hr=12/bucket-0/. Currently there is no code to convert partition values (from BinaryRow) into this filesystem path format.
What needs to be done
-
Create a partition path utility (e.g., spec/partition_utils.rs or similar)
- Given partition keys (names from
TableSchema.partition_keys()) + partition field types (from TableSchema.fields()) + partition values (decoded BinaryRow), generate the partition path string
- Format:
{key1}={value1}/{key2}={value2}/...
-
Handle value formatting per data type
Int/BigInt/SmallInt/TinyInt → integer string
VarChar/Char/String → string value
Date → yyyy-MM-dd format (days since epoch)
Timestamp → appropriate timestamp string
Boolean → true/false
Float/Double → decimal string
Decimal → decimal string
-
Handle special cases
- Null partition values → use
__DEFAULT_PARTITION__
- Special characters in string values → URL encoding if needed (follow Java Paimon's behavior)
-
Unit tests
- Single partition key
- Multiple partition keys
- Different data types (string, int, date)
- Null partition value →
__DEFAULT_PARTITION__
Reference
Affected files
- New file:
crates/paimon/src/spec/partition_utils.rs (or integrated into existing module)
Parent Issue
Part of #124 (support partitioned table)
Depends on #126 (BinaryRow deserialization)
Background
Paimon stores data files under partition directories like
{table_path}/dt=2024-01-01/hr=12/bucket-0/. Currently there is no code to convert partition values (fromBinaryRow) into this filesystem path format.What needs to be done
Create a partition path utility (e.g.,
spec/partition_utils.rsor similar)TableSchema.partition_keys()) + partition field types (fromTableSchema.fields()) + partition values (decodedBinaryRow), generate the partition path string{key1}={value1}/{key2}={value2}/...Handle value formatting per data type
Int/BigInt/SmallInt/TinyInt→ integer stringVarChar/Char/String→ string valueDate→yyyy-MM-ddformat (days since epoch)Timestamp→ appropriate timestamp stringBoolean→true/falseFloat/Double→ decimal stringDecimal→ decimal stringHandle special cases
__DEFAULT_PARTITION__Unit tests
__DEFAULT_PARTITION__Reference
PartitionPathUtilsAffected files
crates/paimon/src/spec/partition_utils.rs(or integrated into existing module)