Skip to content

Support configurable time partition format for auto-partitioned tables #3191

@luoyuxia

Description

@luoyuxia

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

Fluss currently fixes the partition value format based on table.auto-partition.time-unit. For example, DAY uses yyyyMMdd, HOUR uses yyyyMMddHH, and users cannot configure another format.

This becomes awkward when Fluss is used together with Paimon. Paimon commonly uses yyyy-MM-dd style date partitions by default, while Fluss expects yyyyMMdd. As a result, users need extra conversion logic or have to accept inconsistent partition naming between Fluss and Paimon.

Making the time partition format configurable would improve interoperability and reduce friction for lake integration scenarios, especially when users want Fluss and Paimon to use the same partition naming convention.

Solution

Add a table option for configuring the time partition formatter, for example table.auto-partition.time-format or similar.

Expected behavior:

  • Keep the current hard-coded formats as the default to preserve backward compatibility.
  • Allow users to override the formatter when using time-based auto partitioning, for example yyyy-MM-dd for DAY partitions.
  • Use the configured formatter consistently for partition generation, validation, retention, and pre-creation logic.
  • Document the supported formatter patterns and any compatibility constraints with different time units.

Anything else?

There is already related formatter support on the Paimon side, such as paimon.partition.timestamp-formatter, so exposing a similar capability in Fluss would make Fluss-Paimon integration more natural.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions