Skip to content

[fluss-common] Validate partition time when creating partition on auto partition table.#2940

Open
loserwang1024 wants to merge 3 commits intoapache:mainfrom
loserwang1024:fix-partition
Open

[fluss-common] Validate partition time when creating partition on auto partition table.#2940
loserwang1024 wants to merge 3 commits intoapache:mainfrom
loserwang1024:fix-partition

Conversation

@loserwang1024
Copy link
Contributor

Purpose

Linked issue: close #2939

Brief change log

Tests

API and Format

Documentation

@loserwang1024
Copy link
Contributor Author

@platinumhamburg @luoyuxia , CC

@loserwang1024 loserwang1024 changed the title [fluss-common] Validate auto-partition time on creation and write. [fluss-common] Validate partition time when creating partition on auto partition table. Mar 26, 2026
ResolvedPartitionSpec resolvedPartitionSpec =
ResolvedPartitionSpec.fromPartitionName(partitionKeys, partitionName);
PartitionSpec partitionSpec = resolvedPartitionSpec.toPartitionSpec();
validateAutoPartitionTime(partitionSpec, partitionKeys, autoPartitionStrategy);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a state leak issue in DynamicPartitionCreator that could cause confusing behavior.

In createPartition(), validateAutoPartitionTime runs after the path has already been added to inflightPartitionsToCreate (line 93). If validation throws InvalidPartitionException, the path stays in the set permanently — admin.createPartition().whenComplete(...) never fires, so onPartitionCreationFailed (which does the remove()) never runs either.

What happens next: any subsequent write with the same invalid partition value hits inflightPartitionsToCreate.add() → returns false → skips creation entirely → tries to send data to a partition that doesn't exist → blows up with a different error. Pretty hard to debug from the user's perspective.

I think the simplest fix is to move the validation before we touch inflightPartitionsToCreate at all — e.g. at the top of checkAndCreatePartitionAsync, right after the null check:

public void checkAndCreatePartitionAsync(
        PhysicalTablePath physicalTablePath,
        List<String> partitionKeys,
        AutoPartitionStrategy autoPartitionStrategy) {
    String partitionName = physicalTablePath.getPartitionName();
    if (partitionName == null) {
        return;
    }

    // Validate early, before touching any state.
    ResolvedPartitionSpec resolvedPartitionSpec =
            ResolvedPartitionSpec.fromPartitionName(partitionKeys, partitionName);
    validateAutoPartitionTime(
            resolvedPartitionSpec.toPartitionSpec(), partitionKeys, autoPartitionStrategy);

    // ... rest of the existing logic
}

This way invalid partitions get rejected immediately without polluting the inflight set, and as a bonus it removes the redundant ResolvedPartitionSpec construction in createPartition.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Need to validate auto-partition time when creation and write new partition

2 participants