[fluss-common] Validate partition time when creating partition on auto partition table.#2940
[fluss-common] Validate partition time when creating partition on auto partition table.#2940loserwang1024 wants to merge 3 commits intoapache:mainfrom
Conversation
5e85506 to
7d298df
Compare
| ResolvedPartitionSpec resolvedPartitionSpec = | ||
| ResolvedPartitionSpec.fromPartitionName(partitionKeys, partitionName); | ||
| PartitionSpec partitionSpec = resolvedPartitionSpec.toPartitionSpec(); | ||
| validateAutoPartitionTime(partitionSpec, partitionKeys, autoPartitionStrategy); |
There was a problem hiding this comment.
I think there's a state leak issue in DynamicPartitionCreator that could cause confusing behavior.
In createPartition(), validateAutoPartitionTime runs after the path has already been added to inflightPartitionsToCreate (line 93). If validation throws InvalidPartitionException, the path stays in the set permanently — admin.createPartition().whenComplete(...) never fires, so onPartitionCreationFailed (which does the remove()) never runs either.
What happens next: any subsequent write with the same invalid partition value hits inflightPartitionsToCreate.add() → returns false → skips creation entirely → tries to send data to a partition that doesn't exist → blows up with a different error. Pretty hard to debug from the user's perspective.
I think the simplest fix is to move the validation before we touch inflightPartitionsToCreate at all — e.g. at the top of checkAndCreatePartitionAsync, right after the null check:
public void checkAndCreatePartitionAsync(
PhysicalTablePath physicalTablePath,
List<String> partitionKeys,
AutoPartitionStrategy autoPartitionStrategy) {
String partitionName = physicalTablePath.getPartitionName();
if (partitionName == null) {
return;
}
// Validate early, before touching any state.
ResolvedPartitionSpec resolvedPartitionSpec =
ResolvedPartitionSpec.fromPartitionName(partitionKeys, partitionName);
validateAutoPartitionTime(
resolvedPartitionSpec.toPartitionSpec(), partitionKeys, autoPartitionStrategy);
// ... rest of the existing logic
}This way invalid partitions get rejected immediately without polluting the inflight set, and as a bonus it removes the redundant ResolvedPartitionSpec construction in createPartition.
06381ad to
4d7e783
Compare
Purpose
Linked issue: close #2939
Brief change log
Tests
API and Format
Documentation