Add the parameter: metastore.drop-partition-ignore-nonexistent to ignore exceptions when a Hive partition does not exist.#7969
Conversation
…ore exceptions when a Hive partition does not exist.
JingsongLi
left a comment
There was a problem hiding this comment.
A few comments on this PR:
-
.gitignorechange: The.cursor-svg-check/entry looks unrelated to this PR — likely IDE-specific. Please remove it or put it into a separate commit. -
Design question: When
MetaExceptionis caught withignorePartitionNonexistent=true, the code callspartitionExistsInHms()which issues a second HMS RPC (getPartition). If HMS is unhealthy (which is likely the cause of the originalMetaException), this verification call may also fail — and the fallback in that case conservatively returnstrue, causing the original exception to be rethrown anyway. This is a reasonable safety choice, but it means the option won't help when HMS is in a bad state.Consider documenting this behavior (i.e., "the option only helps when the specific partition is genuinely absent, not when HMS is having connectivity issues").
-
Title format: The PR title doesn't follow the project convention of
[module] description. Should be something like:[hive] Add metastore.drop-partition-ignore-nonexistent option. -
Option naming: The Paimon project typically uses shorter, more consistent option names.
metastore.drop-partition-ignore-nonexistentis quite long. Have you consideredmetastore.drop-partition.ignore-not-existsto be more consistent with other Paimon options?
Fixed a bug that caused the Partition Expire operation to fail due to a non-existent Hive partition.