Skip to content

HIVE-29066: PARTITION_NAME_WHITELIST_PATTERN is not honouring session level metaconf #5943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

HIVE-29066: PARTITION_NAME_WHITELIST_PATTERN is not honouring session level metaconf #5943

wants to merge 2 commits into from
+111 −135

Conversation

Aggarwal-Raghav
Copy link
Contributor

What changes were proposed in this pull request?

Check HIVE-29066, Ensure that the partitionValidationPattern is picked from the metaconf instead of HMSHandler Object.

Why are the changes needed?

To honour, modified session level metaconf.

Does this PR introduce any user-facing change?

NO

How was this patch tested?

q file is attached.

@Aggarwal-Raghav Aggarwal-Raghav mentioned this pull request Jul 8, 2025
Closed
@Aggarwal-Raghav
Copy link
Contributor Author

The test failure is flaky as reported in https://issues.apache.org/jira/browse/HIVE-29061, will re-trigger if there are any review commets.

@Aggarwal-Raghav
Copy link
Contributor Author

Can you please help with the review @deniskuzZ @okumin ?

deniskuzZ
deniskuzZ reviewed Jul 10, 2025
View reviewed changes
...ne-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java Outdated
@@ -525,6 +520,15 @@ public String getMetaConf(String key) throws MetaException {
return getConf().get(key, confVar.getDefaultVal().toString());
}

private Pattern getPartitionValidationRegex() {
Copy link
Member

@deniskuzZ deniskuzZ Jul 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move it to MetaStoreServerUtils#validatePartitionNameCharacters

MetaStoreServerUtils.validatePartitionNameCharacters(part.getValues(), conf)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deniskuzZ , can you please check if the latest commit is what you meant by moving it to MetaStoreServerUtils?

@Aggarwal-Raghav
Copy link
Contributor Author

Also, in HS2 DynamicPartitionCtx#whiteListPattern field exist per object. I am seeing the issue in cluster as well.
Testing (when setting the conf in beeline session)
Screenshot 2025-07-12 at 8 26 10 PM

(when i set the config in hive-site.xml)
Screenshot 2025-07-12 at 8 27 52 PM

Unfortunately, its not happening in q file. Can you suggest should I make similar changes in DynamicPartitionCtx and org/apache/hadoop/hive/ql/exec/FileSinkOperator.java:1129?

@Aggarwal-Raghav
Copy link
Contributor Author

The reason why session level hive.metastore.partition.name.whitelist.pattern is because of this line:

hive/ql/src/java/org/apache/hadoop/hive/ql/plan/DynamicPartitionCtx.java

Line 129 in f2fdea0

confVal = Hive.get().getMetaConf(ConfVars.PARTITION_NAME_WHITELIST_PATTERN.getHiveName());

If we are using hive.metastore.partition.name.whitelist.pattern then it should look in:

confVal = Hive.get().getConf().get(ConfVars.PARTITION_NAME_WHITELIST_PATTERN.getHiveName());

instead of getMetaConf

In q files, as HMS runs is embedded the conf object "might be" shared.

@Aggarwal-Raghav Aggarwal-Raghav changed the title HIVE-29066: PARTITION_NAME_WHITELIST_PATTERN is not honouring session level metaconf [WIP] HIVE-29066: PARTITION_NAME_WHITELIST_PATTERN is not honouring session level metaconf Jul 13, 2025
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@Aggarwal-Raghav Aggarwal-Raghav changed the title [WIP] HIVE-29066: PARTITION_NAME_WHITELIST_PATTERN is not honouring session level metaconf HIVE-29066: PARTITION_NAME_WHITELIST_PATTERN is not honouring session level metaconf Jul 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@deniskuzZ deniskuzZ deniskuzZ left review comments

Assignees
No one assigned
Labels
tests passed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Aggarwal-Raghav @deniskuzZ @asf-ci-hive