[MINOR] Make SparkCatalogMetaStoreClient.setMetaConf a no-op#18652
Conversation
HoodieHiveSyncClient invokes IMetaStoreClient.setMetaConf at construction time to forward hive.metastore.callerContext.* properties to the metastore for audit/tracing. With Spark's external catalog there is no remote HMS to receive those values, so throwing UnsupportedOperationException unconditionally breaks every sync client that uses SparkCatalogMetaStoreClient (it fails before any actual catalog operation can run). Accept the call silently. The caller-context properties are diagnostic metadata; dropping them is the correct semantic for a non-thrift catalog backend. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
hudi-agent
left a comment
There was a problem hiding this comment.
🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.
Thanks for the contribution! This PR makes SparkCatalogMetaStoreClient.setMetaConf a no-op so that HoodieHiveSyncClient construction doesn't fail when forwarding diagnostic hive.metastore.callerContext.* properties to a non-thrift Spark catalog backend. No issues flagged from this automated pass — a Hudi committer or PMC member can take it from here for a final review.
cc @yihua
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18652 +/- ##
=========================================
Coverage 68.06% 68.06%
- Complexity 28909 28910 +1
=========================================
Files 2518 2518
Lines 140572 140572
Branches 17422 17422
=========================================
+ Hits 95684 95687 +3
Misses 37032 37032
+ Partials 7856 7853 -3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Describe the issue this Pull Request addresses
Closes #18651
SparkCatalogMetaStoreClient(#18203) throwsUnsupportedOperationExceptionfromsetMetaConf.HoodieHiveSyncClientcallsIMetaStoreClient.setMetaConfunconditionally at construction time (HoodieHiveSyncClient.java:119) to forwardhive.metastore.callerContext.*properties for audit/tracing. Result: any sync client constructed withSparkCatalogMetaStoreClientthrows before any catalog operation can run, blocking the entire reasonSparkCatalogMetaStoreClientwas introduced (avoiding the Hive-on-Spark classloader split during Hive sync).Summary and Changelog
Make
SparkCatalogMetaStoreClient.setMetaConfa no-op.HoodieHiveSyncClient.setMetaConfonly forwardshive.metastore.callerContext.*properties, which are diagnostic metadata sent to a remote thrift HMS. With Spark's external catalog there is no remote HMS to receive them; dropping the call is the correct semantic for a non-thrift catalog backend.Files changed:
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hive/SparkCatalogMetaStoreClient.scala—setMetaConfnow returnsUnitwithout throwing.Other unsupported methods (e.g.
getMetaConf,getTables,getDatabases) continue to throwUnsupportedOperationExceptionbecause they have meaningful semantics that the Spark catalog implementation does not provide. OnlysetMetaConfis changed because:HoodieHiveSyncClient.setMetaConf) only forwards diagnostic context properties.Impact
Unblocks
hoodie.datasource.hive_sync.use_spark_catalog=true. No behavior change for callers that didn't enable this config. No behavior change for the Spark catalog operations themselves (the dropped properties are diagnostic-only, not behavioral).Risk Level
low
Single-method behavior change in a class introduced two months ago (#18203) that is opt-in via a config defaulting to
false. Users with the config disabled see no change. Users with the config enabled were previously unable to use the feature at all, so the only direction the change can go is "from broken to working".Documentation Update
none
Contributor's checklist