Skip to content

HIVE-27473: Rewrite MetaStoreClients to be composable #5771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
892b7ed
HIVE-27473: Rewrite MetaStoreClients to be composable
ngsg Jun 11, 2025
b9d45a5
Fix issues causing test failures
ngsg Jun 12, 2025
7a42f3e
Fix some issues; Extend IMetaStoreClient due to an architectural limi…
ngsg Jun 13, 2025
96e6cfe
Fix remaining test failures
ngsg Jun 16, 2025
5ba1f14
minor
ngsg Jun 17, 2025
d8fbcdc
Resolve some FIXMEs
ngsg Jun 18, 2025
c16f809
Refactor ThriftHiveMetaStoreClient to use NormalizedMetaStoreClient
ngsg Jun 19, 2025
1ed3e6a
minor
ngsg Jun 20, 2025
a34706b
minor
ngsg Jun 20, 2025
904e910
minor
ngsg Jun 20, 2025
362c447
Revise getAggrColStatsFor
ngsg Jun 20, 2025
8aabb11
Address review comments
ngsg Jun 23, 2025
386965f
truncateTempTable should use alterTempTable
ngsg Jun 25, 2025
6e3bb22
Remove some deprecated methods in IMetaStoreClient
ngsg Jun 25, 2025
3d7c64c
Address SonarQube report + minor changes
ngsg Jun 26, 2025
625bff7
minor refactoring
ngsg Jun 26, 2025
9a02f55
minor
ngsg Jun 27, 2025
582659c
Address review comments
ngsg Jun 30, 2025
386c37b
Introduce SynchronizedMetaStoreClientProxy
ngsg Jun 30, 2025
3eb9d73
Drop public getDelegate
ngsg Jun 30, 2025
0b31b6b
fix: RetryingMetaStoreClient requires implements IMetaStoreClient
ngsg Jun 30, 2025
a7a2fd5
Rename: address review comments
ngsg Jul 1, 2025
428fa54
minor fix
ngsg Jul 2, 2025
dfa54e2
address review comments
ngsg Jul 2, 2025
fcea542
minor
ngsg Jul 2, 2025
354f695
Fix failed test cases & restore TestSessionHiveMetastoreClient*
ngsg Jul 3, 2025
4c7ed01
Remove MetaStoreUtils.throwMetaException from base MSC classes
ngsg Jul 4, 2025
e73a680
Revert "Remove MetaStoreUtils.throwMetaException from base MSC classes"
ngsg Jul 5, 2025
25c46a9
Move default alter_partitions & modify getAllMaterializedViewObjectsF…
ngsg Jul 5, 2025
75182f4
minor: remove an unused import
ngsg Jul 5, 2025
591f014
Rename client classes
ngsg Jul 5, 2025
882a5b4
minor
ngsg Jul 6, 2025
76b7dda
Rename MetaStoreClientProxy to MetaStoreClientWrapper
ngsg Jul 6, 2025
7a1f277
Remove hive-exec's SynchronizedMetaStoreClient
ngsg Jul 6, 2025
36f8389
minor
ngsg Jul 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -587,7 +587,8 @@ private void dropPartitionsUsingExpressions(Table table, Map<String, String> par
ExprNodeGenericFuncDesc partitionExpression = new ExpressionBuilder(table, partitionSpec).build();
Pair<Integer, byte[]> serializedPartitionExpression = Pair.of(partitionSpec.size(),
SerializationUtilities.serializeObjectWithTypeInformation(partitionExpression));
hmsClient.dropPartitions(table.getDbName(), table.getTableName(), Arrays.asList(serializedPartitionExpression),
hmsClient.dropPartitions(table.getCatName(), table.getDbName(), table.getTableName(),
Arrays.asList(serializedPartitionExpression),
deleteData && !isExternal(table), // Delete data?
ifExists, // Fail if table doesn't exist?
false); // Need results back?
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hive.common.TableName;
import org.apache.hadoop.hive.metastore.HiveMetaHook;
import org.apache.hadoop.hive.metastore.HiveMetaStoreClient;
import org.apache.hadoop.hive.metastore.PartitionDropOptions;
import org.apache.hadoop.hive.metastore.Warehouse;
import org.apache.hadoop.hive.metastore.api.CreateTableRequest;
Expand All @@ -53,6 +52,7 @@
import org.apache.hadoop.hive.metastore.api.SerDeInfo;
import org.apache.hadoop.hive.metastore.api.StorageDescriptor;
import org.apache.hadoop.hive.metastore.api.hive_metastoreConstants;
import org.apache.hadoop.hive.metastore.client.ThriftHiveMetaStoreClient;
import org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy;
import org.apache.hadoop.hive.metastore.utils.MetaStoreUtils;
import org.apache.hadoop.hive.ql.QueryState;
Expand Down Expand Up @@ -1215,7 +1215,7 @@ public void preDropPartitions(org.apache.hadoop.hive.metastore.api.Table hmsTabl
} catch (IOException e) {
throw new MetaException(String.format("Error while fetching the partitions due to: %s", e));
}
context.putToProperties(HiveMetaStoreClient.SKIP_DROP_PARTITION, "true");
context.putToProperties(ThriftHiveMetaStoreClient.SKIP_DROP_PARTITION, "true");
}

private static void validatePartitionSpec(SearchArgument sarg, PartitionSpec partitionSpec) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -505,7 +505,7 @@ private void withCatalog(String catName, CheckFunc<String> f)
for (String tblName : tblNames) {
client.dropTable(catName, dbName, tblName);
}
client.dropDatabase(catName, dbName);
client.dropDatabase(catName, dbName, true, false, false);
}
client.dropCatalog(catName);
}
Expand Down

This file was deleted.

Loading
Loading