[Bug] HMS incremental events have multiple metadata cache consistency gaps

[wenzhenghu]

Search before asking

I searched existing Apache Doris issues and did not find an issue covering these HMS incremental-event cache consistency problems.

Version

Apache Doris master at 3a75387e61388bd886eeef38b794d5ed4eb298bf.

These problems were found while reviewing #65126, but they are already present on the current master baseline and were not introduced by that PR.

What's Wrong?

There are five independent correctness gaps in HMS incremental event handling.

1. ALTER TABLE/ALTER DATABASE rename can be incorrectly cancelled by a load-through "local existence" check

Status: Fixed by the current revision of #65126 (pending merge).

AlterTableEvent.processRename() calls externalTableExistInLocal() before applying the rename:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/event/AlterTableEvent.java

Lines 107 to 123 in 3a75387

	private void processRename() throws DdlException {
	if (!isRename) {
	return;
	}
	boolean hasExist = Env.getCurrentEnv().getCatalogMgr()
	.externalTableExistInLocal(tableAfter.getDbName(), tableAfter.getTableName(), catalogName);
	if (hasExist) {
	logInfo("AlterExternalTable canceled,because tableAfter has exist, "
	+ "catalogName:[{}],dbName:[{}],tableName:[{}]",
	catalogName, dbName, tableAfter.getTableName());
	return;
	}
	Env.getCurrentEnv().getCatalogMgr()
	.unregisterExternalTable(tableBefore.getDbName(), tableBefore.getTableName(), catalogName, true);
	Env.getCurrentEnv().getCatalogMgr()
	.registerExternalTableFromEvent(
	tableAfter.getDbName(), tableAfter.getTableName(), catalogName, eventTime, true);

However, HMSExternalCatalog.tableExistInLocal() eventually calls the normal load-through table lookup rather than a cache-only lookup:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalCatalog.java

Lines 190 to 198 in 3a75387

	@Override
	public boolean tableExistInLocal(String dbName, String tblName) {
	makeSureInitialized();
	HMSExternalDatabase hmsExternalDatabase = (HMSExternalDatabase) getDbNullable(dbName);
	if (hmsExternalDatabase == null) {
	return false;
	}
	return hmsExternalDatabase.getTable(tblName).isPresent();

After HMS has renamed old_table to new_table, querying new_table from this check can load it from HMS and return true. The event is then cancelled, and cached state for old_table may remain. AlterDatabaseEvent.processRename() has the same structural problem through catalog.getDbNullable(dbAfter.getName()), although database rename is less commonly reachable in standard Hive.

The current revision of #65126 removes the load-through target-existence checks and makes both rename handlers converge local state by unregistering the old identity and registering the new identity. Its focused FE unit tests cover cold and already-hot rename targets.

2. ADD/DROP PARTITION events do not fence an in-flight partition-values load, and DROP may skip lower-level cache invalidation

Both addPartitionsCache() and dropPartitionsCache() return immediately when the partition-values entry is absent:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveExternalMetaCache.java

Lines 707 to 746 in 3a75387

	private void addPartitionsCache(NameMapping nameMapping,
	List<String> partitionNames,
	List<Type> partitionColumnTypes) {
	long catalogId = nameMapping.getCtlId();
	MetaCacheEntry<PartitionValueCacheKey, HivePartitionValues> partitionValuesEntry =
	partitionValuesEntryIfInitialized(catalogId);
	if (partitionValuesEntry == null) {
	return;
	}
	PartitionValueCacheKey key = new PartitionValueCacheKey(nameMapping, partitionColumnTypes);
	HivePartitionValues partitionValues = partitionValuesEntry.getIfPresent(key);
	if (partitionValues == null) {
	return;
	}
	HivePartitionValues copy = partitionValues.copy();
	Map<String, PartitionItem> nameToPartitionItem = copy.getNameToPartitionItem();
	Map<String, List<String>> nameToPartitionValues = copy.getNameToPartitionValues();
	HMSExternalCatalog catalog = hmsCatalog(catalogId);
	String localTblName = nameMapping.getLocalTblName();
	for (String partitionName : partitionNames) {
	if (nameToPartitionItem.containsKey(partitionName)) {
	LOG.info("addPartitionsCache partitionName:[{}] has exist in table:[{}]",
	partitionName, localTblName);
	continue;
	}
	ListPartitionItem listPartitionItem = toListPartitionItem(partitionName, key.types, catalog.getName());
	nameToPartitionItem.put(partitionName, listPartitionItem);
	nameToPartitionValues.put(partitionName, HiveUtil.toPartitionValues(partitionName));
	}
	copy.rebuildSortedPartitionRanges();
	HivePartitionValues partitionValuesCur = partitionValuesEntry.getIfPresent(key);
	if (partitionValuesCur == partitionValues) {
	partitionValuesEntry.put(key, copy);
	}
	}

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveExternalMetaCache.java

Lines 748 to 789 in 3a75387

	private void dropPartitionsCache(ExternalTable dorisTable,
	List<String> partitionNames,
	boolean invalidPartitionCache) {
	NameMapping nameMapping = dorisTable.getOrBuildNameMapping();
	long catalogId = nameMapping.getCtlId();
	MetaCacheEntry<PartitionValueCacheKey, HivePartitionValues> partitionValuesEntry =
	partitionValuesEntryIfInitialized(catalogId);
	if (partitionValuesEntry == null) {
	return;
	}
	PartitionValueCacheKey key = new PartitionValueCacheKey(nameMapping, null);
	HivePartitionValues partitionValues = partitionValuesEntry.getIfPresent(key);
	if (partitionValues == null) {
	return;
	}
	HivePartitionValues copy = partitionValues.copy();
	Map<String, PartitionItem> nameToPartitionItem = copy.getNameToPartitionItem();
	Map<String, List<String>> nameToPartitionValues = copy.getNameToPartitionValues();
	for (String partitionName : partitionNames) {
	if (!nameToPartitionItem.containsKey(partitionName)) {
	LOG.info("dropPartitionsCache partitionName:[{}] not exist in table:[{}]",
	partitionName, nameMapping.getFullLocalName());
	continue;
	}
	nameToPartitionItem.remove(partitionName);
	nameToPartitionValues.remove(partitionName);
	if (invalidPartitionCache) {
	invalidatePartitionCache(nameMapping, partitionName);
	}
	}
	copy.rebuildSortedPartitionRanges();
	HivePartitionValues partitionValuesCur = partitionValuesEntry.getIfPresent(key);
	if (partitionValuesCur == partitionValues) {
	partitionValuesEntry.put(key, copy);
	}
	}

This causes two problems:

• If a partition-values load started before the event, the event does not invalidate/bump the entry, so the pre-event snapshot may still be published afterward.
• If the partition-values entry has been evicted while partition metadata or file listings remain cached, a DROP PARTITION event returns before invalidating those lower-level entries.

The second case can expose stale data when a partition name is later recreated with a different path or files. It also prevents the cache-miss fallback added by #65334 from being reached through this DROP event path.

3. Incremental CREATE_DATABASE/CREATE_TABLE events can bypass include/exclude visibility filters

Full names loading applies include_database_list, exclude_database_list, and include_table_list:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java

Lines 551 to 616 in 3a75387

	private List<Pair<String, String>> getFilteredDatabaseNames() {
	List<String> allDatabases = Lists.newArrayList(listDatabaseNames());
	allDatabases.remove(InfoSchemaDb.DATABASE_NAME);
	allDatabases.add(InfoSchemaDb.DATABASE_NAME);
	allDatabases.remove(MysqlDb.DATABASE_NAME);
	allDatabases.add(MysqlDb.DATABASE_NAME);
	Map<String, Boolean> includeDatabaseMap = getIncludeDatabaseMap();
	Map<String, Boolean> excludeDatabaseMap = getExcludeDatabaseMap();
	lowerCaseToDatabaseName.clear();
	List<Pair<String, String>> remoteToLocalPairs = Lists.newArrayList();
	allDatabases = allDatabases.stream().filter(dbName -> {
	if (!dbName.equals(InfoSchemaDb.DATABASE_NAME) && !dbName.equals(MysqlDb.DATABASE_NAME)) {
	// Exclude database map take effect with higher priority over include database map
	if (!excludeDatabaseMap.isEmpty() && excludeDatabaseMap.containsKey(dbName)) {
	return false;
	}
	if (!includeDatabaseMap.isEmpty() && !includeDatabaseMap.containsKey(dbName)) {
	return false;
	}
	}
	return true;
	}).collect(Collectors.toList());
	for (String remoteDbName : allDatabases) {
	String localDbName = fromRemoteDatabaseName(remoteDbName);
	// Populate lowercase mapping for case-insensitive lookups
	lowerCaseToDatabaseName.put(remoteDbName.toLowerCase(), remoteDbName);
	// Apply lower_case_database_names mode to local name
	int dbNameMode = getLowerCaseDatabaseNames();
	if (dbNameMode == 1) {
	localDbName = localDbName.toLowerCase();
	} else if (dbNameMode == 2) {
	// Mode 2: preserve original remote case for display
	localDbName = remoteDbName;
	}
	remoteToLocalPairs.add(Pair.of(remoteDbName, localDbName));
	}
	// Check for conflicts when lower_case_meta_names = true or lower_case_database_names = 2
	if (Boolean.parseBoolean(getLowerCaseMetaNames()) || getLowerCaseDatabaseNames() == 2) {
	// Map to track lowercase local names and their corresponding remote names
	Map<String, List<String>> lowerCaseToRemoteNames = Maps.newHashMap();
	// Collect lowercased local names and their remote counterparts
	for (Pair<String, String> pair : remoteToLocalPairs) {
	String lowerCaseLocalName = pair.second.toLowerCase();
	lowerCaseToRemoteNames.computeIfAbsent(lowerCaseLocalName, k -> Lists.newArrayList()).add(pair.first);
	}
	// Identify conflicts: multiple remote names mapping to the same lowercase local name
	List<String> conflicts = lowerCaseToRemoteNames.values().stream()
	.filter(remoteNames -> remoteNames.size() > 1) // Conflict: more than one remote name
	.flatMap(List::stream) // Collect all conflicting remote names
	.collect(Collectors.toList());
	// Throw exception if conflicts are found
	if (!conflicts.isEmpty()) {
	throw new RuntimeException(String.format(
	FOUND_CONFLICTING + " database names under case-insensitive conditions. "
	+ "Conflicting remote database names: %s in catalog %s. "
	+ "Please use meta_names_mapping to handle name mapping.",
	String.join(", ", conflicts), name));
	}

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java

Lines 343 to 356 in 3a75387

	* @return names of tables in specified database, filtered by include_table_list if configured
	*/
	public final List<String> listTableNames(SessionContext ctx, String dbName) {
	makeSureInitialized();
	Map<String, List<String>> includeTableMap = getIncludeTableMap();
	if (includeTableMap.containsKey(dbName) && !includeTableMap.get(dbName).isEmpty()) {
	if (LOG.isDebugEnabled()) {
	LOG.debug("get table list from include map. catalog: {}, db: {}, tables: {}",
	name, dbName, includeTableMap.get(dbName));
	}
	return includeTableMap.get(dbName);
	}
	return listTableNamesFromRemote(ctx, dbName);
	}

The incremental register paths directly update the legacy metadata cache without applying the same visibility policy:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalCatalog.java

Lines 205 to 216 in 3a75387

	@Override
	public void registerDatabase(long dbId, String dbName) {
	if (LOG.isDebugEnabled()) {
	LOG.debug("create database [{}]", dbName);
	}
	ExternalDatabase<? extends ExternalTable> db = buildDbForInit(dbName, null, dbId, logType, false);
	if (isInitialized()) {
	metaCache.updateCache(db.getRemoteName(), db.getFullName(), db,
	Util.genIdByName(name, db.getFullName()));
	}

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalDatabase.java

Lines 649 to 663 in 3a75387

	// Only used for sync hive metastore event
	@Override
	public boolean registerTable(TableIf tableIf) {
	makeSureInitialized();
	String tableName = tableIf.getName();
	if (LOG.isDebugEnabled()) {
	LOG.debug("create table [{}]", tableName);
	}
	if (isInitialized()) {
	String localName = extCatalog.fromRemoteTableName(this.remoteName, tableName);
	metaCache.updateCache(tableName, localName, (T) tableIf,
	Util.genIdByName(extCatalog.getName(), name, localName));
	lowerCaseToTableName.put(tableName.toLowerCase(), tableName);
	}
	setLastUpdateTime(System.currentTimeMillis());

When a names cache is already hot, an HMS CREATE event can therefore add a database/table that should remain hidden. The object can subsequently become queryable through the cached name.

4. DROP_PARTITION records a DATABASE deletion in ExternalMetaIdMgr

DropPartitionEvent.transferToMetaIdMappings() uses META_OBJECT_TYPE_DATABASE instead of META_OBJECT_TYPE_PARTITION:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/event/DropPartitionEvent.java

Lines 142 to 152 in 3a75387

	@Override
	protected List<MetaIdMappingsLog.MetaIdMapping> transferToMetaIdMappings() {
	List<MetaIdMappingsLog.MetaIdMapping> metaIdMappings = Lists.newArrayList();
	for (String partitionName : this.getAllPartitionNames()) {
	MetaIdMappingsLog.MetaIdMapping metaIdMapping = new MetaIdMappingsLog.MetaIdMapping(
	MetaIdMappingsLog.OPERATION_TYPE_DELETE,
	MetaIdMappingsLog.META_OBJECT_TYPE_DATABASE,
	dbName, tblName, partitionName);
	metaIdMappings.add(metaIdMapping);
	}
	return ImmutableList.copyOf(metaIdMappings);

ExternalMetaIdMgr interprets this as a request to remove the entire database mapping subtree:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalMetaIdMgr.java

Lines 145 to 169 in 3a75387

	private static void handleDelMetaIdMapping(MetaIdMappingsLog.MetaIdMapping mapping,
	CtlMetaIdMgr ctlMetaIdMgr,
	MetaIdMappingsLog.MetaObjectType objType) {
	TblMetaIdMgr tblMetaIdMgr;
	DbMetaIdMgr dbMetaIdMgr;
	switch (objType) {
	case DATABASE:
	ctlMetaIdMgr.dbNameToMgr.remove(mapping.getDbName());
	break;
	case TABLE:
	dbMetaIdMgr = ctlMetaIdMgr.dbNameToMgr.get(mapping.getDbName());
	if (dbMetaIdMgr != null) {
	dbMetaIdMgr.tblNameToMgr.remove(mapping.getTblName());
	}
	break;
	case PARTITION:
	dbMetaIdMgr = ctlMetaIdMgr.dbNameToMgr.get(mapping.getDbName());
	if (dbMetaIdMgr != null) {
	tblMetaIdMgr = dbMetaIdMgr.tblNameToMgr.get(mapping.getTblName());
	if (tblMetaIdMgr != null) {
	tblMetaIdMgr.partitionNameToMgr.remove(mapping.getPartitionName());
	}
	}

Current production use of the ExternalMetaIdMgr lookup APIs is limited, so the immediate query impact appears limited, but the persisted/replayed external metadata ID state is incorrect.

5. ALTER_DATABASE rename can generate an ID from the remote name instead of the canonical local name

Status: Open. This is present on the master baseline and is not introduced by #65126.

CatalogMgr.registerExternalDatabaseFromEvent() generates the event-created database ID directly from the incoming remote dbName:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java

Lines 772 to 789 in 54e0e97

	public void registerExternalDatabaseFromEvent(String dbName, String catalogName)
	throws DdlException {
	CatalogIf catalog = nameToCatalog.get(catalogName);
	if (catalog == null) {
	throw new DdlException("No catalog found with name: " + catalogName);
	}
	if (!(catalog instanceof ExternalCatalog)) {
	throw new DdlException("Only support create ExternalCatalog databases");
	}
	HMSExternalCatalog hmsCatalog = (HMSExternalCatalog) catalog;
	long dbId = Util.genIdByName(catalogName, dbName);
	// -1L means it will be dropped later, ignore
	if (dbId == ExternalMetaIdMgr.META_ID_FOR_NOT_EXISTS) {
	return;
	}
	hmsCatalog.registerDatabase(dbId, dbName);

Normal external-database object loading instead generates the ID from the canonical localDbName:

doris/fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java

Lines 459 to 466 in 54e0e97

	OptionalLong.of(Config.external_cache_expire_time_seconds_after_access),
	OptionalLong.of(Config.external_cache_refresh_time_minutes * 60L),
	Math.max(Config.max_meta_object_cache_num, 1),
	ignored -> getFilteredDatabaseNames(),
	localDbName -> Optional.ofNullable(
	buildDbForInit(null, localDbName, Util.genIdByName(name, localDbName), logType,
	true)),
	(key, value, cause) -> value.ifPresent(v -> v.resetMetaToUninitialized()));

When lower_case_database_names = 1, or more generally whenever the incoming remote name differs from its canonical local cache key, the same database can therefore have two deterministic IDs:

• Immediately after the event: ID(remote name) -> canonical local name.
• After cache reset or normal reload: ID(canonical local name) -> canonical local name.

This makes the database ID and the local ID-to-name index unstable across refresh/reload and can break ID-based lookup consistency.

What You Expected?

• Rename events should inspect only locally published cache state and should always remove the old local name. This is fixed by the current revision of [refactor](fe) Refactor external metadata cache with MetaCacheEntry #65126.
• Partition events should prevent pre-event snapshots from being published and should invalidate partition/file caches even when partition-values cache state is cold.
• Incremental create events should apply the same visibility rules as full names loading.
• DROP_PARTITION should delete only the corresponding partition metadata-ID mapping.
• Incremental database registration should derive the database ID from the same canonical local name used by normal object loading.

How to Reproduce?

Rename cancellation (fixed by the current revision of #65126)

1. Enable HMS incremental event synchronization.
2. Ensure the target name is not locally cached.
3. Rename old_table to new_table in Hive.
4. Process the ALTER_TABLE event.
5. On the affected baseline, the existence check loads new_table from HMS and cancels the event; cached old_table state may remain.

Partition cache

1. Cache partition metadata/file listings for p=1.
2. Evict or invalidate only the table's partition-values entry, or block an in-flight partition-values load.
3. Drop p=1 through HMS and process the DROP_PARTITION event.
4. Recreate p=1 with a different location/files.
5. The old partition metadata or file listing may still be reused.

Visibility filter

1. Create an HMS catalog with an include/exclude database filter or include_table_list.
2. Warm the corresponding names cache.
3. Create a filtered-out database/table directly in Hive.
4. Process the HMS CREATE event.
5. Observe that the filtered object is inserted into the cached names and becomes visible/queryable.

External metadata ID mapping

1. Add a database/table/partition mapping to ExternalMetaIdMgr.
2. Process the partition's DROP_PARTITION mapping.
3. Observe that the database mapping subtree is removed instead of only the partition mapping.

Database ID stability

1. Configure an HMS catalog with lower_case_database_names = 1.
2. Process an ALTER_DATABASE rename event whose new remote name contains uppercase characters.
3. Observe that registerExternalDatabaseFromEvent() generates the new ID from the remote mixed-case name while the local cache key is lowercase.
4. Reset or reload the catalog so the normal database-object loader runs.
5. Observe that the reloaded database ID is generated from the lowercase local name and differs from the event-created ID.

Suggested Fixes

1. Fixed in the current revision of [refactor](fe) Refactor external metadata cache with MetaCacheEntry #65126: remove the load-through rename target-existence checks and always converge the old/new local identities.
2. For ADD/DROP PARTITION, invalidate or bump the partition-values key when it is cold/loading. DROP should unconditionally invalidate partition metadata/file cache entries for each dropped partition before optionally updating a hot partition-values snapshot.
3. Reuse a shared database/table visibility predicate in both full names loading and incremental registration.
4. Change the DROP_PARTITION mapping type to META_OBJECT_TYPE_PARTITION and add an event mapping unit test.
5. Resolve the canonical local database name with a cache-only helper, and use it consistently for event registration, invalidation, and Util.genIdByName(). Add mode-1 coverage that asserts the event-created ID equals the normal reload ID.

The remaining four open fixes can be implemented as separate PRs if preferred.

[zclllyybb]

Breakwater-GitHub-Analysis-Slot: slot_8fa76c90e7f5
This content is generated by AI for reference only.

Initial triage: I re-read the live issue and checked the referenced code at 3a75387e61388bd886eeef38b794d5ed4eb298bf. I also refreshed upstream/master to 9a5afd4958f2794a2dd458b2e7463315e9746ce3; the relevant files have no diff from the issue commit, so these observations still apply to the current master snapshot I inspected.

Judgment:

• The report looks valid and is best handled as four independent FE HMS incremental-event cache consistency bugs.
• I did not find evidence in the code path that makes any of the four cases intentionally safe.

Code-backed details:

1. Rename cancellation path: confirmed. AlterTableEvent.processRename() calls externalTableExistInLocal(tableAfter...); for HMS, this reaches HMSExternalCatalog.tableExistInLocal(), which calls getDbNullable() and hmsExternalDatabase.getTable(tblName). ExternalCatalog.getDbNullable() and ExternalDatabase.getTableNullable() both use metaCache.getMetaObj(...), and MetaCache.getMetaObj() loads on miss. That makes the check not cache-only. If HMS already exposes the renamed target, the check can materialize the new table and return true; processRename() then returns before unregistering tableBefore, so the old local name can remain. AlterDatabaseEvent.processRename() has the analogous risk because catalog.getDbNullable(dbAfter.getName()) is also load-through; getDbForReplay(...) shows there is already a cache-only pattern available for databases.

2. Partition events: confirmed, with a concurrency nuance. MetaCacheEntry has generation-based protection against stale in-flight manual loads, but only when an invalidation actually happens. In HiveExternalMetaCache.PartitionCacheCoordinator.addPartitionsCache() and dropPartitionsCache(), the code returns when the partition-values entry/key is not present, so no generation is bumped for an in-flight partition-values load. In dropPartitionsCache(), this early return also prevents invalidatePartitionCache(...) from running, so stale partition metadata and file-listing entries can survive after a DROP. The lower-level invalidation helper can already fall back to table-id + partition-values matching when the exact partition metadata is cold, but the current DROP event path can skip calling that helper entirely.

3. Visibility filters: confirmed. Full names loading applies include_database_list, exclude_database_list, and include_table_list through ExternalCatalog.getFilteredDatabaseNames() and ExternalCatalog.listTableNames(). Incremental CREATE paths bypass those predicates: CreateDatabaseEvent -> CatalogMgr.registerExternalDatabaseFromEvent() -> HMSExternalCatalog.registerDatabase() updates the catalog meta cache directly, and CreateTableEvent -> CatalogMgr.registerExternalTableFromEvent() -> ExternalDatabase.registerTable() updates the DB meta cache and lowerCaseToTableName directly. If the names cache is hot, a filtered object can be inserted into the visible cached names.

4. ExternalMetaIdMgr mapping: confirmed. DropPartitionEvent.transferToMetaIdMappings() emits META_OBJECT_TYPE_DATABASE for a partition delete. ExternalMetaIdMgr.handleDelMetaIdMapping() interprets DATABASE delete by removing the whole dbNameToMgr entry, while PARTITION delete would only remove partitionNameToMgr under the table. This is a direct type mismatch.

Missing information / validation needed:

• For (1), a small HMS rename regression test should assert that the target-name existence probe is cache-only and that the old local table/db name is always removed.
• For (2), a regression/unit test should cover a DROP with a hot lower-level partition/file cache but a cold partition-values cache, plus an in-flight partition-values load where the event occurs before the load publishes.
• For (3), tests should cover both database and table include/exclude filters with a pre-warmed names cache before processing CREATE_DATABASE / CREATE_TABLE events.
• For (4), an event mapping replay test should verify DROP_PARTITION deletes only the partition mapping and preserves sibling table/db mappings.

Suggested next steps:

• Split into separate PRs if that is easier to review: rename cache-only existence, partition event fencing/invalidation, shared visibility predicates for incremental register paths, and the DROP_PARTITION meta-id type fix.
• Prefer sharing the same database/table visibility helper between full-list loading and event registration, so incremental events cannot drift from catalog-list semantics.
• For partition events, make DROP invalidate partition metadata/file cache entries before attempting the hot partition-values snapshot update, and make ADD/DROP invalidate or generation-bump the partition-values key when the entry exists but the value is cold/loading.
• For rename, use cache-only lookup APIs similar to getDbForReplay / getTableForReplay for local-existence checks, and avoid returning before removing the old name.

[wenzhenghu]

Update: item 1 in this issue is fixed by the current FE patch set.

What changed:

• AlterTableEvent.processRename() no longer short-circuits on target-name existence. It now always converges local state by unregistering the old table name and registering the new table name.
• AlterDatabaseEvent.processRename() now follows the same converge-state pattern for database rename.
• The old tableExistInLocal() helper chain has been removed entirely because it no longer has any production callers.

Why this closes item 1:

• The original bug was not only the load-through local-existence check itself, but also the broader race where the target name could already be hot before the HMS rename event arrived.
• The new logic makes rename handling idempotent and cache-state convergent, so both cases are covered: the target may be cold or already hot, but the old local name/object/id mapping and engine cache are still cleaned up.

Validation:

• Added FE event-level tests covering:
  • table rename with cold target
  • table rename with already-hot target
  • database rename with cold target
  • database rename with already-hot target
• The cold-target tests also assert that the event publishes the new name/id mapping without warming the cold object cache.

Status of the remaining items in this issue:

• Item 2: not addressed by this patch set.
• Item 3: still open; incremental CREATE_DATABASE/CREATE_TABLE visibility filtering still needs a separate fix.
• Item 4: not addressed by this patch set.

So this issue should remain open, but item 1 can be considered resolved.