Skip to content

[Bug] HMS event listener sync failed for hive catalog #18251

@dutyu

Description

@dutyu

Search before asking

  • I had searched in the issues and found no similar issues.

Version

1.2.1 or higher

What's Wrong?

When creating a hive catalog with automatic update enabled by hms event listener, sometimes we can find the error logs like follows, and hive meta info can not be refreshed.

2023-03-30 15:12:49,340 WARN (org.apache.doris.datasource.hive.event.MetastoreEventsProcessor|34) [PooledHiveMetaStoreClient.getNextNotification():170] Failed to get next notification based on last evv
ent id 0
java.lang.IllegalStateException: Notification events are missing in the meta store.
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getNextNotification(HiveMetaStoreClient.java:2709) ~[hive-standalone-metastore-3.1.3.jar:3.1.3]
        at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_231]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:208) ~[hive-standalone-metastore-3.1.3.jar:3.1.3]
        at com.sun.proxy.$Proxy37.getNextNotification(Unknown Source) ~[?:?]
        at org.apache.doris.datasource.hive.PooledHiveMetaStoreClient.getNextNotification(PooledHiveMetaStoreClient.java:168) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.HMSExternalCatalog.getNextEventResponse(HMSExternalCatalog.java:194) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.hive.event.MetastoreEventsProcessor.getNextHMSEvents(MetastoreEventsProcessor.java:80) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.hive.event.MetastoreEventsProcessor.realRun(MetastoreEventsProcessor.java:132) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.hive.event.MetastoreEventsProcessor.runAfterCatalogReady(MetastoreEventsProcessor.java:117) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]

What You Expected?

Hive meta info should be refreshed when enabling automatic update

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions