[Bug] Not graceful exit when engine idle

[cxzl25]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the bug

It is not graceful to exit when engine is idle, which may cause am to try again.
The current workaround is to configure spark.yarn.maxAppAttempts.

21/11/30 18:35:00 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 16, (reason: Shutdown hook called before final status was reported.)

Affects Version(s)

master

Kyuubi Server Log Output

No response

Kyuubi Engine Log Output

21/11/30 18:34:50 INFO session.SparkSQLSessionManager: Idled for more than 1800000 ms, terminating
21/11/30 18:34:50 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4a134605{/kyuubi,null,STOPPED,@Spark}
21/11/30 18:34:50 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@6d44ce3f{/kyuubi/json,null,STOPPED,@Spark}
21/11/30 18:34:50 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@671a6a5{/kyuubi/session,null,STOPPED,@Spark}
21/11/30 18:34:50 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@39db6d9f{/kyuubi/session/json,null,STOPPED,@Spark}
21/11/30 18:34:50 INFO spark.SparkSQLEngine: Service: [SparkThriftBinaryFrontendService] is stopping.
21/11/30 18:34:50 INFO spark.SparkThriftBinaryFrontendService: SparkThriftBinaryFrontendService has stopped
21/11/30 18:34:50 INFO spark.SparkThriftBinaryFrontendService: Service: [EngineServiceDiscovery] is stopping.

21/11/30 18:34:50 INFO client.EngineServiceDiscovery: Service[EngineServiceDiscovery] is stopped.
21/11/30 18:34:50 INFO spark.SparkThriftBinaryFrontendService: Service[SparkThriftBinaryFrontendService] is stopped.
21/11/30 18:34:50 INFO spark.SparkSQLEngine: Service: [SparkSQLBackendService] is stopping.
21/11/30 18:34:50 INFO spark.SparkSQLBackendService: Service: [SparkSQLSessionManager] is stopping.
21/11/30 18:34:50 INFO session.SparkSQLSessionManager: Service: [SparkSQLOperationManager] is stopping.
21/11/30 18:34:50 INFO operation.SparkSQLOperationManager: Service[SparkSQLOperationManager] is stopped.
21/11/30 18:34:50 INFO session.SparkSQLSessionManager: Service[SparkSQLSessionManager] is stopped.
21/11/30 18:34:50 WARN client.EngineServiceDiscovery: Service[EngineServiceDiscovery] is not started(STOPPED) yet.
21/11/30 18:35:00 INFO spark.SparkSQLBackendService: Service[SparkSQLBackendService] is stopped.
21/11/30 18:35:00 INFO spark.SparkSQLEngine: Service[SparkSQLEngine] is stopped.
21/11/30 18:35:00 INFO events.EventLoggingService: Service[EventLogging] is stopped.
21/11/30 18:35:00 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x57c736b4fff0649
21/11/30 18:35:00 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 16, (reason: Shutdown hook called before final status was reported.)
21/11/30 18:35:00 INFO storage.DiskBlockManager: Shutdown hook called

Kyuubi Server Configurations

No response

Kyuubi Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!