Skip to content

Commit 0857786

Browse files
committed
[KYUUBI #3065] Support to retry the killApplicationByTag for JpsApplicationOperation
### _Why are the changes needed?_ If using jps application manager, for spark job, it might generate two processes. And the first one is only for spark submit args parser and is transient, so if we try to kill the app by tag, it might say `No such process` and the spark main process is still running. So, for JpsApplicationOperation, we need to retry the kill app operation to prevent flaky test. ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [x] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request Closes #3065 from turboFei/jps_appmgr. Closes #3065 3d7503d [Fei Wang] thanks 7b25965 [Fei Wang] if else 9133e80 [Fei Wang] Retry the kill for one time Authored-by: Fei Wang <fwang12@ebay.com> Signed-off-by: Fei Wang <fwang12@ebay.com>
1 parent ce72a50 commit 0857786

File tree

1 file changed

+11
-2
lines changed

1 file changed

+11
-2
lines changed

kyuubi-server/src/main/scala/org/apache/kyuubi/engine/JpsApplicationOperation.scala

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ class JpsApplicationOperation extends ApplicationOperation {
5858
}
5959
}
6060

61-
override def killApplicationByTag(tag: String): KillResponse = {
61+
private def killJpsApplicationByTag(tag: String, retryable: Boolean): KillResponse = {
6262
val commandOption = getEngine(tag)
6363
if (commandOption.nonEmpty) {
6464
val idAndCmd = commandOption.get
@@ -68,13 +68,22 @@ class JpsApplicationOperation extends ApplicationOperation {
6868
(true, s"Succeeded to terminate: $idAndCmd")
6969
} catch {
7070
case e: Exception =>
71-
(false, s"Failed to terminate: $idAndCmd, due to ${e.getMessage}")
71+
// the application might generate multiple processes, ensure that it is killed eventually.
72+
if (retryable && getEngine(tag).nonEmpty) {
73+
killJpsApplicationByTag(tag, false)
74+
} else {
75+
(false, s"Failed to terminate: $idAndCmd, due to ${e.getMessage}")
76+
}
7277
}
7378
} else {
7479
(false, NOT_FOUND)
7580
}
7681
}
7782

83+
override def killApplicationByTag(tag: String): KillResponse = {
84+
killJpsApplicationByTag(tag, true)
85+
}
86+
7887
override def getApplicationInfoByTag(tag: String): Map[String, String] = {
7988
val commandOption = getEngine(tag)
8089
if (commandOption.nonEmpty) {

0 commit comments

Comments
 (0)