-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-46186][CONNECT][TESTS][FOLLOWUP] Remove flakiness of ReattachableExecuteSuite
#44189
[SPARK-46186][CONNECT][TESTS][FOLLOWUP] Remove flakiness of ReattachableExecuteSuite
#44189
Conversation
// make sure the interrupt reaches the server | ||
Eventually.eventually(timeout(eventuallyTimeout)) { | ||
val execution = SparkConnectService.executionManager.listActiveExecutions match { | ||
case Right(list) => list.find(_.operationId == operationId) | ||
case Left(_) => None | ||
} | ||
assert(execution.isDefined && execution.get.status == ExecuteStatus.Canceled) | ||
} | ||
// make sure the client gets the OPERATION_CANCELED error | ||
val e = intercept[StatusRuntimeException] { | ||
while (iter.hasNext) iter.next() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While raising PR I realize this is not needed, because the sleep
query should ensure that we don't get results to iterate through for 30 seconds. By this time, interrupt should reach the server.
Will remove and test for flakiness again.
ReattachableExecuteSuite
Thank you, @juliuszsompolski . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you the following?
Error: /home/runner/work/apache-spark/apache-spark/connector/connect/server/src/test/scala/org/apache/spark/sql/connect/execution/ReattachableExecuteSuite.scala:29: Unused import
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM.
Thank you, @juliuszsompolski . |
…ableExecuteSuite` ### What changes were proposed in this pull request? The test added in apache#44095 could be flaky because `MEDIUM_RESULTS_QUERY` could very quickly finish before interrupt was sent. Replace it with a query that sleeps 30 seconds, so that we are sure that interrupt runs before it finishes. ### Why are the changes needed? Remove test flakiness. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Rerun ReattachableExecuteSuite 100+ times to check it isn't flaky. ### Was this patch authored or co-authored using generative AI tooling? Github Copilot was assisting in some boilerplate auto-completion. Generated-by: Github Copilot Closes apache#44189 from juliuszsompolski/SPARK-46186-followup. Authored-by: Juliusz Sompolski <julek@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
What changes were proposed in this pull request?
The test added in #44095 could be flaky because
MEDIUM_RESULTS_QUERY
could very quickly finish before interrupt was sent. Replace it with a query that sleeps 30 seconds, so that we are sure that interrupt runs before it finishes.Why are the changes needed?
Remove test flakiness.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Rerun ReattachableExecuteSuite 100+ times to check it isn't flaky.
Was this patch authored or co-authored using generative AI tooling?
Github Copilot was assisting in some boilerplate auto-completion.
Generated-by: Github Copilot