Clarify SparkKubernetesOperator.find_spark_job selection logic and intent #61640
+46
−52
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change improves the clarity and intent of
find_spark_joband its tests:find_spark_jobmore explicit about pod selection and prioritization rules.test_find_spark_job_picks_running_podto assert that only non-terminating pods are eligible for selection.test_find_spark_job_picks_pending_podto validate phase-based selection only, without relying ondeletion_timestamp.test_find_spark_job_picks_succeededby removing terminating pods and focusing solely on phase ordering.test_find_spark_job_picks_latest_podby neutralizing pod phases to isolate creation timestamp–based selection.Rationale
find_spark_jobimplements subtle but critical logic used during restarts and retries.This cleanup reduces ambiguity, documents intent directly in tests, and makes the selection logic easier to maintain and review while preserving existing semantics.