Enable Overriding Poll Interval for Kubernetes Jobs #2724
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change simply updates the
KubernetesJobTask
class to enable setting apoll_interval
attribute or property. This allows subclasses to change how often the task polls the Kubernetes server for job status.Motivation and Context
For long-running tasks, polling Kubernetes every 5 seconds for completion wastes resources. If a task author knows that a task is expected to run for many minutes, they can specify a longer interval to poll less frequently.
How I Tested It
I set up a test task by subclassing
KubernetesJobTask
and enabled debug logging. Whenpoll_interval
is not re-defined on the child class, the task polls every 5 seconds for completion. When adding a class attribute forpoll_interval
to the subclass, the task polled according to the new value.