Add a new neverTerminate job behavior just for upgrade #122643
Conversation
k8s-ci-robot
added
release-note-none
k8s-ci-robot
added
kind/regression
|
/sig apps |
k8s-ci-robot
added
sig/apps
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: soltysh The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
priority/important-longterm
| // this job is being used in an upgrade job see test/e2e/upgrades/apps/job.go | ||
| // it should never be optimized, as it always has to restart during an upgrade | ||
| // and continue running | ||
| job.Spec.Template.Spec.Containers[0].Command = []string{"sleep", "1000000"} |
There was a problem hiding this comment.
non-blocking nit: we could have TerminationGracePeriodSeconds: 1 to make it faster :)
There was a problem hiding this comment.
Great idea, updated.
There was a problem hiding this comment.
You probably need = ptr.To[int64](1)
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: 79190a25a9c2cc7d4dd3c3c30baf96ff244c595a
|
soltysh
force-pushed
the
never_terminate
branch
from
60d19f3
to
e6083d4
Compare
k8s-ci-robot
removed
the
lgtm
soltysh
force-pushed
the
never_terminate
branch
from
e6083d4
to
f8abe71
Compare
k8s-ci-robot
added
size/M
| @@ -70,7 +73,7 @@ func (t *JobUpgradeTest) Teardown(ctx context.Context, f *framework.Framework) { | |||
| // rely on the namespace deletion to clean up everything | |||
| } | |||
|
|
|||
| // ensureAllJobPodsRunning uses c to check in the Job named jobName in ns | |||
| // ensureAllJobPodsRunning uses c to check if the Job named jobName in ns | |||
There was a problem hiding this comment.
I know this is an old function, but I'm wondering how do we know the pods are recreated already (and running)? I'm wondering if we are missing a wait here, but even if so this is a separate issue.
There was a problem hiding this comment.
This is being already handled at the upgrade framework level (see here: https://github.com/kubernetes/kubernetes/blob/master/test/e2e/upgrades/upgrade_suite.go). These jobs here are implementing upgrades.Test interface, so you only focus on the Setup (create the job and make sure it's running) and Test (check the job after the upgrade) steps.
k8s-ci-robot
added
the
lgtm
|
LGTM label has been added. Git tree hash: 8c04d42974a6f18fd47dbef03a5ce31f4cfad7bf
|
What type of PR is this?
/kind failing-test
/kind regression
What this PR does / why we need it:
In #121538 the command for
notTerminatejob behavior was modified, from using asleep 1000000to a pause image. The problem is that this behavior is also used in the upgrade job, to ensure the job continues to run during an upgrade, especially when nodes are being rolled over, ie. terminated and the job controller has to keep the job running for the entire time.Since the goal of that other PR was to optimize executions, I'm adding a new
neverTerminatebehavior which will be used only during upgrade (with a comment explaining why we need it that way).Special notes for your reviewer:
/assign @mimowo @sairameshv
Does this PR introduce a user-facing change?