Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[virt-operator] cope with misscheduled virt-handler pods #10254

Merged
merged 1 commit into from Aug 10, 2023

Conversation

rmohr
Copy link
Member

@rmohr rmohr commented Aug 7, 2023

What this PR does / why we need it:

Applying a custom NoSchedule taint to a nod will flip the Available condition of KubeVirt to False:

  - lastProbeTime: "2023-08-07T14:15:43Z"
    lastTransitionTime: "2023-08-07T14:15:43Z"
    message: Deploying version devel with registry registry:5000/kubevirt
    reason: DeploymentInProgress
    status: "False"
    type: Available

This will only resolve back to True if the pod gets manually evicted, or if the daemonset gets updated.

The background is, that we will see more up-to-date virt-handler pods in ready state than we actually want.

Relax the readiness check slightly by counting misscheduled but up-to-date and ready virt-handlers as something which does not trigger a DeploymentInProgress state.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Release note:

Don't mark the KubeVirt "Available" condition as false on up-to-date and ready but misscheduled virt-handler pods.

@kubevirt-bot kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Aug 7, 2023
@rmohr
Copy link
Member Author

rmohr commented Aug 7, 2023

/cc @acardace
/cc @davidvossel

Applying a custom `NoSchedule` taint to a nod will flip the `Available`
condition of KubeVirt to `False`:

```
  - lastProbeTime: "2023-08-07T14:15:43Z"
    lastTransitionTime: "2023-08-07T14:15:43Z"
    message: Deploying version devel with registry registry:5000/kubevirt
    reason: DeploymentInProgress
    status: "False"
    type: Available
```

This will only resolve back to `True` if the pod gets manually evicted,
or if the daemonset gets updated.

The background is, that we will see more up-to-date virt-handler pods in
ready state than we actually want.

Relax the readiness check slightly by counting misscheduled but
up-to-date and ready virt-handlers as something which does not trigger a
`DeploymentInProgress` state.

Signed-off-by: Roman Mohr <rmohr@google.com>
@acardace
Copy link
Member

acardace commented Aug 7, 2023

/approve

@kubevirt-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: acardace

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 7, 2023
@rmohr
Copy link
Member Author

rmohr commented Aug 9, 2023

@acardace any chance you could also review the change?

@acardace
Copy link
Member

acardace commented Aug 9, 2023

/lgtm

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Aug 9, 2023
@acardace
Copy link
Member

acardace commented Aug 9, 2023

/retest-required

@kubevirt-commenter-bot
Copy link

/retest-required
This bot automatically retries required jobs that failed/flaked on approved PRs.
Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@kubevirt-bot kubevirt-bot merged commit c8a8a44 into kubevirt:main Aug 10, 2023
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants