Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1940940: Deploy multiple operand replicas #88

Merged

Conversation

bertinatto
Copy link
Member

@bertinatto bertinatto commented May 20, 2021

Currently, we deploy only one replica for both operands (webhook and controller). This apparently leads to the operator going unavailable when draining the node where an operand is running on.

With this patch we run multiple replicas of each operand, i.e., 1 replica in single-node control-planes or 2 replicas in multi-node control-planes.

Also, we set maxSurge=0 and maxUnavailable=1 to make sure nodes are drained one at a time and to prevent more than one pod running on the same node when nodePorts or hostNetwork are used.

CC @openshift/storage

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 20, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented May 20, 2021

@bertinatto: This pull request references Bugzilla bug 1940940, which is invalid:

  • expected the bug to target the "4.8.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1940940: Fix conditions

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels May 20, 2021
@openshift-ci openshift-ci bot requested review from huffmanca and tsmetana May 20, 2021 08:31
@openshift-ci
Copy link
Contributor

openshift-ci bot commented May 20, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bertinatto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 20, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented May 21, 2021

@bertinatto: This pull request references Bugzilla bug 1940940, which is invalid:

  • expected the bug to target the "4.8.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1940940: Fix conditions

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

1 similar comment
@openshift-ci
Copy link
Contributor

openshift-ci bot commented May 21, 2021

@bertinatto: This pull request references Bugzilla bug 1940940, which is invalid:

  • expected the bug to target the "4.8.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1940940: Fix conditions

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@bertinatto
Copy link
Member Author

/hold cancel

@wking @smarterclayton could you take a look at this approach?

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 21, 2021
@bertinatto bertinatto changed the title Bug 1940940: Fix conditions Bug 1940940: Deploy one replica per master node May 21, 2021
@bertinatto
Copy link
Member Author

/retest

@bertinatto bertinatto changed the title Bug 1940940: Deploy one replica per master node Bug 1940940: Deploy one operand replica per master node May 21, 2021
@bertinatto bertinatto force-pushed the fix-available-false branch 3 times, most recently from a54dfec to a309669 Compare May 21, 2021 13:33
@bertinatto
Copy link
Member Author

/hold
until we agree on the details in openshift/library-go/pull/1056.

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 26, 2021
@bertinatto bertinatto changed the title Bug 1940940: Deploy one operand replica per master node Bug 1940940: Deploy multiple operand replicas Jun 2, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 2, 2021

@bertinatto: This pull request references Bugzilla bug 1940940, which is invalid:

  • expected the bug to target the "4.8.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1940940: Deploy multiple operand replicas

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@bertinatto bertinatto force-pushed the fix-available-false branch 2 times, most recently from c7b59bf to 974dd5a Compare June 2, 2021 12:16
@bertinatto
Copy link
Member Author

/hold cancel
Ready for review.

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 2, 2021
@bertinatto
Copy link
Member Author

/bugzilla refresh

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jun 2, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 2, 2021

@bertinatto: This pull request references Bugzilla bug 1940940, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.8.0) matches configured target release for branch (4.8.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

No GitHub users were found matching the public email listed for the QA contact in Bugzilla (wduan@redhat.com), skipping review request.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Jun 2, 2021
Currently, we deploy only one replica for both operands
(webhook and controller). This apparently leads to the
operator going unavailable when draining the node where
an operand is running on.

With this patch we run multiple replicas of each operand,
i.e., 1 replica in single-node control-planes or 2 replicas
in multi-node control-planes.

Also, we set maxSurge=0 and maxUnavailable=1 to make sure
nodes are drained one at a time and to prevent more than
one pod running on the same node when nodePorts or
hostNetwork are used.
@bertinatto
Copy link
Member Author

bertinatto commented Jun 3, 2021 via email

@jsafrane
Copy link
Contributor

jsafrane commented Jun 3, 2021

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 3, 2021
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 07b3f81 into openshift:master Jun 3, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 3, 2021

@bertinatto: All pull requests linked via external trackers have merged:

Bugzilla bug 1940940 has been moved to the MODIFIED state.

In response to this:

Bug 1940940: Deploy multiple operand replicas

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-cluster-csi-snapshot-controller-operator-container-v4.8.0-202311261141.p0.g07b3f81.assembly.stream for distgit ose-cluster-csi-snapshot-controller-operator.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants