Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug 1824287: OpenStack: Replace remote_group_id with remote_ip_prefix #3461

Merged
merged 1 commit into from Apr 17, 2020
Merged

Bug 1824287: OpenStack: Replace remote_group_id with remote_ip_prefix #3461

merged 1 commit into from Apr 17, 2020

Conversation

mandre
Copy link
Member

@mandre mandre commented Apr 15, 2020 •
edited

Using remote_group_id in the security rules is very inefficient, triggering a lot of computation by ovs agent to generate the flows and possibly exceeding the time allocated for flow generation. In such cases, especially in environments already under stress, masters nodes may be unable to communicate with worker nodes, leading the deployment to fail.

We're seeing this behavior in MOC, the cloud we're using for our CI.

The workaround is to use the more efficient remote_ip_prefix rather than remote_group_id when creating security rules.

This was already done for openshift-ansible in the past: https://bugzilla.redhat.com/show_bug.cgi?id=1703947

@mandre
OpenStack: Replace remote_group_id with remote_ip_prefix
90e77a3 
OpenStack with OVS has an issue where security groups using
remote_group_id can be very slow, leading to OVS dropping packets.

https://bugzilla.redhat.com/show_bug.cgi?id=1703947

Use remote_ip_prefix instead to workaround the issue.
@iamemilio
Copy link

/test e2e-openstack

@iamemilio
Copy link

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 15, 2020
@mandre mandre changed the title OpenStack: Replace remote_group_id with remote_ip_prefix Bug 1824287: OpenStack: Replace remote_group_id with remote_ip_prefix Apr 16, 2020
@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Apr 16, 2020
@openshift-ci-robot
Copy link
Contributor

@mandre: This pull request references Bugzilla bug 1824287, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1824287: OpenStack: Replace remote_group_id with remote_ip_prefix

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link
Contributor

@mandre: This pull request references Bugzilla bug 1824287, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1824287: OpenStack: Replace remote_group_id with remote_ip_prefix

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@luis5tb
Copy link
Contributor

luis5tb commented Apr 16, 2020

/lgtm

@luis5tb
Copy link
Contributor

luis5tb commented Apr 16, 2020

going to give it a try!

@Fedosin
Copy link
Contributor

Fedosin commented Apr 16, 2020

/approve

@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Fedosin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 16, 2020
@Fedosin
Copy link
Contributor

Fedosin commented Apr 16, 2020

Let's make our ci green again!

@Fedosin
Copy link
Contributor

Fedosin commented Apr 16, 2020

/test e2e-aws

2 similar comments
@Fedosin
Copy link
Contributor

Fedosin commented Apr 16, 2020

/test e2e-aws

@Fedosin
Copy link
Contributor

Fedosin commented Apr 16, 2020

/test e2e-aws

@iamemilio
Copy link

/retest

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

2 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@Fedosin
Copy link
Contributor

Fedosin commented Apr 16, 2020

/test e2e-aws

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Apr 16, 2020 •
edited

@mandre: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-aws-scaleup-rhel7 90e77a3 link /test e2e-aws-scaleup-rhel7

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Fedosin Fedosin mentioned this pull request Apr 16, 2020
Merged
@Fedosin
Copy link
Contributor

Fedosin commented Apr 17, 2020

/retest

@pierreprinetti pierreprinetti mentioned this pull request Apr 17, 2020
Merged
@pierreprinetti
Copy link
Member

/bugzilla refresh

@openshift-ci-robot
Copy link
Contributor

@pierreprinetti: This pull request references Bugzilla bug 1824287, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pierreprinetti
Copy link
Member

/label platform/openstack

@openshift-merge-robot openshift-merge-robot merged commit 6decc16 into openshift:master Apr 17, 2020
@openshift-ci-robot
Copy link
Contributor

@mandre: Some pull requests linked via external trackers have merged: openshift/installer#3461. The following pull requests linked via external trackers have not merged:

In response to this:

Bug 1824287: OpenStack: Replace remote_group_id with remote_ip_prefix

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pierreprinetti
Copy link
Member

/cherry-pick release-4.4

@openshift-cherrypick-robot
Copy link

@pierreprinetti: #3461 failed to apply on top of branch "release-4.4":

error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
M	data/data/openstack/topology/sg-master.tf
M	data/data/openstack/topology/sg-worker.tf
Falling back to patching base and 3-way merge...
Auto-merging data/data/openstack/topology/sg-worker.tf
CONFLICT (content): Merge conflict in data/data/openstack/topology/sg-worker.tf
Auto-merging data/data/openstack/topology/sg-master.tf
CONFLICT (content): Merge conflict in data/data/openstack/topology/sg-master.tf
Patch failed at 0001 OpenStack: Replace remote_group_id with remote_ip_prefix

In response to this:

/cherry-pick release-4.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pierreprinetti pierreprinetti mentioned this pull request Apr 18, 2020
Merged
@mandre mandre deleted the openstack-sg branch September 17, 2020 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jhixson74 jhixson74 Awaiting requested review from jhixson74

@patrickdillon patrickdillon Awaiting requested review from patrickdillon

Assignees

@luis5tb luis5tb

@iamemilio iamemilio

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. platform/openstack
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
9 participants
@mandre @iamemilio @openshift-ci-robot @luis5tb @Fedosin @openshift-bot @pierreprinetti @openshift-cherrypick-robot @openshift-merge-robot