Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric #2928

Merged
merged 1 commit into from Jan 26, 2022

Conversation

andreaskaris
Copy link
Contributor

@andreaskaris andreaskaris commented Jan 24, 2022

Setting the default NM route metric for ovs-if-br-ex is problematic
in the presence of other Ethernet ports. In that case, ovs-if-br-ex
as well as the Ethernet ports will have the same route metric, and
the winner is undefined. Lower ovs-if-br-ex route metric to 49 to
avoid ambiguous situations.

Conflicts:
templates/common/_base/files/configure-ovs-network.yaml

Signed-off-by: Andreas Karis ak.karis@gmail.com
(cherry picked from commit e6a673d)

  • What I did

Cause:
When more than one ethernet interface has the default route on startup, then NMs default behavior will assign metrics 100, 101, 102..
and so forth to the Ethernet interfaces to avoid ambiguity. NM will make sure that interfaces of the same type will not
have the same metric: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/managing-the-default-gateway-setting_configuring-and-managing-networking

When ovs-configure.sh OVNKubernetes is run, it sets a metric of 100 to OVS port ovs-if-br-ex (which normally would have metric 800).
The port is no longer an Ethernet type port and the remaining Ethernet interfaces will receive new metrics, assigned in ascending order, starting with 100. This will lead to 2 default routes having metric 100, one of the Ethernet interfaces, and ovs-if-br-ex. It is no longer guaranteed that br-ex will be used as the default interface.

Consequence:
Cluster traffic will not function on clusters with multiple default route ethernet interfaces, such as OpenStack clusters with additional networks.

Fix:
Enforce that the metric configured on the OVN-Kubernetes interface (br-ex and ovs-if-br-ex) is set to 49.

Result:
The default route via br-ex should always be the one with the highest priority (= lowest metric).

  • How to verify it

After deployment, routes via br-ex should have metric 49, e.g.:

# ip r
default via 192.168.20.1 dev br-ex proto dhcp metric 49 
default via 192.168.21.1 dev eth1 proto dhcp metric 100 
169.254.169.254 via 192.168.20.2 dev br-ex proto dhcp metric 49
169.254.169.254 via 192.168.21.2 dev eth1 proto dhcp metric 100 
192.168.20.0/24 dev br-ex proto kernel scope link src 192.168.20.80 metric 49
192.168.21.0/24 dev eth1 proto kernel scope link src 192.168.21.167 metric 100 
  • Description for the changelog

Setting the default NM route metric for ovs-if-br-ex is problematic
in the presence of other Ethernet ports. In that case, ovs-if-br-ex
as well as the Ethernet ports will have the same route metric, and
the winner is undefined. Lower ovs-if-br-ex route metric to 49 to
avoid ambiguous situations.

Setting the default NM route metric for ovs-if-br-ex is problematic
in the presence of other Ethernet ports. In that case, ovs-if-br-ex
as well as the Ethernet ports will have the same route metric, and
the winner is undefined. Lower ovs-if-br-ex route metric to 49 to
avoid ambiguous situations.

Conflicts:
  templates/common/_base/files/configure-ovs-network.yaml

Signed-off-by: Andreas Karis <ak.karis@gmail.com>
(cherry picked from commit e6a673d)
@openshift-ci openshift-ci bot added bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Jan 24, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 24, 2022

@andreaskaris: This pull request references Bugzilla bug 2044503, which is invalid:

  • expected dependent Bugzilla bug 2035326 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kikisdeliveryservice
Copy link
Contributor

PTAL
/assign @cybertron @trozet

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 24, 2022

@andreaskaris: This pull request references Bugzilla bug 2044503, which is invalid:

  • expected dependent Bugzilla bug 2035326 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

4 similar comments
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 24, 2022

@andreaskaris: This pull request references Bugzilla bug 2044503, which is invalid:

  • expected dependent Bugzilla bug 2035326 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 24, 2022

@andreaskaris: This pull request references Bugzilla bug 2044503, which is invalid:

  • expected dependent Bugzilla bug 2035326 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 24, 2022

@andreaskaris: This pull request references Bugzilla bug 2044503, which is invalid:

  • expected dependent Bugzilla bug 2035326 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 24, 2022

@andreaskaris: This pull request references Bugzilla bug 2044503, which is invalid:

  • expected dependent Bugzilla bug 2035326 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@cybertron
Copy link
Member

/label backport-risk-assessed

Seems like a reasonable change to backport. Holding off on lgtm to give Tim a chance to weigh in if he wants, and we have to wait for the original bug to be verified anyway.

@openshift-ci openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Jan 24, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 24, 2022

@andreaskaris: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-ovn-step-registry 3711b7c link false /test e2e-ovn-step-registry
ci/prow/e2e-aws-workers-rhel8 3711b7c link false /test e2e-aws-workers-rhel8
ci/prow/okd-e2e-aws 3711b7c link false /test okd-e2e-aws
ci/prow/okd-e2e-gcp-op 3711b7c link false /test okd-e2e-gcp-op
ci/prow/e2e-aws-workers-rhel7 3711b7c link false /test e2e-aws-workers-rhel7
ci/prow/e2e-aws-upgrade-single-node 3711b7c link false /test e2e-aws-upgrade-single-node
ci/prow/e2e-aws-disruptive 3711b7c link false /test e2e-aws-disruptive
ci/prow/okd-e2e-upgrade 3711b7c link false /test okd-e2e-upgrade
ci/prow/e2e-vsphere-upgrade 3711b7c link false /test e2e-vsphere-upgrade

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@andreaskaris
Copy link
Contributor Author

/bugzilla refresh

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jan 25, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 25, 2022

@andreaskaris: This pull request references Bugzilla bug 2044503, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.9.z) matches configured target release for branch (4.9.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 2035326 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
  • dependent Bugzilla bug 2035326 targets the "4.10.0" release, which is one of the valid target releases: 4.10.0
  • bug has dependents

Requesting review from QA contact:
/cc @anuragthehatter

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Jan 25, 2022
Copy link

@flavio-fernandes flavio-fernandes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 25, 2022
@trozet
Copy link
Contributor

trozet commented Jan 25, 2022

/lgtm
/assign @kikisdeliveryservice

@kikisdeliveryservice
Copy link
Contributor

Thanks all, this seems good to go

/approve

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 25, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreaskaris, flavio-fernandes, kikisdeliveryservice, trozet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [kikisdeliveryservice]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

20 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@rbbratta
Copy link
Contributor

/label cherry-pick-approved

@openshift-ci openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Jan 26, 2022
@openshift-merge-robot openshift-merge-robot merged commit ce39533 into openshift:release-4.9 Jan 26, 2022
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 26, 2022

@andreaskaris: All pull requests linked via external trackers have merged:

Bugzilla bug 2044503 has been moved to the MODIFIED state.

In response to this:

[release-4.9] Bug 2044503: ovs-configuration: use lower than NM default ethernet route metric

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@andreaskaris
Copy link
Contributor Author

/cherrypick

@andreaskaris
Copy link
Contributor Author

/cherrypick 4.8

@openshift-cherrypick-robot

@andreaskaris: cannot checkout 4.8: error checking out 4.8: exit status 1. output: error: pathspec '4.8' did not match any file(s) known to git

In response to this:

/cherrypick 4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@andreaskaris
Copy link
Contributor Author

/cherrypick release-4.8

@openshift-cherrypick-robot

@andreaskaris: #2928 failed to apply on top of branch "release-4.8":

Applying: ovs-configuration: use lower than NM default ethernet route metric
Using index info to reconstruct a base tree...
M	templates/common/_base/files/configure-ovs-network.yaml
Falling back to patching base and 3-way merge...
Auto-merging templates/common/_base/files/configure-ovs-network.yaml
CONFLICT (content): Merge conflict in templates/common/_base/files/configure-ovs-network.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 ovs-configuration: use lower than NM default ethernet route metric
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherrypick release-4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants