Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCPBUGS-17787: Fix sysctl breaking dots in paths #3870

Merged
merged 1 commit into from Aug 23, 2023

Conversation

mkowalski
Copy link
Contributor

Using sysctl to modify paths containing dots is known to cause issues due to the fact that sysctl blindly assumes that dot is always a separator and is changed to /.

This is a problem when we want to configure a network interface containing a dot in its name, e.g. bond0.354, because net.ipv4.conf.bond0.354.forwarding=1 expands to
/proc/sys/net/ipv4/conf/bond0/354/forwarding when in reality it should be /proc/sys/net/ipv4/conf/bond0.354/forwarding.

To remediate this we are changing the call from
sysctl -w net.ipv4.conf.bond0.354.forwarding=1 to echo "1" > /proc/sys/net/ipv4/conf/bond0.354/forwarding as there is no simple way to change the behaviour of sysctl.

In the current form the call is breaking the script as writing a value to a non-existing path returns an error cannot stat [...] No such file or directory

Fixes: OCPBUGS-17787

@openshift-ci-robot openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Aug 16, 2023
@openshift-ci-robot
Copy link
Contributor

@mkowalski: This pull request references Jira Issue OCPBUGS-17787, which is invalid:

  • expected the bug to target the "4.14.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Using sysctl to modify paths containing dots is known to cause issues due to the fact that sysctl blindly assumes that dot is always a separator and is changed to /.

This is a problem when we want to configure a network interface containing a dot in its name, e.g. bond0.354, because net.ipv4.conf.bond0.354.forwarding=1 expands to
/proc/sys/net/ipv4/conf/bond0/354/forwarding when in reality it should be /proc/sys/net/ipv4/conf/bond0.354/forwarding.

To remediate this we are changing the call from
sysctl -w net.ipv4.conf.bond0.354.forwarding=1 to echo "1" > /proc/sys/net/ipv4/conf/bond0.354/forwarding as there is no simple way to change the behaviour of sysctl.

In the current form the call is breaking the script as writing a value to a non-existing path returns an error cannot stat [...] No such file or directory

Fixes: OCPBUGS-17787

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mkowalski
Copy link
Contributor Author

/cc @trozet
as author of the original code

/cc @cybertron
as on-prem networking

/cc @sinnykumari
as MCO owner

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 16, 2023
@mkowalski mkowalski marked this pull request as ready for review August 16, 2023 16:02
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 16, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 16, 2023
@mkowalski
Copy link
Contributor Author

/jira refresh

@openshift-ci-robot
Copy link
Contributor

@mkowalski: This pull request references Jira Issue OCPBUGS-17787, which is invalid:

  • expected the bug to target the "4.14.0" version, but it targets "4.14" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

sysctl -w net.ipv4.conf.${iface}.forwarding=1
sysctl -w net.ipv6.conf.${iface}.forwarding=1
echo "1" > /proc/sys/net/ipv4/conf/${iface}/forwarding
echo "1" > /proc/sys/net/ipv6/conf/${iface}/forwarding
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please quote "${iface}"

Can interfaces ever contain a / ?

Copy link
Contributor Author

@mkowalski mkowalski Aug 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

TLDR: / and white characters are forbidden from interface names

Longer version: Validations for iproute2 are defined in this code -- https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/tree/lib/utils.c?id=1f420318bda3cc62156e89e1b56d60cc744b48ad#n827. Different tools may allow and disallow different variations, but let's also be clear that we cannot support every single tool out there and cover for every corner case. We can fix them as they arrive though

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

" should handle shell meta-characters or globbing. We could always fuzz.

Using sysctl to modify paths containing dots is known to cause issues
due to the fact that sysctl blindly assumes that dot is always a
separator and is changed to `/`.

This is a problem when we want to configure a network interface
containing a dot in its name, e.g. `bond0.354`, because
`net.ipv4.conf.bond0.354.forwarding=1` expands to
`/proc/sys/net/ipv4/conf/bond0/354/forwarding` when in reality it should
be `/proc/sys/net/ipv4/conf/bond0.354/forwarding`.

To remediate this we are changing the call from
`sysctl -w net.ipv4.conf.bond0.354.forwarding=1` to
`echo "1" > /proc/sys/net/ipv4/conf/bond0.354/forwarding` as there is no
simple way to change the behaviour of sysctl.

In the current form the call is breaking the script as writing a value
to a non-existing path returns an error `cannot stat [...] No such file
or directory`

Fixes: OCPBUGS-17787
@cybertron
Copy link
Member

/test e2e-metal-ipi-ovn-ipv6

The OpenStack job should cover the ipv4 case and the metal job will make sure ipv6 still works.

/lgtm

Assuming the ci jobs pass.

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 16, 2023
@mkowalski
Copy link
Contributor Author

mkowalski commented Aug 17, 2023

/test e2e-openstack

level=info msg=Obtaining RHCOS image file from 'https://rhcos.mirror.openshift.com/art/storage/prod/streams/4.14-9.2/builds/414.92.202308032115-0/x86_64/rhcos-414.92.202308032115-0-openstack.x86_64.qcow2.gz?sha256=cb38a9daa14e746f1b6fe4033b583777e86bcea3b44c2a95b3516304fb358ac2'
level=fatal msg=failed to fetch Terraform Variables: failed to generate asset "Terraform Variables": failed to get openstack Terraform variables: gzip: invalid checksum

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 17, 2023

@mkowalski: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-openstack 88fffa2 link false /test e2e-openstack

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@mkowalski
Copy link
Contributor Author

/jira refresh
/retest-required

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Aug 23, 2023
@openshift-ci-robot
Copy link
Contributor

@mkowalski: This pull request references Jira Issue OCPBUGS-17787, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.14.0) matches configured target version for branch (4.14.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @anuragthehatter

In response to this:

/jira refresh
/retest-required

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot removed the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Aug 23, 2023
@mkowalski
Copy link
Contributor Author

/cherry-pick release-4.14

@openshift-cherrypick-robot

@mkowalski: once the present PR merges, I will cherry-pick it on top of release-4.14 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.14

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 23, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cybertron, mkowalski, yuqi-zhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 23, 2023
@openshift-merge-robot openshift-merge-robot merged commit 73d4728 into openshift:master Aug 23, 2023
13 of 15 checks passed
@openshift-ci-robot
Copy link
Contributor

@mkowalski: Jira Issue OCPBUGS-17787: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-17787 has been moved to the MODIFIED state.

In response to this:

Using sysctl to modify paths containing dots is known to cause issues due to the fact that sysctl blindly assumes that dot is always a separator and is changed to /.

This is a problem when we want to configure a network interface containing a dot in its name, e.g. bond0.354, because net.ipv4.conf.bond0.354.forwarding=1 expands to
/proc/sys/net/ipv4/conf/bond0/354/forwarding when in reality it should be /proc/sys/net/ipv4/conf/bond0.354/forwarding.

To remediate this we are changing the call from
sysctl -w net.ipv4.conf.bond0.354.forwarding=1 to echo "1" > /proc/sys/net/ipv4/conf/bond0.354/forwarding as there is no simple way to change the behaviour of sysctl.

In the current form the call is breaking the script as writing a value to a non-existing path returns an error cannot stat [...] No such file or directory

Fixes: OCPBUGS-17787

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-cherrypick-robot

@mkowalski: new pull request created: #3881

In response to this:

/cherry-pick release-4.14

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants