Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor fixes for gather_network_logs #243

Merged

Conversation

npinaeva
Copy link
Member

@npinaeva npinaeva commented Jun 23, 2021

Fix ip route gathering (command should be evaluated inside OVNKUBE_NODE_POD)
Fix evaluating if ipsec is enabled (condition evaluation previously failed, always returning False condition result)
Add How to run section to README.md

… than service's logs)

Fix ip route gathering (command should be evaluated inside OVNKUBE_NODE_POD)
Fix evaluating if ipsec is enabled (condition evaluation previously failed, always returning False condition result)
Add How to run section to README.md
@openshift-ci openshift-ci bot added bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Jun 23, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 23, 2021

@npinaeva: This pull request references Bugzilla bug 1970129, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.9.0) matches configured target release for branch (4.9.0)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

No GitHub users were found matching the public email listed for the QA contact in Bugzilla (anusaxen@redhat.com), skipping review request.

In response to this:

Bug 1970129: Add gathering for ovs log files

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

# Call format: NODE [file_paths]
# Collects log files from given paths,
# path should be relative to the node's /var/logs/ folder,
# e.g. openvswitch/ovs-vswitchd.log will be collected to "$NETWORK_LOG_PATH"/"$NODE"_ovs-vswitchd.log
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already capture ovs-vswitchd.log as part of must-gather now:

ovs-configuration ovsdb-server ovs-vswitchd)

The issue is we need to set the logging levels in it, by doing ovs-vsctl vlog/set INFO for example. It looks like we just use to set it to INFO as shown here:
https://github.com/openshift/cluster-network-operator/blob/release-4.6/bindata/network/ovn-kubernetes/006-ovs-node.yaml#L92

We previously ran this OVS container in 4.6 that took care of this, but we no longer have it since 4.7. So log levels are not getting set correctly. I think you will need a PR in CNO to set the log level for OVS in the ovnkube-node template:
https://github.com/openshift/cluster-network-operator/blob/release-4.6/bindata/network/ovn-kubernetes/ovnkube-node.yaml

or alternatively you could put it in MCO:
https://github.com/openshift/machine-config-operator/blob/master/templates/common/_base/files/configure-ovs-network.yaml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment, I will open different PR for the bug fix, and leave minor fixes in this one

@npinaeva npinaeva changed the title Bug 1970129: Add gathering for ovs log files Minor fixes for gather_network_logs Jun 23, 2021
@openshift-ci openshift-ci bot removed the bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. label Jun 23, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 23, 2021

@npinaeva: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

Minor fixes for gather_network_logs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot removed the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jun 23, 2021
@npinaeva
Copy link
Member Author

/retest

@npinaeva
Copy link
Member Author

Reproducing: run oc adm must-gather -- gather_network_logs

Before
In the stdout:
/usr/bin/gather_network_logs: line 256: [-z: command not found
netlink error: no device matches name (offset 24)
netlink error: No such device

In gathered files:
network_logs/_ethtool_driver
network_logs/_ethtool_offload - empty files

After
No errors in stdout, files are filled with output

@npinaeva
Copy link
Member Author

@davemulford, @sferich888: I think this is the final version of the PR, added a comment about testing changes. I will appreciate if you could take a look.

@dcbw
Copy link
Member

dcbw commented Jul 1, 2021

/lgtm
@sferich888 what do you think?

@@ -10,3 +10,7 @@ Data collection scripts are kept in `./collection-scripts`. The content of that
The data collection scripts should only include collection logic for components that are included as part of the OpenShift
CVO payload. Outside components are encouraged to produce a similar "must-gather" image, but this is not the spot to be
included.

## How to run
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we link to https://docs.okd.io/latest/support/gathering-cluster-data.html#support_gathering_data_gathering-cluster-data (or something similar on how to run advanced commands, later in these docs)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, that may be useful

echo "INFO: No ovn-ipsec pods exist, tunnel traffic will be unencrypted"
else
echo "INFO: ovn-ipsec is enabled, tunnel traffic should be encryted"
echo "INFO: ovn-ipsec is enabled, tunnel traffic should be encrypted"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this being true is running gather_ovn_ipsec_data valuable?
Will we be decrypting it to use what we collect? If we do decrypt it do we need to tell the user? It doesn't look like we collect any traffic only how we shape/direct the traffic so these question may not matter, and if so I don't know if the messages/logs are needed (but they are not harmful).

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 14, 2021
@sferich888
Copy link
Contributor

/lgtm

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 14, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dcbw, npinaeva, sferich888

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 0aa902a into openshift:master Jul 14, 2021
@npinaeva npinaeva deleted the fix-ovs-logs-gathering branch July 20, 2021 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants