Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

templates: Silence audit events from container infra by default #2633

Merged

Conversation

cgwalters
Copy link
Member

I was going to go add a check for "system has an AVC denial"
but the problem today is that every time a container starts or
stops and most notably liveness probes end up generating
audit events.

This very quickly rotates out audit events that we do care
about.

Outside of Kubernetes, workloads can be much more "static"
and it makes sense for "iptables rules changed" to cause an
audit event. For OpenShift, it doesn't make sense.

Silence that and the promiscuous device one so that we can
more easily read the audit logs captured from a CI run to
verify there were no AVC denials.

This will also be useful preparation for e.g. teaching
the MCO do watch for some types of audit event (such as
AVC) and bridge them to Prometheus metrics or so.

I was going to go add a check for "system has an AVC denial"
but the problem today is that every time a container starts or
stops *and* most notably liveness probes end up generating
audit events.

This very quickly rotates out audit events that we *do* care
about.

Outside of Kubernetes, workloads can be much more "static"
and it makes sense for "iptables rules changed" to cause an
audit event.  For OpenShift, it doesn't make sense.

Silence that and the promiscuous device one so that we can
more easily read the audit logs captured from a CI run to
verify there were no AVC denials.

This will also be useful preparation for e.g. teaching
the MCO do watch for some types of audit event (such as
AVC) and bridge them to Prometheus metrics or so.
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 21, 2021
@cgwalters
Copy link
Member Author

/test e2e-aws

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 21, 2021

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@cgwalters cgwalters marked this pull request as ready for review June 22, 2021 00:34
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 22, 2021
@cgwalters
Copy link
Member Author

@JAORMX does compliance use Linux audit at all? WDYT about this?

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 22, 2021

@cgwalters: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-aws-disruptive ec3d89a link /test e2e-aws-disruptive
ci/prow/okd-e2e-aws ec3d89a link /test okd-e2e-aws
ci/prow/e2e-ovn-step-registry ec3d89a link /test e2e-ovn-step-registry
ci/prow/e2e-aws-serial ec3d89a link /test e2e-aws-serial
ci/prow/e2e-vsphere-upgrade ec3d89a link /test e2e-vsphere-upgrade

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@JAORMX
Copy link
Contributor

JAORMX commented Jun 22, 2021

@JAORMX does compliance use Linux audit at all? WDYT about this?

extensively, yes. I think it's a good idea, we also now have a different audit log to see changes done to the SDN flows (at least for OVNKubernetes)

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 22, 2021
Copy link
Contributor

@kikisdeliveryservice kikisdeliveryservice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK yep this works great! Check out the audit file in https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/2633/pull-ci-openshift-machine-config-operator-master-e2e-aws/1407082825038958592/artifacts/e2e-aws/gather-extra/artifacts/nodes/ip-10-0-134-39.us-west-2.compute.internal/
and compare vs e.g. https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.8-e2e-aws/1407024371372920832/artifacts/e2e-aws/gather-extra/artifacts/nodes/ip-10-0-146-171.us-west-1.compute.internal/ (latest accepted 4.8 nightly e2e-aws) which is basically a giant spam of NETFILTER_CFG.

looking at the old logs netfilter_cfg has 3000+ lines, pr brings size down 3411 v 39579 too. since it sounds like those logs can be safely suppressed, makes sense to give this a shot and improve our collection.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 22, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 22, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, kikisdeliveryservice, sinnykumari

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [kikisdeliveryservice,sinnykumari]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 670ea52 into openshift:master Jun 22, 2021
@kikisdeliveryservice
Copy link
Contributor

/cherrypick release-4.7

@openshift-cherrypick-robot

@kikisdeliveryservice: new pull request created: #2792

In response to this:

/cherrypick release-4.7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kikisdeliveryservice
Copy link
Contributor

/cherrypick release-4.8

@openshift-cherrypick-robot

@kikisdeliveryservice: new pull request created: #2793

In response to this:

/cherrypick release-4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants