Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache pods and container IDs in log enricher #509

Merged
merged 1 commit into from
Jul 1, 2021

Conversation

saschagrunert
Copy link
Member

What type of PR is this?

/kind feature

What this PR does / why we need it:

We now use a cache and a default timeout of 6 hours to cache the
container info as well as the PIDs. This should speedup the enricher in
general.

Which issue(s) this PR fixes:

None

Does this PR have test?

None

Special notes for your reviewer:

None

Does this PR introduce a user-facing change?

Added container ID caching to log enricher for performance reasons.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 29, 2021
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 29, 2021
@saschagrunert
Copy link
Member Author

/test pull-security-profiles-operator-test-e2e

Copy link
Contributor

@JAORMX JAORMX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff! I actually didn't know about this caching library, but it looks quite neat! Let's at least add some comments about our cache TTL value reasoning.

@@ -77,7 +88,7 @@ func (e *Enricher) getNodeContainers(logger logr.Logger, nodeName string) (map[s
continue
}

containers[rawContainerID] = containerInfo{
containers[rawContainerID] = &containerInfo{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we update the cache already here? This will cache all the container infos that have been found already, wouldn't it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think this makes sense, changed as suggested 👍

@@ -27,23 +30,39 @@ import (
"sigs.k8s.io/security-profiles-operator/internal/pkg/config"
)

const defaultCacheTimeout time.Duration = 6 * time.Hour
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the ephemeral nature of containers, maybe we should choose a lower cache timeout. e.g. only a couple of hours maybe? or even less, what do you think? We should document our reasoning in comments here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think we should reduce it, I've chosen 1 hour. I also added a comment that it's nothing more than a rough guess 🤷‍♂️

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, at least now people won't wonder why a value was chosen. Maybe one day we'll have enough usage and data for the operator to choose a value based on data :)

We now use a cache and a default timeout of 6 hours to cache the
container info as well as the PIDs. This should speedup the enricher in
general.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Copy link
Contributor

@JAORMX JAORMX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 1, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JAORMX, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit cd71f46 into kubernetes-sigs:master Jul 1, 2021
@saschagrunert saschagrunert deleted the cache branch July 1, 2021 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants