Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROX-15904: migrate caching node scanner #1130

Merged
merged 3 commits into from
Mar 23, 2023

Conversation

Maddosaurus
Copy link
Member

@Maddosaurus Maddosaurus commented Mar 22, 2023

This PR updates the Node Scanner that was introduced in #1116 to a new implementation that caches results in an EmptyDir to not rescan the Node if a new enough cached scan is available in the cache.
The implementation was discussed and developed in stackrox/stackrox#4701, but required some minimal changes to adapt to the new location.

Limitations

Due to the fact that Scanner doesn't currently have a Duration-type environment setting, all settings for the Caching Scanner that were previously env vars are currently hardcoded (see TODO in service.go).
This will be addressed in a follow up, ROX-16095, to keep these PR reviews short, isolated, and quick 😃

Testing Performed

I deployed this version on an OpenShift 4.12 cluster and verified that the caching functionality is still working.
As long as stackrox/stackrox/pull/5292 is not merged, manual changes need to be applied to the Collector DaemonSet to use this container.
The following changes have to be executed, after ACS has been deployed via ./deploy/openshift/deploy.sh:
Update the Collector DaemonSet with this additional container in spec/template/spec/containers:

      - name: node-inventory
        image: quay.io/mmeiding/playground:scanner-8a7f5bf983be
        command: [ "/scanner", "--nodeinventory", "--config=", "" ]
        ports:
        - containerPort: 8444
        env:
        - name: ROX_NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        volumeMounts:
        - mountPath: /host
          name: host-root-ro
          readOnly: true
        - mountPath: /tmp/
          name: tmp-volume
        - mountPath: /cache
          name: cache-volume

Afterwards, set the following env vars to shorten the rescan times for testing:
kubectl -n stackrox set env daemonsets/collector --containers="compliance" ROX_NODE_SCANNING_INTERVAL="20s" ROX_NODE_SCANNING_INTERVAL_DEVIATION="1s" ROX_NODE_SCANNING_MAX_INITIAL_WAIT="10s" LOGLEVEL="DEBUG"

Tests

  1. If a cached scan is used, debug logs will denote the use of a cached scan.
  2. To test the backoff, get a shell on node-inventory and execute: echo -n "{\"CacheValidUntil\":\"0001-01-01T00:00:00Z\",\"RetryBackoffDuration\":\"10s\",\"CachedInventory\":\"\"}" > /cache/inventory-cache. Observe a log entry on node-scanner on the next scan that it found a backoff and will wait for 10 seconds.
  3. To test a corrupted meta cache, get a shell and execute: echo -n "{\"CacheValidUntil\":\"42\",\"RetryBackoffDuration\":\"noDuration\",\"CachedInventory\":\"\"}" > /cache/inventory-cache. Observe a failsafe where the next scan will be delayed by 300 seconds.
  4. To test a corrupted inventory, execute: echo -n "{\"CacheValidUntil\":\"2023-04-01T00:00:00Z\",\"RetryBackoffDuration\":\"0s\",\"CachedInventory\":\"noInventory\"}" > /cache/inventory-cache. Observe a failsafe where the next scan will be delayed by 300 seconds.

@roxbot
Copy link
Contributor

roxbot commented Mar 22, 2023

Images are ready for the commit at 218b500.

To use the images, use the tag 2.28.x-47-g218b500bc7.

Copy link
Contributor

@jvdm jvdm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@Maddosaurus Maddosaurus merged commit 8129118 into master Mar 23, 2023
@Maddosaurus Maddosaurus deleted the mm/ROX-15904-migrate-caching-scanner branch March 23, 2023 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants