CMP-2859: Resolve failing Image-stream-sets-schedule #12895

KaushikOP · 2025-01-24T13:34:26Z

Description:

Resolve failing Image-stream-sets-schedule. Changes to samples operator which were not accounted for, and ClusterVersion managed imagestreams should be excluded.

Rationale:

fixes failing rule
Fixes #CMP-2859

Review Hints:

Part of DISA-STIG profile for OCP

openshift-ci · 2025-01-24T13:34:38Z

Hi @KaushikOP. Thanks for your PR.

I'm waiting for a ComplianceAsCode member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

prb112 · 2025-01-24T13:44:09Z

/jira-refresh

prb112 · 2025-01-24T13:50:35Z

applications/openshift/registry/imagestream_sets_schedule/rule.yml

@@ -32,17 +32,17 @@ references:
    srg: SRG-APP-000456-CTR-001125

 {{% set api_path = '/apis/image.openshift.io/v1/imagestreams' %}}
-{{% set jqfilter = '[.items[] | .spec.tags[]? | select(.from.kind != "ImageStreamTag") | (.importPolicy.scheduled != null and .importPolicy.scheduled != false)] | all' %}}
+{{% set jqfilter = '[.items[] | select( .metadata.ownerReferences? // [] | all(.kind != "ClusterVersion")) | select(.metadata.labels[]? | select("samples.operator.openshift.io/managed: true") | not) | select(.spec.tags[]?.from.kind != "ImageStreamTag" and (.importPolicy.scheduled != null or .importPolicy.scheduled != false))] | any' %}}


Gather all ImageStreams - .items[]

Select only those not managed by ClusterVersion - select( .metadata.ownerReferences? // [] | all(.kind != "ClusterVersion"))

ClusterVersion implies on upgrade that these ImageStream entries are upgraded

From those that remain, check if it's managed by the Samples Operator, if so ignore those - select(.metadata.labels[]? | select("samples.operator.openshift.io/managed: true") | not)

Check the Remaining ones are not tags - select(.spec.tags[]?.from.kind != "ImageStreamTag" and (partial query here)

e.g. ImageStreamTag latest points to v1.2.3

The ones that remain from that check to see if scheduled is false or not set - (.importPolicy.scheduled != null or .importPolicy.scheduled != false))] | any' %}}

If any fail the last condition it returns false for the query.

Thanks a bunch for the breakdown and explanation of the jq filter.

prb112 · 2025-01-27T13:20:21Z

/retest-required

openshift-ci · 2025-01-27T13:20:38Z

@prb112: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest-required

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

prb112 · 2025-02-13T18:35:06Z

/jira-refresh

yuumasato · 2025-02-28T09:07:35Z

/ok-to-test

yuumasato

/LGTM

But could you also update the expected results so that our CI doesn't flag this change?

Change the following lines from FAIL to PASS.

$ grep -rA2 "imagestream-sets-schedule" tests/assertions/ocp4/
tests/assertions/ocp4/ocp4-stig-4.12.yml:  e2e-stig-imagestream-sets-schedule:
tests/assertions/ocp4/ocp4-stig-4.12.yml-    default_result: FAIL
tests/assertions/ocp4/ocp4-stig-4.12.yml-    result_after_remediation: FAIL
--
tests/assertions/ocp4/ocp4-stig-4.13.yml:  e2e-stig-imagestream-sets-schedule:
tests/assertions/ocp4/ocp4-stig-4.13.yml-    default_result: FAIL
tests/assertions/ocp4/ocp4-stig-4.13.yml-    result_after_remediation: FAIL
--
tests/assertions/ocp4/ocp4-stig-4.14.yml:  e2e-stig-imagestream-sets-schedule:
tests/assertions/ocp4/ocp4-stig-4.14.yml-    default_result: FAIL
tests/assertions/ocp4/ocp4-stig-4.14.yml-    result_after_remediation: FAIL
--
tests/assertions/ocp4/ocp4-stig-4.15.yml:  e2e-stig-imagestream-sets-schedule:
tests/assertions/ocp4/ocp4-stig-4.15.yml-    default_result: FAIL
tests/assertions/ocp4/ocp4-stig-4.15.yml-    result_after_remediation: FAIL
--
tests/assertions/ocp4/ocp4-stig-4.16.yml:  e2e-stig-imagestream-sets-schedule:
tests/assertions/ocp4/ocp4-stig-4.16.yml-    default_result: FAIL
tests/assertions/ocp4/ocp4-stig-4.16.yml-    result_after_remediation: FAIL
--
tests/assertions/ocp4/ocp4-stig-4.17.yml:  e2e-stig-imagestream-sets-schedule:
tests/assertions/ocp4/ocp4-stig-4.17.yml-    default_result: FAIL
tests/assertions/ocp4/ocp4-stig-4.17.yml-    result_after_remediation: FAIL
--
tests/assertions/ocp4/ocp4-stig-4.18.yml:  e2e-stig-imagestream-sets-schedule:
tests/assertions/ocp4/ocp4-stig-4.18.yml-    default_result: FAIL
tests/assertions/ocp4/ocp4-stig-4.18.yml-    result_after_remediation: FAIL

$ cat ./applications/openshift/registry/imagestream_sets_schedule/tests/ocp4/e2e.yml 
---
default_result: FAIL

prb112 · 2025-02-28T15:18:42Z

Thanks @yuumasato I'll leave a message with Kaushik letting him know

yuumasato

@KaushikOP tests/assertions/ocp4/ocp4-stig-4.18.yml was left out of the assertion update.

Please update it as well.

yuumasato · 2025-03-03T15:35:12Z

/test 4.17-e2e-aws-ocp4-stig
/test 4.18-e2e-aws-ocp4-stig

prb112 · 2025-03-04T14:09:47Z

/retest-required

KaushikOP · 2025-03-05T07:09:31Z

Hi @yuumasato

I have made the changes. Please re-review.

rhmdnd · 2025-03-05T16:13:20Z

/retest-required

yuumasato · 2025-03-05T16:13:55Z

/test 4.12-e2e-aws-ocp4-stig
/test 4.16-e2e-aws-ocp4-stig

prb112 · 2025-03-05T18:40:13Z

Grabbed the imagestreams from the must-gather. https://gist.github.com/prb112/06768521d08ecf0eba49476252580e8e

Will ask @KaushikOP to test the jq locally to see why it failed.

prb112 · 2025-03-06T20:28:16Z

Analyzed the gist / imagestreams - it should have passed. Perhaps a retest?

prb112 · 2025-03-06T20:28:30Z

/retest-required

prb112 · 2025-03-12T13:26:51Z

/retest-required

prb112 · 2025-03-14T13:37:37Z

/retest-required

KaushikOP · 2025-03-28T06:01:12Z

/retest-required

KaushikOP · 2025-04-02T13:19:14Z

/retest-required

xiaojiey · 2025-04-07T09:47:36Z

@prb112 @KaushikOP I am not sure below imagestream is a good example or not. I created a imagestream without ownerReferences set, and set the importPolicy.scheduled to null. Under this scenario, the rule should FAIL, right?
However, I can see the rule show as PASS. Could you please help to check? Thanks.
I also tried to create a similar imagestream with importPolicy.scheduled set to true. Both filters in the instructions will still return empty.

% oc get imagestream nginx -o yaml -n openshift-compliance
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"image.openshift.io/v1","kind":"ImageStream","metadata":{"annotations":{},"name":"nginx","namespace":"openshift-compliance"},"spec":{"tags":[{"from":{"kind":"DockerImage","name":"quay.io/security-profiles-operator/test-nginx-unprivileged:1.21"},"importPolicy":{"scheduled":false},"name":"latest"}]}}
    openshift.io/image.dockerRepositoryCheck: "2025-04-07T07:16:44Z"
  creationTimestamp: "2025-04-07T07:16:41Z"
  generation: 2
  name: nginx
  namespace: openshift-compliance
  resourceVersion: "108869"
  uid: 8364f1d4-475d-40a4-9f16-60dd13978282
spec:
  lookupPolicy:
    local: false
  tags:
  - annotations: null
    from:
      kind: DockerImage
      name: quay.io/security-profiles-operator/test-nginx-unprivileged:1.21
    generation: 2
    importPolicy:
      importMode: Legacy
    name: latest
    referencePolicy:
      type: Source
status:
  dockerImageRepository: image-registry.openshift-image-registry.svc:5000/openshift-compliance/nginx
  tags:
  - items:
    - created: "2025-04-07T07:16:44Z"
      dockerImageReference: quay.io/security-profiles-operator/test-nginx-unprivileged@sha256:362f8cfc2fed11a2037f29bfcf91c9d9ab5e3b8b46aabb846d8f1f7d81a939d2
      generation: 2
      image: sha256:362f8cfc2fed11a2037f29bfcf91c9d9ab5e3b8b46aabb846d8f1f7d81a939d2
    tag: latest

% oc get rule upstream-ocp4-imagestream-sets-schedule -o=jsonpath={.instructions}
To list all the imagestreams and identify which imagestream tags are
configured to periodically check for updates (imagePolicy = { scheduled: true }), run the following command:
oc get imagestream -A -ojson | jq -r '.items[] | select( .metadata.ownerReferences? // [] | all(.kind == "ClusterVersion")) | select(.metadata.labels[]? | select( "samples.operator.openshift.io/managed: true")) | select( .spec.tags[]?.from.kind != "ImageStreamTag" and .importPolicy.scheduled == true).metadata.name' | sort | uniq
Alternatively, to view a list of ImageStreams that do not schedule updates,
run:
oc get imagestream -A -ojson | jq -r '.items[] | select( .metadata.ownerReferences? // [] | all(.kind != "ClusterVersion")) | select(.metadata.labels[]? | select( "samples.operator.openshift.io/managed: true") | not) | select( .spec.tags[]?.from.kind != "ImageStreamTag" and (.importPolicy.scheduled == null or .importPolicy.scheduled == false)) | "\(.metadata.namespace),\(.metadata.name)"' | sort | uniq
Is it the case that imagestream is not configured to perform periodical updates?%

% oc get imagestream -A -ojson | jq -r '.items[] | select( .metadata.ownerReferences? // [] | all(.kind == "ClusterVersion")) | select(.metadata.labels[]? | select( "samples.operator.openshift.io/managed: true")) | select( .spec.tags[]?.from.kind != "ImageStreamTag" and .importPolicy.scheduled == true).metadata.name' | sort | uniq

% oc get imagestream -A -ojson | jq -r '.items[] | select( .metadata.ownerReferences? // [] | all(.kind != "ClusterVersion")) | select(.metadata.labels[]? | select( "samples.operator.openshift.io/managed: true") | not) | select( .spec.tags[]?.from.kind != "ImageStreamTag" and (.importPolicy.scheduled == null or .importPolicy.scheduled == false)) | "\(.metadata.namespace),\(.metadata.name)"' | sort | uniq
% oc get ccr | grep -i imagestream-sets-schedule
ocp4-stig-imagestream-sets-schedule                             FAIL     medium
upstream-ocp4-stig-imagestream-sets-schedule                    PASS     medium

xiaojiey · 2025-04-07T09:49:32Z

/hold for test

prb112 · 2025-04-07T18:39:53Z

Discussed xiaojiey's comments
refactoring dropped this edge case, we're fixing it.
Thanks :)

KaushikOP · 2025-04-11T18:52:10Z

Hi @xiaojiey,

We have refactored the rule to include cases that mentioned your review.

CC: @prb112

KaushikOP · 2025-04-11T19:03:00Z

applications/openshift/registry/imagestream_sets_schedule/rule.yml

@@ -32,17 +32,17 @@ references:
    srg: SRG-APP-000456-CTR-001125

 {{% set api_path = '/apis/image.openshift.io/v1/imagestreams' %}}
-{{% set jqfilter = '[[.items[] | select( .metadata.ownerReferences? // [] | all(.kind != "ClusterVersion")) | select(.metadata.labels[]? | select("samples.operator.openshift.io/managed: true") | not) | select(.spec.tags[]?.from.kind != "ImageStreamTag" and (.importPolicy.scheduled != null or .importPolicy.scheduled != false))] | any]' %}}
+{{% set jqfilter = '[[.items[] | ( .spec.tags[]?.from.kind != "ImageStreamTag" and ([.spec.tags[]? | (.importPolicy.scheduled == null or .importPolicy.scheduled == false)] | any) ) and ((.metadata.labels == null) or (.metadata.labels."samples.operator.openshift.io/managed" == null) or (.metadata.labels."samples.operator.openshift.io/managed" != "true")) and ((.metadata.ownerReferences == null) or (.metadata.ownerReferences | length == 0) or (.metadata.ownerReferences?[].kind == null) or (isempty(.metadata.ownerReferences?[] | select(.kind == "ClusterVersion"))))] | any]' %}}


Explanation:

Get all ImageStreams: .items[]

Target user-defined streams only: .spec.tags[]?.from.kind != "ImageStreamTag"

Check for non-scheduled tags: (.importPolicy.scheduled == null or .importPolicy.scheduled == false)] | any).

Ignore Sample Operator-managed streams: ((.metadata.labels == null) or (.metadata.labels."samples.operator.openshift.io/managed" == null) or (.metadata.labels."samples.operator.openshift.io/managed" != "true")).

Exclude system-managed ImageStreams: ((.metadata.ownerReferences == null) or (.metadata.ownerReferences | length == 0) or (.metadata.ownerReferences?[].kind == null) or (isempty(.metadata.ownerReferences?[] | select(.kind == "ClusterVersion")).

Summary: This detects user-managed, unmanaged, external ImageStreams that are not auto-updating.

Final condition: | any returns true if any such non-compliant ImageStream exists.

xiaojiey · 2025-05-17T03:07:47Z

Verification pass.
##scenario 1: there is a imagestream with schedule unset, the rule returns FAIL:
% oc get imagestream -A -ojson | jq -r '.items[] | select(( (.spec.tags[]?.from.kind != "ImageStreamTag") and ([.spec.tags[]? | (.importPolicy.scheduled == null or .importPolicy.scheduled == false)] | any) and ((.metadata.labels == null) or (.metadata.labels["samples.operator.openshift.io/managed"] == null) or (.metadata.labels["samples.operator.openshift.io/managed"] != "true")) and ((.metadata.ownerReferences == null) or (.metadata.ownerReferences | length == 0) or (.metadata.ownerReferences[]?.kind == null) or (isempty(.metadata.ownerReferences[]? | select(.kind == "ClusterVersion")))) )) | "(.metadata.namespace),(.metadata.name)"' | sort | uniq
openshift-compliance,nginx
% oc get ccr | grep imagestream-sets-schedule
upstream-ocp4-stig-imagestream-sets-schedule FAIL medium
##Scenario 2: there is no imagestream with schedule unset, the rule returns PASS:
% oc get imagestream -A -ojson | jq -r '.items[] | select(( (.spec.tags[]?.from.kind != "ImageStreamTag") and ([.spec.tags[]? | (.importPolicy.scheduled == null or .importPolicy.scheduled == false)] | any) and ((.metadata.labels == null) or (.metadata.labels["samples.operator.openshift.io/managed"] == null) or (.metadata.labels["samples.operator.openshift.io/managed"] != "true")) and ((.metadata.ownerReferences == null) or (.metadata.ownerReferences | length == 0) or (.metadata.ownerReferences[]?.kind == null) or (isempty(.metadata.ownerReferences[]? | select(.kind == "ClusterVersion")))) )) | "(.metadata.namespace),(.metadata.name)"' | sort | uniq
% oc get ccr | grep imagestream-sets-schedule
upstream-ocp4-stig-imagestream-sets-schedule PASS medium

xiaojiey · 2025-05-17T03:08:26Z

/unhold

xiaojiey · 2025-05-17T03:08:33Z

/lgtm

yuumasato · 2025-05-26T12:56:57Z

@KaushikOP Hi, looks good to me.

Sorry for delay in picking this up.
Since this was pushed there a new CI checks that need to pass.
Could you rebase this?
Thanks

…ator and ClusterVersion Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

Enclosing the jqfilter in array and updating result as false. Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

KaushikOP · 2025-05-26T15:34:03Z

Hi @yuumasato,

I have rebased this with upstream/master.

yuumasato · 2025-06-02T10:15:40Z

@KaushikOP Sorry for delay, I got sick.

There should be no merge commits in the PR:
https://github.com/ComplianceAsCode/content/actions/runs/15256465663/job/43296097548?pr=12895

KaushikOP · 2025-06-04T07:45:56Z

/test all

codeclimate · 2025-06-04T14:08:41Z

Code Climate has analyzed commit 7b81ca2 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 61.9% (0.0% change).

View more on Code Climate.

openshift-ci bot added the needs-ok-to-test Used by openshift-ci bot. label Jan 24, 2025

KaushikOP changed the title ~~CMP2859: Resolve failing Image-stream-sets-schedule~~ CMP-2859: Resolve failing Image-stream-sets-schedule Jan 24, 2025

prb112 reviewed Jan 24, 2025

View reviewed changes

jan-cerny added the OpenShift OpenShift product related. label Jan 27, 2025

yuumasato self-assigned this Feb 28, 2025

openshift-ci bot added ok-to-test Used by openshift-ci bot. and removed needs-ok-to-test Used by openshift-ci bot. labels Feb 28, 2025

yuumasato requested changes Feb 28, 2025

View reviewed changes

yuumasato reviewed Mar 3, 2025

View reviewed changes

KaushikOP requested a review from yuumasato March 5, 2025 07:08

yuumasato added this to the 0.1.77 milestone Mar 5, 2025

openshift-ci bot added the do-not-merge/hold Used by openshift-ci-robot bot. label Apr 7, 2025

KaushikOP commented Apr 11, 2025

View reviewed changes

openshift-ci bot removed the do-not-merge/hold Used by openshift-ci-robot bot. label May 17, 2025

vojtapolasek modified the milestones: 0.1.77, 0.1.78 May 21, 2025

KaushikOP requested review from matusmarhefka, Mab879, vojtapolasek and jan-cerny as code owners May 26, 2025 14:32

KaushikOP and others added 5 commits May 26, 2025 20:46

CMP-2859: filter default ImageStream constructs from the samples oper…

a3c4b17

…ator and ClusterVersion Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

Updating Assertions for Rule

3086e51

Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

Updating missing assertion for rule

29f305d

Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

Updating the condition based on e2e tests results

9431195

Enclosing the jqfilter in array and updating result as false. Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

Refactoring based upon review comments

7b81ca2

Signed-off-by: Kaushik Talathi <kaushik.talathi1@ibm.com>

KaushikOP force-pushed the CMP2859-imagestream-rule-fail branch from 172e7ae to 7b81ca2 Compare June 4, 2025 07:07

yuumasato approved these changes Jun 4, 2025

View reviewed changes

Mab879 approved these changes Jun 4, 2025

View reviewed changes

yuumasato merged commit 3d4eee4 into ComplianceAsCode:master Jun 4, 2025
123 of 131 checks passed

CMP-2859: Resolve failing Image-stream-sets-schedule #12895

CMP-2859: Resolve failing Image-stream-sets-schedule #12895

Uh oh!

Conversation

KaushikOP commented Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description:

Rationale:

Review Hints:

Uh oh!

openshift-ci bot commented Jan 24, 2025

Uh oh!

prb112 commented Jan 24, 2025

Uh oh!

prb112 Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

yuumasato Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

prb112 commented Jan 27, 2025

Uh oh!

openshift-ci bot commented Jan 27, 2025

Uh oh!

prb112 commented Feb 13, 2025

Uh oh!

yuumasato commented Feb 28, 2025

Uh oh!

yuumasato left a comment

Choose a reason for hiding this comment

Uh oh!

prb112 commented Feb 28, 2025

Uh oh!

yuumasato left a comment

Choose a reason for hiding this comment

Uh oh!

yuumasato commented Mar 3, 2025

Uh oh!

prb112 commented Mar 4, 2025

Uh oh!

KaushikOP commented Mar 5, 2025

Uh oh!

rhmdnd commented Mar 5, 2025

Uh oh!

yuumasato commented Mar 5, 2025

Uh oh!

prb112 commented Mar 5, 2025

Uh oh!

prb112 commented Mar 6, 2025

Uh oh!

prb112 commented Mar 6, 2025

Uh oh!

prb112 commented Mar 12, 2025

Uh oh!

prb112 commented Mar 14, 2025

Uh oh!

KaushikOP commented Mar 28, 2025

Uh oh!

KaushikOP commented Apr 2, 2025

Uh oh!

xiaojiey commented Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xiaojiey commented Apr 7, 2025

Uh oh!

prb112 commented Apr 7, 2025

Uh oh!

KaushikOP commented Apr 11, 2025

Uh oh!

KaushikOP Apr 11, 2025

Choose a reason for hiding this comment

Uh oh!

xiaojiey commented May 17, 2025

Uh oh!

xiaojiey commented May 17, 2025

Uh oh!

xiaojiey commented May 17, 2025

Uh oh!

yuumasato commented May 26, 2025

Uh oh!

KaushikOP commented Jan 24, 2025 •

edited

Loading

xiaojiey commented Apr 7, 2025 •

edited

Loading