Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] test Command Not Respecting Autogen Rules for Pod Controllers #1922

Closed
daviddyball opened this issue May 21, 2021 · 8 comments · Fixed by #2093
Closed

[BUG] test Command Not Respecting Autogen Rules for Pod Controllers #1922

daviddyball opened this issue May 21, 2021 · 8 comments · Fixed by #2093
Assignees
@vyankyGH
Labels
bug Something isn't working
Milestone
Kyverno Release 1...

Comments

@daviddyball
Copy link

daviddyball commented May 21, 2021 •
edited

Software version numbers

  • Kubernetes version: N/A (using local kyverno test)
  • Kyverno version: 1.3.6

Describe the bug

When writing test-cases for use with the kyverno test sub-command it appears that autogen rules for pod-controllers aren't respected. If I write a policy that includes match.resources.kinds: ["Pod"]

NOTE: The same policy works fine when using kyverno apply, which suggests that it is only the test sub-command that is not respecting auto-gen rules correctly.

To Reproduce

Create the following files
policy.yaml

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-common-labels
spec:
  validationFailureAction: enforce
  rules:
  - name: check-for-labels
    match:
      resources:
        kinds:
        - Pod
    validate:
      message: "Both `app` and `owner` labels must be set on all workloads"
      pattern:
        metadata:
          labels:
            app: "?*"
            owner: "?*"

resource.yaml

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-labels
  labels:
    app: my-pod
    owner: me
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-missing-labels
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-with-labels
  labels:
    app: my-pod
    owner: me
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-missing-labels

test.yaml

policies:
- policy.yaml
resources:
- resource.yaml
results:
# TEST: Pod with Labels Should Pass
- policy: require-common-labels
  rule: check-for-labels
  status: pass
  resource: pod-with-labels

# TEST: Pod Missing Labels Should Fail
- policy: require-common-labels
  rule: check-for-labels
  status: pass
  resource: pod-missing-labels

# TEST: Deployment with Labels Should Pass
- policy: require-common-labels
  rule: check-for-labels
  status: pass
  resource: deployment-with-labels

# TEST: Deployment with Labels Should Fail
- policy: require-common-labels
  rule: check-for-labels
  status: fail
  resource: deployment-missing-labels

Expected behavior

The run kyverno test . in the same directory. The two tests on Pod resources should succeed, but the two Deployment resource tests will fail.

Using the same policy above when using kyverno apply --resource resource.yaml works fine with the above policy (the auto-gen correctly creates policies for Deployment, DaemonSet and Job as expected.

Debug Output

> kyverno test . -v5

I0521 09:17:40.250283  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.batch.v1.CronJob" "reason"="gvk not found by the given definition name io.k8s.api.batch.v1.CronJob, [batch/v1beta1/CronJob]"
I0521 09:17:40.250550  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.discovery.v1.EndpointSlice" "reason"="gvk not found by the given definition name io.k8s.api.discovery.v1.EndpointSlice, [discovery.k8s.io/v1beta1/EndpointSlice]"
I0521 09:17:40.250705  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.node.v1alpha1.RuntimeClass" "reason"="gvk not found by the given definition name io.k8s.api.node.v1alpha1.RuntimeClass, [node.k8s.io/v1/RuntimeClass node.k8s.io/v1beta1/RuntimeClass]"
I0521 09:17:40.250741  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.policy.v1.PodDisruptionBudget" "reason"="gvk not found by the given definition name io.k8s.api.policy.v1.PodDisruptionBudget, [policy/v1beta1/PodDisruptionBudget]"
I0521 09:17:40.250823  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.rbac.v1alpha1.ClusterRole" "reason"="gvk not found by the given definition name io.k8s.api.rbac.v1alpha1.ClusterRole, [rbac.authorization.k8s.io/v1/ClusterRole rbac.authorization.k8s.io/v1beta1/ClusterRole]"
I0521 09:17:40.250845  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.rbac.v1alpha1.ClusterRoleBinding" "reason"="gvk not found by the given definition name io.k8s.api.rbac.v1alpha1.ClusterRoleBinding, [rbac.authorization.k8s.io/v1/ClusterRoleBinding rbac.authorization.k8s.io/v1beta1/ClusterRoleBinding]"
I0521 09:17:40.250877  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.rbac.v1alpha1.Role" "reason"="gvk not found by the given definition name io.k8s.api.rbac.v1alpha1.Role, [rbac.authorization.k8s.io/v1/Role rbac.authorization.k8s.io/v1beta1/Role]"
I0521 09:17:40.250905  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.rbac.v1alpha1.RoleBinding" "reason"="gvk not found by the given definition name io.k8s.api.rbac.v1alpha1.RoleBinding, [rbac.authorization.k8s.io/v1/RoleBinding rbac.authorization.k8s.io/v1beta1/RoleBinding]"
I0521 09:17:40.250949  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.scheduling.v1alpha1.PriorityClass" "reason"="gvk not found by the given definition name io.k8s.api.scheduling.v1alpha1.PriorityClass, [scheduling.k8s.io/v1/PriorityClass scheduling.k8s.io/v1beta1/PriorityClass]"
I0521 09:17:40.251329  612836 validation.go:186]  "msg"="unable to cache OpenAPISchema"  "definitionName"="io.k8s.api.storage.v1alpha1.VolumeAttachment" "reason"="gvk not found by the given definition name io.k8s.api.storage.v1alpha1.VolumeAttachment, [storage.k8s.io/v1/VolumeAttachment storage.k8s.io/v1beta1/VolumeAttachment]"

Executing ...I0521 09:17:40.261689  612836 common.go:65]  "msg"="reading policies"  "path"="policy.yaml"
I0521 09:17:40.262107  612836 common.go:149]  "msg"="read policies"  "errors"=0 "policies"=1
I0521 09:17:40.262122  612836 policymutation.go:157] apply "msg"="setting default value"  "spec.background"=true
I0521 09:17:40.262144  612836 policymutation.go:174] apply "msg"="generated JSON Patch to set default"  "spec.background"=true
I0521 09:17:40.262166  612836 policymutation.go:248] apply "msg"="auto generating rule for pod controllers"  "controllers"="DaemonSet,Deployment,Job,StatefulSet,CronJob"
I0521 09:17:40.262184  612836 policymutation.go:431] apply/generateRuleForControllers "msg"="processing rule"  "rulename"="check-for-labels"
I0521 09:17:40.262258  612836 cronjob.go:22] apply/handleCronJob "msg"="generating rule for cronJob"  
I0521 09:17:40.262281  612836 policymutation.go:431] apply/handleCronJob/generateRuleForControllers "msg"="processing rule"  "rulename"="check-for-labels"

applying 1 policy to 4 resources... 
I0521 09:17:40.270711  612836 common.go:442]  "msg"="applying policy on resource"  "policy"="require-common-labels" "resource"="default/Pod/pod-with-labels"
I0521 09:17:40.270752  612836 mutation.go:38] EngineMutate "msg"="start policy processing" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.270721737+01:00"
I0521 09:17:40.270780  612836 mutation.go:150] EngineMutate "msg"="finished processing policy" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "mutationRulesApplied"=0 "processingTime"="45.476µs"
I0521 09:17:40.270800  612836 validation.go:27] EngineValidate "msg"="start policy processing" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.270785059+01:00"
I0521 09:17:40.270828  612836 validation.go:105] EngineValidate "msg"="matched validate rule" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" 
I0521 09:17:40.270903  612836 validation.go:227] EngineValidate "msg"="start processing rule" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" "startTime"="2021-05-21T09:17:40.270885609+01:00"
I0521 09:17:40.270954  612836 validation.go:246] EngineValidate "msg"="successfully processed rule" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" 
I0521 09:17:40.270978  612836 validation.go:232] EngineValidate "msg"="finished processing rule" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" "processingTime"="73.227µs"
I0521 09:17:40.271004  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "reason"="rule autogen-check-for-labels not matched:\n 1. kind does not match [DaemonSet Deployment Job StatefulSet]"
I0521 09:17:40.271032  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-cronjob-check-for-labels" "reason"="rule autogen-cronjob-check-for-labels not matched:\n 1. kind does not match [CronJob]"
I0521 09:17:40.271051  612836 validation.go:30] EngineValidate "msg"="finished policy processing" "kind"="Pod" "name"="pod-with-labels" "namespace"="default" "policy"="require-common-labels" "processingTime"="254.37µs" "validationRulesApplied"=1
I0521 09:17:40.271090  612836 common.go:442]  "msg"="applying policy on resource"  "policy"="require-common-labels" "resource"="default/Pod/pod-missing-labels"
I0521 09:17:40.271108  612836 mutation.go:38] EngineMutate "msg"="start policy processing" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.27109404+01:00"
I0521 09:17:40.271126  612836 mutation.go:150] EngineMutate "msg"="finished processing policy" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "mutationRulesApplied"=0 "processingTime"="21.137µs"
I0521 09:17:40.271143  612836 validation.go:27] EngineValidate "msg"="start policy processing" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.271130588+01:00"
I0521 09:17:40.271169  612836 validation.go:105] EngineValidate "msg"="matched validate rule" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" 
I0521 09:17:40.271248  612836 validation.go:227] EngineValidate "msg"="start processing rule" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" "startTime"="2021-05-21T09:17:40.271232969+01:00"
I0521 09:17:40.271276  612836 validate.go:44]  "msg"="Pattern and resource have different structures."  "current"="<nil>" "expected"="map[string]interface {}" "path"="/metadata/labels/"
I0521 09:17:40.271297  612836 validation.go:240] EngineValidate "msg"="validation failed" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" "error"="Pattern and resource have different structures. Path: /metadata/labels/. Expected map[string]interface {}, found <nil>" "path"="/metadata/labels/"
I0521 09:17:40.271312  612836 validation.go:232] EngineValidate "msg"="finished processing rule" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" "processingTime"="68.64µs"
I0521 09:17:40.271335  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "reason"="rule autogen-check-for-labels not matched:\n 1. kind does not match [DaemonSet Deployment Job StatefulSet]"
I0521 09:17:40.271355  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-cronjob-check-for-labels" "reason"="rule autogen-cronjob-check-for-labels not matched:\n 1. kind does not match [CronJob]"
I0521 09:17:40.271371  612836 validation.go:30] EngineValidate "msg"="finished policy processing" "kind"="Pod" "name"="pod-missing-labels" "namespace"="default" "policy"="require-common-labels" "processingTime"="229.696µs" "validationRulesApplied"=1
I0521 09:17:40.271410  612836 common.go:442]  "msg"="applying policy on resource"  "policy"="require-common-labels" "resource"="default/Deployment/deployment-with-labels"
I0521 09:17:40.271428  612836 mutation.go:38] EngineMutate "msg"="start policy processing" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.271415077+01:00"
I0521 09:17:40.271447  612836 mutation.go:150] EngineMutate "msg"="finished processing policy" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "mutationRulesApplied"=0 "processingTime"="20.819µs"
I0521 09:17:40.271463  612836 validation.go:27] EngineValidate "msg"="start policy processing" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.271451657+01:00"
I0521 09:17:40.271485  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" "reason"="rule check-for-labels not matched:\n 1. kind does not match [Pod]"
I0521 09:17:40.271508  612836 validation.go:105] EngineValidate "msg"="matched validate rule" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" 
I0521 09:17:40.271595  612836 validation.go:227] EngineValidate "msg"="start processing rule" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "startTime"="2021-05-21T09:17:40.271576441+01:00"
I0521 09:17:40.271612  612836 validate.go:44]  "msg"="Pattern and resource have different structures."  "current"="<nil>" "expected"="map[string]interface {}" "path"="/spec/"
I0521 09:17:40.271632  612836 validation.go:240] EngineValidate "msg"="validation failed" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "error"="Pattern and resource have different structures. Path: /spec/. Expected map[string]interface {}, found <nil>" "path"="/spec/"
I0521 09:17:40.271647  612836 validation.go:232] EngineValidate "msg"="finished processing rule" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "processingTime"="60.061µs"
I0521 09:17:40.271668  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-cronjob-check-for-labels" "reason"="rule autogen-cronjob-check-for-labels not matched:\n 1. kind does not match [CronJob]"
I0521 09:17:40.271682  612836 validation.go:30] EngineValidate "msg"="finished policy processing" "kind"="Deployment" "name"="deployment-with-labels" "namespace"="default" "policy"="require-common-labels" "processingTime"="220.835µs" "validationRulesApplied"=1
I0521 09:17:40.271720  612836 common.go:442]  "msg"="applying policy on resource"  "policy"="require-common-labels" "resource"="default/Deployment/deployment-missing-labels"
I0521 09:17:40.271736  612836 mutation.go:38] EngineMutate "msg"="start policy processing" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.271724878+01:00"
I0521 09:17:40.271753  612836 mutation.go:150] EngineMutate "msg"="finished processing policy" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "mutationRulesApplied"=0 "processingTime"="18.829µs"
I0521 09:17:40.271772  612836 validation.go:27] EngineValidate "msg"="start policy processing" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "startTime"="2021-05-21T09:17:40.271757866+01:00"
I0521 09:17:40.271799  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="check-for-labels" "reason"="rule check-for-labels not matched:\n 1. kind does not match [Pod]"
I0521 09:17:40.271822  612836 validation.go:105] EngineValidate "msg"="matched validate rule" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" 
I0521 09:17:40.271920  612836 validation.go:227] EngineValidate "msg"="start processing rule" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "startTime"="2021-05-21T09:17:40.271904235+01:00"
I0521 09:17:40.272027  612836 validate.go:44]  "msg"="Pattern and resource have different structures."  "current"="<nil>" "expected"="map[string]interface {}" "path"="/spec/"
I0521 09:17:40.272055  612836 validation.go:240] EngineValidate "msg"="validation failed" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "error"="Pattern and resource have different structures. Path: /spec/. Expected map[string]interface {}, found <nil>" "path"="/spec/"
I0521 09:17:40.272078  612836 validation.go:232] EngineValidate "msg"="finished processing rule" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-check-for-labels" "processingTime"="157.949µs"
I0521 09:17:40.272171  612836 validation.go:199] EngineValidate "msg"="resource does not match rule" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "rule"="autogen-cronjob-check-for-labels" "reason"="rule autogen-cronjob-check-for-labels not matched:\n 1. kind does not match [CronJob]"
I0521 09:17:40.272210  612836 validation.go:30] EngineValidate "msg"="finished policy processing" "kind"="Deployment" "name"="deployment-missing-labels" "namespace"="default" "policy"="require-common-labels" "processingTime"="425.369µs" "validationRulesApplied"=1
│───│───────────────────────────────────────────────────────────────────────│────────│
│ # │ TEST                                                                  │ RESULT │
│───│───────────────────────────────────────────────────────────────────────│────────│
│ 1 │ pod-with-labels with require-common-labels/check-for-labels           │ Pass   │
│ 2 │ pod-missing-labels with require-common-labels/check-for-labels        │ Pass   │
│ 3 │ deployment-with-labels with require-common-labels/check-for-labels    │ Fail   │
│ 4 │ deployment-missing-labels with require-common-labels/check-for-labels │ Fail   │
│───│───────────────────────────────────────────────────────────────────────│────────│

Additional context

N/A
@daviddyball daviddyball added the bug Something isn't working label May 21, 2021
@daviddyball daviddyball mentioned this issue May 21, 2021
Closed
@vyankyGH vyankyGH self-assigned this May 21, 2021
@realshuting realshuting added this to the Kyverno Release 1.3.7 milestone May 21, 2021
@vyankyGH
Copy link
Contributor

@daviddyball Extremely sorry for late reply. Autogen rule apply only for Kind Pod. If we want policy for deployment we have create another policy specific for deloyment, or we can add deployment in match-resource-kind.

@daviddyball
Copy link
Author

daviddyball commented Jun 22, 2021 •
edited

@vyankyGH What you suggested is the workaround I've implemented in my tests... but it's not a fix.

Isn't the point of auto-gen rules to take policies that target Pods and automatically make them fit with Deployment, Job and Daemonset objects so you don't have to add duplicate policies?

As far as I'm aware this is only affecting the test subcommand and auto-gen works normally against Deployment, Job and DaemonSet resources when you use kyverno apply --resource policy.yaml

Edit: I've added notes about the different behaviour of kyverno apply vs kyverno test in the original bug description.

@github-actions
Copy link
Contributor

Thanks for opening this issue!
I have applied any labels matching special text in your title and description.

Please review the labels and make any necessary changes.

@github-actions github-actions bot added the API Call Use case for API Server calls to fetch JSON data label Jun 22, 2021
@realshuting realshuting removed the API Call Use case for API Server calls to fetch JSON data label Jun 22, 2021
@realshuting
Copy link
Member

@vyankd - any updates?

@vyankyGH
Copy link
Contributor

vyankyGH commented Jul 1, 2021 •
edited

Hi @daviddyball I have tried the above policy and resource with apply command I get the same behavior as test command.
applying 1 policy to 4 resources...
Here what I see:

policy require-common-labels -> resource default/Pod/pod-missing-labels failed:
1. check-for-labels: validation error: Both `app` and `owner` labels must be set on all workloads. Rule check-for-labels failed at path /metadata/labels/

policy require-common-labels -> resource default/Deployment/deployment-with-labels failed:
1. autogen-check-for-labels: validation error: Both `app` and `owner` labels must be set on all workloads. Rule autogen-check-for-labels failed at path /spec/

policy require-common-labels -> resource default/Deployment/deployment-missing-labels failed:
1. autogen-check-for-labels: validation error: Both `app` and `owner` labels must be set on all workloads. Rule autogen-check-for-labels failed at path /spec/

pass: 1, fail: 3, warn: 0, error: 0, skip: 0

There are 3 resources fail and 1 pass for pod with labels.

And we want test command against Pod auto-gen rules,
Rule which is created are has autogen- added to the rule name and the pattern created for deployment is

 name: autogen-check-for-labels
    validate:
      message: Both `app` and `owner` labels must be set on all workloads
      pattern:
        spec:
          template:
            metadata:
              labels:
                app: ?*
                owner: ?*

I made following changes in deployment resource and test.yaml accordingly and it works.

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-labels
  labels:
    app: my-pod
    owner: me
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-missing-labels
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-with-labels
spec:
  template:
    metadata:
      labels:
        app: my-pod
        owner: me
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-missing-labels

I have added autogen- to the rule name
test.yaml

- policy: require-common-labels
  rule: check-for-labels
  status: pass
  resource: pod-with-labels

# TEST: Pod Missing Labels Should Fail
- policy: require-common-labels
  rule: check-for-labels
  status: pass

# TEST: Deployment with Labels Should Pass
- policy: require-common-labels
  rule: autogen-check-for-labels
  status: pass
  resource: deployment-with-labels

# TEST: Deployment with Labels Should Fail
- policy: require-common-labels
  rule: autogen-check-for-labels
  status: fail
  resource: deployment-missing-labels

Result:

Executing ...
applying 1 policy to 4 resources...
│───│──────────────────────────────────────────────────────────────────────────
│ # │ TEST                                                                       │ RESULT │
│───│──────────────────────────────────────────────────────────────────────────
│ 1 │ pod-with-labels with require-common-labels/check-for-labels                │ Pass   │
│ 2 │ pod-missing-labels with require-common-labels/check-for-labels             │ Fail   │
│ 3 │ deployment-with-labels with require-common-labels/autogen-check-for-labels │ Pass   │
│ 4 │ deployment-missing-labels with require-common-labels/autogen-check-for-labels      │ Fail   │
│───│──────────────────────────────────────────────────────────────────────────

Please Let me know, If its work's.
Thanks

@daviddyball
Copy link
Author

@vyankyGH thanks for coming back to me.... I see what you are saying.... I have to manually specify the autogen- prefix for the Deployment resources in my test.yaml e.g.

- rule: autogen-check-for-labels
vs.
- rule: check-for-labels

I guess this confusion came about because it's not documented anywhere for the end user that, when doing tests on auto-gen rules you have to use the rule name containing the autogen-{rule} prefix (and possibly autogen-cronjob-{rule} if you're testing a CronJob).

This isn't great from a UX perspective because, as a user, I have to understand the auto-gen naming format and anticipate/guess what the resulting name of my policy will be at runtime. What's to stop that from behaving differently in the future and breaking all of my tests?

@vyankyGH
Copy link
Contributor

vyankyGH commented Jul 1, 2021

@daviddyball yes your correct we should documented that and will do that.
@realshuting what's your thought's on this?

@vyankyGH
Copy link
Contributor

vyankyGH commented Jul 2, 2021

Hi @daviddyball I will raised PR for support autogen-naming for test command. After that, for test against autogen rule's, No need to add autogen- prefix for rule names.it will work as you defined in test.yaml.
Thanks.

This was referenced Jul 2, 2021
Merged
Closed
This was referenced Sep 3, 2021
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vyankyGH vyankyGH

Labels
bug Something isn't working
Projects
None yet
Milestone
Kyverno Release 1.4.2
Development

Successfully merging a pull request may close this issue.

Add autogen-support for test command vyankyGH/kyverno
3 participants
@daviddyball @vyankyGH @realshuting