Implement global anchor

[kacejot]

Related issue

closes #2201

Milestone of this PR

/kind feature

Proposed Changes

Added global anchor logic for both validation and strategic merge patch.
Changed e2e tests and updated add-safe-to-evict policy with global anchor.
Global anchor is a key wrapped with <( and ) symbols. For example <(image).
When global anchor condition fails, entire policy application is skipped.

Proof Manifests

Resource:

apiVersion: v1
kind: Pod
metadata:
  name: static-web
  labels:
    role: myrole
spec:
  containers:
    - name: web
      image: 1nginx
      ports:
        - name: web
          containerPort: 80
          protocol: TCP

Validation:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: sample
spec:
  validationFailureAction: enforce
  rules:
  - name: check-container-image
    match:
      resources:
        kinds:
        - Pod
    validate:
      pattern:
        spec:
          containers:
          - name: "*"
            <(image): "nginx"

Strategic merge patch:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: 
  name: add-safe-to-evict
  annotations:
    policies.kyverno.io/category: Workload Management
    policies.kyverno.io/description: The Kubernetes cluster autoscaler does not evict pods that 
      use hostPath or emptyDir volumes. To allow eviction of these pods, the annotation 
      cluster-autoscaler.kubernetes.io/safe-to-evict=true must be added to the pods. 
spec: 
  rules: 
  - name: annotate-empty-dir
    match: 
      resources: 
        kinds: 
        - Pod
    mutate: 
      patchStrategicMerge:
        metadata:
          annotations:
            +(cluster-autoscaler.kubernetes.io/safe-to-evict): "true"
        spec:          
          volumes: 
          - <(emptyDir): {}
  - name: annotate-host-path
    match: 
      resources: 
        kinds: 
        - Pod
    mutate: 
      patchStrategicMerge:
        metadata:
          annotations:
            +(cluster-autoscaler.kubernetes.io/safe-to-evict): "true"
        spec:          
          volumes: 
          - hostPath:
              <(path): "*"

^ Here is the updated policy variant

Checklist

• I have read the contributing guidelines.
• I have added tests that prove my fix is effective or that my feature works.
• My PR contains new or altered behavior to Kyverno and
  • I have added or changed the documentation myself in an existing PR and the link is:
  • I have raised an issue in kyverno/website to track the doc update and the link is:
    [Docs] Add global anchor documentation website#271
  • I have read the PR documentation guide and followed the process including adding proof manifests to this PR.

[chipzoller]

How would or does this work with other anchors in a policy? I assume there can't be two global anchors, correct? It would be good to better understand this change and what impact it may have to both existing policies in the docs/policies page and for users.

[chipzoller]

I'd really like to see this presented at a contributors/community meeting so as to ask some live Q&A without having to go back and forth over GitHub and take two weeks.

[kacejot]

@chipzoller, unfortunately I can't participate today. The reason is that I need some time to prepare manifests and test them. We could have a call tomorrow.

[realshuting]

Hi @kacejot - got a few questions regarding the proof manifests:

1. what does the following policy validate?

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: sample
spec:
  validationFailureAction: enforce
  rules:
  - name: check-container-image
    match:
      resources:
        kinds:
        - Pod
    validate:
      pattern:
        spec:
          containers:
          - name: "*"
            <(image): "nginx"

2. I tested the above mutate policy with the following pod, but it doesn't seem to work:

apiVersion: v1
kind: Pod
metadata:
  name: static-web
  labels:
    role: myrole
spec:
  containers:
    - name: web
      image: 1nginx
      ports:
        - name: web
          containerPort: 80
          protocol: TCP
      volumeMounts:
      - mountPath: /cache
        name: cache-volume
  volumes: 
  - emptyDir: {}
    name: cache-volume

✗ k get pod static-web -o yaml | grep "annotations:" -A5
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"role":"myrole"},"name":"static-web","namespace":"default"},"spec":{"containers":[{"image":"1nginx","name":"web","ports":[{"containerPort":80,"name":"web","protocol":"TCP"}],"volumeMounts":[{"mountPath":"/cache","name":"cache-volume"}]}],"volumes":[{"emptyDir":{},"name":"cache-volume"}]}}
  creationTimestamp: "2021-09-07T06:37:33Z"
  labels:
    role: myrole
...

[kacejot]

@realshuting, in your case global condition fails and policy application is skipped.

[realshuting]

@kacejot - the mutate policy works as expected now.

Having a question about validate:
Testing a validate policy, I would expect the pod creation to be blocked while it didn't:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: sample
spec:
  validationFailureAction: enforce
  rules:
  - name: check-container-image
    match:
      resources:
        kinds:
        - Pod
    validate:
      pattern:
        spec:
          containers:
          - name: "*"
            <(image): "nginx"
          imagePullSecrets:
          - name: my-registry-secret

apiVersion: v1
kind: Pod
metadata:
  name: static-web
  labels:
    role: myrole
spec:
  containers:
    - name: web
      image: nginx
      ports:
        - name: web
          containerPort: 80
          protocol: TCP
  imagePullSecrets:
  - name: other-registry

✗ k apply -f pod.yaml 
pod/static-web created

[realshuting]

Isn't the conditional anchor works only on its child tags? Why the imagePullSecrets is added to the Pod? Same for the next unit test.

[kacejot]

Because imagePullSecrets is not a child tag for image.
image condition applies only to its list element, but not the elements from the other list.

[kacejot]

@realshuting, OK, I will take a look on Monday.

[kacejot]

I re-tested several times, I have global anchor logic working here. Here are my manifests:

Resource:

apiVersion: v1
kind: Pod
metadata:
  name: static-web
  labels:
    role: myrole
spec:
  containers:
    - name: web
      image: nginx
      ports:
        - name: web
          containerPort: 80
          protocol: TCP
  imagePullSecrets:
  - name: other-registry

Resource is blocked:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: sample
spec:
  validationFailureAction: enforce
  rules:
  - name: check-container-image
    match:
      resources:
        kinds:
        - Pod
    validate:
      pattern:
        spec:
          containers:
          - name: "*"
            <(image): "nginx"
          imagePullSecrets:
          - name: my-registry-secret

Resource is validated:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: sample
spec:
  validationFailureAction: enforce
  rules:
  - name: check-container-image
    match:
      resources:
        kinds:
        - Pod
    validate:
      pattern:
        spec:
          containers:
          - name: "*"
            <(image): "nginx1"
          imagePullSecrets:
          - name: my-registry-secret

In last policy global anchor condition fails and the policy application is skipped.

P.S. I see there some merge conflict appeared. I'm going to resolve them immediately.

[kacejot]

Here is the output for the first case where the resource should be blocked:

umka@polarbear ~/Workspace/kyverno/global-anchor-validation                                                                      [16:38:03] 
> $ kubectl create -f ./resource.yaml                                                                                                      
Error from server: error when creating "./resource.yaml": admission webhook "validate.kyverno.svc" denied the request: 

resource Pod/default/static-web was blocked due to the following policies

sample:
  check-container-image: 'validation error: rule check-container-image failed at path
    /spec/imagePullSecrets/0/name/'                                                                                   

umka@polarbear ~/go/src/github.com/kyverno/kyverno                                                                               [18:02:32] 
> $ git status                                                                                                           [±global-anchor ✓]
On branch global-anchor
Your branch is up to date with 'kacejot/global-anchor'.

nothing to commit, working tree clean