How to configure webhook matching rules for subresources

[ycao56]

Found this issue #1056 but it's not clear to me if other subresrouces are supported or not. e.g. /scale

If it is supported, how to configure it?

[maxsmythe]

We've done some preliminary digging on this. So far, it looks like subresources are supported, as long as the specific subresource is supported by ValidatingWebhookConfiguation and validating webhooks generally.

Steps to get specific subresources:

• Add the subresource to Gatekeeper's ValidatingWebhookConfigration object.
• Figure out what the kind associated with that subresource is
• Write a constraint/template against that kind

I tried this out for deployments/scale...

ValidatingWebhookConfiguration changes

• add deployments/scale underthe path: webhooks[name:"validation.gatekeeper.sh"].rules[].resources

Note that other resources might support the scale subresource, such as StatefulSet

What kind to validate against

For my kind cluster, I got:

group: autoscaling
kind: Scale
version: v1

Note that it may be possible that there are other versions of scale that should be validated.

Here is the validation request I received:

{
  "_unstable": {
    "namespace": {
      "apiVersion": "v1",
      "kind": "Namespace",
      "metadata": {
        "annotations": {
          "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{},\"name\":\"local-path-storage\"}}\n"
        },
        "creationTimestamp": "2021-01-13T01:30:37Z",
        "managedFields": [
          {
            "apiVersion": "v1",
            "fieldsType": "FieldsV1",
            "fieldsV1": {
              "f:metadata": {
                "f:annotations": {
                  ".": {},
                  "f:kubectl.kubernetes.io/last-applied-configuration": {}
                }
              },
              "f:status": {
                "f:phase": {}
              }
            },
            "manager": "kubectl",
            "operation": "Update",
            "time": "2021-01-13T01:30:37Z"
          }
        ],
        "name": "local-path-storage",
        "resourceVersion": "247",
        "selfLink": "/api/v1/namespaces/local-path-storage",
        "uid": "8f2e3d04-24c7-4f29-adc2-592a8de7ecb6"
      },
      "spec": {
        "finalizers": [
          "kubernetes"
        ]
      },
      "status": {
        "phase": "Active"
      }
    }
  },
  "dryRun": false,
  "kind": {
    "group": "autoscaling",
    "kind": "Scale",
    "version": "v1"
  },
  "name": "local-path-provisioner",
  "namespace": "local-path-storage",
  "object": {
    "apiVersion": "autoscaling/v1",
    "kind": "Scale",
    "metadata": {
      "creationTimestamp": "2021-01-13T01:30:37Z",
      "name": "local-path-provisioner",
      "namespace": "local-path-storage",
      "resourceVersion": "3246355",
      "uid": "a6d1a90d-becc-432c-afb1-388e5ca06a41"
    },
    "spec": {
      "replicas": 1
    },
    "status": {
      "replicas": 1,
      "selector": "app=local-path-provisioner"
    }
  },
  "oldObject": {
    "apiVersion": "autoscaling/v1",
    "kind": "Scale",
    "metadata": {
      "creationTimestamp": "2021-01-13T01:30:37Z",
      "name": "local-path-provisioner",
      "namespace": "local-path-storage",
      "resourceVersion": "3246355",
      "uid": "a6d1a90d-becc-432c-afb1-388e5ca06a41"
    },
    "spec": {
      "replicas": 1
    },
    "status": {
      "replicas": 1,
      "selector": "app=local-path-provisioner"
    }
  },
  "operation": "UPDATE",
  "options": {
    "apiVersion": "meta.k8s.io/v1",
    "kind": "UpdateOptions"
  },
  "requestKind": {
    "group": "autoscaling",
    "kind": "Scale",
    "version": "v1"
  },
  "requestResource": {
    "group": "apps",
    "resource": "deployments",
    "version": "v1"
  },
  "requestSubResource": "scale",
  "resource": {
    "group": "apps",
    "resource": "deployments",
    "version": "v1"
  },
  "subResource": "scale",
  "uid": "f81c338d-87fb-45f5-ac07-1f0e0d616792",
  "userInfo": {
    "groups": [
      "system:masters",
      "system:authenticated"
    ],
    "username": "kubernetes-admin"
  }
}

It looks like object and oldObject are defined by:

https://github.com/kubernetes/api/blob/0d6bea24dc491757b8ef25113eec746f0634d3e9/autoscaling/v1/types.go#L113-L147

[ycao56]

Thanks @maxsmythe ! It did work!

[maxsmythe]

Glad it worked!

@ritazh @sozercan @brycecr @shomron

Should we have documentation around this? Do we want to watch subresources by default? I know we came to the conclusion "probably not" WRT PodExecOptions, scale seems like it may be a more common request.

[brycecr]

What about the status subresource? It seems like some subresources are commonly accessed and others are obscure

I like saying "worse is better" and requiring subresources to be explicitly added in order to be watched, unless we get more requests for watching subresources. It might be useful to provide some documentation / example around this, though, to make the pattern more salient.

[maxsmythe]

"worse is better" is definitely safer in terms of policy not interfering with cluster operations.

I wonder if we could include certain common subresources that may be important "/scale", and maybe "/status"

Being concerned about policy against status seems weird, but may be relevant per

open-policy-agent/gatekeeper-library#45

Where apparently changing status can have some effect on the state of the world.

I'm wondering how/if status updates are subject to the VWH when modified via the status subresource.