Upgrade gatekeeper to 3.6.0

[lsviben]

What does this PR do / Why do we need it:
Upgrades the OPA integration Gatekeeper from 3.5.2 to 3.6.0.

Important change here is that ConstraintTemplates are now v1 from v1beta1

Does this PR close any issues?:
Fixes #8832

Does this PR introduce a user-facing change?:

Upgraded OPA integration Gatekeeper version from 3.5.2 to 3.6.0. Important change here is that the OPA ConstraintTemplates are upgraded from v1beta1 to v1. What this means for Kubermatic ConstraintTemplates is that when creating new ones, the `spec.crd.spec.validation.openAPIV3Schema` needs to be structurally correct. The old ConstraintTemplates will have a `legacySchema` flag in the `spec.crd.spec.validation` so they wont need to be migrated yet, although we suggest editing them and fixing the schema to be structurally correct.

More info about change from v1beta1 to v1 in gatekeeper docs:  https://open-policy-agent.github.io/gatekeeper/website/docs/constrainttemplates#v1-constraint-template

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

/retest

[xrstf]

/override pre-kubermatic-test-helm-charts

This one is broken because of the new docker registry mirroring stuff, because of the dangling socat process at the end. The charts in this PR are actually fine.

/test pre-kubermatic-opa-e2e

This one is flaky.

Are we sure this can be applied over an existing OPA installation without issues? Changes to CRDs are always scary to me.

[kubermatic-bot]

@xrstf: Overrode contexts on behalf of xrstf: pre-kubermatic-test-helm-charts

In response to this:

/override pre-kubermatic-test-helm-charts

This one is broken because of the new docker registry mirroring stuff, because of the dangling socat process at the end. The charts in this PR are actually fine.

/test pre-kubermatic-opa-e2e

This one is flaky.

Are we sure this can be applied over an existing OPA installation without issues? Changes to CRDs are always scary to me.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lsviben]

/override pre-kubermatic-test-helm-charts

This one is broken because of the new docker registry mirroring stuff, because of the dangling socat process at the end. The charts in this PR are actually fine.

/test pre-kubermatic-opa-e2e

This one is flaky.

Are we sure this can be applied over an existing OPA installation without issues? Changes to CRDs are always scary to me.

Nor sure. This opa test became flaky again after the crd migration, i have to check it to see why.

So i dont know if now its flaky or its something real. But no rush with this PR ill check it on monday

[xrstf]

let my quote my notes from team A's slack channel 2 hours ago:

i just checked and the situation was

• gatekeeper pods were scheduled, but could not run because the nodes weren't ready yet. the test should wait for node==Ready, not node==Exists
• the gatekeeper webhook seems to block the gatekeeper deployments, because k8s fails to create events because the webhook is not yet ready. chicken & egg problem.

maybe the webhook should only be deployed after the gatekeeper pods are ready.

we already have 2 statusses on the ClusterStatus for the 2 gatekeeper deployments, so tieing the webhook deployment to the status should be trivial.

there's a chance the CRD mirgation made it flaky, but it would guess more that the improved ClusterStatus handling made things so fast that we now experience these issues 😁

[lsviben]

let my quote my notes from team A's slack channel 2 hours ago:

i just checked and the situation was

• gatekeeper pods were scheduled, but could not run because the nodes weren't ready yet. the test should wait for node==Ready, not node==Exists
• the gatekeeper webhook seems to block the gatekeeper deployments, because k8s fails to create events because the webhook is not yet ready. chicken & egg problem.

maybe the webhook should only be deployed after the gatekeeper pods are ready.
we already have 2 statusses on the ClusterStatus for the 2 gatekeeper deployments, so tieing the webhook deployment to the status should be trivial.

there's a chance the CRD mirgation made it flaky, but it would guess more that the improved ClusterStatus handling made things so fast that we now experience these issues grin

Right, it could be the ClusterStatus.

Anyway the webhook shouldnt block the deployment, because it has a failure policy set to Ignore (allow if validation request fails), but maybe something changed, ill check it out.

In the meantime, regarding this PR, I found 1 place in the e2e tests where the old v1beta1 CT was being used, maybe it helps here.

[lsviben]

let my quote my notes from team A's slack channel 2 hours ago:

i just checked and the situation was

• gatekeeper pods were scheduled, but could not run because the nodes weren't ready yet. the test should wait for node==Ready, not node==Exists
• the gatekeeper webhook seems to block the gatekeeper deployments, because k8s fails to create events because the webhook is not yet ready. chicken & egg problem.

maybe the webhook should only be deployed after the gatekeeper pods are ready.
we already have 2 statusses on the ClusterStatus for the 2 gatekeeper deployments, so tieing the webhook deployment to the status should be trivial.

there's a chance the CRD mirgation made it flaky, but it would guess more that the improved ClusterStatus handling made things so fast that we now experience these issues grin

Right, it could be the ClusterStatus.

Anyway the webhook shouldnt block the deployment, because it has a failure policy set to Ignore (allow if validation request fails), but maybe something changed, ill check it out.

In the meantime, regarding this PR, I found 1 place in the e2e tests where the old v1beta1 CT was being used, maybe it helps here.

Ill first focus on fixing the OPA tests flakiness, then check here. It seems this one is failing for a different issue, so Ill put this on hold for now.

/hold

[lsviben]

There is some issue with the OPA e2e tests in this PR beside the flakiness. So I will be checking it here, keeping the PR on hold

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

/retest

[lsviben]

@xrstf this is ready for re-review. The failing opa test was fixed, it was fixed in the commit, the issue was that dynamically you cant (or at least i havent found a way) properly create openAPIschema for arrays which includes its type. And as the new v1 ConstraintTemplates demand this, it was failing.

Btw the opa test is still flaky because of #9047 .

[lsviben]

@xrstf PTAL

[lsviben]

/retest

[xrstf]

/approve

[kubermatic-bot]

LGTM label has been added.

Git tree hash: 557d8027fbd8d1c520041e63efd4cdc25c686350

[kubermatic-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lsviben, xrstf

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [lsviben,xrstf]
• charts/kubermatic-operator/OWNERS [xrstf]
• cmd/kubermatic-api/OWNERS [lsviben]
• pkg/api/OWNERS [lsviben]
• pkg/apis/OWNERS [xrstf]
• pkg/controller/OWNERS [xrstf]
• pkg/ee/allowed-registry-controller/OWNERS [xrstf]
• pkg/handler/OWNERS [lsviben]
• pkg/provider/kubernetes/OWNERS [lsviben]
• pkg/resources/OWNERS [xrstf]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment