Add structural information to Constraint Kind CRDs

[julianKatz]

Kubernetes v1 CRDs require structural schemas. This means that, where
possible, JSONSchemas should identify the type of a field. Fields
without this type information will require the key: value pair of
x-kubernetes-unknown-fields: true to be set. This signals the API
server to save the content found in that field to etcd without
validation.

This PR adds the remaining type information to the MatchSchema()
function, rendering its output structural. This func's output is what
populates the match section of the CRD for a Constraint kind.

It also imports Constraint Framework, including the updates made in
open-policy-agent/frameworks#114. This change transforms the schema
information provided by users in a v1beta1 ConstraintTemplate to be
structural.

Contributes to #550

Signed-off-by: juliankatz juliankatz@google.com

[maxsmythe]

1 comment, we'll need to explicitly add matchLabels as part of moving to v1 CRDs because we wont be able to rely on preserveUnknownFields

[sozercan]

e2e is failing with

Error from server ([spec.validation.openAPIV3Schema.properties[metadata].type: Invalid value: "": must be object, spec.validation.openAPIV3Schema.properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[parameters].properties[labels].items: Required value: must be specified, spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]): error when creating "test/bats/tests/templates/k8srequiredlabels_template.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [spec.validation.openAPIV3Schema.properties[metadata].type: Invalid value: "": must be object, spec.validation.openAPIV3Schema.properties[metadata].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].properties[parameters].properties[labels].items: Required value: must be specified, spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.properties[spec].type: Required value: must not be empty for specified object fields, spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]

[julianKatz]

@maxsmythe , I'm still figuring out the earlier e2e test failures. That's why I've got code commented out. I believe the failures were related to this new code. Will confirm now.

[julianKatz]

I've confirmed that enabling the MatchLabels code (at least in its present form) triggers these errors in the e2e tests:

spec.validation.openAPIV3Schema.properties[metadata].type: Invalid value: "": must be object
spec.validation.openAPIV3Schema.properties[metadata].type: Required value: must not be empty for specified object fields
spec.validation.openAPIV3Schema.properties[spec].properties[parameters].properties[labels].items: Required value: must be specified
spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields
spec.validation.openAPIV3Schema.properties[spec].type: Required value: must not be empty for specified object fields
spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root]): error when creating "test/bats/tests/templates/k8srequiredlabels_template.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [spec.validation.openAPIV3Schema.properties[metadata].type: Invalid value: "": must be object
spec.validation.openAPIV3Schema.properties[metadata].type: Required value: must not be empty for specified object fields
spec.validation.openAPIV3Schema.properties[spec].properties[parameters].properties[labels].items: Required value: must be specified
spec.validation.openAPIV3Schema.properties[spec].properties[parameters].type: Required value: must not be empty for specified object fields
spec.validation.openAPIV3Schema.properties[spec].type: Required value: must not be empty for specified object fields
spec.validation.openAPIV3Schema.type: Required value: must not be empty at the root

[julianKatz]

Tests are passing with MatchLabels turned on, but the x-kuberentes-preserve-unknown-fields line commented out. Trying again with it uncommented.

[julianKatz]

I've confirmed that adding the x-kubernetes-preserve-unknown-fields: true key/value pair (or at least this line of go: XPreserveUnknownFields: &trueBool) to the matchLabels section triggers the e2e failure described earlier.

[maxsmythe]

Weird that it's validating the entire resource using "v1" rules because of that. Let's put this PR on ice for a bit until the CF returns V1 CRDs, then we can revisit?

[julianKatz]

This is blocked until open-policy-agent/frameworks#113 is merged

[shomron]

Do we need to move to apiextensionsv1 here?

[maxsmythe]

No reason not to, IMO

[julianKatz]

It turns out that doing this upgrade (apiextensionsv1beta1 --> apiextensionsv1) actually causes a number of test breakages. As this PR is already large, I'm going to save those for a follow-up PR.

Does that work for you? @shomron

[ritazh]

There are alot more places in GK that are still usingk8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1 so +1 on followup.

[maxsmythe]

LGTM