Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apiextensions: validation for customresources #47263

Merged
merged 6 commits into from Aug 30, 2017

Conversation

@nikhita
Copy link
Member

commented Jun 9, 2017

  • Add types for validation of CustomResources
  • Fix conversion-gen: #49747
  • Fix defaulter-gen: kubernetes/gengo#61
  • Convert to OpenAPI types
  • Validate CR using go-openapi
  • Validate CRD Schema
  • Add integration tests
  • Fix round trip tests: #51204
  • Add custom fuzzer functions
  • Add custom conversion functions
  • Fix data race while updating CRD: #50098
  • Add feature gate for CustomResourceValidation
  • Fix protobuf generation

Proposal: kubernetes/community#708
Additional discussion: #49879, #50625

Release note:

Add validation for CustomResources via JSON Schema.

/cc @sttts @deads2k

@k8s-ci-robot k8s-ci-robot requested review from sttts and deads2k Jun 9, 2017

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2017

Hi @nikhita. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with @k8s-bot ok to test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@luxas

This comment has been minimized.

Copy link
Member

commented Jun 11, 2017

@nikhita Please use two commits for this PR. One for your code and one for automatically generated code

@k8s-bot ok to test

😄

@nikhita nikhita force-pushed the nikhita:crd-01-validation-types branch from 1cdc672 to 6ef70fe Jun 11, 2017

@nikhita nikhita force-pushed the nikhita:crd-01-validation-types branch from 5044557 to 5e1c126 Jun 12, 2017

@k8s-github-robot k8s-github-robot added size/XL and removed size/XXL labels Jun 12, 2017

@nikhita nikhita force-pushed the nikhita:crd-01-validation-types branch from 5e1c126 to 3bf86ac Jun 12, 2017

@k8s-github-robot k8s-github-robot added size/XXL and removed size/XL labels Jun 12, 2017

@nikhita nikhita force-pushed the nikhita:crd-01-validation-types branch from 3bf86ac to c70f918 Jun 22, 2017

@nikhita nikhita changed the title [WIP] apiextensions: Add types for validation [WIP] apiextensions: validation for customresources Jun 22, 2017

@nikhita nikhita force-pushed the nikhita:crd-01-validation-types branch 2 times, most recently from 4ce3290 to 8192977 Jul 2, 2017

@nikhita

This comment has been minimized.

Copy link
Member Author

commented Jul 2, 2017

This will not pass tests until kubernetes/gengo#61 is merged.

@nikhita nikhita force-pushed the nikhita:crd-01-validation-types branch from 8192977 to 604f48b Jul 2, 2017

in, out := unwrapAlias(a), unwrapAlias(b)
switch {
case in == out:
return true
case in.Kind == out.Kind:
// if the type exists already, return early to avoid recursion
if existingTypes[in] {

This comment has been minimized.

Copy link
@sttts

sttts Jul 3, 2017

Contributor

also here: alreadyVisitedTypes

}

func (s JSON) MarshalJSON() ([]byte, error) {
if len(s.Raw) > 0 {

This comment has been minimized.

Copy link
@deads2k

deads2k Aug 29, 2017

Contributor

note that you're compressing empty into "null". That's reasonable given the rest of the API, it just catches my eye.

func Convert_v1beta1_JSON_To_apiextensions_JSON(in *JSON, out *apiextensions.JSON, s conversion.Scope) error {
if in != nil {
var i interface{}
if err := json.Unmarshal(in.Raw, &i); err != nil {

This comment has been minimized.

Copy link
@deads2k

deads2k Aug 29, 2017

Contributor

This makes sense. The end result is a protobuf shell with a delicious json nougat-y center, right?

This comment has been minimized.

Copy link
@sttts

sttts Aug 29, 2017

Contributor

exactly :)

This comment has been minimized.

Copy link
@nikhita

nikhita Aug 29, 2017

Author Member

Yes :)

@sttts

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2017

@deads2k

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2017

As the JSON protobuf change is squashed already, here are the relevant lines of code:

the internal JSON type (just an alias for interface{}): https://github.com/kubernetes/kubernetes/pull/47263/files#diff-48b8b5215523ba995d8b41c2dfdf68afR192
the external type: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-3029a18192feb70af52de676209da9b6R194,
an (optional) JSON field: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-48b8b5215523ba995d8b41c2dfdf68afR159,
the json marshal/unmarshal: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-3fe7a62396a7708e034adc5c0bbff3fdR121
the conversion: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-a48a0d878266c677cd0c5c9987b327b6R40

This looks fairly straightforward. protobuf wrapping a limited json field with simple deserialization and a little validation. This may actually have been what the service catalog was looking for a while back.

}

// JSONSchemaProps is a JSON-Schema following Specification Draft 4 (http://json-schema.org/).
type JSONSchemaProps struct {

This comment has been minimized.

Copy link
@smarterclayton

smarterclayton Aug 29, 2017

Contributor

Maybe move this into its own types.go file - types_jsonschema.go

@smarterclayton

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2017

Looks like I expected. This has high level approval from me.

nikhita added some commits Jun 11, 2017

Add types for validation of CustomResources
Remove protobuf generation because of the interface type

Add custom fuzzer funcs

Add custom marshalling

Add custom conversion functions

move jsonschema types to separate file
Add generated code
update generated proto
Validate CustomResource
* convert our types to openAPI types
* update strategy to include crd
* use strategy to validate customresource
* add helper funcs
* Fix conversion of empty ref field
* add validation for forbidden fields
* add defaulting for schema field
* Validate CRD Schema
Add integration tests
Update test schema

Add polling for TestCRValidationOnCRDUpdate

Add tests for forbidden fields

Enable featureGate for CustomResourceValidation
Add feature gate for CustomResourceValidation
update feature gates for generic apiserver

Add apiextensions-apiserver features to golint_failures

Ignore alpha feature if gate is disabled

@nikhita nikhita force-pushed the nikhita:crd-01-validation-types branch from 73203bd to 6ba1523 Aug 29, 2017

@sttts

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2017

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 29, 2017

@smarterclayton

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2017

/approve

@k8s-github-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2017

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, nikhita, smarterclayton, sttts

Associated issue: 49747

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 30, 2017

@nikhita: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-gce-bazel 6ba1523 link /test pull-kubernetes-e2e-gce-bazel

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-github-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 30, 2017

Automatic merge from submit-queue

@k8s-github-robot k8s-github-robot merged commit 4457e43 into kubernetes:master Aug 30, 2017

11 of 13 checks passed

pull-kubernetes-e2e-gce-bazel Job failed.
Details
Submit Queue Required Github CI test is not green: pull-kubernetes-e2e-gce-etcd3
Details
cla/linuxfoundation nikhita authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-e2e-gce-etcd3 Jenkins job succeeded.
Details
pull-kubernetes-e2e-gce-gpu Jenkins job succeeded.
Details
pull-kubernetes-e2e-kops-aws Jenkins job succeeded.
Details
pull-kubernetes-federation-e2e-gce Jenkins job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce Jenkins job succeeded.
Details
pull-kubernetes-node-e2e Jenkins job succeeded.
Details
pull-kubernetes-unit Jenkins job succeeded.
Details
pull-kubernetes-verify Jenkins job succeeded.
Details

@tamalsaha tamalsaha referenced this pull request Aug 31, 2017

Open

Improve CRD support #440

2 of 4 tasks complete

@enisoc enisoc moved this from Assigned to Done in CustomResourceDefinition Sep 8, 2017

@munnerz munnerz referenced this pull request Oct 27, 2017

Open

Support OpenAPI v3 #51163

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.