Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apiextensions: validation for customresources #47263

Merged
merged 6 commits into from
Aug 30, 2017

Conversation

nikhita
Copy link
Member

@nikhita nikhita commented Jun 9, 2017

Proposal: kubernetes/community#708
Additional discussion: #49879, #50625

Release note:

Add validation for CustomResources via JSON Schema.

/cc @sttts @deads2k

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 9, 2017
@k8s-ci-robot
Copy link
Contributor

Hi @nikhita. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with @k8s-bot ok to test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 9, 2017
@k8s-github-robot k8s-github-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. release-note-none Denotes a PR that doesn't merit a release note. labels Jun 9, 2017
@luxas
Copy link
Member

luxas commented Jun 11, 2017

@nikhita Please use two commits for this PR. One for your code and one for automatically generated code

@k8s-bot ok to test

😄

@k8s-github-robot k8s-github-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 11, 2017
@k8s-github-robot k8s-github-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 12, 2017
@k8s-github-robot k8s-github-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 12, 2017
@nikhita nikhita changed the title [WIP] apiextensions: Add types for validation [WIP] apiextensions: validation for customresources Jun 22, 2017
@nikhita nikhita force-pushed the crd-01-validation-types branch 2 times, most recently from 4ce3290 to 8192977 Compare July 2, 2017 22:37
@nikhita
Copy link
Member Author

nikhita commented Jul 2, 2017

This will not pass tests until kubernetes/gengo#61 is merged.

in, out := unwrapAlias(a), unwrapAlias(b)
switch {
case in == out:
return true
case in.Kind == out.Kind:
// if the type exists already, return early to avoid recursion
if existingTypes[in] {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also here: alreadyVisitedTypes

}

func (s JSON) MarshalJSON() ([]byte, error) {
if len(s.Raw) > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that you're compressing empty into "null". That's reasonable given the rest of the API, it just catches my eye.

func Convert_v1beta1_JSON_To_apiextensions_JSON(in *JSON, out *apiextensions.JSON, s conversion.Scope) error {
if in != nil {
var i interface{}
if err := json.Unmarshal(in.Raw, &i); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense. The end result is a protobuf shell with a delicious json nougat-y center, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes :)

@sttts
Copy link
Contributor

sttts commented Aug 29, 2017

@deads2k
Copy link
Contributor

deads2k commented Aug 29, 2017

As the JSON protobuf change is squashed already, here are the relevant lines of code:

the internal JSON type (just an alias for interface{}): https://github.com/kubernetes/kubernetes/pull/47263/files#diff-48b8b5215523ba995d8b41c2dfdf68afR192
the external type: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-3029a18192feb70af52de676209da9b6R194,
an (optional) JSON field: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-48b8b5215523ba995d8b41c2dfdf68afR159,
the json marshal/unmarshal: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-3fe7a62396a7708e034adc5c0bbff3fdR121
the conversion: https://github.com/kubernetes/kubernetes/pull/47263/files#diff-a48a0d878266c677cd0c5c9987b327b6R40

This looks fairly straightforward. protobuf wrapping a limited json field with simple deserialization and a little validation. This may actually have been what the service catalog was looking for a while back.

}

// JSONSchemaProps is a JSON-Schema following Specification Draft 4 (http://json-schema.org/).
type JSONSchemaProps struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move this into its own types.go file - types_jsonschema.go

@smarterclayton
Copy link
Contributor

Looks like I expected. This has high level approval from me.

Remove protobuf generation because of the interface type

Add custom fuzzer funcs

Add custom marshalling

Add custom conversion functions

move jsonschema types to separate file
update generated proto
* convert our types to openAPI types
* update strategy to include crd
* use strategy to validate customresource
* add helper funcs
* Fix conversion of empty ref field
* add validation for forbidden fields
* add defaulting for schema field
* Validate CRD Schema
Update test schema

Add polling for TestCRValidationOnCRDUpdate

Add tests for forbidden fields

Enable featureGate for CustomResourceValidation
update feature gates for generic apiserver

Add apiextensions-apiserver features to golint_failures

Ignore alpha feature if gate is disabled
@sttts
Copy link
Contributor

sttts commented Aug 29, 2017

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 29, 2017
@smarterclayton
Copy link
Contributor

/approve

@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, nikhita, smarterclayton, sttts

Associated issue: 49747

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@k8s-github-robot k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 29, 2017
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Aug 30, 2017

@nikhita: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-gce-bazel 6ba1523 link /test pull-kubernetes-e2e-gce-bazel

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-github-robot
Copy link

k8s-github-robot commented Aug 30, 2017

Automatic merge from submit-queue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet