Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apiextensions: implement structural schema CRD pruning #77333

Merged
merged 8 commits into from May 17, 2019

Conversation

@sttts
Copy link
Contributor

commented May 2, 2019

Based on #77207.

KEP kubernetes/enhancements#709

TODO:

  • default behaviour for preserveUnknownFields:false, but no schema
  • do pruning benchmarks
BenchmarkUnmarshal-8   	  100000	     13638 ns/op	    4672 B/op	      78 allocs/op
BenchmarkDeepCopy-8   	  300000	      4879 ns/op	    2448 B/op	      16 allocs/op
BenchmarkPrune-8   	  500000	      3053 ns/op	    1568 B/op	      14 allocs/op
  • add metadata pruning tests
  • add status subresource pruning tests
Add CRD spec.preserveUnknownFields boolean, defaulting to true in v1beta1 and to false in v1 CRDs. If false, fields not specified in the validation schema will be removed when sent to the API server or when read from etcd.
  • [ ] check that converted resources are pruned, possibly with an e2e test opened follow-up #77922 which is blocked on CRD conversion integration tests.
  • check that webhook admitted resources are pruned via integration tests
  • fix pruning e2e test

@sttts sttts changed the title apiextensions: implement structural schema CRD pruning WIP: apiextensions: implement structural schema CRD pruning May 2, 2019

@sttts sttts added this to the v1.15 milestone May 2, 2019

@k8s-ci-robot k8s-ci-robot removed the needs-sig label May 2, 2019

@sttts sttts added the kind/feature label May 2, 2019

@k8s-ci-robot k8s-ci-robot removed the needs-kind label May 2, 2019

@k8s-ci-robot k8s-ci-robot requested review from brendandburns and krzyzacy May 2, 2019

@sttts

This comment has been minimized.

Copy link
Contributor Author

commented May 2, 2019

@sttts sttts force-pushed the sttts:sttts-structural-crd-pruning branch 4 times, most recently from 75b1a90 to 3e3a85c May 2, 2019

@liggitt liggitt added this to Required for GA, in progress in Custom Resource Definitions May 2, 2019

@apelisse

This comment has been minimized.

Copy link
Member

commented May 2, 2019

/assign @jpbetz

@k8s-ci-robot k8s-ci-robot removed the lgtm label May 15, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented May 15, 2019

New changes are detected. LGTM label has been removed.

@sttts sttts added the lgtm label May 15, 2019

@sttts

This comment has been minimized.

Copy link
Contributor Author

commented May 15, 2019

/retest

@fejta-bot

This comment has been minimized.

Copy link

commented May 16, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@sttts

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

/retest

@sttts

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

/retest

Empty build log.

@sttts

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

/retest

integration failed. No log, no artifacts.

@fejta-bot

This comment has been minimized.

Copy link

commented May 17, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented May 17, 2019

@sttts: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-local-e2e 0dbe8e3 link /test pull-kubernetes-local-e2e

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sttts sttts force-pushed the sttts:sttts-structural-crd-pruning branch from 533873e to d10f8c1 May 17, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented May 17, 2019

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot removed the lgtm label May 17, 2019

@sttts sttts added the lgtm label May 17, 2019

@k8s-ci-robot k8s-ci-robot merged commit 3193e78 into kubernetes:master May 17, 2019

19 of 20 checks passed

pull-kubernetes-kubemark-e2e-gce-big Job triggered.
Details
cla/linuxfoundation sttts authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-image-test Skipped.
pull-kubernetes-cross Skipped.
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-csi-serial Skipped.
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gce-storage-slow Skipped.
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details

@sttts sttts deleted the sttts:sttts-structural-crd-pruning branch May 17, 2019

@liggitt liggitt moved this from Required for GA, in progress to Complete in Custom Resource Definitions May 17, 2019

@liggitt liggitt moved this from Changes requested to Completed, 1.15 in API Reviews May 17, 2019

for i, v := range spec.Versions {
schemaPath := fldPath.Child("versions").Index(i).Child("schema", "openAPIV3Schema")
if v.Served && (v.Schema == nil || v.Schema.OpenAPIV3Schema == nil) {
allErrs = append(allErrs, field.Required(schemaPath, "because otherwise all fields are pruned"))

This comment has been minimized.

Copy link
@tedyu

tedyu May 17, 2019

Contributor

Should schemaPath be included in the error message ?

This comment has been minimized.

Copy link
@liggitt

liggitt May 17, 2019

Member

it gets included when printed

} else {
delete(x, k)
}
fmt.Printf("deleting %q => %#v\n", k, x)

This comment has been minimized.

Copy link
@tedyu

tedyu May 17, 2019

Contributor

deleting -> deleted

This comment has been minimized.

Copy link
@liggitt

liggitt May 17, 2019

Member

actually, shouldn't this debug output be dropped?

This comment has been minimized.

Copy link
@tedyu

tedyu May 17, 2019

Contributor

Created #78043 to remove this log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.