Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize away repeated structural schema and CEL DeclType construction in CRD validation #110135

Merged
merged 1 commit into from
Jul 23, 2022
Merged

Optimize away repeated structural schema and CEL DeclType construction in CRD validation #110135

merged 1 commit into from
Jul 23, 2022

Conversation

jpbetz
Copy link
Contributor

@jpbetz jpbetz commented May 19, 2022 •
edited

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

This optimizes away repeated structural schema and CEL "decls" construction, which was flagged as a potential blocker for beta for CRD Validation using CEL (kubernetes/enhancements#3266).

Which issue(s) this PR fixes:

Fixes: #106438

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

Please use the following format for linking documentation:

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 19, 2022
@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch from 8fecfcf to 6c7560e Compare May 19, 2022 16:50
@jpbetz jpbetz changed the title build and reuse structural schema and cel decls during CRD validation optimizes away repeated structural schema and CEL DeclType construction in CRD validation May 19, 2022
@jpbetz jpbetz changed the title optimizes away repeated structural schema and CEL DeclType construction in CRD validation Optimize away repeated structural schema and CEL DeclType construction in CRD validation May 19, 2022
@jpbetz
Copy link
Contributor Author

jpbetz commented May 19, 2022

@liggitt would you be willing to review? This is one of the optimizations we had listed to do before taking CEL CRD validation to beta (@DangerOnTheRanger will be working on the other one-- eliminating repeated calls to calculate the min size of a schema).

@jpbetz
Copy link
Contributor Author

jpbetz commented May 19, 2022

/sig api-machinery
/priority important-soon
/triage accepted

@k8s-ci-robot k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 19, 2022
@k8s-triage-robot
Copy link

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch 2 times, most recently from f49b78c to a5ae4cb Compare May 20, 2022 01:02
@cici37 cici37 added this to the v1.25 milestone Jul 13, 2022
@cici37 cici37 mentioned this pull request Jul 14, 2022
Closed
10 tasks
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 15, 2022
@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch from a5ae4cb to 8bf80d8 Compare July 15, 2022 23:19
@k8s-ci-robot k8s-ci-robot removed the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 15, 2022
@jpbetz
Copy link
Contributor Author

jpbetz commented Jul 18, 2022

/retest

1 similar comment
@jpbetz
Copy link
Contributor Author

jpbetz commented Jul 18, 2022

/retest

liggitt
liggitt reviewed Jul 18, 2022
View reviewed changes
Copy link
Member

@liggitt liggitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't review the tests yet but wanted to do a sweep in the impl

staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/validation.go Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/validation.go Outdated Show resolved Hide resolved
@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch 3 times, most recently from 09dcc39 to fed572b Compare July 20, 2022 02:08
@jpbetz
Copy link
Contributor Author

jpbetz commented Jul 20, 2022

/retest

1 similar comment
@jpbetz
Copy link
Contributor Author

jpbetz commented Jul 20, 2022

/retest

@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch 2 times, most recently from 660d441 to 799baef Compare July 20, 2022 17:56
liggitt
liggitt reviewed Jul 21, 2022
View reviewed changes
staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch 2 times, most recently from 673a7e6 to 222e36a Compare July 21, 2022 21:44
@jpbetz
Copy link
Contributor Author

jpbetz commented Jul 21, 2022

/retest

liggitt
liggitt reviewed Jul 22, 2022
View reviewed changes
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/validation.go Outdated Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/schema/cel/validation.go Show resolved Hide resolved
staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/validation/cel_validation.go Show resolved Hide resolved
@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch from 222e36a to 744083a Compare July 22, 2022 16:21
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 22, 2022
@liggitt
Copy link
Member

liggitt commented Jul 22, 2022

#110135 (comment) is the only outstanding comment I have. Go ahead and squash once that's done, then lgtm

Thanks for all the work here, the expanded test coverage looks good

@jpbetz jpbetz force-pushed the efficient-crd-cel-validation branch from 744083a to 81bd325 Compare July 23, 2022 00:24
@jpbetz
Copy link
Contributor Author

jpbetz commented Jul 23, 2022

#110135 (comment) is the only outstanding comment I have. Go ahead and squash once that's done, then lgtm

Thanks for all the work here, the expanded test coverage looks good

Fixed, squashed and rebased. Thanks for the review!

@liggitt
Copy link
Member

liggitt commented Jul 23, 2022

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 23, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jpbetz, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 23, 2022
@k8s-ci-robot k8s-ci-robot merged commit 5e14c5d into kubernetes:master Jul 23, 2022
@cici37 cici37 mentioned this pull request Jul 29, 2022
Merged
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liggitt liggitt liggitt left review comments

@logicalhan logicalhan Awaiting requested review from logicalhan

@pwittrock pwittrock Awaiting requested review from pwittrock

Assignees

@liggitt liggitt

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.25
Development

Successfully merging this pull request may close these issues.

Benchmark cel.UnstructuredToVal and optimize away repeated wrapper object construction
5 participants
@jpbetz @k8s-triage-robot @liggitt @k8s-ci-robot @cici37