Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove estimateMinSizeJSON calls for CEL #111156

Merged

Conversation

DangerOnTheRanger
Copy link
Contributor

@DangerOnTheRanger DangerOnTheRanger commented Jul 14, 2022

What type of PR is this?

/kind feature

What this PR does / why we need it:

estimateMinSizeJSON is a function that is used to estimate the minimum valid size for a given schema node when serialized into JSON (this is used in turn to calculate how many values things like maps and arrays can have). The function works by walking through everything below the schema node it was given, though since SchemaDeclType (which is what calls estimateMinSizeJSON) is already walking the entire schema and calls estimateMinSizeJSON for all arrays/maps, this results in a lot of redundant estimateMinSizeJSON calls. This PR removes the need entirely for estimateMinSizeJSON by adding a MinSerializedSize attribute to DeclTypes instead.

The below two tables show the performance gained from this change; the tables were generated with the benchmark included in this branch (the estimateMinSizeJSON table used the benchmark patched onto master), with the depth parameter passed to genNestedSchema adjusted as per the values in the n column. All runs were performed on a 16-core Intel Xeon clocked at 2.3Ghz with 66GB of RAM.

with estimateMinSizeJSON

n (schema depth) ns/op
1 538
5 10275
10 385079
15 22950231

with MinSerializedSize

n (schema depth) ns/op
1 482
5 2274
10 4863
15 6750

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jul 14, 2022
@k8s-triage-robot
Copy link

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@cici37
Copy link
Contributor

cici37 commented Jul 14, 2022

Would you mind try with existing benchmark test(or write your own if needed) to show the performance different?

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 14, 2022
@liggitt
Copy link
Member

liggitt commented Jul 15, 2022

/assign @jpbetz
for initial review

@jpbetz
Copy link
Contributor

jpbetz commented Jul 18, 2022

Per offline discussion, we should close this in favor of #110135 but still need a PR to optimize the minSize calculations performed by

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jul 19, 2022
@leilajal
Copy link
Contributor

/triage accepted

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jul 19, 2022
@DangerOnTheRanger DangerOnTheRanger changed the title Cache SchemaDeclType calls for CEL Remove estimateMinSizeJSON calls for CEL Jul 20, 2022
Copy link
Contributor

@jpbetz jpbetz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see the code for this change fell out roughly as I had hoped. It's a nice simplification and optimization combined into one change.

@jpbetz
Copy link
Contributor

jpbetz commented Jul 21, 2022

/lgtm
/assign @liggitt for approval

@k8s-ci-robot
Copy link
Contributor

@jpbetz: GitHub didn't allow me to assign the following users: for, approval.

Note that only kubernetes members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/lgtm
/assign @liggitt for approval

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 21, 2022
Copy link
Member

@liggitt liggitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this cleanup... putting the size computation in the recursive construction we were already doing makes a ton of sense.

@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jul 21, 2022
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 22, 2022
@DangerOnTheRanger
Copy link
Contributor Author

/test pull-kubernetes-e2e-gce-ubuntu-containerd

1 similar comment
@DangerOnTheRanger
Copy link
Contributor Author

/test pull-kubernetes-e2e-gce-ubuntu-containerd

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 25, 2022
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 25, 2022
@liggitt
Copy link
Member

liggitt commented Jul 25, 2022

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 25, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: DangerOnTheRanger, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 25, 2022
@DangerOnTheRanger
Copy link
Contributor Author

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants