Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

openapi: reference shared parameters #118204

Merged
merged 4 commits into from Jul 18, 2023
Merged

openapi: reference shared parameters #118204

merged 4 commits into from Jul 18, 2023

Conversation

sttts
Copy link
Contributor

@sttts sttts commented May 23, 2023 •
edited

Reducing the v2 spec size by 55%.

Shrink the OpenAPI v2 spec by more than 50%, especially for less CPU resource consumption.

Depends on kubernetes/kube-openapi#411.

The OpenAPI diff: https://gist.github.com/sttts/97df5c47d2a68084cf9f65d09f9f59f6

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. approved Indicates a PR has been approved by an approver from all required OWNERS files. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 23, 2023
@Jefftree
Copy link
Member

This looks great, some overall questions:

  • Have you looked into integrating this functionality in the builder as well? If we can say reduce the size of the individual specs to be downloaded and aggregated, would that have a big impact on performance numbers? Not suggesting that we should do it, just wondering if it was something considered.
  • Do clients support resolving parameters (do we need to update them + @seans3)? By definition, this new version adheres to the swagger specification, but clients may still attempt to naively try to access all the parameters.

@apelisse
Copy link
Member

Have you looked into integrating this functionality in the builder as well? If we can say reduce the size of the individual specs to be downloaded and aggregated, would that have a big impact on performance numbers? Not suggesting that we should do it, just wondering if it was something considered.

Yeah, ideally I think that should be done much earlier in the process if possible, since this is looking at all paths and marshalling the parameters to see how they are, I'm concerned this is going to be super expensive to run on every single aggregation.

Do clients support resolving parameters (do we need to update them + @seans3)? By definition, this new version adheres to the swagger specification, but clients may still attempt to naively try to access all the parameters.

I'm pretty sure Sean's client is going to fail unfortunately, and I suspect there might be others that would fail too.

@sttts
Copy link
Contributor Author

sttts commented May 23, 2023

I'm pretty sure Sean's client is going to fail unfortunately, and I suspect there might be others that would fail too.

It's valid OpenAPI. If they fail, they should fix it. Nothing should depend on our concrete shape of the spec.

@sttts
Copy link
Contributor Author

sttts commented May 23, 2023

Have you looked into integrating this functionality in the builder as well? If we can say reduce the size of the individual specs to be downloaded and aggregated, would that have a big impact on performance numbers? Not suggesting that we should do it, just wondering if it was something considered.

I looked into the builder. Doing that in the builder is much trickier. The builder might differ between aggregated apiserver and kube-apiserver. And we would have to know the set of shared parameters very early, maybe statically via generation. It's certainly possible to push it earlier in generation, but it's a lot more complicated. Am not sure about the value vs. effort.

@apelisse
Copy link
Member

It's valid OpenAPI. If they fail, they should fix it. Nothing should depend on our concrete shape of the spec.

I don't disagree, it shouldn't stop us from doing the right thing, but we need to consider it.

@sttts
Copy link
Contributor Author

sttts commented May 23, 2023

Also, if we support shared parameters from aggregation (today we don't, we just drop those and leave dangling references), we need unification, i.e. something similar we do for the definitions today. An aggregated apiserver can have its own ObjectMeta schema, and the code figures it out to have another definition for that special ObjectMeta. That will add another cost to the merging.

@apelisse
Copy link
Member

@Jefftree Do we have anything useful to benchmark the aggregation loop? I think I remember you had mentioned something like this at some point?

@Jefftree
Copy link
Member

It's valid OpenAPI. If they fail, they should fix it. Nothing should depend on our concrete shape of the spec.

I don't disagree, it shouldn't stop us from doing the right thing, but we need to consider it.

Right, we don't exactly want to be breaking clients using a GA feature so we'll have to carefully how we want to move forward.

@Jefftree Do we have anything useful to benchmark the aggregation loop? I think I remember you had mentioned something like this at some point?

I had some basic code to run our OpenAPI tests in benchmark mode. Unfortunately it had some pretty high variance because of the setup overhead all the other stuff going on in the background. I do really want to come up with a good way of benchmarking, but at the moment pprof is still the most reliable way. @sttts out of curiosity, how did you perform the benchmarks for this PR?

@sttts
Copy link
Contributor Author

sttts commented May 23, 2023

out of curiosity, how did you perform the benchmarks for this PR?

Have installed a few hundred CRDs and curl'ed /openapi/v2 and compared the size. Not super systematic. But it's pretty obvious when you look at a spec that a lot of it is just listing parameters. The measurement proved that.

@apelisse
Copy link
Member

apelisse commented May 23, 2023 •
edited

Yeah, I was looking at it too and it's clear that there's A LOT of parameters being repeated.

@Jefftree
Copy link
Member

ack, for the purposes of this PR, running ./hack/update-openapi-spec.sh should suffice for displaying the delta in swagger.json for reviewers.

@cici37
Copy link
Contributor

cici37 commented May 23, 2023

/triage accepted

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 23, 2023
@sttts
Copy link
Contributor Author

sttts commented May 24, 2023

ack, for the purposes of this PR, running ./hack/update-openapi-spec.sh should suffice for displaying the delta in swagger.json for reviewers.

pushed.

@k8s-ci-robot k8s-ci-robot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 24, 2023
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2023
@sttts
Copy link
Contributor Author

sttts commented Jul 16, 2023 •
edited

Rebased to test client generators and compare this PR against master. No difference in output with simple parameter expansion in #118204.

@sttts sttts added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2023
@sttts
Copy link
Contributor Author

sttts commented Jul 16, 2023

Yeah this will break at least the QueryParamVerifier that we use to check
if validate is available on cluster/resources.

In kubernetes/kube-openapi#415, I exclude the fieldValidation query parameter.

I am worried that this PR has been green. The apply code lacks test coverage it seems.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jul 17, 2023
@sttts
Copy link
Contributor Author

sttts commented Jul 17, 2023

/retest

@liggitt liggitt added this to the v1.28 milestone Jul 17, 2023
@sttts sttts force-pushed the sttts-openapi-v2-parameter-refs branch from 4fc4f97 to e7be841 Compare July 18, 2023 01:23
@alexzielenski alexzielenski mentioned this pull request Jul 18, 2023
Merged
@liggitt
Copy link
Member

liggitt commented Jul 18, 2023

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 18, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: e83dde7a96b5e4565f04052e737708e66e936246

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alexzielenski, liggitt, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 18, 2023
@k8s-ci-robot k8s-ci-robot merged commit f42ff86 into kubernetes:master Jul 18, 2023
16 checks passed
SIG Node PR Triage automation moved this from WIP to Done Jul 18, 2023
@Jefftree Jefftree mentioned this pull request Nov 29, 2023
Open
@shyamjvs shyamjvs mentioned this pull request Feb 20, 2024
Closed
sushanth0910 added a commit to sushanth0910/kubernetes that referenced this pull request Feb 28, 2024
@sushanth0910
Revert "Merge pull request kubernetes#118204 from sttts/sttts-openapi…
b36f5d7 
…-v2-parameter-refs"

This reverts commit f42ff86, reversing
changes made to b4d793c.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liggitt liggitt liggitt left review comments

@cheftako cheftako Awaiting requested review from cheftako

@jpbetz jpbetz Awaiting requested review from jpbetz

@alexzielenski alexzielenski Awaiting requested review from alexzielenski

@apelisse apelisse Awaiting requested review from apelisse

@Jefftree Jefftree Awaiting requested review from Jefftree

Assignees

@alexzielenski alexzielenski

@liggitt liggitt

@apelisse apelisse

@Jefftree Jefftree

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver area/cloudprovider area/code-generation area/dependency Issues or PRs related to dependency changes area/kube-proxy area/kubectl cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
SIG Auth
Archived in project
SIG Node PR Triage
Done
[sig-release] Bug Triage
Archived in project
Milestone
v1.28
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@sttts @Jefftree @apelisse @cici37 @k8s-ci-robot @dims @liggitt @alexzielenski