Use OpenAPI V3 for client side SMP #120707

Jefftree · 2023-09-15T20:47:46Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Use OpenAPI V3 for client side apply

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Client-side apply will use OpenAPI V3 by default

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

staging/src/k8s.io/apimachinery/pkg/util/strategicpatch/meta.go

staging/src/k8s.io/apimachinery/pkg/util/strategicpatch/patch_test.go

apelisse · 2023-09-19T17:13:39Z

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go

+
+// TODO: GVK parser should be moved elsewhere
+// Is this duplicated somewhere?
+func gvkMatches(targetGVK schema.GroupVersionKind, ext spec.Extensions) bool {


There's a strong possibility that Alex has already written a piece of code to do that, I also suspect maybe sean has written a layer that goes on top of the OpenAPI client to find the object directly.

I saw Sean's snippet in query params verifier v3. It's really concise and I'm going to opt to duplicate it because patch requires checking for two different kinds of gvks. Schemas can have a list of gvks while the patch endpoint can only have a single gvk.

apelisse · 2023-09-19T17:15:19Z

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go

+		if gvkSupported, _ := p.gvkSupportsPatchOpenAPIV3(p.Mapping.GroupVersionKind); gvkSupported {
+			patch, err = p.buildStrategicMergePatchFromOpenAPIV3(original, modified, current)


it seems like these two things are running together yet they have a lot of common code? Can we actually return the PatchMeta if we find one?

I like keeping the two separate because they look at different components of the OpenAPI. gvkSupportsPatchOpenAPIV3 looks at the PATCH verb for a resource path (does not look at schema at all) while buildStrategicMergePatchFromOpenAPIV3 looks at Component definitions and traverses the schema resource.

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go

seans3 · 2023-10-24T02:28:51Z

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go

+	if ext == nil {
+		return false
+	}
+	f, ok := ext["x-kubernetes-group-version-kind"]


Is there a constant somewhere for this string (x-kubernetes-group-version-kind)?

There is in util/openapi in kubectl but that's coupled with OpenAPI V2 and I was hoping we can keep OpenAPI V2 and V3 coupling to a minimum. I declared a constant in this file instead.

seans3

I think this approach looks good. I would hope for more kubectl tests.

Jefftree · 2023-10-24T15:47:07Z

@seans3 I added kubectl testing by extending the existing apply tests and showing that OpenAPI V3 produces the same result. Let me know if there are other cases you'd like to see.

Jefftree · 2023-10-24T16:12:06Z

/triage accepted

staging/src/k8s.io/apimachinery/pkg/util/strategicpatch/meta.go

staging/src/k8s.io/kubectl/pkg/cmd/apply/apply.go

apelisse · 2023-10-25T17:11:10Z

How would we disable the feature if this doesn't work though? Like if openapi v3 generates the wrong patch, is there a way to revert back to v2?

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go

apelisse · 2023-10-30T21:07:01Z

Only minor nits, feel free to push back.

staging/src/k8s.io/kubectl/pkg/cmd/apply/apply_test.go

Jefftree · 2023-10-31T17:15:40Z

I'll send a follow up PR to for linter hints since it involves changing invocations of existing tests. It's a non blocking CI.

k8s-ci-robot · 2023-10-31T17:23:33Z

@Jefftree: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubernetes-linter-hints	`eb32969`	link	false	`/test pull-kubernetes-linter-hints`
pull-kubernetes-verify-lint	`eb32969`	link	false	`/test pull-kubernetes-verify-lint`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

staging/src/k8s.io/kubectl/pkg/cmd/util/factory_client_access.go

apelisse · 2023-10-31T17:46:30Z

/lgtm
/approve

k8s-ci-robot · 2023-10-31T17:46:38Z

LGTM label has been added.

Git tree hash: 045d3242f41da2d99c5b92e9bf31da352b9bb1d9

k8s-ci-robot · 2023-10-31T17:46:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: apelisse, Jefftree

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~staging/src/k8s.io/apimachinery/pkg/util/strategicpatch/OWNERS~~ [apelisse]
~~staging/src/k8s.io/kubectl/OWNERS~~ [apelisse]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot requested review from apelisse and brianpursley September 15, 2023 20:48

apelisse reviewed Sep 19, 2023

View reviewed changes

seans3 reviewed Oct 24, 2023

View reviewed changes

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 24, 2023

Jefftree changed the title ~~[WIP] Use OpenAPI V3 for client side SMP~~ Use OpenAPI V3 for client side SMP Oct 24, 2023

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 24, 2023

k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Oct 24, 2023

apelisse reviewed Oct 24, 2023

View reviewed changes

staging/src/k8s.io/apimachinery/pkg/util/strategicpatch/meta.go Show resolved Hide resolved

apelisse reviewed Oct 25, 2023

View reviewed changes

staging/src/k8s.io/kubectl/pkg/cmd/apply/apply.go Outdated Show resolved Hide resolved

Jefftree force-pushed the csa-openapiv3 branch 2 times, most recently from fa5150b to 1b78deb Compare October 25, 2023 19:52

apelisse reviewed Oct 30, 2023

View reviewed changes

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go Outdated Show resolved Hide resolved

apelisse reviewed Oct 30, 2023

View reviewed changes

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go Outdated Show resolved Hide resolved

apelisse reviewed Oct 30, 2023

View reviewed changes

staging/src/k8s.io/kubectl/pkg/cmd/apply/patcher.go Outdated Show resolved Hide resolved

Jefftree force-pushed the csa-openapiv3 branch from 5b5d595 to fbbf215 Compare October 30, 2023 22:24

seans3 reviewed Oct 30, 2023

View reviewed changes

staging/src/k8s.io/kubectl/pkg/cmd/apply/apply_test.go Outdated Show resolved Hide resolved

Jefftree force-pushed the csa-openapiv3 branch 5 times, most recently from 363e417 to 3b44bdd Compare October 31, 2023 16:42

Jefftree added 4 commits October 31, 2023 12:45

Use OpenAPI V3 for client side SMP

4f3b0b1

use OpenAPIV3 for kubectl diff

e7216c6

Add feature toggle for OpenAPI V3 apply in kubectl

f23ab82

Lazy load OpenAPIV2

eb32969

Jefftree force-pushed the csa-openapiv3 branch from 3b44bdd to eb32969 Compare October 31, 2023 16:45

apelisse reviewed Oct 31, 2023

View reviewed changes

staging/src/k8s.io/kubectl/pkg/cmd/util/factory_client_access.go Show resolved Hide resolved

k8s-ci-robot assigned apelisse Oct 31, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 31, 2023

k8s-ci-robot merged commit 07d2da7 into kubernetes:master Oct 31, 2023
11 of 14 checks passed

k8s-ci-robot added this to the v1.29 milestone Oct 31, 2023

This was referenced Dec 19, 2023

Avoid nilref panic in kubectl apply #122364

Closed

nil reference in kubectl apply on v0.29.0 kubernetes/kubectl#1537

Closed

Jefftree deleted the csa-openapiv3 branch January 25, 2024 22:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use OpenAPI V3 for client side SMP #120707

Use OpenAPI V3 for client side SMP #120707

Jefftree commented Sep 15, 2023

apelisse Sep 19, 2023

Jefftree Oct 23, 2023

apelisse Sep 19, 2023

Jefftree Oct 23, 2023

seans3 Oct 24, 2023

Jefftree Oct 24, 2023

seans3 left a comment

Jefftree commented Oct 24, 2023

Jefftree commented Oct 24, 2023

apelisse commented Oct 25, 2023

apelisse commented Oct 30, 2023

Jefftree commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023 •

edited

apelisse commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023

		if gvkSupported, _ := p.gvkSupportsPatchOpenAPIV3(p.Mapping.GroupVersionKind); gvkSupported {
		patch, err = p.buildStrategicMergePatchFromOpenAPIV3(original, modified, current)

Use OpenAPI V3 for client side SMP #120707

Use OpenAPI V3 for client side SMP #120707

Conversation

Jefftree commented Sep 15, 2023

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

apelisse Sep 19, 2023

Choose a reason for hiding this comment

Jefftree Oct 23, 2023

Choose a reason for hiding this comment

apelisse Sep 19, 2023

Choose a reason for hiding this comment

Jefftree Oct 23, 2023

Choose a reason for hiding this comment

seans3 Oct 24, 2023

Choose a reason for hiding this comment

Jefftree Oct 24, 2023

Choose a reason for hiding this comment

seans3 left a comment

Choose a reason for hiding this comment

Jefftree commented Oct 24, 2023

Jefftree commented Oct 24, 2023

apelisse commented Oct 25, 2023

apelisse commented Oct 30, 2023

Jefftree commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023 • edited

apelisse commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023

k8s-ci-robot commented Oct 31, 2023 •

edited