Enable kustomize in kubectl #70875

Liujingfang1 · 2018-11-09T17:32:42Z

What type of PR is this?
/kind feature

What this PR does / why we need it:
This PR is the implementation of KEP to enable kustomize in kubectl.

When -f <dir> is passed to a kubectl command, it will look for a kustomization.yaml file. If kustomization.yaml is found, a kustomize build will be run to get the list of expanded resources. This list of resources is then passed to kubectl commands. If there is no kustomization.yaml in the directory, kubectl will behave the same as current.

To apply a kustomization directory

kubectl apply -f <dir>

To get resources of a kustomization directory applied to a cluster

kubectl get -f <dir>

To delete a kustomization directory applied to a cluster

kubectl delete -f <dir>

Special notes for your reviewer:

This PR contains 6 commits.

The first three of them is to vendor kustomize.
The 4th commit is the change in resource builder to use Kustomization when it it enabled.
The 5th one adds some unit test for Builder.
The 6th commit is to enable kustomization in kubectl commands.

Kubectl will have kustomization enabled by default.
Other cli-runtime consumers may choose if they want to enable or not kustomization by a Boolean variable.

Does this PR introduce a user-facing change?:
NONE

Enable customize in kubectl: kubectl will be able to recognize directories with kustomization.YAML

k8s-ci-robot · 2018-11-09T17:32:50Z

Hi @Liujingfang1. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Liujingfang1 · 2018-11-09T17:34:05Z

/cc @pwittrock @soltysh @seans3

Liujingfang1 · 2018-11-09T17:45:24Z

/cc @monopole

smarterclayton · 2018-11-10T23:53:45Z

staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/visitor.go

@@ -452,7 +457,10 @@ func ExpandPathsToFileVisitors(mapper *mapper, paths string, recursive bool, ext
 		if err != nil {
 			return err
 		}
-
+		if isKustomizationDir(path) {


I don't think this is backwards compatible. Anyone using kubectl create -f DIR will see something different happen after this change lands, which means existing CLI workflows could break.

I just tested the behaviour. kubectl create and kubectl apply both fail with a kustomization.yaml present:

error: error validating ".../k8s.io/examples/guestbook-go/kustomization.yaml": error validating data: [apiVersion not set, kind not set]; if you choose to ignore these errors, turn validation off with --validate=false

It looks like we have a separate issue, which is that we don't validate all files before starting to apply, which is contrary to what I would have expected.

But I don't think users can be using kubectl create -f DIR or kubectl apply -f DIR today with a dir containing kustomization.yaml.

@smarterclayton The existing kubectl doesn't work with directories with a kustomization.yaml as @justinsb explained. With this PR, kubectl will be able to recognize a directory with a kustomization.yaml. For any directories without kustomization.yaml, there is no change in kubectl's behavior.

If someone has validate=false off, what happens? If it also fails, then my primary concern is addressed.

With validate=false, it fails with similar error

error: unable to decode "kustomization.yaml": Object 'Kind' is missing in `<truncated>`

seans3 · 2018-11-12T21:48:22Z

/ok-to-test

Liujingfang1 · 2018-12-17T23:34:24Z

As discussed offline, we can remove the opt out. I added a commit for that. @pwittrock PTAL

Liujingfang1 · 2018-12-17T23:54:28Z

/retest

Liujingfang1 · 2018-12-18T00:43:49Z

/retest

Liujingfang1 · 2018-12-18T18:03:27Z

/retest

BenTheElder · 2018-12-18T22:30:37Z

/test pull-kubernetes-godeps

pwittrock · 2018-12-18T22:34:32Z

staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/visitor.go

@@ -463,7 +471,10 @@ func ExpandPathsToFileVisitors(mapper *mapper, paths string, recursive bool, ext
 		if path != paths && ignoreFile(path, extensions) {
 			return nil
 		}
-
+		if filepath.Base(path) == constants.KustomizationFileName {


This will be changed to check the GVK of the kustomization file in an immediate follow up.

pwittrock · 2018-12-18T22:36:31Z

staging/src/k8s.io/cli-runtime/pkg/genericclioptions/resource/visitor.go

+	fSys := fs.MakeRealFS()
+	f := k8sdeps.NewFactory()
+	var out bytes.Buffer
+	cmd := build.NewCmdBuild(&out, fSys, f.ResmapF, f.TransformerF)


In an immediate follow up this should be a library that takes options.

pwittrock · 2018-12-18T22:43:44Z

/approve

pwittrock · 2018-12-18T22:43:51Z

/lgtm

pwittrock · 2018-12-18T22:44:07Z

Follow up issues tracked here:
kubernetes/kubectl#570

k8s-ci-robot · 2018-12-18T22:44:11Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Liujingfang1, pwittrock, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~Godeps/OWNERS~~ [thockin]
~~hack/OWNERS~~ [pwittrock,thockin]
~~staging/OWNERS~~ [thockin]
~~vendor/OWNERS~~ [thockin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pwittrock · 2018-12-18T23:33:12Z

/hold cancel

pwittrock · 2018-12-18T23:33:52Z

@soltysh removing hold. @JiangtianLi is working on a follow up.

Liujingfang1 · 2018-12-19T02:26:40Z

/test pull-kubernetes-integration

pwittrock · 2018-12-19T05:33:54Z

/test

pwittrock · 2018-12-19T05:34:12Z

/test pull-kubernetes-e2e-gce

anguslees · 2019-01-11T03:51:03Z

I'm only just learning about kustomize, but I am a bit alarmed about what I'm reading and the implications for this PR. With this PR, can I just pwn the world by putting a malicious kustomize.yaml in a popular manifest examples site somewhere?

Consider:

# kustomize.yaml - don't try this at home.
secretGenerator:
- name: allyourbase
  commands:
    # or any other malicious command
    foo: "echo backdoorkey >> $HOME/.ssh/authorized_keys"

In particular, with this PR, I think kubectl apply -f http://that/repo" suddenly becomes able to modify the local machine, not just the target cluster, even with --dry-run.

BenTheElder · 2019-01-11T05:00:31Z

FWIW I think that particular issue will be covered by kubernetes-sigs/kustomize#683. This PR as-is was merged early in the current release cycle so it will not be in any releases (not even an alpha) for a bit.

…

On Thu, Jan 10, 2019 at 7:52 PM Angus Lees ***@***.***> wrote: I'm only just learning about kustomize, but I am a bit alarmed about what I'm reading and the implications for this PR. With this PR, can I just pwn the world by putting a malicious kustomize.yaml in a popular manifest examples site somewhere? Consider: secretGenerator: - name: allyourbase commands: # or any other malicious command foo: "echo backdoorkey >> $HOME/.ssh/authorized_keys" In particular, with this PR, I think kubectl apply -f http://that/repo" suddenly becomes able to modify the *local* machine, not just the target cluster, even with --dry-run`. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70875 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA4Bq4qCEXWAABrseJFWln6ACEJJAxlkks5vCApugaJpZM4YW_gT> .

pwittrock · 2019-01-11T05:33:02Z

Discussion of changes to the UX integration were raised and are being talked through in kubernetes/kubectl#570. The outcome of those discussions will be folded into the KEP before reintegrating: kubernetes/enhancements#684

Additionally there were some security concerns raised that require changes, such as limiting process callouts: kubernetes-sigs/kustomize#683. The new capabilities added by kustomize will be reviewed from a security perspective prior to reintegration.

PR to revert: #72805

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. sig/cli Categorizes an issue or PR as relevant to SIG CLI. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 9, 2018

k8s-ci-robot requested review from juanvallejo and lavalamp November 9, 2018 17:32

k8s-ci-robot requested review from pwittrock, seans3 and soltysh November 9, 2018 17:34

k8s-ci-robot requested a review from monopole November 9, 2018 17:45

Liujingfang1 changed the title ~~Enable kustomize~~ Enable kustomize in kubectl Nov 9, 2018

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 10, 2018

smarterclayton requested changes Nov 10, 2018

View reviewed changes

Liujingfang1 force-pushed the enable-kustomize branch 2 times, most recently from a401eca to b426d83 Compare November 12, 2018 21:46

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 12, 2018

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 12, 2018

Liujingfang1 force-pushed the enable-kustomize branch 2 times, most recently from 90bac14 to 3c4901b Compare November 12, 2018 23:56

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 16, 2018

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 17, 2018

pwittrock reviewed Dec 18, 2018

View reviewed changes

k8s-ci-robot assigned pwittrock Dec 18, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 18, 2018

Liujingfang1 mentioned this pull request Dec 18, 2018

[WIP] Make BuildOptions exported kubernetes-sigs/kustomize#646

Closed

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 18, 2018

k8s-ci-robot merged commit be5a1fb into kubernetes:master Dec 19, 2018

Liujingfang1 mentioned this pull request Dec 19, 2018

Determine kustomization file by GVK kubernetes-sigs/kustomize#650

Closed

tobru mentioned this pull request Dec 26, 2018

Kustomize integration support fluxcd/flux#1261

Closed

pwittrock mentioned this pull request Jan 11, 2019

Kustomize changes are required, reverting until those are worked through #72805

Merged

seh mentioned this pull request Jan 12, 2019

add options to disallow executing commands in SecretGenerators kubernetes-sigs/kustomize#683

Closed

blaggacao mentioned this pull request Aug 7, 2020

Support pre-post hooks for build, deploy and sync GoogleContainerTools/skaffold#1441

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable kustomize in kubectl #70875

Enable kustomize in kubectl #70875

Liujingfang1 commented Nov 9, 2018 •

edited

k8s-ci-robot commented Nov 9, 2018

Liujingfang1 commented Nov 9, 2018

Liujingfang1 commented Nov 9, 2018

smarterclayton Nov 10, 2018

justinsb Nov 11, 2018

Liujingfang1 Nov 12, 2018

smarterclayton Nov 13, 2018

Liujingfang1 Nov 13, 2018

seans3 commented Nov 12, 2018

Liujingfang1 commented Dec 17, 2018

Liujingfang1 commented Dec 17, 2018

Liujingfang1 commented Dec 18, 2018

Liujingfang1 commented Dec 18, 2018

BenTheElder commented Dec 18, 2018

pwittrock Dec 18, 2018

pwittrock Dec 18, 2018

pwittrock commented Dec 18, 2018

pwittrock commented Dec 18, 2018

pwittrock commented Dec 18, 2018

k8s-ci-robot commented Dec 18, 2018

pwittrock commented Dec 18, 2018

pwittrock commented Dec 18, 2018

Liujingfang1 commented Dec 19, 2018

pwittrock commented Dec 19, 2018

pwittrock commented Dec 19, 2018

anguslees commented Jan 11, 2019 •

edited

BenTheElder commented Jan 11, 2019 via email

pwittrock commented Jan 11, 2019

Enable kustomize in kubectl #70875

Enable kustomize in kubectl #70875

Conversation

Liujingfang1 commented Nov 9, 2018 • edited

k8s-ci-robot commented Nov 9, 2018

Liujingfang1 commented Nov 9, 2018

Liujingfang1 commented Nov 9, 2018

smarterclayton Nov 10, 2018

Choose a reason for hiding this comment

justinsb Nov 11, 2018

Choose a reason for hiding this comment

Liujingfang1 Nov 12, 2018

Choose a reason for hiding this comment

smarterclayton Nov 13, 2018

Choose a reason for hiding this comment

Liujingfang1 Nov 13, 2018

Choose a reason for hiding this comment

seans3 commented Nov 12, 2018

Liujingfang1 commented Dec 17, 2018

Liujingfang1 commented Dec 17, 2018

Liujingfang1 commented Dec 18, 2018

Liujingfang1 commented Dec 18, 2018

BenTheElder commented Dec 18, 2018

pwittrock Dec 18, 2018

Choose a reason for hiding this comment

pwittrock Dec 18, 2018

Choose a reason for hiding this comment

pwittrock commented Dec 18, 2018

pwittrock commented Dec 18, 2018

pwittrock commented Dec 18, 2018

k8s-ci-robot commented Dec 18, 2018

pwittrock commented Dec 18, 2018

pwittrock commented Dec 18, 2018

Liujingfang1 commented Dec 19, 2018

pwittrock commented Dec 19, 2018

pwittrock commented Dec 19, 2018

anguslees commented Jan 11, 2019 • edited

BenTheElder commented Jan 11, 2019 via email

pwittrock commented Jan 11, 2019

Liujingfang1 commented Nov 9, 2018 •

edited

anguslees commented Jan 11, 2019 •

edited