Table of Contents
Declarative specification of Kubernetes objects is the recommended way to manage Kubernetes production workloads, however gaps in the kubectl tooling force users to write their own scripting and tooling to augment the declarative tools with preprocessing transformations. While most of these transformations already exist as imperative kubectl commands, they are not natively accessible from a declarative workflow.
This KEP describes how
kustomize addresses this problem by providing a declarative format for users to access
the imperative kubectl commands they are already familiar natively from declarative workflows.
The kubectl command provides a cli for:
- accessing the Kubernetes apis through json or yaml configuration
- porcelain commands for generating and transforming configuration off of command line flags.
Generate a configmap or secret from a text or binary file
kubectl create configmap,
kubectl create secret
- Users can manage their configmaps and secrets text and binary files
Create or update fields that cut across other fields and objects
- Users can add and update labels for all objects composing an application
Transform an existing declarative configuration without forking it
- Users may generate multiple variations of the same workload
Transform live resources arbitrarily without auditing
To create a Secret from a binary file, users must first base64 encode the binary file and then create a Secret yaml config from the resulting data. Because the source of truth is actually the binary file, not the config, users must write scripting and tooling to keep the 2 sources consistent.
Instead, users should be able to access the simple, but necessary, functionality available in the imperative kubectl commands from their declarative workflow.
Long standing issues
Kustomize addresses a number of long standing issues in kubectl.
- Declarative enumeration of multiple files kubernetes/kubernetes#24649
- Declarative configmap and secret creation: kubernetes/kubernetes#24744, kubernetes/kubernetes#30337
- Configmap rollouts: kubernetes/kubernetes#22368
- Name/label scoping and safer pruning: kubernetes/kubernetes#1698
- Template-free add-on customization: kubernetes/kubernetes#23233
- Declarative support for defining ConfigMaps and Secrets generated from binary and text files
- Declarative support for adding or updating cross-cutting fields
- labels & selectors
- names (as transformation of the original name)
- Declarative support for applying patches to transform arbitrary fields
- use strategic-merge-patch format
- Ease of integration with CICD systems that maintain configuration in a version control repository as a single source of truth, and take action (build, test, deploy, etc.) when that truth changes (gitops).
Exposing every imperative kubectl command in a declarative fashion
The scope of kustomize is limited only to functionality gaps that would otherwise prevent users from
defining their workloads in a purely declarative manner (e.g. without writing scripts to perform pre-processing
or linting). Commands such as
kubectl create deployment and
kubectl edit are unnecessary
in a declarative workflow because a Deployment can easily be managed as declarative config.
Providing a simpler facade on top of the Kubernetes APIs
The community has developed a number of facades in front of the Kubernetes APIs using templates or DSLs. Attempting to provide an alternative interface to the Kubernetes API is a non-goal. Instead the focus is on:
- Facilitating simple cross-cutting transformations on the raw config that would otherwise require other tooling such as sed
- Generating configuration when the source of truth resides elsewhere
- Patching existing configuration with transformations
Note: This proposal has already been implemented in
Define a new meta config format called kustomization.yaml.
kustomization.yaml will allow users to reference config files
- Path to config yaml file (similar to
kubectl apply -f <file>)
- Urls to config yaml file (similar to
kubectl apply -f <url>)
- Path to kustomization.yaml file (takes the output of running kustomize)
kustomization.yaml will allow users to generate configs from files
- ConfigMap (
kubectl create configmap)
- Secret (
kubectl create secret)
kustomization.yaml will allow users to apply transformations to configs
- Label (
- Annotate (
- Strategic-Merge-Patch (
Kustomize will also contain subcommands to facilitate authoring kustomization.yaml.
The edit subcommands will allow users to modify the kustomization.yaml through cli commands containing helpful messaging and documentation.
- Add ConfigMap - like
kubectl create configmapbut declarative in kustomization.yaml
- Add Secret - like
kubectl create secretbut declarative in kustomization.yaml
- Add Resource - adds a file reference to kustomization.yaml
- Set NamePrefix - adds NamePrefix declaration to kustomization.yaml
The diff subcommand will allow users to see a diff of the original and transformed configuration files
- Generated config (configmap) will show the files as created
- Transformations (name prefix) will show the files as modified
Implementation Details/Notes/Constraints [optional]
Kustomize has already been implemented in the
github.com/kubernetes/kubectl repo, and should be moved to a
separate repo for the subproject.
Kustomize was initially developed as its own cli, however once it has matured, it should be published as a subcommand of kubectl or as a statically linked plugin. It should also be more tightly integrated with apply.
- Create the kustomize sig-cli subproject and update sigs.yaml
- Move the existing kustomize code from
Risks and Mitigations
Risks of Not Having a Solution
By not providing a viable option for working directly with Kubernetes APIs as json or yaml config, we risk the ecosystem becoming fragmented with various bespoke API facades. By ensuring the raw Kubernetes API json or yaml is a usable approach for declaratively managing applications, even tools that do not use the Kubernetes API as their native format can better work with one another through transformation to a common format.
- Dogfood kustomize by either:
- moving one or more of our own (OSS Kubernetes) services to it.
- getting user feedback from one or more mid or large application deployments using kustomize.
- Publish kustomize as a subcommand of kubectl.
kustomize was implemented in the kubectl repo before subprojects became a first class thing in Kubernetes. The code has been fully implemented, but it must be moved to a proper location.
- Users write their own bespoke scripts to generate and transform the config before it is applied.
- Users don't work with the API directly, and use or develop DSLs for interacting with Kubernetes.