An application is a group of k8s resources related by some common purpose, e.g. a load balancer in front of a webserver backed by a database. Resource labelling, naming and metadata schemes have historically served to group resources together for collective operations like list and remove.
This proposal describes a new k8s resource called application to more formally describe this idea and provide support for application-level operations and dashboards.
kustomize configures k8s resources, and the proposed application resource is just another resource.
The verb apply in the context of k8s refers to a kubectl command and an in-progress API endpoint for mutating a cluster.
One applies a statement of what one wants to a cluster in the form of a complete resource list.
The cluster merges this with the previously applied state and the actual state to arrive at a new desired state, which the cluster's reconcilation loop attempts to create. This is the foundation of level-based state management in k8s.
Any target, including an overlay, can be a base to another target.
A base has no knowledge of the overlays that refer to it.
For simple gitops management, a base configuration could be the sole content of a git repository dedicated to that purpose. Same with overlays. Changes in a repo could generate a build, test and deploy cycle.
The workflow associated with a bespoke config is simpler than the workflow associated with an off-the-shelf config, because there's no notion of periodically capturing someone else's upgrades to the off-the-shelf config.
custom resource definition
One can extend the k8s API by making a Custom Resource Definition (CRD spec).
This defines a custom resource (CD), an entirely new resource that can be used alongside native resources like ConfigMaps, Deployments, etc.
Kustomize can customize a CD, but to do so kustomize must also be given the corresponding CRD so that it can interpret the structure correctly.
declarative application management
Kustomize aspires to support Declarative Application Management, a set of best practices around managing k8s clusters.
In brief, kustomize should
- Work with any configuration, be it bespoke, off-the-shelf, stateless, stateful, etc.
- Support common customizations, and creation of variants (e.g. development vs. staging vs. production).
- Expose and teach native k8s APIs, rather than hide them.
- Add no friction to version control integration to support reviews and audit trails.
- Compose with other tools in a unix sense.
- Eschew crossing the line into templating, domain specific languages, etc., frustrating the other goals.
Devops or CICD workflows that use a git repository as a single source of truth and take action (e.g., build, test or deploy) when that truth changes.
A kustomization is a file called
describes a configuration consumable by kustomize.
Here's an example.
A kustomization contains fields falling into these categories:
Customization operators for modifying operands, e.g. namePrefix, commonLabels, patches, etc.
- resources - completely specified k8s API objects,
- bases - paths or github URLs specifying directories containing a kustomization. These bases may be subjected to more customization, or merely included in the output.
- CRDs - custom resource definition files, to allow use of custom resources in the resources list. Not an actual operand - but allows the use of new operands.
- resources - completely specified k8s API objects, e.g.
Generators, for creating more resources (configmaps and secrets) which can then be customized.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
It's often abbreviated as k8s.
An object, expressed in a YAML or JSON file, with the fields required by kubernetes. Basically just a kind field to identify the type, a metadata/name field to identify the particular instance, and an apiVersion field to identify the version (if there's more than one version).
kustomize is a command line tool supporting template-free customization of declarative configuration targetted to k8s-style objects.
Targetted to k8s means that kustomize may need some limited understanding of API resources, k8s concepts like names, labels, namespaces, etc. and the semantics of resource patching.
kustomize is an implementation of DAM.
An off-the-shelf configuration is a kustomization and resources intentionally published somewhere for others to use.
E.g. one might create a github repository like this:
github.com/username/someapp/ kustomization.yaml deployment.yaml configmap.yaml README.md
Someone could then fork this repo (on github) and clone their fork to their local disk for customization.
An overlay is a target that modifies (and thus depends on) another target.
An overlay is unusable without its base.
An overlay may act as a base to another overlay.
Overlays make the most sense when there is more than one, because they create different variants of a common base - e.g. development, QA, staging and production environment variants.
These variants use the same overall resources, and vary in relatively simple ways, e.g. the number of replicas in a deployment, the CPU to a particular pod, the data source used in a configmap, etc.
One configures a cluster like this:
kustomize build someapp/overlays/staging |\ kubectl apply -f - kustomize build someapp/overlays/production |\ kubectl apply -f -
Usage of the base is implicit - the overlay's kustomization points to the base.
General instructions to modify a resource.
A patchStrategicMerge is strategic-merge-style patch (SMP).
An SMP looks like an incomplete YAML specification of
a k8s resource. The SMP includes
fields to establish the group/version/kind/name of the
resource to patch, then just enough remaining fields
to step into a nested structure to specify a new field
value, e.g. an image tag.
By default, an SMP replaces values. This usually desired when the target value is a simple string, but may not be desired when the target value is a list.
To change this default behavior, add a directive. Recognized directives include replace (the default), merge (avoid replacing a list), delete and a few more (see these notes).
Fun fact - any resource file can be used as an SMP, overwriting matching fields in another resource with the same group/version/kind/name, but leaving all other fields as they were.
TODO(monopole): add ptr to example.
A patchJson6902 can do almost everything a patchStrategicMerge can do, but with a briefer syntax. See this example.
A resource in the context of a REST-ful API is the target object of an HTTP operation like GET, PUT or POST. k8s offers a REST-ful API surface to interact with clients.
More generally, a resource can be any correct YAML file that defines an object with a kind and a metadata/name field.
sub-target / sub-application / sub-package
The target is the argument to
kustomize build, e.g.:
kustomize build $target
$target must be a path or a url to a directory that
immediately contains a kustomization.
The target contains, or refers to, all the information needed to create customized resources to send to the apply operation.
E.g., a staging and production overlay both modify some common base to create distinct variants.
The staging variant is the set of resources exposed to quality assurance testing, or to some external users who'd like to see what the next version of production will look like.
The production variant is the set of resources exposed to production traffic, and thus may employ deployments with a large number of replicas and higher cpu and memory requests.