Skip to content

Commit

Permalink
feat: GA of Kustomize and K8s manifests support (#1172)
Browse files Browse the repository at this point in the history
This is the GA version of the helm-x integration #673 developed last year.

You get all the following benefits without an extra helm plugin:

- Ability to add ad-hoc chart dependencies/aliases, without forking the chart (Fixes #876 )
- Ability to patch resulting K8s resources before installing the helm chart
- Ability to install a kustomization as a chart (Requires `kustomize` binary to be available in `$PATH`
- Ability to install a directory of K8s manifests as a chart
- etc.
  • Loading branch information
mumoshu committed May 27, 2020
1 parent 969de1e commit 16288df
Show file tree
Hide file tree
Showing 11 changed files with 400 additions and 184 deletions.
91 changes: 91 additions & 0 deletions docs/advanced-features.md
@@ -0,0 +1,91 @@
## Advanced Features

- [Import Configuration Parameters into Helmfile](#import-configuration-parameters-into-helmfile)

### Import Configuration Parameters into Helmfile

Helmfile integrates [vals]() to import configuration parameters from following backends:

- AWS SSM Parameter Store
- AWS SecretsManager
- Vault
- SOPS

See [Vals "Suported Backends"](https://github.com/variantdev/vals#suported-backends) for the full list of available backends.

This feature was implemented in https://github.com/roboll/helmfile/pull/906.
If you're curious how it's designed and how it works, please consult the pull request.

### Deploy Kustomizations with Helmfile

You can deploy [kustomize](https://github.com/kubernetes-sigs/kustomize) "kustomization"s with Helmfile.

Most of Kustomize operations that is usually done with `kustomize edit` can be done declaratively via Helm values.yaml files.

Under the hood, Helmfile transforms the kustomization into a local chart in a temporary directory so that it can be `helm upgrade --install`ed.

The transformation is done by generating (1)a temporary kustomization from various options and (2)temporary chart from the temporary kustomization.

An example pseudo code for the transformation logic can be written as:

```console
$ TMPCHART=/tmp/sometmpdir
$ mkdir -p ${TMPCHART}/templates
$ somehow_generate_chart_yaml ${TMPCHART}/Chart.yaml

$ TMPKUSTOMIZATION=/tmp/sometmpdir2
$ somehow_generate_temp_kustomization_yaml ${TMPKUSTOMIZATION}/kustomization.yaml
$ kustomize build ${TMPKUSTOMIZATION}/kustomization.yaml > ${TMPCHART}/templates/all.yaml
```

Let's say you have a `helmfile.yaml` that looks like the below:

```yaml
releases:
- name: myapp
chart: mykustomization
values:
- values.yaml
```

Helmfile firstly generates a temporary `kustomization.yaml` that looks like:

```yaml
bases:
- $(ABS_PATH_TO_HELMFILE_YAML}/mykustomization
```

Followed by the below steps:

- Running `kustomize edit set image $IMAGE` for every `$IMAGE` generated from your values.yaml
- Running `kustomize edit set nameprefix $NAMEPREFIX` with the nameprefix specified in your values.yaml
- Running `kustomize edit set namesuffix $NAMESUFFIX` with the namesuffix specified in your values.yaml
- Running `kustomize edit set namespace $NS` with the namespace specified in your values.yaml

A `values.yaml` file for kustomization would look like the below:

```yaml
images:
# kustomize edit set image mysql=eu.gcr.io/my-project/mysql@canary
- name: mysql
newName: eu.gcr.io/my-project/mysql
newTag: canary
# kustomize edit set image myapp=my-registry/my-app@sha256:24a0c4b4a4c0eb97a1aabb8e29f18e917d05abfe1b7a7c07857230879ce7d3d3
- name: myapp
digest: sha256:24a0c4b4a4c0eb97a1aabb8e29f18e917d05abfe1b7a7c07857230879ce7d3d3
newName: my-registry/my-app

# kustomize edit set nameprefix foo-
namePrefix: foo-

# kustomize edit set namesuffix -bar
nameSuffix: -bar

# kustomize edit set namespace myapp
namespace: myapp
```

At this point, Helmfile can generate a complete kustomization from the base kustomization you specified in `releases[].chart` of your helmfile.yaml and `values.yaml`,
which can be included in the temporary chart.

After all, Helmfile just installs the temporary chart like standard charts, which allows you to manage everything with Helmfile regardless of each app is declared using a Helm chart or a kustomization.
1 change: 1 addition & 0 deletions go.mod
Expand Up @@ -23,6 +23,7 @@ require (
github.com/r3labs/diff v0.0.0-20190801153147-a71de73c46ad
github.com/tatsushid/go-prettytable v0.0.0-20141013043238-ed2d14c29939
github.com/urfave/cli v1.20.0
github.com/variantdev/chartify v0.3.0
github.com/variantdev/dag v0.0.0-20191028002400-bb0b3c785363
github.com/variantdev/vals v0.4.1-0.20200501114609-9cebe482281c
go.mozilla.org/sops v0.0.0-20190912205235-14a22d7a7060 // indirect
Expand Down
27 changes: 27 additions & 0 deletions go.sum
Expand Up @@ -278,6 +278,7 @@ github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2
github.com/go-ldap/ldap v3.0.2+incompatible h1:kD5HQcAzlQ7yrhfn+h+MSABeAy/jAJhvIJ/QDllP44g=
github.com/go-ldap/ldap v3.0.2+incompatible/go.mod h1:qfd9rJvER9Q0/D/Sqn1DfHRoBp40uXYvFoEVrNEPqRc=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas=
github.com/go-martini/martini v0.0.0-20170121215854-22fa46961aab h1:xveKWz2iaueeTaUgdetzel+U7exyigDYBryyVfV/rZk=
github.com/go-martini/martini v0.0.0-20170121215854-22fa46961aab/go.mod h1:/P9AEU963A2AYjv4d1V5eVL1CQbEJq6aCNHDDjibzu8=
github.com/go-sql-driver/mysql v1.4.1 h1:g24URVg0OFbNUTx9qqY1IRZ9D9z3iPyi5zKhQZpNwpA=
Expand Down Expand Up @@ -646,6 +647,12 @@ github.com/oracle/oci-go-sdk v7.0.0+incompatible h1:oj5ESjXwwkFRdhZSnPlShvLWYdt/
github.com/oracle/oci-go-sdk v7.0.0+incompatible/go.mod h1:VQb79nF8Z2cwLkLS35ukwStZIg5F66tcBccjip/j888=
github.com/ory/dockertest v3.3.4+incompatible h1:VrpM6Gqg7CrPm3bL4Wm1skO+zFWLbh7/Xb5kGEbJRh8=
github.com/ory/dockertest v3.3.4+incompatible/go.mod h1:1vX4m9wsvi00u5bseYwXaSnhNrne+V0E6LAcBILJdPs=
github.com/otiai10/copy v1.1.1 h1:PH7IFlRQ6Fv9vYmuXbDRLdgTHoP1w483kPNUP2bskpo=
github.com/otiai10/copy v1.1.1/go.mod h1:rrF5dJ5F0t/EWSYODDu4j9/vEeYHMkc8jt0zJChqQWw=
github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95/go.mod h1:9qAhocn7zKJG+0mI8eUu6xqkFDYS2kb2saOteoSB3cE=
github.com/otiai10/curr v1.0.0/go.mod h1:LskTG5wDwr8Rs+nNQ+1LlxRjAtTZZjtJW4rMXl6j4vs=
github.com/otiai10/mint v1.3.0/go.mod h1:F5AjcsTsWUqX+Na9fpHb52P8pcRX2CI6A3ctIT91xUo=
github.com/otiai10/mint v1.3.1/go.mod h1:/yxELlJQ0ufhjUwhshSj+wFjZ78CnZ48/1wtmBH1OTc=
github.com/oxtoacart/bpool v0.0.0-20150712133111-4e1c5567d7c2 h1:CXwSGu/LYmbjEab5aMCs5usQRVBGThelUKBNnoSOuso=
github.com/oxtoacart/bpool v0.0.0-20150712133111-4e1c5567d7c2/go.mod h1:L3UMQOThbttwfYRNFOWLLVXMhk5Lkio4GGOtw5UrxS0=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
Expand Down Expand Up @@ -774,6 +781,20 @@ github.com/ulikunitz/xz v0.5.5 h1:pFrO0lVpTBXLpYw+pnLj6TbvHuyjXMfjGeCwSqCVwok=
github.com/ulikunitz/xz v0.5.5/go.mod h1:2bypXElzHzzJZwzH67Y6wb67pO62Rzfn7BSiF4ABRW8=
github.com/urfave/cli v1.20.0 h1:fDqGv3UG/4jbVl/QkFwEdddtEDjh/5Ov6X+0B/3bPaw=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/variantdev/chartify v0.0.0-20200330123007-ddc79388188c h1:PiAOZUdKtf8tIfps1fLaEsP9BCTT8hR9FawTfAuLY5o=
github.com/variantdev/chartify v0.0.0-20200330123007-ddc79388188c/go.mod h1:DzFQ1XeNBpbLdbze39ai3auS4b+Xo/KzRluEY/LyqlM=
github.com/variantdev/chartify v0.1.0 h1:RHo/di1tXvydrJwwbHoWduvG3r9nU0oj7RvuxmQLCW0=
github.com/variantdev/chartify v0.1.0/go.mod h1:DzFQ1XeNBpbLdbze39ai3auS4b+Xo/KzRluEY/LyqlM=
github.com/variantdev/chartify v0.1.1 h1:4K+6XotrJ/U5BoQc29BTOWpNGnzsCBEDahApiOoPURc=
github.com/variantdev/chartify v0.1.1/go.mod h1:DzFQ1XeNBpbLdbze39ai3auS4b+Xo/KzRluEY/LyqlM=
github.com/variantdev/chartify v0.1.2 h1:BAaZawFXJfcgk6HVM3o+/Q9Xzyv1rkPd9m+YeWM7+dg=
github.com/variantdev/chartify v0.1.2/go.mod h1:DzFQ1XeNBpbLdbze39ai3auS4b+Xo/KzRluEY/LyqlM=
github.com/variantdev/chartify v0.1.3 h1:3WEaT1ZYLaivcVdlE2poxBCw2KxNEaOzHB9k4rj5zXk=
github.com/variantdev/chartify v0.1.3/go.mod h1:DzFQ1XeNBpbLdbze39ai3auS4b+Xo/KzRluEY/LyqlM=
github.com/variantdev/chartify v0.2.0 h1:oaG/o+juw1C5PTAbRlsxUTgGYPmIneoqnYI7KQyVJJU=
github.com/variantdev/chartify v0.2.0/go.mod h1:0tw+4doFHsNnhttYx7I9Pv/dsZ82BD4UuTV9saBOcfw=
github.com/variantdev/chartify v0.3.0 h1:vm9cY+Amj44g7scz/FVpa3s13XqzFvTJrdM6iufiKEM=
github.com/variantdev/chartify v0.3.0/go.mod h1:0tw+4doFHsNnhttYx7I9Pv/dsZ82BD4UuTV9saBOcfw=
github.com/variantdev/dag v0.0.0-20191028002400-bb0b3c785363 h1:KrfQBEUn+wEOQ/6UIfoqRDvn+Q/wZridQ7t0G1vQqKE=
github.com/variantdev/dag v0.0.0-20191028002400-bb0b3c785363/go.mod h1:pH1TQsNSLj2uxMo9NNl9zdGy01Wtn+/2MT96BrKmVyE=
github.com/variantdev/vals v0.4.0 h1:O1O7/sWhlvozcY2DjZBzlE1notxwVo6UBT1+w7HsO/k=
Expand Down Expand Up @@ -1105,6 +1126,10 @@ gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20190709130402-674ba3eaed22 h1:0efs3hwEZhFKsCoP8l6dDB1AZWMgnEl3yWXWRZTOaEA=
gopkg.in/yaml.v3 v3.0.0-20190709130402-674ba3eaed22/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20191026110619-0b21df46bc1d h1:LCPbGQ34PMrwad11aMZ+dbz5SAsq/0ySjRwQ8I9Qwd8=
gopkg.in/yaml.v3 v3.0.0-20191026110619-0b21df46bc1d/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20200121175148-a6ecf24a6d71/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
Expand All @@ -1128,6 +1153,8 @@ k8s.io/apimachinery v0.0.0-20190409092423-760d1845f48b h1:fVkKJL9FIpA8LSJyHVM00M
k8s.io/apimachinery v0.0.0-20190409092423-760d1845f48b/go.mod h1:FW86P8YXVLsbuplGMZeb20J3jYHscrDqw4jELaFJvRU=
k8s.io/klog v0.0.0-20190306015804-8e90cee79f82 h1:SHucoAy7lRb+w5oC/hbXyZg+zX+Wftn6hD4tGzHCVqA=
k8s.io/klog v0.0.0-20190306015804-8e90cee79f82/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/klog v1.0.0 h1:Pt+yjF5aB1xDSVbau4VsWe+dQNzA0qv1LlXdC2dF6Q8=
k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I=
k8s.io/kube-openapi v0.0.0-20190228160746-b3a7cee44a30/go.mod h1:BXM9ceUBTj2QnfH2MK1odQs778ajze1RxcmP6S8RVVc=
layeh.com/radius v0.0.0-20190322222518-890bc1058917 h1:BDXFaFzUt5EIqe/4wrTc4AcYZWP6iC6Ult+jQWLh5eU=
layeh.com/radius v0.0.0-20190322222518-890bc1058917/go.mod h1:fywZKyu//X7iRzaxLgPWsvc0L26IUpVvE/aeIL2JtIQ=
Expand Down

0 comments on commit 16288df

Please sign in to comment.