helm install <umbrella chart> should do individual deployments as an option

[jroberts235]

Deployments using an umbrella, parent, chart for a large number of child charts, currently results in single deployment-name. This makes it difficult, and impossible in some cases, to make changes to the child deployments, without deleting the entire umbrella deployment. While, this could and I'm sure will, be addressed by improvements to helm upgrade, there is also the issue of visibility into what is actually deployed when looking at the output of helm list.

While I can see the value in having a single deployment, there should be an option to deploy each child chart as an individual. This would have the benefit of a single chart to install but the greater manageability of individual deployments and deployment-names.

[technosophos]

I like this idea. It will be very tricky to implement, but it would be a valuable addition.

[prydonius]

I actually think there is value in general to being able to selectively filter out chart dependencies at install time. For example, a WordPress chart may support configuring the database host/port to point to a DBaaS instead of a MariaDB deployment in-cluster, or make use of service catalog. Currently, you would have to deploy MariaDB with the WordPress chart whilst configuring WordPress to ignore the MariaDB deployment. It would be great to have something like:

helm install stable/wordpress --skip-dependencies=mariadb --set databaseHost=my.rds.com

This usecase is definitely more complicated than the default chart deployment, and it may be easier to solve this problem by providing a separate chart, but I just wanted to add my 2-cents.

[databus23]

This would be super helpful when it comes to packaging more complicated applications like lets say OpenStack... :)

As of now we are installing a complete OpenStack as one chart consisting of 10 subcharts describing distinct OpenStack services. We are still not entirely certain if this is feasible or if we should break things up in smaller chunks which creates other problems in terms of duplication and complexity.

We still chase the vision of bringing up a complete OpenStack environment by doing helm install openstack instead of configuring 10 separate services like helm install keystone, helm install nova, etc. Being able to "zoom-in" on different parts of an application when it comes to upgrading or reconfiguring would be a part of the puzzle we are currently missing.

[abattye]

👍 @databus23

[technosophos]

I think @prydonius 's idea is very workable. The question, though, is what we should do about upgrades and rollbacks.

For example: Say I install foo with helm install foo --skip--dependencies=bar. If I later run helm upgrade foo, does it still skip bar? Is there any way to later turn on or off bar?

[sbezverk]

@databus23 that is exactly the reason why I submitted #1570 . In my case, we have global values/variables which are shared between bunch of Openstack projects. Each main service/project like neutron for example has variables shared between sub-neutron services and so on. Right now to deploy using this hierarchical variables model I need to helm install the top level parent, which is very very inconvenient. Having a feature which allows to use all levels of defined variables and install a specific subchart would be great. I really hope helm community would accept this feature.

[jroberts235]

I wrote a wrapper script in Ruby(2.x.x) over the weekend that adds interactive deployment and some other features to helm. This is a work in progress, and ultimately should be a patch, but I don't know Go-lang. If I get the time to port it, I give it a whirl.

Feel free to comment, improve: https://github.com/jroberts235/helmPlus

[jroberts235]

I added, '--skip' feature

[sbezverk]

@jroberts235 would your wrapper allow using global variables defined in the parent chart to be used for rendering when deploying child or grand-child chart?

[jroberts235]

I did think about that and I really like the idea, but it does break from helm convention, which I'm trying to adhere to.
My script is Ruby and while I could use ERB, I don't like the idea of putting a different tempting language into the parent values.yaml. BUT, using Go substitution seems more palatable. A college found this: https://github.com/tsg/gotpl/blob/master/tpl.go and while I don't really like having to shell out again to call the Go code, I might add that feature in, because not have to do a bulk search and replace on the parent values, to change the namespace is super convenient.
I really need to learn go.

[prydonius]

@sbezverk @jroberts235 Helm already supports global values (https://github.com/kubernetes/helm/blob/master/docs/chart_template_guide/subcharts_and_globals.md#global-chart-values), would this solve your problem?

[sbezverk]

@prydonius Unfortunately it does not or at least not completely. When you define global variable in parent, then child's chart only see it when the parent gets deployed but when you deploy just child's chart then parent's global variables are not visible and helm install fails. I hope it clarified the issue I need to resolve.

[jroberts235]

@sbezverk, my script does do that. I mis-understood what you were asking above. I thought you wanted an extra layer of templating on top of the parent values file, so that when installed, it too would have values that were replaced, like namespace.

If you put a section called global: into the parent values.yaml, it will get passed to all children, using my script, even when only deploying a child, or children. BUT the children have to already be referring to those values as .Values.global.

Does that work for your case?

[jroberts235]

@technosophos, regarding upgrades to skipped charts, what about a directory of deployment manifests, each link to the deployment by filename. When an upgrade is run, helm could check the manifest dir and find what was installed, what versions, where it came from and what was skipped. Upon successful upgrades, the manifest could either be updated, moved to a new filename that indicates that it's superseded, or overwritten entirely.

[roboll]

One potential solution is to not try to package "umbrella charts" as charts.

In my case, I reached limitations with deploying to multiple namespaces (required explicitly handling namespaces on resources in each chart), as well as inability to manage lifecycle on subcharts.

helmfile is my solution for managing several charts with independent lifecycles. It's a simple wrapper around the helm command to manage chart deployments, with a declarative spec that maps to command line arguments.

[technosophos]

@jroberts235 A Helm plugin PR is in the works. This PR would allow you to write custom Helm plugins (in any language) that could do this sort of thing. It might be possible to simply turn your Ruby script into a plugin, which would be really cool.

I mentioned this ticket to both @sbezverk and @adamreese, asking if they thought it might be possible to implement this functionality on the client by basically taking a chart as input, rewriting the chart contents on the fly (client-side), and then pushing the modified chart to Tiller.

So say we create a plugin called install-some whose purpose is to install some parts of a chart.
It might get invoked like this:

$ helm install-some --skip="swift" openstack-1.2.3.tgz

The idea would be that it skips installing openstack/charts/swift.

The implementation could be that it reads the openstack-1.2.3.tgz chart into memory, drops openstack/charts/swift from the in-memory copy, and then sends that modified version to Tiller.

It seems like it may even be possible to do this as a shell script just by unpacking the chart into a temp dir, rming the files you don't want, and then running helm install on the remaining chart.

Anyway... that's just an idea of how we could do something like this as a plugin. We are planning on putting a lot of effort into making Helm plugins a first-class thing so that people can quickly extend Helm without having to learn Go and jump through all the "official" hoops. But of course we would still welcome a contribution like this to Helm itself.

[technosophos]

@roboll and @jroberts235 Would you like to add your tools on #1564? I'm trying to improve documentation on extra tooling around the Helm ecosystem. Pointers to other tools will be really helpful for other users. And your tools are looking really interesting.

[jroberts235]

@technosophos - yes, of course.

[alanmeadows]

@technosophos If this is an actual feature in progress, please make sure that it functions the opposite way as well:

$ helm install-some --include-only="swift" openstack-1.2.3.tgz

I have concerns though that this would really deliver what we and I think many others may be after. Reading your explanation and the wordpress example, this seems like a way to exclude certain subcharts from an uber parent chart install. Imagine I repeated my command above 3 times for 3 different OpenStack components, I believe what I would end up with is 3 different deployments associated with the "openstack" parent chart, even though they may include different elements. That seems difficult to manage.

What we are trying to achieve is all the convenience an "uber" parent chart has to offer with its common defines and values but the capability of individually installing, upgrading, and managing children charts without massive global values duplication.

There are a massive number of OpenStack components and some of them we want in different namespaces (especially infrastructure elements, like ceph and databases) and some of them we want to repeatedly install and remove (read: develop) or upgrade knowing for sure we are not touching any other deployment. This is why the granularity of individual charts is best for us. We consider the parent chart a development "all-in-one" convenience only, not our ideal production solution.

We have struggled through a myriad of ways, finally settling on a _common.tpl with defines that is copied into each child chart so that we can helm install any individual component, and manage those individually but still enjoy some common global "values" and snippets. Initially, we tried to include a common "utility" child chart to all charts with global definitions but found that the unreliable chart loading (presumably a map) made includes from the child chart unreliable.

[syhol]

I'm currently wrapping a massive mysql template file in a big 'ol if block

{{ if .Values.mysql.enabled }}
# MySql Service
# MySql Pods
...
{{ end }}

...rather than using subcharts.

I like managing dependencies using values (via files or --set) instead of a new command line flag, but wish that I could use proper subcharts instead.

Here's an idea:

Set conditions using values

values.yaml

mysql:
  enabled: true

and helm install --set=mysql.enabled=false ./mychart

Declare subchart conditions

Now we have a couple of ways we can declare subchart conditions:

Option 1

Add a new top level item to Chart.yaml

name: mychart
version: 1.0.0
subcharts:
  - path: ./charts/mysql
    condition: mysql.enabled

On a helm install ./charts/mysql would only be installed if the mysql.enabled resolved to true

Option 2

Add a new item to the dependencies object in requirements.yaml. Although I think this would mean that all charts must be stored in a repository if you wanted to use this feature.

dependencies:
  - name: mysql
    version: 3.2.1
    repository: http://another.example.com/charts
    condition: mysql.enabled

On a helm install http://another.example.com/charts/mysql-3.2.1.tgz would only be installed if the mysql.enabled resolved to true

I think option this would be my preference, if I could swap the repository value for a path value pointing to the chart inside my subchart directory.

dependencies:
  - name: mysql
    version: 3.2.1
    path: ./charts/mysql
    condition: mysql.enabled

Sorry for the massive post.

[technosophos]

I'm looking into this idea of adding conditions.

[thomastaylor312]

@technosophos: I really like @syhol's idea for handling this. It is exactly what I was thinking of.

[bacongobbler]

@syhol's suggestion looks a lot like how we do our cluster installs as well on the Workflow side of things, though we call it "on-cluster" vs "off-cluster" for things like the database, redis, registry etc. and it's a flag on the global field.

That would simplify some of the logic we have in our templates.

[chancez]

I have this issue as well. I've got 4-5 components all contained within a umbrella chart. I used to just deploy the umbrella chart. Now I've got another chart I want to deploy, but separate from the rest of the charts for DB migrations. I want to run my db migrations chart separately, since i requires more manual verification and validation.

The way I ended up solving this was to structure my chart with 2 levels of nested charts to properly group things, and retain the values.yaml mapping I had.

Before:

.
├── Chart.yaml
├── Jenkinsfile
├── README.md
├── bin
├── charts
│   ├── app-web
│   ├── dex
│   ├── app-hookhandler
│   ├── app-server
│   └── app-worker
├── templates
│   ├── _helpers.tpl
│   └── dockerconfig.yaml
└── values.yaml

After (note the charts/app-migrate/charts/app-migrate nesting).

.
├── Chart.yaml
├── Jenkinsfile
├── README.md
├── bin
├── charts
│   ├── app-deploy
│   │   ├── Chart.yaml
│   │   ├── charts
│   │   │   ├── app-web
│   │   │   ├── dex
│   │   │   ├── app-hookhandler
│   │   │   ├── app-server
│   │   │   └── app-worker
│   │   ├── templates
│   │   │   ├── _helpers.tpl
│   │   │   └── dockerconfig.yaml
│   │   └── values.yaml
│   └── app-migrate
│       ├── Chart.yaml
│       ├── charts
│       │   └── app-migrate
│       ├── templates
│       │   ├── _helpers.tpl
│       │   └── dockerconfig.yaml
│       └── values.yaml
└── values.yaml

The alternative I was considering, and still am, is to flatten everything back out into what I had the first time, and to run helm upgrade <chart> for each of the subchart components, but that requires duplication of some created resources and this will result in a release for each component, which isn't really want I'm looking for.

[technosophos]

I believe the core of this proposal is now easily accomplished with conditions and tags (Helm 2.2.0).

https://github.com/kubernetes/helm/blob/master/docs/charts.md#tags-and-condition-fields-in-requirementsyaml