feature request: create configmaps from directories

[stapelberg]

Using kubectl, I can use kubectl create configmap prometheus --from-file=prometheus/ to create a ConfigMap with all files in the prometheus directory.

AFAICT, helm does not support creating ConfigMaps from static files yet — instead, I would need to create a YAML template and inline the files into that config. For easier modification of these files and more clarity, I’d like to keep the files as they are.

Is this something which helm could support in the future?

[technosophos]

We're talking about how we might handle cases like this without introducing a lot of special-casing into Helm. (For example, if we support ConfigMaps, do we support similar for secrets? What about taking a shell script and passing it to a Job...?)

We're also exploring using the template system to make this possible, where you might be able to do something like this:

{{ $params := tuple "configmap_name" "path/to/file"}}
{{template "common.configmap" $params}}

The idea there would be that a standard configmap template could accept information to build named configmaps.

[stapelberg]

Including data from external files in templates would certainly work for me. In general though, I can see how that becomes a bit cumbersome once the directory contains a lot of files, as you need to create an equivalent template file for each one.

[technosophos]

I'm adding partial support for this by exposing the files in a chart ("pkg/protoo/hapi".Chart.Files) in the charts.

[technosophos]

Alright, so the new way of doing this is {{.Files.GetString "myfile.txt"}}. The data can then be piped into other functions ({{.Files.GetString "foo.yaml" | indent 4}}).

This is probably as close as we will get to the requested feature, though there is work in https://github.com/kubernetes/charts to add some high-level structures that may make it even easier to create configmaps.

[Quentin-M]

@technosophos For these solutions to be exceptionally interesting, the included files should be treated as templates and rendered.

The issue with {{.Files.GetString}}``is that the files aren't parsed/parametrized. The issue with{{template ...}` is that you can't pipe it to indent (afaik) and that's it is a burden to define a template for each included file.

[stapelberg]

Thanks for implementing the Files map. I’ve tried using it, but I can’t figure out how to get around the following issue (or what the issue actually is):

$ cat rlt/templates/prometheus.configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: "prometheus"
data:
  prometheus.conf: {{.Files.Get "static/prometheus/repro.conf" | printf "%s" | quote}}

$ cat rlt/static/prometheus/repro.conf 
global:
  scrape_interval:     15s
  evaluation_interval: 15s

rule_files:
  - "robustirc.rules"

$ echo -n "robustircImage: 'bleh/foo'" |  helm install rlt --debug --dry-run
Created tunnel using local port: '44318'                                                                                        
Server: ":44318"
Chart path: /home/michael/gocode/src/github.com/robustirc/benchmark/rlt
Error: YAML parse error on rlt/templates/prometheus.configmap.yaml: error converting YAML to JSON: yaml: line 10: did not find expected key

Once I remove the “rule_files:” section from rlt/static/prometheus/repro.conf, the command works as expected. I suspect it’s a quoting issue.

Is there a way to see the result of template expansion before it’s converted from YAML to JSON?

Am I using the quote function wrong?

[stapelberg]

Looking at https://github.com/Masterminds/sprig/blob/abe09979bcb1ec0a50b2d7bbf67e6aaa11787417/functions.go#L583, the code uses fmt.Sprintf("\"%v\"", s), i.e. the quote sprig function doesn’t escape quotes.

Indeed, replacing the - "robustirc.rules" line with - 'robustirc.rules' works around the issue.

[stapelberg]

The following pattern seems to work:

apiVersion: v1
kind: ConfigMap
metadata:
  name: "prometheus"
data:
  prometheus.conf: |
{{.Files.Get "static/prometheus/repro.conf" | printf "%s" | indent 4}}

See also http://stackoverflow.com/a/21699210/712014 for details about multi-line strings in YAML.

[yllierop]

What's the current status of this issue?

[technosophos]

The current status of this issue is that it's assigned for triaging post-2.0.0.

We've been recommending that people use @stapelberg 's method for importing configuration files into ConfigMaps. And I have talked a little about implementing this feature as a plugin. But I can explain why the issue isn't moving too quickly.

It's hard to justify adding special casing logic for this issue, or even introducing new commands, when the same outcome can be achieved via the template system. But that's a good reason why the solution to this problem might be a plugin.

Now, there may be a use case that I don't understand -- one that would warrant replicating kubectl's behavior within Helm. If you feel like you've got a new use case, I'd appreciate hearing it.

[technosophos]

I believe #1666 fixes this.

[technosophos]

Closed via #1666