helm install does not use the url value in the repository index.yaml files

[jstrachan]

Here's an example of the issue:

$ helm repo add fabric8 https://fabric8.io/helm
fabric8 has been added to your repositories
$ helm search chaos
fabric8/chaos-monkey-2.2.233.tgz
fabric8/chaos-monkey-2.2.245.tgz
fabric8/chaos-monkey-2.2.236.tgz
fabric8/chaos-monkey-2.2.238.tgz
fabric8/chaos-monkey-2.2.247.tgz
fabric8/chaos-monkey-2.2.237.tgz
fabric8/chaos-monkey-2.2.240.tgz
fabric8/chaos-monkey-2.2.242.tgz
fabric8/chaos-monkey-2.2.239.tgz
fabric8/chaos-monkey-2.2.243.tgz
fabric8/chaos-monkey-2.2.230.tgz
fabric8/chaos-monkey-2.2.231.tgz
fabric8/chaos-monkey-2.2.234.tgz
fabric8/chaos-monkey-2.2.225.tgz
fabric8/chaos-monkey-2.2.223.tgz
$ helm install fabric8/chaos-monkey-2.2.245.tgz
Error: file "fabric8/chaos-monkey-2.2.245.tgz" not found

The issue is that the the code right now ignores the url setting in the index.yaml and instead assumes that the charts are at repositoryURL + "/" + chartname + "-" + version + ".tgz"

PR on its way

[jstrachan]

ah I just spotted the text here: https://github.com/kubernetes/helm/blob/master/docs/chart_repository.md#the-chart-repository-structure I'm not sure if thats new or I just missed it before!

If there's a canonical place to look based on the repository URL and the chart name-and-version, there's no need to bother with the URL inside the index.yaml I suppose?

It does seem a little more limiting though; the chart tgz could be anywhere and the index.yaml could just link to the actual tgz file. (This is how we did the first cut of the fabric8 registry at http://fabric8.io/helm/ since we host all our chart tgz files in http://central.maven.org/ :)

If you have many charts each with many versions and we wanted to use a simple static file server for the helm repository, then we'd need to start using '/' instead of '.' as a separator in the chart key to avoid having crazy numbers of files in a single directory. Would this approach be valid?

e.g. to have charts like this:

$ helm repo add fabric8 https://fabric8.io/helm
fabric8 has been added to your repositories
$ helm search chaos
fabric8/chaos-monkey/2.2.233.tgz
fabric8/chaos-monkey/2.2.245.tgz
fabric8/chaos-monkey/2.2.236.tgz
fabric8/chaos-monkey/2.2.238.tgz
fabric8/chaos-monkey/2.2.247.tgz
fabric8/chaos-monkey/2.2.237.tgz
fabric8/chaos-monkey/2.2.240.tgz
fabric8/chaos-monkey/2.2.242.tgz
fabric8/chaos-monkey/2.2.239.tgz
fabric8/chaos-monkey/2.2.243.tgz
fabric8/chaos-monkey/2.2.230.tgz
fabric8/chaos-monkey/2.2.231.tgz
fabric8/chaos-monkey/2.2.234.tgz
fabric8/chaos-monkey/2.2.225.tgz
fabric8/chaos-monkey/2.2.223.tgz

so that in the file system each chart would have a separate folder of versions?

Forcing all charts to be in the same logical directory in the file server would mean we'd have to run proxies in front just to convert the dots into / ;)

[michelleN]

The original intent was to always fetch the chart url from the index file. The original architecture of the chart repo was that it had a flat structure and you could assume the path to a chart if you wanted to structure your repo that way. However, that definitely doesn't need to be the case anymore. If we continue relying on the index file, then that should be the canonical source for the chart url.
I think we need some further discussion on this point and the repo structure in general at Thursday's dev call. Thank you for bringing this up. Would love to have you on the call Thursday if you're around.
cc/ @technosophos @jackfrancis @prydonius

P.S. I also get what you're saying about having a folder of versions. I think that's a good idea. I think we could make the search output even better in that scenario. Will also need to bring this up in a similar vein.

[jstrachan]

@michelleN sorry I couldn't make this weeks call; I'll try make next weeks though

I spotted this comment on the possible use of URLs #1197 (comment) - still a proposal right now but if it happened then we'd definitely need to start using the URL(s) to support repository mirrors.

TL;DR: I'd like to use github releases, bintray or maven/npm repositories to host helm tgz files for open source charts so they can get released with other project artefacts.

My main argument with of using the URL in the Index.yaml is it gives the implementor more freedom to use different repository tools.

e.g. folks using java / node / python or whatever can reuse their existing release mechanisms and repositories (maven or npm or whatever) to release helm charts along with any other artefacts. e.g. all those fabric8 charts, we release jars, pom.xml files, kubernetes yaml files, helm charts all in the usual maven/java way (with separate docker images in a docker registry).

e.g. here's all the chaos-money releases: http://central.maven.org/maven2/io/fabric8/devops/apps/chaos-monkey/

then here's a specific version (with jar, kubernetes yaml, openshift template and helm chart):
http://central.maven.org/maven2/io/fabric8/devops/apps/chaos-monkey/2.2.254/

By forcing the URL to be ignored it means we can't reuse the mirrored maven repositories as a hosting implementation of helm charts. I can't see folks being able to use services like npm repos, github releases or bintray to distribute helm charts either; since we can't force those repos to use the helm layout. Folks would have to spin up their own custom website repository; rather than reuse maven, npm, github, bintray et al repositories which just makes it harder for OSS folks to create & maintain repositories.

While I totally grok the local file system layout conventions; I guess I don't grok why we care how or where the tgz's live so long as the Index.yaml is valid - since we have the URL/URLs field. Its not like we iterate through the remote repository like a file system; I mean otherwise there's not much point of having the Index.yaml in the first place ;)

[technosophos]

I changed a bunch of things in #1198, #1197, and #1199 so that now the URLs are extracted from the index.yaml instead of being constructed, you can have more than one URL per chart, and the index file is hierarchical instead of flat.

Please let me know if this refactoring gets you closer to where you want to go.

Specifically, there is no no relation between the location of the index and the location of the chart unless you use relative paths for the chart URL. So you should be able to structure your chart hierarchy however you want.

[jstrachan]

@technosophos many thanks I'll give it a try!

[jstrachan]

@technosophos this all looks fixed for me now in alpha 5 - many thanks!

All our charts now seem to be usable with alpha 5: http://fabric8.io/helm/ I've regenerated the index.yaml to use the new alpha 5 layout and so far helm's working great. I'm sure there are issues in our specific charts - just testing them now - but helm looks good for us now!