Flag --repository-cache used for finding helm-manager-* ‘ad hoc’ repository indexes, but not for creating them

[vovtz]

Output of helm version:

version.BuildInfo{Version:"v3.9.4", GitCommit:"dbc6d8e20fe1d58d50e6ed30f09a04a77e4c68db", GitTreeState:"clean", GoVersion:"go1.19"}

But this bug applies to older versions of Helm 3 equally, also on Linux.

Output of kubectl version: Client Version: v1.23.5

Cloud Provider/Platform (AKS, GKE, Minikube etc.): Any and all (N/A)

When using the --repository-cache flag in conjunction with a helm dependency update command while the Helm chart affected has a dependency for which there is no repo ‘alias’ defined (i.e. its URL has not been added with a helm repo add (…) command; often called an ‘unmanaged’ repo), an ‘ad hoc’ repository index file, named helm-manager-*-index.yaml (along with a file with the same base name, but ends with -charts.txt), will be created in the default repository cache directory, but a later step tries to find it in the directory that was set via the flag.

Example of the error message (this was on macOS):

Update Complete. ⎈Happy Helming!⎈
Error: no cached repository for helm-manager-0199bcabd1392eb1d12164d7fc1f56a975ac01570d110f202cb3a78614ce224b found. (try 'helm repo update'): open /Users/someone/tmp/x/helm-manager-0199bcabd1392eb1d12164d7fc1f56a975ac01570d110f202cb3a78614ce224b-index.yaml: no such file or directory

How to reproduce

The default directories mentioned in the instructions are the ones for Linux.

Setup and check the ‘standard’ usage

• Remove all files from the default repo index cache dir: rm -f ~/.cache/helm/repository/*
• Create a new Helm chart in a temporary directory: helm create test
• Change the current working directory to that chart: cd test
• Add a (can be fake) dependency to the Chart.yaml file, referring to a repository you don’t have registered with Helm yet, e.g.:

dependencies:
- name: test
  repository: https://cetic.github.io/helm-charts
  version: "1.0.0"

• Run helm dependency update (that works, but if you have a fake dependency it’ll tell you that it can’t be found in the repo)
• List the default repo cache dir: ls -lat ~/.cache/helm/repository; you’ll find indexes for all ‘managed’ repos, e.g. for Bitnami, SonarQube, etc.; there’s an ‘ad hoc’ index for that ‘unmanaged’ repo we added as a dependency as well, named helm-manager-<some-hash>-charts.txt and helm-manager-<some-hash>-index.yaml
• To start with a clean slate, remove the indexes again: rm -f ~/.cache/helm/repository/*

Reproduce the bug with the flag

• Not needed, but to make the scenario more realistic and to prove this doesn’t make a difference, explicitly update the repo indexes, but in the alternative cache dir: helm --repository-cache /tmp/helm-cache repo update
• Check that the default repo cache dir is still empty: ls -lat ~/.cache/helm/repository
• Check that the repo indexes have been created in the specified alternative dir: ls -lat /tmp/helm-cache; obviously, no index files were created for the ‘unmanaged’ repo
• Wait at least a minute (because of the minimal delta between file time stamps)
• Run helm --repository-cache /tmp/helm-cache dependency update (it will fail with a message complaining that it can’t find a helm-manager-*-index.yaml file in directory /tmp/helm-cache)
• List the contents of the default repo index cache dir: ls -lat ~/.cache/helm/repository; you’ll notice the file mentioned in the error message is present there (along with newly created indexes for the ‘managed’ repos)
• List the contents of the alternative repo cache dir that was specified: ls -lat /tmp/helm-cache; notice that the (specific) helm-manager-*-index.yaml is not present and that the indexes for the ‘managed’ repos were not updated (touched)

[joejulian]

You'll need to populate your new cache:

helm --repository-cache ~/tmp/x repo update

[vovtz]

@joejulian In practice that’s exactly what we do in CI, but it doesn’t make any difference, because that will only download the indexes for ‘managed’ repos into the cache. I’ll add it to the steps to reproduce, to prevent confusion, though.

[vovtz]

Maybe the root cause becomes clearer with a more limited PoC (mentioned default directories are again for Linux):

• Start from an empty directory:

mkdir test
cd test

• Add an empty values.yaml file: touch values.yaml
• Add a basic Chart.yaml file with a dependency on an ‘unmanaged’ Helm repo:

cat > Chart.yaml <<EOF
apiVersion: v2
name: test
description: Helm chart to reproduce a bug with regard to the --repository-cache flag
type: application
version: 1.0.0
appVersion: "1.0.0"
dependencies:
- name: test
  repository: https://cetic.github.io/helm-charts
  version: "1.0.0"
EOF

• Empty the default repo index cache dir: rm -f ~/.cache/helm/repository/*
• Try to install the dependency located in an ‘unmanaged’ repo, using an alternate index cache dir: helm --repository-cache /tmp/helm-cache dependency update; it will fail, complaining that it can’t find the index file for the ‘unmanaged’ repo, that has been created on-the-fly
• Check the contents of the default repo index cache dir: ls -lat ~/.cache/helm/repository; notice that the index files have been created there, instead of in the alternate dir (including the ‘helm-manager--index.yaml` for the ‘unmanaged’ repo)
• Check the contents of the alternative dir: ls -lat /tmp/helm-cache; notice that it hasn’t been even created (or will be empty if pre-existing)

A step that updates the indexes for the ‘managed’ Helm repos, in the alternative cache dir, can be included before the dependency update step: helm --repository-cache /tmp/helm-cache repo update. But the only difference will be that initial cache files will be created in the alternative dir. They will not be touched on the dependency update step, nor will the index for the ‘unmanaged’ repo be created there.

[github-actions]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[Oicho]

Hi,

Can we fast-track the merge of the PR that is fixing this issue ? We are encountering this issue and it would be nice to get this fix.

[sitilge]

Same here, can this be merged? Please?

[felipecrs]

I'm having the exact same issue, except that I'm not using --repository-cache or HELM_REPOSITORY_CACHE. Also, I tested the PR and it doesn't solve my issue either.

Any opinions if I should open a different issue?

[thesuperzapper]

You can get helm dependency update (or helm dependency build) to succeed by simply adding the repos from the Chart.yaml to the repo cache using helm repo add.

This can be automated by parsing the result of helm dependency list:

# add the helm repos for the chart dependencies
helm dependency list --max-col-width 10000 "./path/to/chart" | awk 'NR>1 {print $1,$3}' | while read -r name url; do
    if [[ -n "$name" && -n "$url" ]]; then
        helm repo add "$name" "$url"
    fi
done

# update the helm dependencies
helm dependency update "./path/to/chart"