helm --set-file does not work

[davidkarlsen]

If you are trying to resolve an environment-specific issue or have a one-off question about the edge case that does not require a feature then please consider asking a
question in argocd slack channel.

Checklist:

• I've searched in the docs and FAQ for my answer: http://bit.ly/argocd-faq.
• I've included steps to reproduce the bug.
• I've pasted the output of argocd version.

Describe the bug

According to #2752 it should be possible to do --set-file paramname=filename

To Reproduce

docker run --rm -v $(pwd):/workspace:ro -w /workspace fsnexus.evry.com:8085/argoproj/argocd:v1.5.0 argocd --server argocd.global.finods.com --grpc-web --auth-token ${TOKEN} app set eos-jfr-srv-d2 --helm-set image.tag=10.2.1-SNAPSHOT-2b8b344b --helm-set-file configmap.application-envconfig\.yaml=/workspace/argocd/application-g-d1.yml --loglevel=debug

the file is there:

docker run --rm -v $(pwd):/workspace:ro -w /workspace fsnexus.evry.com:8085/argoproj/argocd:v1.5.0 ls -l /workspace/argocd/application-g-d1.yml
-rw-r--r-- 1 argocd argocd 1020 Apr  4 20:47 /workspace/argocd/application-g-d1.yml

Expected behavior

It should work.

Screenshots

See logs

Version

argocd version
argocd: v1.5.0+bdda410
  BuildDate: 2020-04-02T16:39:07Z
  GitCommit: bdda41046378a855e289b5f1602d5c923a3f914a
  GitTreeState: clean
  GoVersion: go1.14
  Compiler: gc
  Platform: darwin/amd64
argocd-server: v1.5.0+bdda410
  BuildDate: 2020-04-02T16:38:24Z
  GitCommit: bdda41046378a855e289b5f1602d5c923a3f914a
  GitTreeState: clean
  GoVersion: go1.14
  Compiler: gc
  Platform: linux/amd64
  Ksonnet Version: v0.13.1
  Kustomize Version: Version: {Version:kustomize/v3.2.1 GitCommit:d89b448c745937f0cf1936162f26a5aac688f840 BuildDate:2019-09-27T00:10:52Z GoOs:linux GoArch:amd64}
  Helm Version: version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}
  Kubectl Version: v1.14.0

Logs

time="2020-04-04T23:02:57Z" level=fatal msg="rpc error: code = InvalidArgument desc = application spec is invalid: InvalidSpecError: Unable to generate manifests in : rpc error: code = Unknown desc = `helm2 template . --name eos-jfr-srv-d2 --namespace eos-jfr-srv-d1 --kube-version 1.13 --set image.tag=10.2.1-SNAPSHOT-2b8b344b --set-file configmap.application-envconfig.yaml=/workspace/argocd/application-g-d1.yml --values /tmp/values-654267902.yaml --api-versions v1 --api-versions apiregistration.k8s.io/v1 --api-versions apiregistration.k8s.io/v1beta1 --api-versions extensions/v1beta1 --api-versions apps/v1 --api-versions apps/v1beta2 --api-versions apps/v1beta1 --api-versions events.k8s.io/v1beta1 --api-versions authentication.k8s.io/v1 --api-versions authentication.k8s.io/v1beta1 --api-versions authorization.k8s.io/v1 --api-versions authorization.k8s.io/v1beta1 --api-versions autoscaling/v1 --api-versions autoscaling/v2beta1 --api-versions autoscaling/v2beta2 --api-versions batch/v1 --api-versions batch/v1beta1 --api-versions batch/v2alpha1 --api-versions certificates.k8s.io/v1beta1 --api-versions networking.k8s.io/v1 --api-versions policy/v1beta1 --api-versions rbac.authorization.k8s.io/v1 --api-versions rbac.authorization.k8s.io/v1beta1 --api-versions storage.k8s.io/v1 --api-versions storage.k8s.io/v1beta1 --api-versions admissionregistration.k8s.io/v1beta1 --api-versions admissionregistration.k8s.io/v1alpha1 --api-versions apiextensions.k8s.io/v1beta1 --api-versions scheduling.k8s.io/v1beta1 --api-versions coordination.k8s.io/v1beta1 --api-versions servicecatalog.k8s.io/v1beta1 --api-versions clusterhealth.ibm.com/v1 --api-versions comcast.github.io/v1 --api-versions helm.fluxcd.io/v1 --api-versions icp.ibm.com/v1 --api-versions monitoring.coreos.com/v1 --api-versions monitoringcontroller.cloud.ibm.com/v1 --api-versions oidc.security.ibm.com/v1 --api-versions volumesnapshot.external-storage.k8s.io/v1 --api-versions argoproj.io/v1alpha1 --api-versions audit.policies.ibm.com/v1alpha1 --api-versions authentication.istio.io/v1alpha1 --api-versions bitnami.com/v1alpha1 --api-versions certmanager.k8s.io/v1alpha1 --api-versions forecastle.stakater.com/v1alpha1 --api-versions iam.policies.ibm.com/v1alpha1 --api-versions kubeapps.com/v1alpha1 --api-versions openebs.io/v1alpha1 --api-versions policies.ibm.com/v1alpha1 --api-versions rbac.istio.io/v1alpha1 --api-versions config.istio.io/v1alpha2 --api-versions networking.istio.io/v1alpha3 --api-versions admission.certmanager.k8s.io/v1beta1 --api-versions securityenforcement.admission.cloud.ibm.com/v1beta1 --api-versions metrics.k8s.io/v1beta1` failed exit status 1: Error: failed parsing --set-file data: open /workspace/argocd/application-g-d1.yml: no such file or directory"

[jannfis]

Hi @davidkarlsen, the error message stems from the server, not from the client. I think the file specified must exist at given path in the container of argocd-repo-server, so most likely should be included in your repository and then referenced with a relative path.

[jon-walton]

Hi @jannfis , I'm also having trouble with this. An example of what in trying to do is here

https://github.com/jon-walton/linkerd2-gitops-example/blob/master/apps/linkerd.yaml#L23

If I use fileParameters, argocd gives me a file not found error for example-resources/ca.crt

My expectation is for argocd to be able to use --set-file to include files relative to the root of the repo containing the Application manifest

[mayzhang2000]

Tried with both argocd app set and create app declaratively.
Result: app is created and the right value is updated.

argocd app set test-helm-set-file --helm-set foo2=mayHelmSet --helm-set-file foo=/Users/mayz985/go/src/github.com/argoproj/argo-cd/test/mayfile --plaintext --server localhost:8080 " dir= execID=ShqsW
time="2020-04-21T09:14:02-07:00" level=debug duration=473.508013ms execID=ShqsW
time="2020-04-21T09:14:02-07:00" level=info msg="expectation succeeded: no error and output contained ''"

Also tried with creating app declaratively and app is created with right value set.

kind: Application
metadata:
  name: {{.Name}}
  namespace: {{.ArgoCDNamespace}}
spec:
  project: {{.Project}}
  source:
    Helm:
      File Parameters:
        Name:  foo
        Path:  mayfile
    repoURL: {{.RepoURL}}
    targetRevision: HEAD
    path: {{.Path}}
  destination:
    server: https://kubernetes.default.svc
    namespace: {{.DeploymentNamespace}}

[alexmt]

@jon-walton . your application references chart using Helm repo URL, not Git repository. In this case, Argo CD downloads chart content only so other files are not available.

In order to use chart and file in git repo I would suggest to use git URL. You can untar linkerd2-2.7.1.tgz into the folder in the git repo and create Argo CD app using Git repo URL. File parameter path should be relative to the chart folder. E.g. ../../example-resources/ca.crt if chart is stored in ./charts/linkerd.

[jon-walton]

Hi @alexmt

That's unfortunate and in my mind, a little counter-intuitive. Fetching and untarring the charts is currently what we're doing (not shown in the example repo) but I would prefer not to continue maintaining a copy of the charts.

We currently have a single repository that contains an argocd folder for each cluster. Those folders contain the ArgoCD Application manifests. The untarred charts are also in that repository.

I'll use the linkerd chart as an example...

Due to having multiple clusters, we're having to maintain multiple copies of the linkerd chart and each cluster references the folder containing the desired version of the chart (a chart version is tied to a released version of linkerd, we can't just set the linkerd version in the values.yaml file).

Having multiple versions of the charts is also beneficial for other deployed resources (let's say nginx-ingress). We wouldn't want to roll out the new chart to all clusters a the same time.

The reason for maintaining multiple copies of the chart, rather then setting the targetRevision on the git url is because sometimes we do make changes to the values.yaml file (we don't use overrides in the ArgoCD UI).

We could set targetRevision to a branch, but we've found that all clusters being configured from master to be the simplest workflow for us and reduces mistakes.

My hope was to start moving away from fetching and untarring charts, over to using helm urls and be able to reference files in the repo where the Application is defined.

In my mind, it makes sense to be able to do that because if I understand correctly, ArgoCD already cloned the git repo containing the Application manifests, therefore also has the files referenced for --set-file

[alexmt]

I suggest considering using the umbrella chart approach: https://github.com/argoproj/argocd-example-apps/tree/master/helm-dependency . The idea is to keep real charts in helm repository and keep only value files and requirements.yaml in Git. The requirements.yaml references required chart versions.

This way you don't have to untar charts and can use values stored in git as parameters.

[alexmt]

Looks like issue can be closed since we found the reason why Argo CD could not find the file.

[davidkarlsen]

@alexmt having to create "wrapper" umbrella-charts adds cruft. What do you think about the solution like @jon-walton describes - i.e. keeping the chart in the chart-repo, referring to it by repo, chart name and version, while keeping value-files in the argo git-repo - passing an array of them to argo? That would be clean-cut and more natural to users (looks like a normal helm workflow).

[abdennour]

getting same issue & we found a typo as following:

      fileParameters:
        - name:
        - path

Where it should be

      fileParameters:
        - name:
          path: 

Yaml typo