dial tcp: lookup $(ARGOCD_REDIS_SERVICE): no such host

[azkore]

Checklist:

• I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• I've included steps to reproduce the bug.
• I've pasted the output of argocd version.

Describe the bug

After updating ArgoCD to v2.7.0-rc2 there are errors in argocd-repo-server logs:

argocd-repo-server time="2023-04-13T08:07:37Z" level=warning msg="manifest cache error &ApplicationSource{RepoURL:https://***,Path:***,TargetRevision:HEAD,Helm:nil,Kust
omize:nil,Directory:nil,Plugin:nil,Chart:,Ref:,}: dial tcp: lookup $(ARGOCD_REDIS_SERVICE): no such host"

Looks like the issue was introduced here 25badeb#diff-13c44d9d61d8213832a7e70890e8d2a6fdab6b3c2170d364e3b71ff3ac73805aL15834-R15837

I think env vars in k8s manifests are not supposed to be expanded

To Reproduce

There are no specific steps to reproduce, I guess. Just deploy ArgoCD v2.7.0-rc2 using https://github.com/argoproj/argo-cd/blob/v2.7.0-rc2/manifests/install.yaml and look for the error in the logs

Expected behavior

argocd-repo-server is able to connect to redis, no errors in the logs

Version

argocd: v2.6.7+5bcd846.dirty
  BuildDate: 2023-03-23T17:25:30Z
  GitCommit: 5bcd846fa16e4b19d8f477de7da50ec0aef320e5
  GitTreeState: dirty
  GoVersion: go1.20.2
  Compiler: gc
  Platform: linux/amd64
argocd-server: v2.7.0-rc2+483d26b.dirty

Logs

argocd-repo-server time="2023-04-13T08:07:37Z" level=warning msg="manifest cache error &ApplicationSource{RepoURL:https://***,Path:***,TargetRevision:HEAD,Helm:nil,Kust
omize:nil,Directory:nil,Plugin:nil,Chart:,Ref:,}: dial tcp: lookup $(ARGOCD_REDIS_SERVICE): no such host"

[yasirub]

I also got the issue and was able to fix it by manually editing argocd-repo-server deployment

[crenshaw-dev]

Thanks for testing! I think this should fix it: #13252

[3bbbeau]

In the meantime, after looking at @crenshaw-dev's PR to see the underlying issue I was able to fix it with:

kubectl patch deploy argocd-repo-server -n argocd --type "json" -p '[
{"op":"replace","path":"/spec/template/spec/containers/0/args","value":["/usr/local/bin/argocd-repo-server"]}]'

[azkore]

Although the issue has been resolved in the master branch, it has not been addressed in the 2.7.0 tag, and as a result, it is not included in the manifest file available at https://raw.githubusercontent.com/argoproj/argo-cd/v2.7.0/manifests/install.yaml. This raises the question of whether this was an oversight or if the bug was intentionally left in the 2.7.0 release.

Would it be possible to update the tag to include the fix, or was the decision made to release 2.7.0 with the bug?

[ldacey]

Just ran into this issue today when I upgraded to 2.7.0 as well

[crenshaw-dev]

The fix should have been in 2.7.0... maybe I fired off the release too fast?

[crenshaw-dev]

I simply failed to cherry-pick the fix. It's what I get for cutting releases after 5pm. :-) Will cut 2.7.1 to fix it.