Error from git fetch cannot lock ref caused by incorrect branch paths

[ADustyOldMuffin]

Checklist:

• I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• I've included steps to reproduce the bug.
• I've pasted the output of argocd version.

Describe the bug

When 2 branches are created that are in a format of mybranch/mychange mybranch/mychange/subchange this causes issues in Git and as such also causes issues in ArgoCD when running a fetch.

To Reproduce

1. Create a branch my-cool-branch/sub on a repo
2. Create a second branch my-cool-branch/sub/another-one (perform any cleanup needed to allow this to push)
3. Try and refresh an application using this repo in ArgoCD

Expected behavior

ArgoCD should run a --prune option which would fix this issue.

Version

argocd: v2.4.12+41f54aa.dirty
  BuildDate: 2022-09-16T03:19:53Z
  GitCommit: 41f54aa556f3ffb3fa4cf93d784fb7d30c15041c
  GitTreeState: dirty
  GoVersion: go1.19.1
  Compiler: gc
  Platform: darwin/amd64
argocd-server: v2.4.6+a48bca0
  BuildDate: 2022-07-12T22:31:17Z
  GitCommit: a48bca03c79b6d63be0c34d6094831bc6916b3bc
  GitTreeState: clean
  GoVersion: go1.18.3
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: v4.4.1 2021-11-11T23:36:27Z
  Helm Version: v3.8.1+g5cb9af4
  Kubectl Version: v0.23.1
  Jsonnet Version: v0.18.0

Logs

Unable to generate manifests in path/to/my/application: rpc error: code = Internal desc = Failed to fetch default: `git fetch origin --tags --force` failed exit status 1: error: cannot lock ref 'refs/remotes/origin/badbranch/subbranch/extrasub': 'refs/remotes/origin/badbranch/subbranch' exists; cannot create 'refs/remotes/origin/badbranch/subbranch/extrasub' From <Link to repository> ! [new branch] badbranch/subbranch/extrasub -> origin/badbranch/subbranch/extrasub  (unable to update local ref) Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. 

Extra Notes
Git prune is currently being used to fix other issues, as seen here but I believe it would be better to just run --prune by default to remove any hanging branches not around on the remote anyway. I'm open to hearing why this might cause issues.

[jessesuen]

I think you're right about this. I also can't think of a reason why prune would be bad, since our local copy should mirror the remote.

[ADustyOldMuffin]

@jessesuen Would you mind if I made a PR for this? Is there a process such as a proposal I need to follow for this?

[crenshaw-dev]

@ADustyOldMuffin a PR is fine, no proposal needed!

I'm a little worried about whether there's a performance impact associated with running --prune on every fetch. The repo-server has the potential to do a lot of fetches. Personally I'd rather see the cannot lock ref error added to the messages which trigger a one-off prune here. But I'm not strongly opposed to always using --prune.

Either way, happy to review a PR! :-)

[ADustyOldMuffin]

👍🏻 When this was initially mentioned in the chat on Slack I did a bunch of timing with fetch and prune on various repositories we use since they change very often.

I'll do some more testing to double check, but on initial viewing the difference was in the milliseconds for a repo getting hundreds to thousands of commits a day and lots of branches coming and going and on less busy repos the difference was close to un-measurable (do not that I waited significant time between each fetch to properly load up something that would require pruning).

I can also check into possibly building and running a custom version of ArgoCD with this change and monitor metrics to see if we have a change if needed.

[crenshaw-dev]

the difference was in the milliseconds

Cool, that sounds worth it. Would also get rid of the weird fallback logic we have now.

I can also check into possibly building and running a custom version of ArgoCD with this change and monitor metrics

The extra validation would be helpful!

[ADustyOldMuffin]

Posted the small PR to start the process, I'll note if I can manage to get the custom build working and/or any findings I get.