Argocd and velero, repo-server CrashLoopBackOff during restore

[JDHawken]

Hello there,
I have an little issue with argocd and velero, we are using velero (set up on kopia) to make backup and restore of the application, everything is fine during backup but when we trying to restore every pods in the namespace argocd, argocd-repo-server goes into error : CrashLoopBackOff.
When I look log from the pod, I have the following messages :

Defaulted container "repo-server" out of: repo-server, avp, avp-helm, restore-wait (init), copyutil (init), download-tools (init)
time="2023-06-26T13:41:36Z" level=info msg="ArgoCD Repository Server is starting" built="2023-03-07T22:48:16Z" commit=7be094f38d06859b594b98eb75c7c70d39b80b1e port=8081 version=v2.6.4+7be094f
time="2023-06-26T13:41:36Z" level=info msg="Generating self-signed TLS certificate for this session"
time="2023-06-26T13:41:36Z" level=info msg="Initializing GnuPG keyring at /app/config/gpg/keys"
time="2023-06-26T13:41:36Z" level=fatal msg="re-initializing keyring at /app/config/gpg/keys failed: unlinkat /app/config/gpg/keys/.velero/e044b47e-ad60-424a-a9a1-9c26922464f4: permission denied"

I think I understand a right problem between argocd and velero during the restore, when we reach the key initialization phase, if someone has already tried to restore argocd with velero or has a lead.

Thanks in advance !

[jannfis]

I suggest to exclude the volume mount from backup/restore procedures, as it is ephemeral anyway (i.e. reconstructed by the repo server process) on each startup. It seems to me that velero creates some meta-data on the file system with permissions that interfere with the repo server pod's activties.

This may help you going forward.