Received cert error when configuring AroCD SSO to use OIDC with self signed certificate

[ayalam26]

If you are trying to resolve an environment-specific issue or have a one-off question about the edge case that does not require a feature then please consider asking a
question in argocd slack channel.

Checklist:

• I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• I've included steps to reproduce the bug.
• I've pasted the output of argocd version.

Describe the bug

When configuring ArgoCD to use SSO using an existing OIDC provider that is using a self-signed certificate, you receive the following error message when attempting to authenticate to the ArgoCD UI:
"Failed to query provider "https://exmaple.com/adfs": Get "https://example.com/adfs/.well-known/openid-configuration": x509: certificate signed by unknown authority". The argocd-server deployment also had the --insecure flag set.

To Reproduce

Configure ArgoCD to use SSO with an IDP that uses a self-signed certificate.

Expected behavior

When logging into ArgoCD using SSO, you should be able to authenticated and based on RBAC policy see projects available to you.

Screenshots

If applicable, add screenshots to help explain your problem.

Version

v1.5.8

Logs

Paste any relevant application logs here.

[jessesuen]

We don't currently support configuring argo cd to a identity provider with a self-signed cert.

[ayalam26]

Thank you for the information. Are there any plans to support a identity provider that uses a self-signed cert in the future, if so any idea of the time frame.

[Timoses]

We don't currently support configuring argo cd to a identity provider with a self-signed cert.

@jessesuen Does this also apply when the SSO host is using a certificate from a private/custom CA? If not, how would one permit the CA? This sounds like it only applies to repositories.
Background: We face this problem occasionally right after deploying ArgoCD. Usually it was enough to restart the argocd-server pod and it worked (we use a repository which has a certificate from the same CA which is configured in argocd-tls-certs-cm.yaml for the repository host). (Using ArgoCD 1.7).

[jbarrick-mesosphere]

@jessesuen does ArgoCD also not support custom CAs for oidc?

[jpertejo]

We are also facing this same issue. We need to configure SSO using an IdM that uses self-signed certificates (for development environments) and certificates from a private CA (for upper environments).
Any news or plans to support this feature?

[skuethe]

Today I also stumbled upon this problem while deploying argo-cd in a new staging environment. The OIDC provider (Keycloak) is deployed using a custom CA signed wildcard certificate (argo-cd is using the same wildcard). We are using argo version v1.8.4+28aea3d.

Now comes the interesting part:
We have argo-cd up and running with the (more or less) same setup in a DEV environment. And there it works without problems! So if I press the LOG IN VIA KEYCLOAK button, it redirects me to the Keycloak instance, I can authenticate, get redirected back to argo-cd and have access based on my user groups.

The only differences between these two environments are:

• the OIDC provider in DEV (keycloak) is not using a wildcard certificate
• DEV argo-cd is running version v1.8.3+0f9c684

And that's it (as far as I currently know).
All certificates are signed by the same custom CA (so wildcard STAGE and the ones for DEV).
The wildcard certificate in STAGE is configured as default cert in the nginx ingress. So it is also used for argo-cd.

I would need to see if I get more testing details, but it seems argo-cd CAN work with custom CA signed certs. It would be nice to get some insights as to what the requirements are.
I hope I can request a new (non wildcard) certificate for STAGE and see if that changes anything. Maybe also downgrade to the argo-cd version used in DEV.

@AnyBody who also has this issue: are you guys using wildcard certificates? Could you also check your arog-cd versions? Maybe we can pinpoint the problem and find the solution to always get this running.

[Timoses]

We do not use wildcard certs for neither keycloak nor ArgoCD.

Keycloak is running on a VM outside of the cluster with a cert signed by our custom root CA.

ArgoCD is naturally running in an ArgoCD cluster with a certificate created by Cert-Manager through a custom CA signed by our custom root CA.

As noted above, previously it seemed like restarting argocd-server helped a bit sometimes. We have multiple instances of the same layout. Some work, some don't.

("custom" CA/cert above relates to our own managed x509 PKI)

[rgarcia6520]

Posting to also request support for this. This feature is much needed.
Either this or ability to specify a CA to be trusted by the system/pod/container.

[jerryli-ctc]

I am using this workaround when deploying argocd with helm chart (inspired by devops-stack approaches)

1. Create a TLS secret in the argued namespace with the issuing or Root CA in it for verifying your self-signed Keycloak certificate or other OIDC certs. i.e. "argocd-tls-ca".
2. In the argocd override chart value file, customize the argocd "server" block with:

  volumeMounts:
     - name: certificate
       mountPath: /etc/ssl/certs/ca.crt # your self-signed CA part inside the secret 
       subPath: ca.crt 
  volumes:
     - name: certificate
       secret:
         secretName: argocd-tls-ca
         defaultMode: 420 

Deploy argocd and configure Keycloak ... then your argocd login will trust your self-signed cert for keycloak.

[wagfeliz]

Posting to also request this feature. For me this is an security issue because it forces you to expose both, keycloak and argocd into extranet, and that is not an option for us.

[perezjasonr]

would also like to request this, its pretty common for people to run something like keycloak internally and with a custom ca. we just ran into this as well

[cubanp3te]

Weirdly I wasn't getting this issue, but since upgrading to 2.4.7 (from 2.4.3 I think) I am now getting this problem.

For full disclosure, the CA cert is loaded in arcocd-tls-certs-cm for my internal git system (gitea) with a key matching the internal dns name for this system but the certificate being the CA certificate, so I don't know if this was being used previously to validate keycloak's certificate in any way?

[cubanp3te]

Weirdly I wasn't getting this issue, but since upgrading to 2.4.7 (from 2.4.3 I think) I am now getting this problem.

For full disclosure, the CA cert is loaded in arcocd-tls-certs-cm for my internal git system (gitea) with a key matching the internal dns name for this system but the certificate being the CA certificate, so I don't know if this was being used previously to validate keycloak's certificate in any way?

Ok ignore this, I've just found #6712 which allows specifying the root CA of a private CA Server specifically for oidc, but it's not on the Keycloak page, it's on https://argo-cd.readthedocs.io/en/stable/operator-manual/user-management/#configuring-a-custom-root-ca-certificate-for-communicating-with-the-oidc-provider

[crenshaw-dev]

Weirdly I wasn't getting this issue, but since upgrading to 2.4.7 (from 2.4.3 I think) I am now getting this problem.

Yeah, there was a security issue where we simply didn't validate the cert at all. Now it's necessary to set the root CA.

[RossComputerGuy]

I deployed Keycloak through the Bitnami Help and I am having this problem. Internally, everything is HTTP but externally, its HTTPS. Not sure what the solution to fix this is. This is my repository.

[onceinarow]

this rootCA configuration value worked perfectly for me: https://argo-cd.readthedocs.io/en/stable/operator-manual/user-management/#configuring-a-custom-root-ca-certificate-for-communicating-with-the-oidc-provider

[zillani]

argocd version: 2.8.3

We can close this, I'm able to login to argocd via keycloak with rootCA, example,

oidc.config: |
    name: Keycloak
    issuer: https://keycloak.mgmt.local/realms/master
    clientID: argocd
    clientSecret: $oidc.keycloak.clientSecret
    requestedScopes: ["openid", "profile", "email", "groups"]
    rootCA: |
      -----BEGIN CERTIFICATE-----
      MIIDtDCCApygAwIBAgIUeMHt21YY2H0+qWUfhVB2Zq4sAhIwDQYJKoZIhvcNAQEL
      BQAwcjELMAkGA1UEBhMCVVMxETAPBgNVBAgTCENvbG9yYWRvMQ8wDQYDVQQHEwZE
      ZW52ZXIxEDAOBgNVBAoTB01hdmVuaXIxEDAOBgNVBAsTB01XUENJQ0QxGzAZBgNV
      BAMTEiBtd3AtbWF2ZW5pci5sb2NhredactedeFw0yMzA1MjQyMTQyMDBaFw0yODA1MjIy
      MTQyMDBaMHIxCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhDb2xvcmFkbzEPMA0GA1UE
      BxMGRGVudmVyMRAwDgYDVQQKEwdNYXZlbmlyMRAwDgYDVQQLEwdNV1BDSUNEMRsw
      GQYDVQQDExIgbXdwLW1hdmVuaXIubG9jYWwwggEiMA0GCSqGSIb3DQEBAQUAA4IB
      DwAwggEKAoIBAQDP//nPJhj56lGxVEABcxeVN74woGLAcpTZ33R/00IEoUL3272g
      rojd0i4Oo75iC0hiNMLq2oiqeyATtsvaf34xHRP993It8Yb74LU5cYy97EziiNuZ
      Cm5JYM8ecgWEksjr3ZBZSLKbEzr5+TsXuzS2fNUZHh6W9JeqRxRCMy4pCBUI7mMx
      tn9y1cHNPD2SB0R2m8HKo7AiIGyPwakYXymk/NLgVwHiEG/HNoJC5nWrmG/iFIs/
      bwGGS8k/SkNga9gm5clcbeEsjI1om2j2rK2RWWMXNWsESNRRl+xh7QCGVN2L/8X6
      a/Svj/bo8kJFOiHqZhZYW20iJDfSYij12mhZAgMBAAGjQjBAMA4GA1UdDwEB/wQE
      AwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBSp3+o7SKm8DIoOxkqr6sHK
      plxcITANBgkqhkiG9w0BAQsFAAOCAQEAQ/zP+8njYhBkHQsfMIvzJljSFGx40TVK
      z+swVEj+EhiL0hL03FD6W3FPfJP+wGzC+vQ5GMa0MgyoB8AzOe7qhNVRAza6DlJz
      6Y++AugonG7J/6ofcmqLRZ3JQCzAiRnVv1lwdP6zAysO+sx5FmyHw6VYSEgx48v9
      VwGrSoawHtYOvm22CwSRl7s5kOcrJGjhHJmXoTOaHeFohiCy9PdP35mI38of9Txn
      ljVx/iboLjajLO6hQPXLrWoA+pZv0iAD9U9NRgiRvYIOEu+/AWpQpMtdxzUyqX/m
      4H/UOmIdt50J/gtY9K8fMVMrujR/cRN66/U5tnbJHEOdjBBV1gJC7g==
      -----END CERTIFICATE-----

reference: https://argo-cd.readthedocs.io/en/stable/operator-manual/user-management/#configuring-a-custom-root-ca-certificate-for-communicating-with-the-oidc-provider

commit: https://github.com/argoproj/argo-cd/pull/6712/commits