Vault Injector TLS Handshake error: bad certificate

[gunnypatel]

I have a preexisting vault cluster that I'd like to hook into using the vault injector webhook. I used helm to generate the manifests from vault-helm chart and applied to to the cluster. The webhook is getting called but it's running into the following error: 2020/03/03 23:41:20 http: TLS handshake error from 172.17.1.43:46482: remote error: tls: bad certificate. I'm not sure which certificate is bad. Is the injector having issues validating the certificate of the apiserver or is the apiserver not trusting the webhook certificate? I didn't configure any certificates in the vault-helm chart. It sounds like if no certificate is specified then the injector will generate a certificate using the service account.

 [March 3, 2020 at 6:37:04 PM GMT-5]
Listening on ":8080"...
[March 3, 2020 at 6:37:04 PM GMT-5]
2020-03-03T23:37:04.249Z [INFO] handler: Starting handler..
[March 3, 2020 at 6:37:04 PM GMT-5]
Updated certificate bundle received. Updating certs...
[March 3, 2020 at 6:38:59 PM GMT-5]
2020/03/03 23:38:59 http: TLS handshake error from 172.17.1.43:43028: remote error: tls: bad certificate
[March 3, 2020 at 6:41:20 PM GMT-5]
2020/03/03 23:41:20 http: TLS handshake error from 172.17.1.43:46482: remote error: tls: bad certificate 

[gunnypatel]

I was able to figure out what the issue was. I finally figured out how the auto generation process worked for the TLS certificate. That led me to re-evaluate my manifest files to make sure everything was good. I found a missing letter in the last host in the AGENT_INJECT_TLS_AUTO_HOSTS environment variable. I guess that just happened to be the host that the api-server was using to call the webhook.

[whume]

What dd you miss in your configs I have a similar setup and am seeing the same log?

I have vault-agent-injector-svc,vault-agent-injector-svc.$(NAMESPACE),vault-agent-injector-svc.$(NAMESPACE).svc in mine

[gunnypatel]

When I generated the manifest I used the wrong namespace so I manually changed it to "vault" afterwards. In the last host I had missed the "t" in "vault".

My namespace is "vault" and the value I have is vault-agent-injector-svc,vault-agent-injector-svc.vault,vault-agent-injector-svc.vault.svc so your looks fine (assuming $(NAMESPACE) is rendered to the correct value). You also need to make sure your service is named vault-agent-injector-svc. If you're seeing the Updated certificate bundle received. Updating certs... message in your log then I think the agent injector is successfully generating a CA cert and updating the MutatingWebhookConfiguration in your cluster.

[JnMik]

Same issue here.
My vault cluster is in kubernetes using the the most recent helm chart.
I generated the certificates with cert-manager, the vault cluster seems to boot up properly (well I don't see any flagrant error in the logs), but the agent-injector doesn't like the certificate.

2020-06-28T17:50:02.572Z [INFO]  handler: Starting handler..
Listening on ":8080"...
Updated certificate bundle received. Updating certs...
2020/06/28 17:50:11 http: TLS handshake error from 10.8.2.253:46456: remote error: tls: bad certificate
2020/06/28 17:50:11 http: TLS handshake error from 10.8.2.253:46458: remote error: tls: bad certificate
2020/06/28 17:50:16 http: TLS handshake error from 10.8.2.253:46470: remote error: tls: bad certificate
2020/06/28 17:50:16 http: TLS handshake error from 10.8.6.44:46692: remote error: tls: bad certificate
2020/06/28 17:50:16 http: TLS handshake error from 10.8.2.253:46472: remote error: tls: bad certificate
2020/06/28 17:50:16 http: TLS handshake error from 10.8.2.253:46476: remote error: tls: bad certificate
2020/06/28 17:50:17 http: TLS handshake error from 10.8.2.253:46480: remote error: tls: bad certificate