Add support for Hashicorp Vault mTLS

[ambis]

Describe the solution you'd like
My Vault needs a client certificate & key to be presented when communicating with it (vault cli envs VAULT_CLIENT_CERT and VAULT_CLIENT_KEY, vault server config tls_require_and_verify_client_cert and tls_client_ca_file).

Looking at the API reference documentation, it seems that the ESO does not provide a way to supply these client certificates.

What is the added value?
I keep my publicly visible Vault extra protected via mTLS.

Give us examples of the outcome

SecretRefs for client cert+key, just like for so many other secretRefs in the config already.

Thanks.

[moolen]

Vault has cert auth:

https://external-secrets.io/v0.5.3/spec/#external-secrets.io/v1beta1.VaultCertAuth

Does that work for you?

I realized this is missing on the provider docs

[ambis]

@moolen Thanks, but that is a completely different thing to mTLS.

mTLS requires you to have a key+cert present for ALL requests to Vault. They do not provide any identity (altough they can, but Vault does not support that), only the possibility to even talk to the Vault.

mTLS is not Vault spesific technology, it is basic mutual-TLS feature supported by pretty much any server (nginx, haproxy etc).

You can read more about it here: https://www.cloudflare.com/learning/access-management/what-is-mutual-tls/

[gusfcarvalho]

Hi @ambis ! We are aware of what mTLS means, but thank you for the link anyways 😅 . CertAuth currently is a method that allows mTLS communication with Vault, but it considers the client to be authenticated through the mTLS itself. What is your current way to authenticate to vault right now? Are you only using the client certificate as a way to authenticate / authorize ESO?

@moolen even the description of certAuth is wrong within the specs... 😓

[ambis]

For authentication I use OIDC (personal login), AppRole, JWT (gitlab $CI_JOB_JWT) and also the Kubernetes backend for Kubernetes auth.

Altough I fail to see the relevance of my auth methods with the simple fact, that I cannot even query vault status without providing mTLS cert+key.

I run Vault behind Cloudflare, which provides the mTLS cert+key and their CA cert. It is the first place they are verified, but they are also passed to my Vault backend where Vault will revalidate them.

This is my Vault listener config:

listener "tcp" {
  address = "0.0.0.0:8443"
# ...
  tls_client_ca_file = "xx/cf_ca.crt"
  tls_require_and_verify_client_cert = true
# ...
}

Which means, that no connection that does not provide cert+key signed by the tls_client_ca_file will even end up in the Vault's API.

Maybe you are familiar with the Vault's helm chart and it's csi, which I have configured like so:

csi:
  image:
    tag: "1.1.0"
  enabled: true
  extraArgs:
    - -vault-addr=https://my.vault.address.example.com
    - -vault-mount=path-to-my-kube-mount
    **- -vault-tls-client-key=/vault/tls/client.key**
    **- -vault-tls-client-cert=/vault/tls/client.crt**
  volumes:
    - name: tls
      secret:
        **secretName: vault-tls-client**
  volumeMounts:
    - name: tls
      mountPath: "/vault/tls"
      readOnly: true

Here's another example:

No client cert+key provided for vault, the error code, 1020 is Cloudflare's error code to tell that the client certificate is missing.

~ $ VAULT_ADDR=https://my.vault.address.example.com vault status
Error checking seal status: Error making API request.

URL: GET https://my.vault.address.example.com/v1/sys/seal-status
Code: 403. Raw Message:

error code: 1020
exit 1

Here I provide the client cert+key:

~ $ VAULT_CLIENT_CERT=/some/path/tls.crt VAULT_CLIENT_KEY=/some/path/tls.key VAULT_ADDR=https://my.vault.address.example.com vault status
Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    1
Threshold                1
Version                  1.10.3
Storage Type             raft
Cluster Name             vault
Cluster ID               4ef3f670-873c-9f88-xx
HA Enabled               true
HA Cluster               https://10.0.0.10:8201
HA Mode                  active
Active Since             2022-05-15T10:36:49.04326185Z
Raft Committed Index     13856
Raft Applied Index       13856

Would this be enough information to make it clear that we are not talking about Vault's mTLS auth method?

[gusfcarvalho]

The relevance was that it is possible to assign permissions directly to client certs, hence the certAuth method :). Unfortunately for other methods, I don’t believe we can set a client cert at that time for Kubernetes, Approle, or JWT/OIDC.

[gusfcarvalho]

Of course, as always, PRs are welcome :)

[ambis]

@gusfcarvalho Can you please read this carefully: I am not talking about auth methods right now. I am talking about the actual TLS connection to the Vault itself.

In my example above, the Vault CSI plugin provides a way to add client cert+key to any request that goes to a Vault. And it is NOT about any particular Vault auth method.

This issue is to ask if ESO would be kind enough to add mTLS support for communicating with a Vault protected by mTLS in general. It would have absolutely nothing to do with Vault's auth methods.

[gusfcarvalho]

Yes, I read it and fully understood it.

Now, if you can read my messages carefully, we do not support it as of right now. I changed the tags to clarify this is actually a new feature request.

I’m not sure when we (maintainers) can look at it, though, as our main focus are with some core features (you can check our public board if you like). Hence, you are welcome to contribute the feature yourself, and we will review it.

[ambis]

Ok, I misunderstood what you were saying, my bad!

[gusfcarvalho]

No worries 😁. It is something that we generally are trying to make sure every provider has..

honestly I’m not sure how Hashivault doesn’t have it already 😅

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity.

[rodrigorfk]

Hi @gusfcarvalho , I am working on implementing support for Vault mTLS as discussed in this issue and will be submitting a PR shortly, do you mind reopening this issue? or would you prefer that I fill a new one? thanks in advance.