Not supplying static root CA "effectively making istio-csr TOFU"

[joshmue]

First, thanks for this project! It glues together istio and cert-manager really well.

I am curious about the recent change in the README.md: It now highly recommends to explicitly supply a static root CA (PR #101):

It is highly recommended that the root CA certificates are statically defined in istio-csr. If they are not, istio-csr will "discover" the root CA certificates when requesting its serving certificate, effectively making istio-csr TOFU.

I am not sure though, how the loading of the root CA cert from the serving certificate request's CertificateRequest.status would make istio-csr TOFU.

• When supplying it statically, istio-csr would trust K8s to supply the correct root CA file.
• When not supplying it statically, it would rely on the configured K8s RBAC to prevent anything other than the cert-manager or other trusted components to make changes to CertificateRequest resources. At least in my understanding, this does not mean TOFU, as trust was already established via K8s RBAC mechanisms and service account authentication.

So, as far as I understand, both methods should be reasonably secure, granted RBAC is configured well.

Am I missing something here? Is the goal to protect against a compromised cert-manager?

[JoshVanL]

Hi @joshmue, thanks for the kind words! 🙂

I think everything you said here is correct, and perhaps the wording ("effectively") could be changed here to be less alarming..

Indeed, the idea here is we are trying to protect against a malicious actor who, by some means, is able to highjack signing (update permissions on CertificateRequests) for the istio-csr signing service and workload's issuer. This could be a stolen token from an existing signer (cert-manager-controller is one), or by other means. With this ability, the actor could completely change or inject an additional CA which would be a vector for attack.

By statically defining the root CAs, this attack doesn't work. An attacker would only be able to stop workloads trusting istio-csr's service or prevent workloads from joining the mesh- a safe failure state. Of course one could argue that this is just moving the problem, however I think that there are some good volume options out there which have arguably better security profiles (the secrets-store csi driver backed by Vault or a cloud provider paired with workload identity comes to mind).

Whilst having the behavior of changing root CAs over time on the fly isn't necessarily bad, root CAs should be fairly static things and their delivery pipeline tightly controlled.

[joshmue]

Hi @JoshVanL, thank you very much for the rapid and detailed response! This really clears this topic up for me.

So it is more like a tradeoff between operational simplicity via automation vs. implementing CA pinning to reduce the security impact of e. g. cert-manager tokens being leaked.

When reading "TOFU" and its Wikipedia page, I got the impression that istio-csr would be vulnerable to simple MITM attacks somewhere - at least in the setup phase. Kind of like when using a SSH client with a lax configuration regarding host key checking. Or trusting selfsigned certificates in web browsers. Glad to see that this is not the case. Maybe this could also be clarified in the README like you kindly did in your comment.

Of course one could argue that this is just moving the problem, however I think that there are some good volume options out there which have arguably better security profiles (the secrets-store csi driver backed by Vault or a cloud provider paired with workload identity comes to mind).

Do you mean fetching the root CA certificate via one of these mechanisms?

[JoshVanL]

So it is more like a tradeoff between operational simplicity via automation vs. implementing CA pinning to reduce the security impact of e. g. cert-manager tokens being leaked.

Yes, I think that is a fair assessment.

Maybe this could also be clarified in the README like you kindly did in your comment.

I agree, I'll open a PR to change this wording and clarify.

Do you mean fetching the root CA certificate via one of these mechanisms?

Yes, exactly, One could store their org CAs in Google Secret Manager (other cloud solutions are available 😜), and use kube secret store to mount the CA data into istio-csr.

[joshmue]

Yes, exactly, One could store their org CAs in Google Secret Manager (other cloud solutions are available stuck_out_tongue_winking_eye), and use kube secret store to mount the CA data into istio-csr.

👍

As everything is clarified and the PR is merged, I'll close this issue. Thank you again, @JoshVanL!