Create trust bundle based on Debian bookworm

[SgtCoDFish]

We currently provide cert-manager-package-debian which contains the trust store from Debian bullseye. That trust store hasn't been updated for a while - since it hasn't needed to be - but it won't be updated to add new CA certificates or to prune CA certificates which have expired.

Since there's a new stable version of Debian - Debian bookworm - we could provide a new trust bundle which uses the ca-certificates package from that new version. That would be more "up to date" for users who want that.

We can create a new bundle for bookworm and update to use it by default in trust-manager. This would involve copying the "debian" bundle here into a new folder ("debian-bookworm") and then updating the new bundle to use bookworm rather than bullseye.

There's some GCB CI/CD for our trust bundles, which is manually configured in the cert-manager GCP org, but I can set that up. The GCB file here would need to be copied and the new version would need to be updated to bookworm.

ℹ️ This would be a great issue for an open source community member to tackle!

[arsenalzp]

Hello,
I can pick it up, but I would like to know - is there any terms for this issue?

[SgtCoDFish]

I can pick it up, but I would like to know - is there any terms for this issue?

Sorry, I didn't see this until now!

Could you explain what you mean by "terms" there?

[arsenalzp]

I can pick it up, but I would like to know - is there any terms for this issue?

Sorry, I didn't see this until now!

Could you explain what you mean by "terms" there?

I meant "deadlines".

[SgtCoDFish]

Ah, no deadlines! We can be flexible, no rush 😁

[arsenalzp]

Could you please assign it to me?

[erikgb]

/assign @arsenalzp

I think you should be able to self-assign:

Ref. https://github.com/jpmcb/prow-github-actions/blob/main/docs/commands.md

[arsenalzp]

Hello,
I'm back from vacation and ready to work.
Could you be so kind to explain me why we need to copy old trust package stuff (in debian folder) instead just change base image directive to use the latest Debian docker package?

[SgtCoDFish]

Could you be so kind to explain me why we need to copy old trust package stuff (in debian folder) instead just change base image directive to use the latest Debian docker package?

Sure, I'll try to explain my thinking!

Basically, the list of CAs trusted by each version of Debian will vary. Bookworm trusts new certs that bullseye doesn't.

I'd prefer for people to be able to make the choice of what they trust by themselves. It's OK for us to change the default in trust-manager (and there has to be a default), but I'd like to give people who already use the bullseye images to continue using those.

Plus if people are going to keep on using bullseye then we need a build pipeline for updating it in the future!

Does that make sense?

[arsenalzp]

Yes, sure it is clear for me.
But the issue is that Debian trusted bundle sets up during the building/deployment process by using Init containers features.
Any suggestion/wishes how to allow end users to define bundle version?
P.S.
We should change help charts as well, there is 'hard-coded' version

# -- Tag for the default package image
  tag: "20210119.0"

[erikgb]

Basically, the list of CAs trusted by each version of Debian will vary. Bookworm trusts new certs that bullseye doesn't.

@SgtCoDFish could you please explain why any user would prefer to trust an outdated trust bundle?

[arsenalzp]

I guess it is for some reason such as the certificate chain of trust which is based on old CA. Some applications/certificates were not updated to use new CA, something like that.

[erikgb]

I guess it is for some reason such as the certificate chain of trust which is based on old CA. Some applications/certificates were not updated to use new CA, something like that.

But the Debian trust roots are based on publicly trusted roots? And I don't think any root certs will be removed before they expire or are revoked. As a user, I would prefer to use an updated public trust bundle.

[arsenalzp]

I guess it is for some reason such as the certificate chain of trust which is based on old CA. Some applications/certificates were not updated to use new CA, something like that.

But the Debian trust roots are based on publicly trusted roots? And I don't think any root certs will be removed before they expire or are revoked. As a user, I would prefer to use an updated public trust bundle.

You are right but some CAs might be revoked so it would lead to issue with the new CA bundle. I guess that's why @SgtCoDFish created this issue.
Of course we should indulge users to use outdated CA bundles, however we shouldn't break users applications as well.
Anyway, decision is fully up to you, colleagues! :-)

[SgtCoDFish]

could you please explain why any user would prefer to trust an outdated trust bundle?

I think that they'd probably not use the term "outdated" - Debian bullseye will be supported for years still.

It's kinda as simple as: if you use the new version, you're potentially trusting more stuff. I just like that people can have the choice, you know?

[arsenalzp]

Hello,
What is the final decision regarding this issue?

[erikgb]

I am not sure if there is a "final decision". 😉 But I will try to summarize my view on this, taking the comments from @SgtCoDFish into account.

Should which defaultCAs trust bundle to use be configurable on the Bundle API level? That will require changes to the API as it is now just a boolean true/false. IMO this is a complexity we don't want/need.

IMO the defaultCAs trust bundle should be configurable when provisioning trust-manager. I also think the easiest solution is to just allow the user to choose image for the init-container to select between the bullseye and bookworm trust bundle. And that we should have CI providing updated tags for both images.

[arsenalzp]

The best and sophisticated solution is to allow user to leverage trusted bundle image with helm chart parameters.
However, @SgtCoDFish has his own point of view on this issue :-)

[SgtCoDFish]

I absolutely agree people should be able to configure this in Helm parameters - which they can today, right?

trust-manager/deploy/charts/trust-manager/values.yaml

Line 33 in ad150e5

defaultPackageImage:

I also think the easiest solution is to just allow the user to choose image for the init-container to select between the bullseye and bookworm trust bundle. And that we should have CI providing updated tags for both images.

This is the way! We can update the default to bookworm, but leave the bullseye image for older versions of trust-manager and for anyone who chooses that.

[arsenalzp]

I realised it.
So the task is just create new docker image by using the latest debian base image, push it to the registry, label it as latest, change Helm chart to use the latest image. Am I right?
Sorry for so many questions, thanks all of you for your patience!

[erikgb]

I don't think we should use latest tags, as they are not reproducible.

[arsenalzp]

I don't think we should use latest tags, as they are not reproducible.

I agree, they don't reflect exact version of image, which is Debian release in our case.
In your code I see a number I guess it reflects some date. Can I use something like that to define distinct Debian release for docker image with CA bundle?

[SgtCoDFish]

In your code I see a number I guess it reflects some date

This is the version number of the package - Debian uses dates for ca-certificates.

$ docker run -it --rm docker.io/library/debian:bookworm-slim bash
$ root@91a2868ce75f:/# apt update && apt install ca-certificates
...
$ apt show ca-certificates
...
Version: 20230311
...

We also add a number on the end so that we can release new versions of the same package (e.g. if we rebuild the go binary).

So I guess for bookworm our first version should look like 20230311.0