Allow to inject custom certificate authorities

[ereslibre]

Some users might want to pull policies from https/OCI registries that are secured using self-signed certificates.

This can be done by starting the Policy Server process with a specially crafted sources.yml file.
The format of the file is described here.

To address this issue, we will have to produce a sources.yml file that looks like that:

source_authorities:
  "internal-registry.dev.my-company.com": /local/path/to/internal-registry-dev-mycompany-com.pki
  "other-internal-registry.dev.my-company.com:5001": /local/path/to/other-internal-registry-dev-mycompany-com.pki

Design

We will extend the Policy Server CRD to have a new attribute named sourceAuthorities. This attribute is a map with:

• key (string): the name of the registrie (e.g.: "internal-registry.dev.my-company.com")
• value (string): the PEM encoded certificate authority that has to be used to verify the certificate used by the endpoint

The user can change at any time the value of the sourceAuthorities attribute, this will lead to a rollout of the PolicyServer Deployment.

Behind the scenes, the controller will update a ConfigMap that has a key named sources.yml, with a string as value. The string will hold the contents of the sources.yml file.
The ConfigMap will also have one entry per certificate specified with:

• key (string): the name of the file that has to be generated
• value (string): the actual certificate

This ConfigMap is then mounted into the Policy Server Pods. The Policy Server Pod Template is updated to make use of this sources.yml file.

We can probably extend the already existing ConfigMap that each Policy Server already uses to store the contents of the policies.yml file.

Given the following sourceAuthorities:

source_authorities:
  "internal-registry.dev.my-company.com": <pem cert1>
  "other-internal-registry.dev.my-company.com:5001": <pem cert2>

The ConfigMap created by the controller will be the following one:

sources.yaml: |
  source_authorities:
    "internal-registry.dev.my-company.com": internal-registry.dev.my-company.com.pem
    "other-internal-registry.dev.my-company.com:5001": other-internal-registry.dev.my-company.com:5001.pem
internal-registry.dev.my-company.com.pem: <pem cert1>
other-internal-registry.dev.my-company.com:5001.pem: <pem cert2>

Acceptance criteria

• The PolicyServer Custom Resource is extended to have an attribute named sourceAuthorities
• By default the map has no entries
• User can add custom CAs by populating the sourceAuthorities attribute
• Custom CAs can be added/remove at any time. Any change done to this attribute leads to the rollout of the PolicyServer Deployment

[flavio]

Blocked, waiting for the new architecture to be in place

[viccuad]

Configmap keys can only contain alphanumeric characters, -, _ or . ("unix fully portable filenames") so I will go with:

sources.yaml: |
  source_authorities:
    "internal-registry.dev.my-company.com": /sources/<sha256 of URI>
    "other-internal-registry.dev.my-company.com:5001": /sources/<sha256 of URI>
<sha256 of URI>: <pem cert1>
<sha256 of URI>: <pem cert2>

And take care to mount the cert files as follows, or similar:

/sources/sources.yaml
/sources/<sha256 of URI1>
/sources/<sha256 of URI2>

[flavio]

That sounds good to me, thanks for having noticed that!

[viccuad]

@kubewarden/kubewarden-developers the more I go with it, the less I like it. I would like to change policy-server as explained in option 3 below.

I have an implementation: see viccuad@680433b.

That implementation currently leaks entries in sha512 in the configmap. See the FIXME
viccuad@680433b#diff-27a2d1900a29003599483bb04572874f55b0198334d916d5c26e566bc63d8cf1R92-R111

On an update to the PolicyServer confimap, we have:

data:
  b0e018a10878ee63f1dad41be8a972d3c1b23a0109b7ef671589acda0abed0d5: pem cert 1
  fe7faa899a162befbe70285b92a2ee5a57dac329720f2adfde184b4e267ac2bc: pem cert 2
  policies.yml: '{}'
  sources.yml: '{"source_authorities":{"host.k3d.internal:5000":"/sources/b0e018a10878ee63f1dad41be8a972d3c1b23a0109b7ef671589acda0abed0d5","host.k3d.internal:999":"/sources/fe7faa899a162befbe70285b92a2ee5a57dac329720f2adfde184b4e267ac2bc"}}'
kind: ConfigMap 

Those sha512 entries cannot be easily removed on reconciling an update. Options that I see:

1. Prepend sha512 with cert-, iterate through Data map, delete()
   keys that start with cert-. Eek.
2. Instead of doing Client.Patch(cfg), do Client.Delete(cfg) &&
   Client.Create(cfg). Also eek.
3. Change behaviour of policy-server binary, from expecting paths to certs
   in sources.yaml, to accept certs there. With that, we don't need the
   sha512 entries in Data, nor mounting them as files.

[viccuad]

We went with option 3, implemented in:
kubewarden/policy-fetcher#17

turns out that rust's serde has a limitation, and either we remove the untagged and deserialize by implementing ourselves the Deserialize() for Rawsources, or we use an internally tagged struct for easy deserialization.

Went with the second option, see:
kubewarden/policy-fetcher#22

This changes the sources.yml format to:

---
 insecure_sources:
  - "registry.dev.my-corp.com"
  - "registry-2.dev.my-corp.com:5001"
source_authorities:
  "registry.pre.my-corp.com":
    - type: PathBased
      path: "/path/to/ca.pem"
  "registry-2.pre.my-corp.com:5001":
    - type: PathBased
      path: "/path/to/ca.pem"
    - type: PathBased
      path: "/path/to/ca.der"
    - type: DataBased
      data: |
            our PEM encoded cert

TODO:

• fix: Deserialize RawSources correctly policy-fetcher#22, tag new version
• policy-server: consume policy-fetcher, update sources.yml.example, tag new version, feat: Bump policy-fetcher v0.1.17, new sources.yml format policy-server#116
• update policy-server docs, wiki: Update documentation about the sources.yml file format docs#62
• controller PR for building sources.yml correctly: feat: Add PolicyServer.spec.sourceAuthorities to CR #114
• Bump controller, make release, bump chart: Bump charts: controller 0.3.1, crds 0.1.1 helm-charts#43

[ereslibre]

Note: data cannot be DER, given DER is a binary format. I would explicitly mention that if the user wants to use DER, they need to use PathBased.

[viccuad]

Doh, that's a good point. Edited the previous comment to reflect it.

[viccuad]

Missing approvals for kubewarden/helm-charts#43, which unblocks the docs. With that, the card could be closed.

[viccuad]

Closing, all PRs merged, docs live, new releases live, sanity tests performed on the kubewarden chart repo.