POC - Load certificates by mounting into the containers instead of into the Docker images

[applejag]

This is an old ticket moved from our internal ticketing system. The idea was to use OPA but if we could find a different way to solve this without introducting yet another dependency then that would be swell.

This deserves an RFC once POC is working

---

Currently the certificate added by the Wharf build into the images when building with kaniko is loaded via the following lines:

We want to add the certificates into the containers inside kubernetes instead, leaving the images intact of any self-signed CAs.

Suggestion is to use OPA (Open Policy Agent) in our kubernetes cluster to dynamically add in the mounting of our CA certs, for example via a configmap.

Steps:

• RnD how to use OPA in k8s (tip: https://www.openpolicyagent.org/docs/v0.12.2/kubernetes-admission-control)
• The rules we want to impose are something like:
  1. If not containing some flag, for example an attribute or label on the pod or on the container with name & value wharf-inject-certs: false (to be able to turn it off)
  2. Add configmap containing certs as volume to pod
  3. Add mounting of volume to container into designated paths
• Alternatively we want rules like this:
  1. (same as above) If not containing some flag, for example an attribute or label on the pod or on the container with name & value wharf.iver.com/inject-certs: false (to be able to turn it off)
  2. (same as above) Add configmap containing certs as volume to pod
  3. Add init-container to each container that prepares the /etc/ssl/certs folder in a temp volume using the configmap volume of our certs.
  4. Mount the temp volume into /etc/ssl/certs on all containers.

After some research (mostly by looking in the update-ca-certificates script), certs needs to go into

# certs are imported from here
/usr/local/share/ca-certificates/**/*.pem

# certs are stored here
/usr/share/ca-certificates/**/*.pem

# conf file tracking all added and ignored certs
/etc/ca-certificates.conf

# all certs in one file
/etc/ssl/certs/ca-certificates.crt

# all certs one by one, name hashed via `openssl rehash .`
/etc/ssl/certs/{HASH-OF-CERT}

That's why it's probably best to do it with an init-container, or maybe an operator. Needs further RnD!

This issue is requesting a tracerbullet. Just something that works good enough for regular GNU/Linux containers. Disregard windows containers for now.