Local spath-as-a-service in a cluster

[kvaps]

Hi @WanzenBug,

We're faced with the problem of using kernel-module-injector in closed enviroments (enviroments without public internet connectivity)
So we decided to run spatch-as-a-service locally in same namespace for our platform.

I prepared patch for DRBD LINBIT/drbd#41 (I didn't sent it to drbd-dev mailing list yet)

The patch allows to set SPAAS_URL enviroment variable to communicate to while building drbd kernel module.
Currently we specify this URL as constant in our Dockerfile, but it does not allow to mount actual CA to the pod and specify CA file.

I'm thinking about modify operator to provide extraVolumeMounts functionality, eg:

operator:
  satelliteSet:
    additionalEnv:
    - name: SPAAS_URL
       value: "--cacert /etc/spaas/certs/ca.crt https://spaas:2020"
    kernelModuleInjectionExtraVolumeMounts:
    - mountPath: /etc/spaas/certs/ca.crt
      name: spaas-certs
      readOnly: true
      subPath: ca.crt
    extraVolumes:
    - name: spaas-certs
      secret:
        secretName: spaas-certs

or adding two new variables like:

operator:
  satelliteSet:
    spaasURL: "--cacert /etc/spaas/certs/ca.tls https://spaas:2020"
    spaasSecret: "spaas-certs"

which option do you prefer or maybe do you have better idea?

[WanzenBug]

Since we already have the sidecars: thing, would it be an option to also have a extraInitContainers: and just use that to define whatever custom loading is needed. The injection mode can then just by DepsOnly, and the extra init container can do the actual DRBD building.

Also, as a heads-up, I've started prototyping what an Operator 2.0 could look like, and all these kinds of "I need to slightly modify the pods" use-cases are top priority to figure out.

[kvaps]

option to also have a extraInitContainers

Nice idea, but I worry that having another []corev1.Container in spec will not fit CRD into etcd.
Last time I was trying and faced with some internal limitations 🙂

I'll try also another option to use Kubernetes Certificate Signing Requests feature, and hardcode these variables in my dockerfile:

ENV SPAAS_URL="https://spaas:2020"
ENV SSL_CERT_FILE="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"

I hope this will work

Also, as a heads-up, I've started prototyping what an Operator 2.0 could look like, and all these kinds of "I need to slightly modify the pods" use-cases are top priority to figure out.

Glad to hear that, I'm very looking forward 👍

[kvaps]

No way, there is no opportunity to issue valid serving certificate

invalid usage for client certificate: server auth

[kvaps]

I found that there is additionalEnv option, however setting SSL_CERT_FILE globally is not a good idea, so I use workaround to pass curl arguments in SPAAS_URL variable.