Skip to content

Operator hardening: non-root runtime, scoped RBAC, dedicated DB credentials #297

Description

@v0l

Operator hardening: non-root runtime, scoped RBAC, dedicated DB credentials

Security audit follow-up. The lnvps_operator is the single highest-value target in the apps system: one pod holding (a) a cluster-wide Kubernetes token, (b) the MySQL root DSN, and (c) the LNVPS_ENCRYPTION_KEY that decrypts every tenant's app_deployment.config. Compromise of this one container reads every tenant's generated DB passwords, TLS private keys, and stored config cluster-wide. Three independent weaknesses stack to make that blast radius total.

1. Operator container runs as root

The runtime base image (debian:trixie-slim in the root Dockerfile) sets no USER, and lnvps_operator/k8s-minimal.yaml's Deployment has no securityContext. The operator process — with all the credentials above — runs as uid 0.

Fix:

  • Add USER (or a numeric non-root user) to the lnvps-operator image stage in the root Dockerfile.
  • Add a pod securityContext to the Deployment manifest:
    securityContext:
      runAsNonRoot: true
      runAsUser: 65534
      fsGroup: 65534
    and per-container:
    securityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities: { drop: ["ALL"] }
  • Verify the operator can still read its config (/etc/config/config.yaml ConfigMap mount) and, when using a key file instead of the env var, /etc/lnvps/encryption.key — mount/perm accordingly.

2. ClusterRole is broader than the operator needs

k8s-minimal.yaml grants, cluster-wide:

  • secrets get/list/watch in all namespaces — only needed because the operator reads the generated secret in each app-* namespace it created. Cluster-wide read of every Secret (including every app-tls private key and every tenant's generated DB passwords) is not required.
  • events: create, patch — the operator never emits events; dead permission.
  • Full create/delete on namespaces, deployments, persistentvolumeclaims, ingresses, networkpolicies — needed, but worth a second pass once secret access is scoped.

Fix:

  • Remove the unused events rule.
  • Scope secret access: prefer per-namespace Roles in each app-* namespace the operator creates (created/bound at reconcile time alongside the NetworkPolicy), so a stolen token reads only tenant namespaces the operator itself provisioned — not kube-system, not cert-manager's, not other deployments' TLS keys it doesn't own. If per-namespace Roles are operationally awkward, document the accepted risk and at minimum drop watch (the operator uses get only in read_generated).
  • Re-audit the remaining verbs against actual usage (grep the operator for Api::<...> calls) and trim.

3. Operator connects to MySQL as root

The example ConfigMap ships db: "mysql://root:password@...". The operator needs DML on the lnvps schema only — not MySQL root.

Fix:

  • Create a dedicated MySQL user (e.g. lnvps_operator) with SELECT/INSERT/UPDATE/DELETE on lnvps.* only.
  • Document it in lnvps_operator/README.md and update the example manifests/configs.
  • Move the DSN out of the plaintext ConfigMap into a Secret (it currently sits in k8s-minimal.yaml's ConfigMap, readable by anyone with configmaps get in lnvps-system).

Acceptance criteria

  • Operator image runs as a non-root user; Deployment enforces runAsNonRoot + readOnlyRootFilesystem + drop: ALL.
  • events RBAC rule removed; secret access no longer cluster-wide (or risk explicitly documented).
  • Example configs/manifests use a dedicated least-privilege MySQL user; DSN moved from ConfigMap to Secret.
  • README.md updated to describe the hardened deployment.
  • Verify app-deployment reconcile still works end-to-end on a hardened pod (config read, generated secret read/write, encryption key file read).

Notes

  • Independent of the planned OVH MKS migration — this hardening applies identically to the current bare-metal cluster and any managed one; do it before or during the move.
  • Related audit findings tracked separately: tenant secrets currently rendered as inline env values (should be secretKeyRef), custom_domain uniqueness check, image pinning.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions