mtls rust stack

mTLS for the Rust noetl stack

Mutual TLS on the control-plane API between noetl-server-rust and noetl-worker-rust — the transport that authenticates + encrypts the worker→server credential channel (GET /api/credentials/<alias>) so a resolved secret no longer travels plaintext on the wire.

Part of the Secrets Wallet umbrella (noetl/ai-meta#61), Phase 4:

Phase	Where	What
4a	noetl/server#103 (v2.30.0)	Server opt-in TLS/mTLS listener — NOETL_TLS_CERT / NOETL_TLS_KEY / NOETL_TLS_CLIENT_CA.
4b	noetl/worker#56 (v5.12.0)	Worker mTLS client — NOETL_TLS_CLIENT_CERT / NOETL_TLS_CLIENT_KEY / NOETL_TLS_CA.
4c	noetl/ops#163	cert-manager issues the certs in-cluster; manifests wire the deployments.

The certs are minted in-cluster by cert-manager — no manual openssl / kubectl create secret, nothing secret in git.

Manifests

ci/manifests/noetl/tls/ in noetl/ops:

File	What
certificates.yaml	self-signed Issuer → CA Certificate → CA Issuer → noetl-server-tls (serverAuth, SAN = service DNS + localhost) + noetl-worker-tls (clientAuth). cert-manager materializes both Secrets (tls.crt / tls.key / ca.crt).
server-rust-mtls-patch.yaml	mounts noetl-server-tls, sets the server NOETL_TLS_* env + https public URL, swaps probes to tcpSocket.
worker-rust-mtls-patch.yaml	mounts noetl-worker-tls, sets the worker NOETL_TLS_CLIENT_* env + https NOETL_SERVER_URL, rewrites wait-for-api to curl the mTLS endpoint with the client cert.

Enable (kind)

# cert-manager (once)
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml
kubectl -n cert-manager rollout status deploy/cert-manager-webhook --timeout=180s

# issue the certs
kubectl apply -f ci/manifests/noetl/tls/certificates.yaml
kubectl -n noetl wait --for=condition=Ready certificate/noetl-server-tls certificate/noetl-worker-tls --timeout=120s

# flip the rust deployments to mTLS
kubectl -n noetl patch deploy noetl-server-rust --type strategic --patch-file ci/manifests/noetl/tls/server-rust-mtls-patch.yaml
kubectl -n noetl patch deploy noetl-worker-rust --type strategic --patch-file ci/manifests/noetl/tls/worker-rust-mtls-patch.yaml

Verify: server logs Server listening (TLS) tls=true mtls=true; worker logs TLS enabled mtls=true ca=true → Worker registered. The full runbook (verify + revert) is in the manifest directory's README.md.

The two probe/health caveats

A client-cert-requiring listener rejects the certless handshake that Kubernetes liveness/readiness probes and curl-based init containers do by default. Two consequences this setup handles:

1. Server probes → tcpSocket. An httpGet/HTTPS probe can't present a client cert, so mTLS fails it → the pod never goes Ready. A TCP port-open probe is the pragmatic fix; a separate non-mTLS health port is the production-grade alternative.
2. Worker wait-for-api init → mTLS curl. The init container's plain-HTTP curl can't complete against an mTLS server → the pod hangs in Init. It's rewritten to curl https://…/api/health with the mounted client cert.

Scope + GKE

Opt-in today (patches on top of the base rust deployments). Folding mTLS into the Helm chart (automation/helm/noetl) as a values-gated default — and using cert-manager's GCP/Kubernetes issuers (or SPIFFE/SPIRE) instead of this self-signed root for prod — is the follow-up.

Related

• GKE Helm install
• System worker pool
• Server deployment-specification (§ Transport security)
• Worker deployment-specification (§ Transport security)