Skip to content

deployment

Jump to bottom
Kadyapam edited this page May 29, 2026 · 2 revisions

Deployment

How noetl-gateway is built, packaged, and deployed.

The production target is GKE behind a Cloudflare Tunnel at https://gateway.mestumre.dev. The gateway is one component of the NoETL Helm release; the full GKE install lives at ops wiki / GKE Helm install.

Build

Cargo

cargo build --release

The release binary lands at target/release/noetl-gateway.

Container image

A multi-stage Dockerfile is at the repo root. Build stage runs cargo build --release against alpine+rust; runtime stage copies the resulting binary onto an alpine base with libgcc + libxslt + ca-certificates + openssl. Image size is around 100 MB.

docker build -t noetl-gateway:<tag> .
docker push <registry>/noetl-gateway:<tag>

The image is the primary distribution channel for the gateway. GKE deployments pin to a tag; rolling forward is a tag bump.

History note: pre-v2.12.0 the runtime image also bundled a Python virtualenv (/opt/firestore-sidecar) for the Firestore listener referenced under Subscriptions. The Firestore subsystem was removed in v2.12.0 (noetl/gateway#18

  • #19); the resulting image is ~30 MB lighter and build time dropped from ~17 min to ~5 min.

Crate

cargo publish

Releases the noetl-gateway crate to crates.io. Useful for downstream consumers embedding the gateway or running it under their own supervision.

Kubernetes manifests

Reference manifests live in k8s/ in the gateway repo. Two concerns:

  1. Deployment — the gateway container. Single container per pod (no sidecar since v2.12.0).
  2. Service — typically ClusterIP; an Ingress or Cloudflare Tunnel forwards external traffic.

These manifests are reference snapshots; the actual production deployment for noetl/ops-managed clusters is templated by the Helm chart at noetl/ops/automation/helm/gateway/. The gateway's k8s/ manifests document the contract; the Helm chart implements it for the GKE profile.

Health checks

The gateway exposes /health for liveness/readiness probes. Default GKE probe config:

livenessProbe:
  httpGet:
    path: /health
    port: 8090
  initialDelaySeconds: 30
  periodSeconds: 10
readinessProbe:
  httpGet:
    path: /health
    port: 8090
  initialDelaySeconds: 5
  periodSeconds: 5

The /health endpoint returns 200 when the gateway has:

  • a valid Postgres connection,
  • a valid NATS connection.

Cloudflare Tunnel

External traffic for gateway.mestumre.dev enters via a Cloudflare Tunnel pod in the cluster's cloudflare namespace, which forwards to the in-cluster gateway service. The tunnel configuration sits in noetl/ops alongside the Helm chart; the gateway itself does not need TLS — TLS terminates at Cloudflare.

If you bring up your own gateway without Cloudflare, you can front the gateway with a Kubernetes Ingress + cert-manager (the same cert-manager the Helm chart already provisions).

Rolling forward

# 1. Bump version in Cargo.toml
# 2. Build + push the container
docker build -t <registry>/noetl-gateway:<new-tag> .
docker push <registry>/noetl-gateway:<new-tag>

# 3. Helm upgrade (from the ops repo)
helm upgrade --install noetl ./automation/helm/noetl \
  --namespace noetl \
  --reuse-values \
  --set gateway.image.tag=<new-tag>

# 4. Verify
kubectl rollout status deployment/gateway -n gateway
kubectl exec -n gateway deploy/gateway -- curl -s http://localhost:8090/health

The Helm chart handles the rolling update. Verify SSE clients reconnect cleanly after the new pod takes traffic.

Rolling back

helm history noetl -n noetl
helm rollback noetl <previous-revision> -n noetl

Atomic; restores the previous gateway image tag along with the rest of the chart.

Local development

cp env.example .env
# edit .env for your local NoETL + NATS + Postgres
cargo run

For a one-shot local NoETL stack, follow the kind install path in the ops repo.

Tests

cargo test

Runs the unit and integration tests.

Related

Gateway

Surfaces

Operations

See also

Clone this wiki locally