Skip to content

configuration

Kadyapam edited this page May 24, 2026 · 6 revisions

Configuration

Environment variables and runtime configuration for noetl-gateway. The env.example file in the repo is the authoritative reference; this page groups variables by concern and explains each one.

Application

Var Default What
ROUTER_PORT 8090 TCP port the gateway listens on.
APP_BIND (axum default) Bind address. Override to 0.0.0.0 to expose externally; leave unset for default.
APP_WORKERS (tokio default) Worker thread count. Leave unset unless tuning.
LOG_FORMAT text text or json. Use json in production for log aggregation.
LOG_LEVEL debug debug / info / warn / error. Use info or warn in production.

NoETL connection

Var Default What
NOETL_BASE_URL http://localhost:8082 noetl-server HTTP base. The gateway proxies authenticated /noetl/* requests here and dispatches playbook executions via /api/execute.
NOETL_TIMEOUT_SECS 120 Request timeout for outbound calls to noetl-server.
AUTH_PLAYBOOK_TIMEOUT_SECS 60 How long the gateway waits for the auth0_login playbook callback before returning a timeout error. Tuned up from 12 during the 2026-05-24 incident.

NATS

Var Default What
NATS_URL nats://localhost:4222 NATS connection URL for RequestStore (K/V) and the playbook/state subscriber.
NATS_UPDATES_SUBJECT_PREFIX playbooks.executions. Prefix the gateway subscribes to for execution lifecycle events. The full subject pattern is <prefix><execution_id>.*.

The NATS credential, if used, is a platform-runtime credential — same classification as POSTGRES_PASSWORD below. Bind via the gateway pod's k8s Secret, not via the NoETL keychain.

Database (gateway's own state)

The gateway holds session state in its own Postgres database (separate concern from noetl-server's event log). Fields:

Var Default What
POSTGRES_HOST localhost
POSTGRES_PORT 54321
POSTGRES_USER demo
POSTGRES_PASSWORD demo Override in production via k8s Secret.
POSTGRES_DATABASE demo_noetl
DATABASE_URL derived Used by sqlx migrations; set explicitly only when running migrations or running the build with sqlx::query! macros.
SQLGEN_MODEL_OUTPUT_FOLDER ./src/models/ Dev-tooling path; not used at runtime.
SQLGEN_MODEL_FOLDER ./src/models/ Dev-tooling path; not used at runtime.
SQLGEN_MIGRATION_OUTPUT ./migrations Dev-tooling path; not used at runtime.

The gateway database is the gateway's runtime. Per the secrets-and-credentials rule, its credentials live with the gateway pod, not in the NoETL keychain.

Auth0 (for the auth0_login playbook the gateway dispatches)

The gateway itself does not embed Auth0 client secrets — the verification happens inside the noetl-server-dispatched playbook. The gateway only needs the public Auth0 tenant identifiers to verify the JWT issuer and audience claims match. These are passed to it via the auth middleware config; exact env names depend on the auth subcrate version. Check src/auth/ in your deployed version.

SSE

Var Default What
GATEWAY_HEARTBEAT_INTERVAL_SECS 15 How often ping frames are sent over /events.
GATEWAY_CONNECTION_TIMEOUT_SECS 300 Idle timeout for an SSE connection.

Firestore subscriptions (v2.11.0)

Var Required What
GATEWAY_FIRESTORE_CREDENTIALS_PATH yes (for subscriptions) Path to the Firestore service-account JSON inside the pod. Mount via a k8s Secret.
GATEWAY_FIRESTORE_PROJECT_ID yes (for subscriptions) GCP project ID owning the Firestore database the subscription endpoint serves.
GATEWAY_FIRESTORE_LISTENER_CMD optional Command to spawn the Firestore listener sidecar. Defaults to the bundled Python script invocation. Override for testing or for a non-default sidecar.

If GATEWAY_FIRESTORE_CREDENTIALS_PATH is unset, the subscription endpoints return 503 with a clear error rather than crashing on first request. Deployments that don't need SPA subscriptions can leave this unset.

See Subscriptions for the credential provisioning recipe.

Production overrides (typical)

A minimal set of overrides for a GKE deployment:

LOG_FORMAT=json
LOG_LEVEL=info
ROUTER_PORT=8080
NOETL_BASE_URL=http://noetl.noetl.svc.cluster.local:8082
NATS_URL=nats://gateway:<secret>@nats.nats.svc.cluster.local:4222
POSTGRES_HOST=pgbouncer.postgres.svc.cluster.local
POSTGRES_PORT=5432
POSTGRES_DATABASE=gateway
GATEWAY_FIRESTORE_CREDENTIALS_PATH=/var/run/secrets/firestore/service-account.json
GATEWAY_FIRESTORE_PROJECT_ID=<your-gcp-project>
GATEWAY_HEARTBEAT_INTERVAL_SECS=15
AUTH_PLAYBOOK_TIMEOUT_SECS=60

Postgres password, NATS password, and Firestore credentials come from k8s Secrets mounted as env or files, not from the ConfigMap.

What does not go in env

For pushback discipline (per secrets-and-credentials rule):

  • Third-party API tokens that playbooks need (OpenAI, Anthropic, Duffel, Amadeus, etc.) — these live in the NoETL keychain.
  • Tenant database DSNs that playbooks use — keychain.
  • OAuth client secrets used by playbook steps — keychain.

The gateway's env only carries the gateway's own runtime credentials (Postgres, NATS, Firestore service account) and the public Auth0 tenant identifiers.

Related

Clone this wiki locally