configuration

Configuration

Environment variables and runtime configuration for noetl-gateway. The env.example file in the repo is the authoritative reference; this page groups variables by concern and explains each one.

Application

Var	Default	What
ROUTER_PORT	8090	TCP port the gateway listens on.
APP_BIND	(axum default)	Bind address. Override to 0.0.0.0 to expose externally; leave unset for default.
APP_WORKERS	(tokio default)	Worker thread count. Leave unset unless tuning.
LOG_FORMAT	text	text or json. Use json in production for log aggregation.
LOG_LEVEL	debug	debug / info / warn / error. Use info or warn in production.

NoETL connection

Var	Default	What
NOETL_BASE_URL	http://localhost:8082	noetl-server HTTP base. The gateway proxies authenticated /noetl/* requests here and dispatches playbook executions via /api/execute. When the shard map is empty (the default), every request goes here — current single-replica behavior, unchanged.
NOETL_TIMEOUT_SECS	120	Request timeout for outbound calls to noetl-server.
AUTH_PLAYBOOK_TIMEOUT_SECS	60	How long the gateway waits for the auth0_login playbook callback before returning a timeout error. Tuned up from 12 during the 2026-05-24 incident.

NoETL shard map (Phase F R3a)

The noetl.shards config field is the gateway-side counterpart of noetl-server's ShardConfig (server v2.11.0+). When populated, the gateway routes requests carrying an execution_id in the path (/noetl/executions/{id}/..., /noetl/vars/{id}/...) to the matching shard's base_url using hash(execution_id) % N consistent with the server-side shard_for(). Empty by default — no behavior change from pre-R3a single-replica deployments.

Configured as a TOML array in the gateway config file (not yet exposed as an env var; see Phase F sharding design):

[[noetl.shards]]
shard_index = 0
base_url = "http://noetl-server-0.noetl.svc.cluster.local:8082"

[[noetl.shards]]
shard_index = 1
base_url = "http://noetl-server-1.noetl.svc.cluster.local:8082"

Constraints (enforced at startup; gateway panics on a bad shard map rather than silently mis-route):

• Indices MUST be contiguous from 0 to N-1.
• No duplicates.
• No holes.

Routes covered (path-param execution_id, R3a):

• GET /noetl/executions/{id} and /noetl/executions/{id}/... (status, cancel, finalize, events/stream)
• GET / POST / DELETE /noetl/vars/{execution_id} and /noetl/vars/{execution_id}/...

Routes covered (body-param execution_id, R3a-2):

• POST /noetl/events — gateway parses the JSON body, extracts the top-level execution_id field, routes to the matching shard. Accepts both string (the noetl-server wire shape) and number encodings.
• POST /noetl/events/batch — same body shape; same extraction path.

Routes NOT covered (fall back to NOETL_BASE_URL):

• POST /noetl/execute — server mints the execution_id; pre-routing is impossible. The gateway forwards to the default upstream; the server's response carries the freshly-minted execution_id for subsequent requests.
• GET /noetl/commands/{event_id} — event_id is not an execution_id; needs a DB lookup. R3b or later.
• POST /noetl/catalog/*, /noetl/credentials/*, /noetl/keychain/*, /noetl/worker/pool/* — cluster-wide tables per the sharding-design; any shard's replica answers, so the default upstream is correct.

Hash function: twox_hash::XxHash64 with fixed seed 0, applied to the little-endian byte encoding of the i64 execution_id. Wire contract: gateway's src/sharding.rs::shard_for MUST produce the same output as noetl-server's src/sharding.rs::shard_for for every input. Both implementations pin the same (eid, N) → shard expected values in their unit tests. Extracting a shared noetl-sharding crate to the noetl/cli workspace is a deferred follow-up (R3.5 / Phase G).

Diagnostic endpoint (Phase F R3b-2)

Added in noetl-gateway v3.2.0 (noetl/gateway#25). Twin of noetl-server's GET /api/runtime/shard-info (R3b-1). Public, deterministic, no auth gate.

GET /sharding/preview?execution_id=<i64>&shard_count=<u32>

Computes locally via the gateway's own shard_for() — NOT a proxy passthrough. The point of the R3b drift-guard is that both sides compute independently so the integration test (R3b-3, ops repo) can verify they agree.

Response shape (mirrors the server's R3b-1 shape with source: "noetl-gateway"):

{
  "execution_id": 320816801799737344,
  "shard_count": 4,
  "shard_index": 2,
  "source": "noetl-gateway",
  "hash_function": "twox_hash::XxHash64",
  "seed": 0
}

Validation:

• execution_id parses as i64; non-numeric → 400.
• shard_count is required, range 1..=1024; 0 or > 1024 → 400.

Wire contract: the shard_index field MUST equal the noetl-server twin's response for every input. The R3b-3 integration test in noetl/ops POSTs to both and asserts equality across a battery of (execution_id, N) pairs.

The endpoint sits OUTSIDE the /noetl/* proxy catchall and OUTSIDE the auth middleware — it's a local-compute diagnostic, intentionally cheap and reachable from internal-network probes.

NATS

Var	Default	What
NATS_URL	nats://localhost:4222	NATS connection URL for RequestStore (K/V) and the playbook/state subscriber.
NATS_UPDATES_SUBJECT_PREFIX	playbooks.executions.	Prefix the gateway subscribes to for execution lifecycle events. The full subject pattern is <prefix><execution_id>.*.

The NATS credential, if used, is a platform-runtime credential — same classification as POSTGRES_PASSWORD below. Bind via the gateway pod's k8s Secret, not via the NoETL keychain.

Database (gateway's own state)

The gateway holds session state in its own Postgres database (separate concern from noetl-server's event log). Fields:

Var	Default	What
POSTGRES_HOST	localhost
POSTGRES_PORT	54321
POSTGRES_USER	demo
POSTGRES_PASSWORD	demo	Override in production via k8s Secret.
POSTGRES_DATABASE	demo_noetl
DATABASE_URL	derived	Used by sqlx migrations; set explicitly only when running migrations or running the build with sqlx::query! macros.
SQLGEN_MODEL_OUTPUT_FOLDER	./src/models/	Dev-tooling path; not used at runtime.
SQLGEN_MODEL_FOLDER	./src/models/	Dev-tooling path; not used at runtime.
SQLGEN_MIGRATION_OUTPUT	./migrations	Dev-tooling path; not used at runtime.

The gateway database is the gateway's runtime. Per the secrets-and-credentials rule, its credentials live with the gateway pod, not in the NoETL keychain.

Auth0 (for the auth0_login playbook the gateway dispatches)

The gateway itself does not embed Auth0 client secrets — the verification happens inside the noetl-server-dispatched playbook. The gateway only needs the public Auth0 tenant identifiers to verify the JWT issuer and audience claims match. These are passed to it via the auth middleware config; exact env names depend on the auth subcrate version. Check src/auth/ in your deployed version.

SSE

Var	Default	What
GATEWAY_HEARTBEAT_INTERVAL_SECS	15	How often ping frames are sent over /events.
GATEWAY_CONNECTION_TIMEOUT_SECS	300	Idle timeout for an SSE connection.

Production overrides (typical)

A minimal set of overrides for a GKE deployment:

LOG_FORMAT=json
LOG_LEVEL=info
ROUTER_PORT=8080
NOETL_BASE_URL=http://noetl.noetl.svc.cluster.local:8082
NATS_URL=nats://gateway:<secret>@nats.nats.svc.cluster.local:4222
POSTGRES_HOST=pgbouncer.postgres.svc.cluster.local
POSTGRES_PORT=5432
POSTGRES_DATABASE=gateway
GATEWAY_HEARTBEAT_INTERVAL_SECS=15
AUTH_PLAYBOOK_TIMEOUT_SECS=60

Note: GATEWAY_FIRESTORE_* env vars were removed in v2.12.0. The gateway no longer carries a Firestore subscription subsystem. See Subscriptions for the historical reference.

Postgres password and NATS password come from k8s Secrets mounted as env, not from the ConfigMap.

What does not go in env

For pushback discipline (per secrets-and-credentials rule):

• Third-party API tokens that playbooks need (OpenAI, Anthropic, Duffel, Amadeus, etc.) — these live in the NoETL keychain.
• Tenant database DSNs that playbooks use — keychain.
• OAuth client secrets used by playbook steps — keychain.

The gateway's env only carries the gateway's own runtime credentials (Postgres, NATS) and the public Auth0 tenant identifiers.

Related

• Deployment — GKE / Helm wiring of these vars.
• SSE events — the heartbeat env in context, and the FORWARDED_EVENT_TYPES allowlist (including calendar.event.touched).
• Subscriptions — the GATEWAY_FIRESTORE_* env group (removed in v2.12.0; historical reference).