Skip to content

subscriptions

Jump to bottom
Kadyapam edited this page May 24, 2026 · 2 revisions

Firestore subscriptions

POST /api/subscriptions/firestore lets an authenticated client subscribe to live Firestore updates without holding a Firestore SDK in the browser. The gateway runs the watch server-side and pushes changes to the client over the existing SSE channel.

Added in v2.11.0 as the gateway-side companion to travel#50, which removed firebase/firestore from the SPA bundle.

Source: src/firestore_subscriptions.rs, requirements-firestore.txt.

Why this exists

Per the Ephemeral Blueprints + Compute-Data Boundary principle, clients must not access the domain database directly. Before v2.11.0, the travel SPA shipped the Firebase JS SDK and opened onSnapshot listeners against Firestore. That bypassed the gateway and the keychain.

The subscription endpoint moves the listener into a process that already has the right identity (the gateway pod, with its own service account or credential reference), keeps the SPA bundle free of database SDKs, and routes live updates over the same SSE channel the SPA already uses for playbook/result.

API

Open a subscription

POST /api/subscriptions/firestore
Cookie / Header: Authorization: Bearer <gateway_session_token>
Content-Type: application/json

{
  "path": "chat_threads/<thread_id>/calendar_events",
  "scope": "owner"
}

Response:

{
  "subscription_id": "<uuid>",
  "client_id": "<existing_sse_client_id>"
}

The client must already have an open SSE connection. The client_id returned in the response matches the one assigned by the SSE init frame and is included for symmetry.

Receive updates

Updates arrive over the existing SSE channel as subscription/event frames keyed by subscription_id. See SSE events.

Cancel a subscription

DELETE /api/subscriptions/{subscription_id}
Authorization: Bearer <gateway_session_token>

Closes the server-side watch and stops forwarding frames for that subscription. The SSE channel itself stays open.

Authorization

The gateway validates that the requested path is allowed for the authenticated session before opening the watch. The current check is:

  • scope must be owner.
  • The path is expected to start with chat_threads/<thread_id>/... where thread_id is reachable to the user (verified against the same tables NoETL's executePlaybook uses).

Paths outside the allowed pattern return 403.

This authorization step is the load-bearing reason a server- side subscription exists in the first place. The SDK in the browser cannot enforce per-tenant scoping — Firestore Security Rules can, but they put the policy in two places. Putting it in the gateway keeps the policy with the rest of the authorization model.

Server-side architecture: the Python sidecar

The gateway is written in Rust, but there is no production- ready Rust Firestore Admin SDK with watch semantics. Rather than introduce a binding ourselves, v2.11.0 ships a thin Python sidecar process that handles the Firestore watch and streams JSON lines back to the gateway.

┌─────────────────────────────┐
│   noetl-gateway (Rust)      │
│   - validates session       │
│   - validates path scope    │
│   - spawns / shares sidecar │
└────┬────────────────────────┘
     │ stdin: { "op": "subscribe", "path": ..., "id": ... }
     │ stdin: { "op": "unsubscribe", "id": ... }
     │ stdout: { "id": ..., "doc_id": ..., "data": ..., "op": "added" }
     ▼
┌─────────────────────────────┐
│   firestore_listener (Py)   │
│   - google-cloud-firestore  │
│   - one onSnapshot per id   │
└─────────────────────────────┘
  • The sidecar is invoked according to GATEWAY_FIRESTORE_LISTENER_CMD (default points at the bundled Python script).
  • It uses google-cloud-firestore. Credentials come from GATEWAY_FIRESTORE_CREDENTIALS_PATH (a service-account JSON mounted into the pod via a k8s Secret) and GATEWAY_FIRESTORE_PROJECT_ID.
  • The gateway parses each stdout JSON line and forwards it as subscription/event to the right client_id.

The Rust gateway never imports Firebase libraries. The sidecar never serves HTTP. The boundary is the JSON-line IPC.

Credentials

Per the secrets-and-credentials rule:

  • The Firestore service-account credential is the gateway's platform credential (the gateway is the process that needs to authenticate to Firestore). It lives in a Kubernetes Secret mounted at GATEWAY_FIRESTORE_CREDENTIALS_PATH.
  • It is not a business-logic credential and does not go in the NoETL keychain. The keychain holds credentials playbook steps need; the gateway sidecar's Firestore handle is part of the gateway runtime.
  • Provision via: 
    kubectl create secret generic gateway-firestore-credentials \
  --from-file=service-account.json=<your-sa-key.json> \
  -n gateway
    Then mount as a volume in the gateway Deployment at /var/run/secrets/firestore/service-account.json.

Never commit the service-account JSON. The Cargo crate and the container image are public-distributable; the credential is per-deployment.

Sidecar lifecycle

  • One sidecar process per gateway pod (today). The sidecar can hold many concurrent watches multiplexed by id.
  • On gateway pod restart, the sidecar restarts with it, and all active subscriptions need to be re-established by their clients. SSE auto-reconnect handles the channel; clients are responsible for re-POSTing their subscriptions.
  • The sidecar reports its readiness via the gateway's /health endpoint (the gateway considers itself healthy only when the sidecar's IPC handle is open).

What this is not

  • Not a write path. The gateway does not accept Firestore writes; writes go through playbooks (mcp/firestore.append_event, mcp/firestore.set_doc).
  • Not a replacement for Firestore Security Rules. Rules still apply server-side. The gateway's authorization is an additional gate, not a substitute.
  • Not a generic webhook surface. Subscriptions are scoped to paths the gateway can validate; arbitrary watch URLs are refused.

Related

Gateway

Surfaces

Operations

See also

Clone this wiki locally