Skip to content

architecture

Jump to bottom
Kadyapam edited this page May 29, 2026 · 2 revisions

Architecture

How the travel SPA, the NoETL gateway, the worker pool, and the playbook catalog fit together. This page is the entry point for engineers who are about to read source code in this repo.

For the underlying platform principle, see Ephemeral Blueprints + Compute-Data Boundary. This page applies that principle to one concrete domain SPA.

The shape

┌──────────────────────────────┐     ┌──────────────────────────────┐
│   Browser (this repo)        │     │   Cloudflare Pages           │
│                              │◄────│   (static SPA hosting)       │
│   - Auth0 SPA login          │     └──────────────────────────────┘
│   - React shell + widgets    │
│   - SSE EventSource          │
└────┬────────────────────┬────┘
     │ session_token       │ SSE /events
     │ + GraphQL execute   │   playbook/result
     ▼                     ▼   playbook/state
                               subscription/event
┌──────────────────────────────────────────────────┐
│   NoETL Gateway (Rust, gatekeeper only)          │
│                                                  │
│   - Auth0 ID token -> gateway session_token      │
│   - GraphQL executePlaybook                      │
│   - SSE /events fan-out (per client_id)          │
│   - POST/DELETE /api/subscriptions/firestore     │
│   - NATS subject bridge for playbook/state       │
└────┬─────────────────────────────────────────────┘
     │ HTTP /api/execute        SSE callback
     │ NATS subscribe           POST /api/internal/callback
     ▼                          ▲
┌──────────────────────────────────────────────────┐
│   NoETL Server (FastAPI)                         │
│                                                  │
│   - Catalog of playbooks                         │
│   - Execution dispatcher (NATS publish)          │
│   - Event log (Postgres)                         │
└────┬─────────────────────────────────────────────┘
     │ NATS JetStream NOETL_COMMANDS
     ▼
┌──────────────────────────────────────────────────┐
│   Worker pool (Kubernetes Deployment)            │
│                                                  │
│   - Claims commands, runs one tool per step      │
│   - Tools: python, postgres, nats, agent, http   │
│   - Emits events back to noetl-server            │
│   - Calls nested playbooks for MCP servers       │
└──────────────────────────────────────────────────┘

The diagram is not "what's nice"; it is the actual contract. Each arrow is a real network hop, each box is a separately- deployable runtime, and nothing skips a layer. The SPA never talks to the worker pool directly; the worker never talks to the browser directly; the gateway never opens a database connection on behalf of the SPA.

Four layers, one direction

1. SPA shell — domain-neutral

src/components/shell/

The shell holds:

  • ChatThread.tsx — list of messages, scroll behavior, message composition.
  • InputBar.tsx — text + dictation input.
  • Sidebar.tsx — searches/orders history list.
  • RightPane.tsx — secondary panel for full-detail views.

The shell does not know what "hotel" or "flight" means. It knows about messages, threads, and a generic "widget envelope" shape it can render. Replace every widget with logistics-domain widgets, and the shell still works.

2. Widgets — the visual vocabulary

src/components/widgets/ + playbooks/widget-contract/*.schema.json

Each widget is two files:

  • A JSON schema in playbooks/widget-contract/ declaring the widget's type and its data shape.
  • A React component in src/components/widgets/ consuming that shape.

The schemas are the source of truth. Running npm run contracts regenerates src/contracts/widgets.ts so TypeScript catches any drift. The smoke harness (npm run smoke:widgets) round-trips an example envelope of every widget through the dispatcher and asserts it renders.

There is no other contract between playbooks and the SPA. A playbook can return any widget the SPA has a schema and a component for, and the SPA renders it without code changes per-request.

See Widget contract for the details and the workflow to add a new widget.

3. Playbooks — the control flow

playbooks/itinerary-planner.yaml + playbooks/agent/ + playbooks/widget-contract/

The itinerary-planner playbook is the orchestrator. It is invoked once per user message and runs through ~16 steps:

  • normalize_input
  • append_input_event (writes to Firestore via MCP)
  • load_slot_state
  • extract_turn (LLM JSON-mode extraction)
  • persist_slot_state
  • append_slot_update_event
  • (one of) call_google_places / call_duffel_offers / call_amadeus_hotels / call_duffel_create_order
  • normalize_tool_response
  • persist_tool_slot_state
  • persist_api_call
  • append_tool_response_event
  • render_widget_chat (emits the widget envelope)
  • persist_calendar_event_* (fans out to calendar entries)
  • append_widget_event
  • final_result

Every step is one tool call. Every external system access goes through a tool (postgres, nats, http, agent). The tool: agent steps dispatch nested playbooks under automation/agents/mcp/<server> in noetl/ops — the MCP servers themselves are playbooks in the catalog, not running processes.

See Playbook: itinerary-planner for the step-by-step walkthrough.

4. Gateway integration — narrow, well-tested

src/api/

Three modules:

  • gatewaySession.ts — Auth0 ID token to gateway session exchange (POST /api/auth/login), session validation, logout.
  • noetlClient.ts — executePlaybook over GraphQL, SSE connection + playbook/result and playbook/state frame handling, callback fallback.
  • calendarSubscription.ts — live calendar refresh. On mount the SPA calls the travel/playbooks/catalog/calendar/list read playbook through executePlaybook; on each forwarded calendar.event.touched SSE frame from the orchestrator it re-runs the read. Falls back to playbook.completed when a turn finishes without a calendar write (clears loading state).

The SPA never holds a database connection or a Firebase SDK handle. All reads and writes flow through the gateway, which routes them through playbooks — there is no gateway-side subscription proxy anymore (the historical gatewaySubscriptions.ts + POST /api/subscriptions/firestore path was removed in noetl/ai-meta#23 Round 03).

See Gateway integration for the wire protocol.

What this SPA never does

A list to push back against, when changes come in:

  • Never connect to a database directly. There is no pg, mysql, firebase-admin, or any other database client in the SPA bundle. Subscriptions flow through the gateway.
  • Never hold a third-party API token in the browser. Tokens for Auth0 (yes, the SPA holds an ID token, but Auth0 is the identity provider), and all other domain tokens (Duffel, Amadeus, OpenAI, Google Places) live in the NoETL keychain and are referenced by alias inside playbook steps.
  • Never run business logic in the SPA. Pricing rules, filtering rules, "which provider to use" — all in playbooks.
  • Never poll for state when an SSE subscription is available. The 200-attempt polling fallback in ChatThread.waitForExecution exists for the cold-start / SSE-error case only and will be removed once the gateway subscription path is fully proven in production.

State lives in two places

For a given user session, state is split:

  • Conversation / slot state — Firestore documents at chat_threads/<thread_id>/slot_state/current, chat_threads/<thread_id>/events/*. Written by the itinerary-planner playbook through the firestore MCP. Read by the SPA through gateway subscriptions.
  • Auth session — gateway-issued session_token in browser storage; gateway maintains the session in its own store.

The browser holds neither a Firestore credential nor a database credential; it holds the session token and an SSE connection.

Cross-references

Travel SPA

Architecture

Integration

Operations

See also

Clone this wiki locally