Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs-web/architecture/custom-dashboard-foundation.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ Primary records:

Dashboard status values are `draft`, `validating`, `validated`, `published`, `rejected`, and `archived`. Validation status values are `queued`, `building`, `running`, `passed`, `failed`, and `cancelled`.

### Feature baseline and bounded addition

Repository history provides the negative baseline for this subsystem: at the pre-feature `dev` commit `716ac2c55`, `CustomDashboardManifest` had no `credentialSlots`, and mutable dashboard and immutable revision records had no `credentialBindings` or binding revision. The implemented change is intentionally limited to bounded manifest declarations, credential-ID bindings in dedicated draft/revision columns, metadata-only compatibility review, optimistic binding mutation, and validation/publication gates. It does not migrate provider secrets and it does not add custom-dashboard secret injection.

Declarations are normalized and bounded for count, slot ID, label, phase (`build` or `runtime`), allowed kinds, and required capabilities. Bindings contain only `slotId` and `credentialId`; generic draft/revision writes cannot set them, and immutable revisions snapshot them. The phase is policy metadata for review and validation, not permission to inject a value into build or runtime artifacts.

## Persistence

SQLite tables are created in both the initial schema and startup migrations:
Expand Down Expand Up @@ -59,7 +65,9 @@ Validation flow:
- A validation session is marked `passed` only after install, build, start, and root URL health checks succeed. Build/start/health failures are recorded as failed validation reports with bounded log excerpts.
- Runtime metadata persists the workspace path, log path, host port, container id/name, image, validation URL path, commands, latest error/log excerpt, and a browser-ready Vite `dist` artifact for passed revisions so the published viewer can render TSX-based drafts without a live validation container.

Validation does not publish or activate dashboards. A successful run only marks the revision validation status as `passed`; publication remains gated by `publishRevision`. REST and MCP publication re-run metadata-only binding review immediately before calling the repository, then require either a revision already marked `passed` with a valid report or an explicit passed validation session for that revision. Failed binding review returns sanitized slot-specific issues without credential IDs or values; queued/running/cancelled validation, missing state, and cross-revision sessions are rejected before the publication pointer changes.
Validation does not publish or activate dashboards. A successful run only marks the revision validation status as `passed`; publication remains gated by `publishRevision`. REST and MCP publication re-run metadata-only binding review immediately before calling the repository, then require either a revision already marked `passed` with a valid report or an explicit passed validation session for that revision. Failed binding review, queued/running/cancelled validation, missing state, and cross-revision sessions are rejected before the publication pointer changes.

No custom-dashboard service resolves credential plaintext. Build workspaces, generated files and Vite artifacts, Docker arguments/mounts/environment, validation reports/logs, generic REST/MCP records, viewer configuration, iframe `srcdoc`, data-bridge payloads, and `postMessage` traffic receive neither credential values nor binding IDs. Only the dedicated metadata-management response may return binding IDs alongside non-secret credential metadata.

## REST and MCP Surface

Expand Down
69 changes: 53 additions & 16 deletions docs-web/architecture/node-flow-builtins-and-security.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,66 @@
# Node Flow Built-ins and External-Effect Security

The governed catalog adds deterministic branches, bounded collection processing, durable approvals, and replay-safe external effects while keeping the versioned definition registry as the executable authority.
The governed built-in catalog extends publication-based node-flow execution with deterministic control nodes and durable boundaries for external effects. The definition registry remains the executable authority; a graph can only run a node when its versioned manifest is registered and executable.

## Control and integration nodes
## Built-in catalog

- `condition` selects `true` or `false`; `switch` selects one named case or `default`. Unselected branches persist as skipped node runs.
- `foreach` rejects inputs above its configured bound (never more than 1,000), then runs downstream nodes once per deterministic logical item. Configured concurrency defaults to one and is capped at 64; zero items explicitly select `empty`. `merge` supports `object`, `array`, and `first` strategies.
- `delay` is cancellable and capped at one hour. `execute_subflow` requires same-project ownership, rejects direct self-reference, and caps depth at eight.
- `approval` persists an idempotent operator decision and continues the exact pinned run after approval. `email_draft` never sends. `email_send` requires approval and uses the idempotent outbox.
- `webhook_trigger` emits payloads accepted through secret-authenticated webhook ingress.
| Node | Contract |
| --- | --- |
| `condition` | Evaluates a bounded operator and selects exactly the `true` or `false` output port. Unselected branches persist as skipped node runs. |
| `switch` | Evaluates no more than 100 configured cases and selects one named case or `default`. |
| `foreach` | Validates an array, rejects inputs above the configured bound (at most 1,000), and executes the selected downstream branch once per logical item with bounded concurrency. |
| `merge` | Combines active upstream values with `object`, `array`, or `first` strategy. |
| `delay` | Waits for a cancellable duration from zero through one hour. |
| `approval` | Creates or reuses a durable approval keyed by run, node, and logical item. |
| `email_draft` | Produces a draft only and never contacts a provider. |
| `email_send` | Requires an approved decision, then dispatches through the idempotent outbox. |
| `execute_subflow` | Executes a published flow owned by the same project, rejects direct self-reference, and caps nesting at eight. |
| `webhook_trigger` | Emits input accepted by a secret-authenticated webhook configuration. |

## Network policy
The existing `input`, `set_fields`, `template`, `provider_prompt`, `http_request`, and `output` nodes retain their previous contracts. Typed manifest ports identify branch handles, many-valued merge inputs, and trigger outputs. Branch routing only runs a node when at least one incoming edge is active, allowing merges to join a selected path without treating an unselected sibling as a failure.

HTTP nodes and future custom nodes use the same `EgressPolicyService`. HTTPS is required unless HTTP is explicitly enabled. Private, loopback, link-local, metadata, multicast, and other non-public addresses remain blocked in both modes. Credentials in URLs and raw restricted headers are rejected.
## Credential-bound execution

Every redirect is manually revalidated. DNS is checked for private results and rebinding. Host and port allowlists, response-size and content-type limits, propagated cancellation and timeouts, capped retries, idempotency requirements for unsafe retry, normalized headers, and per-key rate windows keep requests bounded.
Versioned definition manifests declare credential slots by required state, allowed kinds, and required capabilities. Draft review and publication use metadata-only broker compatibility; the canonical graph stores only slot-to-credential-ID bindings. Required missing bindings and bindings denied for unavailable custody, configuration, status, project access, kind, or capability stop publication. Optional unbound slots remain valid.

## OAuth, approvals, and outbox
At runtime, the immutable published graph is revalidated and the broker repeats authorization immediately before resolving a value for the active attempt. A revoked, restricted, rebound, wrong-kind, insufficiently capable, or unavailable credential fails the attempt closed. Exact resolved values are redacted from built-in output, invocation/attempt records, diagnostics, retries, HTTP/provider responses, and external-effect persistence; neither publication nor MCP inspection injects or returns plaintext.

Pending approvals preserve the run, governed node, logical item, and numbered attempt. Approved decisions resume at that node boundary; rejected and expired decisions terminate durably. Repeated decisions and restart recovery do not create a second approval request, attempt, or external delivery.
Foreach assigns deterministic logical-item identities from the published node id and item index. Each downstream node run and numbered attempt persists that identity together with the item-specific input. The `concurrency` setting defaults to one and is capped at 64; `maxItems` is a rejection bound rather than a truncation rule. A zero-item input selects `empty`, while the `items` branch is persisted as skipped. Per-item failures retain their own retry history, successful siblings are not replayed during approval or restart continuation, and aggregated output preserves input order.

Foreach descendant node runs and attempts persist item-specific inputs and logical identity. Retries remain item-local, completed siblings are reconstructed rather than replayed after restart, and aggregation preserves input order.
## Governed egress

OAuth authorization uses PKCE S256 and short-lived AES-256-GCM state tied to an allowlisted callback origin. Tokens live behind the connection store, rotate on refresh, enforce scopes and expiry, and are never written into graph JSON or agent-visible output. Revocation, reconnect, and health checks expose no token values.
`EgressPolicyService` is the single request boundary for HTTP nodes and future custom-node network calls. HTTPS is required by default. A node must explicitly opt into HTTP, and even then private networking remains blocked. The service rejects credentials embedded in URLs; loopback, private, link-local, carrier-grade NAT, benchmarking, multicast, and cloud-metadata addresses; metadata hostnames; restricted raw headers; and ports or hosts outside configured allowlists.

Approvals are unique per run, node, and logical item. Outbox entries use a unique key derived from publication, run, node, and logical item, and store the provider message id after success. A restart while an entry is sending changes it to `attention_required`; Code UX does not automatically replay an unknown provider outcome.
Each redirect is handled manually and fully revalidated. DNS is resolved twice before dispatch, and a changed or newly private result is treated as rebinding. Cross-origin redirects remove credential headers. Response bodies are streamed into a bounded buffer, content types are allowlisted, timeouts and caller cancellation propagate, retry counts are capped, unsafe methods require an idempotency key before retry, and an in-process rate window bounds requests per project and host.

Webhook configuration returns a newly rotated path token and secret while persisting only their hashes. Ingress requires `x-codeux-webhook-secret` and dispatches the latest published flow version.
## OAuth boundary

`OAuthBroker` implements authorization-code flow with PKCE S256. Authorization state is authenticated AES-256-GCM ciphertext containing a short expiry, callback origin, redirect URI, verifier, connection id, and nonce. Callback origins must be explicitly allowlisted and match the state. Token exchange and refresh results are stored behind an `OAuthConnectionStore`; access and refresh tokens are returned only to provider-bound execution code, never to graph JSON or agent-visible output.

Refresh happens shortly before expiry and rotates the stored refresh token when the provider returns one. Required scopes are checked before access. Revocation deletes local state after provider revocation; reconnect begins from a revoked local connection; health checks refresh when necessary and expose only health, expiry, and scopes.

## Approvals and outbox

`automation_approvals` persists pending and terminal decisions. Repeating the same run, node, and logical item returns the existing decision, so restarts do not create a second prompt. Repeating an identical decision is also idempotent. Approval or rejection through the decision endpoint resumes or terminates the exact waiting run; approval preserves its publication, run id, logical item, and attempt number. Email sending is approval-gated by default; `email_draft` is the non-irreversible default.

`automation_outbox` has a unique SHA-256 idempotency key derived from publication id, run id, node id, and logical item. Provider message ids are stored after success. A process restart while an entry is `sending` changes it to `attention_required`, because the provider may have accepted the operation; Code UX does not replay an unknown external outcome automatically.

## Webhook routes

Creating `POST /api/node-flows/:flowId/webhook` rotates and returns a path token and secret once. Only their hashes are persisted. `POST /api/webhooks/node-flows/:pathToken` requires the secret in `x-codeux-webhook-secret`, uses constant-time digest comparison, and dispatches the latest published version with `triggerType: webhook`. The response returns only run identity and status.

Example condition edges use explicit handles:

```json
{
"nodes": [
{ "id": "check", "type": "condition", "title": "Check", "data": { "path": "input.enabled" } },
{ "id": "draft", "type": "email_draft", "title": "Draft", "data": { "to": "owner@example.test", "subject": "Ready", "body": "Review this draft." } },
{ "id": "done", "type": "output", "title": "Done" }
],
"edges": [
{ "fromNodeId": "check", "fromHandle": "true", "toNodeId": "draft" },
{ "fromNodeId": "check", "fromHandle": "false", "toNodeId": "done" }
]
}
```
Loading
Loading