From fe86a4692dec11b01809c29a2f182e65386a5e27 Mon Sep 17 00:00:00 2001 From: Code UX Date: Mon, 13 Jul 2026 23:43:50 +0000 Subject: [PATCH 1/2] feat(task T09): implement via codex --- .../custom-dashboard-foundation.md | 10 +- .../node-flow-builtins-and-security.md | 69 +++-- docs-web/architecture/node-flows.md | 156 +++++++++-- ...chitecture-custom-dashboard-foundation.mdx | 10 +- ...ecture-node-flow-builtins-and-security.mdx | 69 +++-- .../content/docs/architecture-node-flows.mdx | 156 +++++++++-- .../docs/operations-credential-security.mdx | 64 +++-- .../content/docs/operations-server-mode.mdx | 250 ++++++++++++++++-- docs-web/content/docs/registry.ts | 14 +- .../content/docs/settings-integrations.mdx | 36 ++- .../docs/user-dashboard-custom-dashboards.mdx | 159 ++++++++--- .../docs/user-dashboard-node-flows.mdx | 36 +-- docs-web/operations/credential-security.md | 64 +++-- docs-web/operations/server-mode.md | 250 ++++++++++++++++-- docs-web/settings/integrations.md | 44 ++- docs-web/user/dashboard/custom-dashboards.md | 159 ++++++++--- docs-web/user/dashboard/node-flows.md | 36 +-- docs/SUMMARY.md | 1 + .../custom-dashboard-foundation.md | 8 + .../node-flow-builtins-and-security.md | 6 + docs/architecture/node-flows.md | 8 + docs/dashboard/custom-dashboards.md | 4 + docs/dashboard/node-flows.md | 2 + docs/index.md | 1 + docs/operations/credential-security.md | 12 + docs/operations/server-mode.md | 5 + docs/settings/integrations.md | 17 +- 27 files changed, 1358 insertions(+), 288 deletions(-) diff --git a/docs-web/architecture/custom-dashboard-foundation.md b/docs-web/architecture/custom-dashboard-foundation.md index ae43447fa4..ba3a63d42d 100644 --- a/docs-web/architecture/custom-dashboard-foundation.md +++ b/docs-web/architecture/custom-dashboard-foundation.md @@ -15,6 +15,12 @@ Primary records: Dashboard status values are `draft`, `validating`, `validated`, `published`, `rejected`, and `archived`. Validation status values are `queued`, `building`, `running`, `passed`, `failed`, and `cancelled`. +### Feature baseline and bounded addition + +Repository history provides the negative baseline for this subsystem: at the pre-feature `dev` commit `716ac2c55`, `CustomDashboardManifest` had no `credentialSlots`, and mutable dashboard and immutable revision records had no `credentialBindings` or binding revision. The implemented change is intentionally limited to bounded manifest declarations, credential-ID bindings in dedicated draft/revision columns, metadata-only compatibility review, optimistic binding mutation, and validation/publication gates. It does not migrate provider secrets and it does not add custom-dashboard secret injection. + +Declarations are normalized and bounded for count, slot ID, label, phase (`build` or `runtime`), allowed kinds, and required capabilities. Bindings contain only `slotId` and `credentialId`; generic draft/revision writes cannot set them, and immutable revisions snapshot them. The phase is policy metadata for review and validation, not permission to inject a value into build or runtime artifacts. + ## Persistence SQLite tables are created in both the initial schema and startup migrations: @@ -59,7 +65,9 @@ Validation flow: - A validation session is marked `passed` only after install, build, start, and root URL health checks succeed. Build/start/health failures are recorded as failed validation reports with bounded log excerpts. - Runtime metadata persists the workspace path, log path, host port, container id/name, image, validation URL path, commands, latest error/log excerpt, and a browser-ready Vite `dist` artifact for passed revisions so the published viewer can render TSX-based drafts without a live validation container. -Validation does not publish or activate dashboards. A successful run only marks the revision validation status as `passed`; publication remains gated by `publishRevision`. REST and MCP publication re-run metadata-only binding review immediately before calling the repository, then require either a revision already marked `passed` with a valid report or an explicit passed validation session for that revision. Failed binding review returns sanitized slot-specific issues without credential IDs or values; queued/running/cancelled validation, missing state, and cross-revision sessions are rejected before the publication pointer changes. +Validation does not publish or activate dashboards. A successful run only marks the revision validation status as `passed`; publication remains gated by `publishRevision`. REST and MCP publication re-run metadata-only binding review immediately before calling the repository, then require either a revision already marked `passed` with a valid report or an explicit passed validation session for that revision. Failed binding review, queued/running/cancelled validation, missing state, and cross-revision sessions are rejected before the publication pointer changes. + +No custom-dashboard service resolves credential plaintext. Build workspaces, generated files and Vite artifacts, Docker arguments/mounts/environment, validation reports/logs, generic REST/MCP records, viewer configuration, iframe `srcdoc`, data-bridge payloads, and `postMessage` traffic receive neither credential values nor binding IDs. Only the dedicated metadata-management response may return binding IDs alongside non-secret credential metadata. ## REST and MCP Surface diff --git a/docs-web/architecture/node-flow-builtins-and-security.md b/docs-web/architecture/node-flow-builtins-and-security.md index 80324199ed..18174020be 100644 --- a/docs-web/architecture/node-flow-builtins-and-security.md +++ b/docs-web/architecture/node-flow-builtins-and-security.md @@ -1,29 +1,66 @@ # Node Flow Built-ins and External-Effect Security -The governed catalog adds deterministic branches, bounded collection processing, durable approvals, and replay-safe external effects while keeping the versioned definition registry as the executable authority. +The governed built-in catalog extends publication-based node-flow execution with deterministic control nodes and durable boundaries for external effects. The definition registry remains the executable authority; a graph can only run a node when its versioned manifest is registered and executable. -## Control and integration nodes +## Built-in catalog -- `condition` selects `true` or `false`; `switch` selects one named case or `default`. Unselected branches persist as skipped node runs. -- `foreach` rejects inputs above its configured bound (never more than 1,000), then runs downstream nodes once per deterministic logical item. Configured concurrency defaults to one and is capped at 64; zero items explicitly select `empty`. `merge` supports `object`, `array`, and `first` strategies. -- `delay` is cancellable and capped at one hour. `execute_subflow` requires same-project ownership, rejects direct self-reference, and caps depth at eight. -- `approval` persists an idempotent operator decision and continues the exact pinned run after approval. `email_draft` never sends. `email_send` requires approval and uses the idempotent outbox. -- `webhook_trigger` emits payloads accepted through secret-authenticated webhook ingress. +| Node | Contract | +| --- | --- | +| `condition` | Evaluates a bounded operator and selects exactly the `true` or `false` output port. Unselected branches persist as skipped node runs. | +| `switch` | Evaluates no more than 100 configured cases and selects one named case or `default`. | +| `foreach` | Validates an array, rejects inputs above the configured bound (at most 1,000), and executes the selected downstream branch once per logical item with bounded concurrency. | +| `merge` | Combines active upstream values with `object`, `array`, or `first` strategy. | +| `delay` | Waits for a cancellable duration from zero through one hour. | +| `approval` | Creates or reuses a durable approval keyed by run, node, and logical item. | +| `email_draft` | Produces a draft only and never contacts a provider. | +| `email_send` | Requires an approved decision, then dispatches through the idempotent outbox. | +| `execute_subflow` | Executes a published flow owned by the same project, rejects direct self-reference, and caps nesting at eight. | +| `webhook_trigger` | Emits input accepted by a secret-authenticated webhook configuration. | -## Network policy +The existing `input`, `set_fields`, `template`, `provider_prompt`, `http_request`, and `output` nodes retain their previous contracts. Typed manifest ports identify branch handles, many-valued merge inputs, and trigger outputs. Branch routing only runs a node when at least one incoming edge is active, allowing merges to join a selected path without treating an unselected sibling as a failure. -HTTP nodes and future custom nodes use the same `EgressPolicyService`. HTTPS is required unless HTTP is explicitly enabled. Private, loopback, link-local, metadata, multicast, and other non-public addresses remain blocked in both modes. Credentials in URLs and raw restricted headers are rejected. +## Credential-bound execution -Every redirect is manually revalidated. DNS is checked for private results and rebinding. Host and port allowlists, response-size and content-type limits, propagated cancellation and timeouts, capped retries, idempotency requirements for unsafe retry, normalized headers, and per-key rate windows keep requests bounded. +Versioned definition manifests declare credential slots by required state, allowed kinds, and required capabilities. Draft review and publication use metadata-only broker compatibility; the canonical graph stores only slot-to-credential-ID bindings. Required missing bindings and bindings denied for unavailable custody, configuration, status, project access, kind, or capability stop publication. Optional unbound slots remain valid. -## OAuth, approvals, and outbox +At runtime, the immutable published graph is revalidated and the broker repeats authorization immediately before resolving a value for the active attempt. A revoked, restricted, rebound, wrong-kind, insufficiently capable, or unavailable credential fails the attempt closed. Exact resolved values are redacted from built-in output, invocation/attempt records, diagnostics, retries, HTTP/provider responses, and external-effect persistence; neither publication nor MCP inspection injects or returns plaintext. -Pending approvals preserve the run, governed node, logical item, and numbered attempt. Approved decisions resume at that node boundary; rejected and expired decisions terminate durably. Repeated decisions and restart recovery do not create a second approval request, attempt, or external delivery. +Foreach assigns deterministic logical-item identities from the published node id and item index. Each downstream node run and numbered attempt persists that identity together with the item-specific input. The `concurrency` setting defaults to one and is capped at 64; `maxItems` is a rejection bound rather than a truncation rule. A zero-item input selects `empty`, while the `items` branch is persisted as skipped. Per-item failures retain their own retry history, successful siblings are not replayed during approval or restart continuation, and aggregated output preserves input order. -Foreach descendant node runs and attempts persist item-specific inputs and logical identity. Retries remain item-local, completed siblings are reconstructed rather than replayed after restart, and aggregation preserves input order. +## Governed egress -OAuth authorization uses PKCE S256 and short-lived AES-256-GCM state tied to an allowlisted callback origin. Tokens live behind the connection store, rotate on refresh, enforce scopes and expiry, and are never written into graph JSON or agent-visible output. Revocation, reconnect, and health checks expose no token values. +`EgressPolicyService` is the single request boundary for HTTP nodes and future custom-node network calls. HTTPS is required by default. A node must explicitly opt into HTTP, and even then private networking remains blocked. The service rejects credentials embedded in URLs; loopback, private, link-local, carrier-grade NAT, benchmarking, multicast, and cloud-metadata addresses; metadata hostnames; restricted raw headers; and ports or hosts outside configured allowlists. -Approvals are unique per run, node, and logical item. Outbox entries use a unique key derived from publication, run, node, and logical item, and store the provider message id after success. A restart while an entry is sending changes it to `attention_required`; Code UX does not automatically replay an unknown provider outcome. +Each redirect is handled manually and fully revalidated. DNS is resolved twice before dispatch, and a changed or newly private result is treated as rebinding. Cross-origin redirects remove credential headers. Response bodies are streamed into a bounded buffer, content types are allowlisted, timeouts and caller cancellation propagate, retry counts are capped, unsafe methods require an idempotency key before retry, and an in-process rate window bounds requests per project and host. -Webhook configuration returns a newly rotated path token and secret while persisting only their hashes. Ingress requires `x-codeux-webhook-secret` and dispatches the latest published flow version. +## OAuth boundary + +`OAuthBroker` implements authorization-code flow with PKCE S256. Authorization state is authenticated AES-256-GCM ciphertext containing a short expiry, callback origin, redirect URI, verifier, connection id, and nonce. Callback origins must be explicitly allowlisted and match the state. Token exchange and refresh results are stored behind an `OAuthConnectionStore`; access and refresh tokens are returned only to provider-bound execution code, never to graph JSON or agent-visible output. + +Refresh happens shortly before expiry and rotates the stored refresh token when the provider returns one. Required scopes are checked before access. Revocation deletes local state after provider revocation; reconnect begins from a revoked local connection; health checks refresh when necessary and expose only health, expiry, and scopes. + +## Approvals and outbox + +`automation_approvals` persists pending and terminal decisions. Repeating the same run, node, and logical item returns the existing decision, so restarts do not create a second prompt. Repeating an identical decision is also idempotent. Approval or rejection through the decision endpoint resumes or terminates the exact waiting run; approval preserves its publication, run id, logical item, and attempt number. Email sending is approval-gated by default; `email_draft` is the non-irreversible default. + +`automation_outbox` has a unique SHA-256 idempotency key derived from publication id, run id, node id, and logical item. Provider message ids are stored after success. A process restart while an entry is `sending` changes it to `attention_required`, because the provider may have accepted the operation; Code UX does not replay an unknown external outcome automatically. + +## Webhook routes + +Creating `POST /api/node-flows/:flowId/webhook` rotates and returns a path token and secret once. Only their hashes are persisted. `POST /api/webhooks/node-flows/:pathToken` requires the secret in `x-codeux-webhook-secret`, uses constant-time digest comparison, and dispatches the latest published version with `triggerType: webhook`. The response returns only run identity and status. + +Example condition edges use explicit handles: + +```json +{ + "nodes": [ + { "id": "check", "type": "condition", "title": "Check", "data": { "path": "input.enabled" } }, + { "id": "draft", "type": "email_draft", "title": "Draft", "data": { "to": "owner@example.test", "subject": "Ready", "body": "Review this draft." } }, + { "id": "done", "type": "output", "title": "Done" } + ], + "edges": [ + { "fromNodeId": "check", "fromHandle": "true", "toNodeId": "draft" }, + { "fromNodeId": "check", "fromHandle": "false", "toNodeId": "done" } + ] +} +``` diff --git a/docs-web/architecture/node-flows.md b/docs-web/architecture/node-flows.md index 1dce58b423..4a20a15ffe 100644 --- a/docs-web/architecture/node-flows.md +++ b/docs-web/architecture/node-flows.md @@ -1,30 +1,138 @@ # Node Flows -Node flows are project-owned, versioned Graph v2 workflows. +Node flows are project-scoped, repeatable workflow graphs for turning an operator or agent-defined procedure into a saved Code UX workflow. They are not a generic n8n compatibility layer. A good flow uses Code UX concepts, project-owned records, provider settings, execution invocations, and editable widget schemas so the same workflow can be inspected, rerun, scheduled, and attached to agents. -Authenticated dashboard routes resolve persisted project ownership from flow, run, or approval ids before authorizing the request. Drafts, publications, comparisons, rollbacks, attachments, webhook configuration, debugger data, attempts, cancellation, retry, and approvals cannot be accessed by presenting a different body or query project id. Webhook ingress remains on its path-token and webhook-secret scheme, with dashboard host and browser-origin protections still enforced. +The foundation page in [Node Flow Foundation](./node-flow-foundation.md) lists the low-level contracts. This page describes the end-to-end architecture and runtime expectations for developers and specialist agents. -## Implemented runtime nodes +## Data Model -| Type | Execution | +Node-flow persistence is owned by `NodeFlowRepository` and stored in SQLite: + +| Table | Purpose | +| --- | --- | +| `node_flows` | Current project-scoped flow record: id, project id, title, description, normalized `graph_json`, current version, and timestamps. | +| `node_flow_versions` | Immutable edit snapshots written on create and every update. | +| `node_flow_publications` | Immutable executable graph and execution-policy snapshots selected by pinned or latest-published runs. | +| `node_flow_agent_skills` | Agent attachment table keyed by flow and agent preset. It stores the skill display name and description used when exposing the flow as a repeatable agent capability. | +| `node_flow_runs` | Flow run records with status, version, trigger type, redacted trigger payload, redacted input/output, error message, timestamps, and optional `execution_invocation_id`. | +| `node_flow_node_runs` | Per-node run records with status, node id, redacted input/output, error message, timestamps, and optional `execution_invocation_id`. | +| `node_flow_node_attempts` | Numbered attempts with executor/invocation identity, artifact digest, credential ids, redacted payloads, failure class, and retry decision. | + +All graphs, widget schemas, run inputs, outputs, and trigger payloads are stored as JSON text and hydrated into typed contracts at the repository boundary. Flow, version, run, and attachment records belong to a project. Agent attachment operations verify that the target agent preset belongs to the same project as the flow. + +Authenticated dashboard requests resolve project ownership from the persisted flow, run, or approval record before role and project authorization. This applies to ID-only draft, publication, comparison, rollback, attachment, webhook-configuration, run debugger, attempt, cancellation, retry, and approval routes; a caller-supplied body or query project id is not treated as proof of ownership. Webhook ingress is the exception to dashboard bearer authentication and continues to use its path-token and webhook-secret scheme, while dashboard host and browser-origin protections still apply. + +## Graph Contract + +The shared contract lives in `src/contracts/node-flow-types.ts`. + +A `NodeFlowGraph` contains: + +- `nodes`: stable node ids, string node `type`, title, optional description, optional position, optional `widgetSchema`, and JSON `data`. +- `edges`: directed links from `fromNodeId` to `toNodeId`. +- `inputSchema`: optional graph-level widget schema for run input. +- `metadata`: optional JSON object for non-secret descriptive data. + +Validation is owned by `src/domain/node-flows/node-flow-validation.ts`. It normalizes ids, labels, positions, widget defaults, and graph shape; rejects missing node/edge arrays; rejects duplicate node ids; rejects edges that point at missing nodes; requires at least one node; and rejects cycles. Widget validation supports `text`, `textarea`, `number`, `boolean`, `select`, `json`, `secretRef`, and `keyValue` fields. + +Migration and validation treat persisted Graph v1 and canonical Graph v2 as untrusted input. Malformed collection members are rejected at their original index, such as `nodes[1].ports[0]` or `edges[2]`, while structurally valid siblings remain available to the rest of normalization. Definition references, credential bindings, capabilities, policies, port and graph schemas, and JSON metadata emit deterministic field-level issues instead of throwing. Revalidating the same graph produces the same ordered issue list. + +Validation requires every node's type/version reference to resolve through the registry and rejects unknown definitions. Runtime execution then dispatches according to the registered definition's executable state and execution kind; a planning concept is not runnable merely because it has a string type. + +The dashboard uses the same backend-owned Graph v2 record as the runtime. The selected project controls library loading; no project means no flow, credential, publication, or run requests. The registry list endpoint returns flat palette summaries, while the node-type detail endpoint returns a complete `NodeDefinitionManifest` with nested `ui`, schemas, policies, documentation, and deprecation metadata. The inspector consumes that full manifest. Draft saves use optimistic `draftRevision` checks and surface conflicts without overwriting the newer record. + +`dashboard/src/v2/lib/nodes-canvas-state.ts` remains only a compatibility and pure graph-state layer. Its legacy browser graph can be imported once into a project draft. The adapter translates `trigger`/`agent`/`task` into registered `input`/`set_fields`/`provider_prompt` nodes, remaps legacy handles to governed ports, and retains non-secret canvas metadata. Import failure is isolated from the normal library load; only a successful draft creation removes the old graph key and records the project marker. Browser storage is not the workflow source of truth. + +### Credential binding lifecycle + +Each versioned node definition is the slot-policy authority: every slot declares whether it is required, its allowed credential kinds, and all required capabilities. The picker lists project-visible metadata, then filters each candidate through secure-backend readiness, configured/active state, project access, kind, and capability compatibility. It never resolves a value. + +`NodeFlowNode.credentialBindings` is the only persisted binding source. Selecting, replacing, or unbinding a credential changes the matching `{ slot, credentialId }` entry in the complete canonical graph and saves with the current `draftRevision`. The dashboard adopts the returned graph and revision, then refreshes governed review. A `409`-style revision conflict refreshes the latest draft and requires a deliberate retry; it never replays a stale binding over sibling changes. + +Required unbound slots and any bound credential denied by backend readiness, configuration, active status, project access, allowed kind, or required capabilities block publication. Optional unbound slots do not. Runtime revalidates the immutable publication and repeats the same policy immediately before direct credential-ID resolution, so revocation, restriction, rotation/rebinding races, missing custody, or incompatible policy deny the node attempt rather than injecting stale plaintext. Graph, review, publication, MCP, and dashboard payloads contain IDs and non-secret policy metadata only. + +## Runtime + +`NodeFlowRuntimeService.runFlow(projectId, flowId, input, options)` resolves an explicit pinned or latest-published snapshot, revalidates that immutable graph, claims a durable lease, and executes nodes in topological order. See [Node Flow Durable Execution](./node-flow-durable-execution.md) for queue, retry, lease, recovery, quota, and redaction guarantees. + +Runtime-supported node types are: + +| Node type | Behavior | | --- | --- | -| `input` | Emits run input. | -| `set_fields` | Transforms object fields. | -| `template` | Renders text templates. | -| `provider_prompt` | Invokes a configured CLI provider. | -| `http_request` | Performs a bounded HTTP/HTTPS request. | -| `condition`, `switch` | Selects one explicit output branch and persists unselected branches as skipped. | -| `foreach`, `merge` | Bounds item fan-out and combines active inputs with an explicit strategy. | -| `delay`, `approval` | Waits with cancellation or persists an operator decision gate. | -| `email_draft`, `email_send` | Produces a draft, or sends only after approval through the idempotent outbox. | -| `execute_subflow` | Executes a same-project published flow with recursion bounds. | -| `webhook_trigger` | Emits secret-authenticated webhook input. | -| `output` | Selects the result. | - -These are the executable definitions. Other custom palette concepts remain non-executable until a versioned handler is registered. Graph v1 migration preserves the legacy snapshot and appends deterministic v2. Both migrated v1 and canonical v2 graphs are validated as untrusted input: malformed nested members fail closed with stable paths at their original array indices, valid siblings are retained where safe, and repeated validation returns the same ordered issues instead of throwing. - -The separate browser-canvas compatibility bridge translates legacy `trigger`, `agent`, and `task` kinds into registered `input`, `set_fields`, and `provider_prompt` nodes and remaps their handles before draft creation. Import failure is reported without blocking the selected project's existing flow library. - -Execution uses immutable publications rather than the mutable editor row. Runs select a pinned publication or the latest published version, then use durable queue claims, leases, bounded quotas, timeout/cancellation propagation, and numbered retry attempts. Expired external attempts with unknown outcomes require operator attention and are not silently replayed. See [Node Flow Durable Execution](./node-flow-durable-execution.md). - -HTTP and future custom-node requests share one HTTPS-first egress policy with URL-credential rejection, DNS and redirect revalidation, private-network and metadata blocking, host/port allowlists, bounded content, retries, timeouts, and rate limits. See [Built-ins and External-Effect Security](./node-flow-builtins-and-security.md). +| `input` | Emits the run input object. | +| `set_fields` | Merges upstream object output with configured `fields` or `values`; set `replace: true` to ignore upstream output. | +| `template` | Renders `template` or `prompt` into `outputKey` (default `text`). | +| `provider_prompt` | Renders a prompt and calls an existing CLI provider configuration through `ProviderExecutionService`. | +| `http_request` | Performs bounded HTTP/HTTPS requests with method, URL, headers, query, body, timeout, and optional JSON path extraction. | +| `condition`, `switch` | Select one explicit output branch; non-selected branches are persisted as skipped. | +| `foreach` | Validates and emits a bounded item list. | +| `merge` | Combines active inputs with `object`, `array`, or `first` strategy. | +| `delay` | Waits for a bounded cancellable duration. | +| `approval` | Persists an operator decision gate. | +| `email_draft`, `email_send` | Creates a draft, or sends only after approval through the idempotent outbox. | +| `execute_subflow` | Executes a same-project published subflow with recursion bounds. | +| `webhook_trigger` | Emits authenticated webhook input. | +| `output` | Selects final output from a path, configured fields, or upstream output. | + +Template interpolation reads from `{{ input.path }}` and `{{ nodes.nodeId.path }}`. Node config is built from widget defaults, node `data`, and optional `data.values`, with later values overriding defaults. + +Provider prompt nodes require a configured CLI provider. HTTP nodes require HTTPS unless HTTP is explicitly enabled and support `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, and `HEAD`. Requests pass through the shared SSRF, redirect, DNS, response-size, content-type, retry, timeout, and rate-limit policy described in [Node Flow Built-ins and External-Effect Security](./node-flow-builtins-and-security.md). + +## Invocation Tracking + +Node flows use `execution_invocations` as the observable runtime surface: + +- each flow run creates a parent invocation with `type: "node_flow"` +- externally observable node steps create invocation rows with `type: "node_flow_node"` +- `node_flow_runs.execution_invocation_id` links the run record to the parent invocation +- `node_flow_node_runs.execution_invocation_id` links provider and HTTP node rows to their invocation record + +Only `provider_prompt` and `http_request` nodes currently create `node_flow_node` invocation rows. Deterministic local nodes still create `node_flow_node_runs` rows, but do not create extra execution invocations. + +Provider prompt nodes pass an existing invocation id into `ProviderExecutionService` and disable prompt/assistant transcript capture for raw prompt content. HTTP nodes append a redacted request summary. Flow run inputs, outputs, node input/output payloads, trigger payloads, graph data, and MCP responses redact secret-shaped keys such as `apiKey`, `authorization`, `cookie`, `password`, `secret`, and `token`. + +## Failure Semantics + +A failed node fails the flow and persists skipped records for downstream descendants by default. If a node has `data.continueOnError = true`, the failed node records `{ "error": "" }` as output and downstream nodes may continue. + +Cancellation records cancelled node rows for the current and remaining nodes. At completion, the parent invocation is updated to `completed`, `failed`, or `cancelled` to match the flow outcome. + +## Scheduling + +Scheduler entries with `targetType: "node_flow"` persist an explicit `versionSelection`: pinned schedules continue to execute version N after N+1 is published, while latest-published schedules resolve the newest publication at dispatch time. Legacy `flowVersion` values normalize to pinned selection and are executable semantics, not audit-only metadata. Ownership is validated when entries are created or updated and again before due-run execution. + +Due runs call `NodeFlowRuntimeService.runFlow` with `triggerType = "scheduler"` and trigger payload metadata for the scheduler entry id, scheduled occurrence time, target type, and persisted flow version when present. Node-flow schedules advance only when `runFlow` returns a run status of `succeeded`. Returned `failed` or `cancelled` runs mark the scheduler entry `failed` with the run error and still count the attempted occurrence in `lastRunAt` and `runCount`; runtime startup rejections mark failure without creating a false successful schedule run. + +## Agent Skill Attachment + +`node_flow_agent_skills` exposes a saved flow to an agent preset as a repeatable skill. Attachment stores `flow_id`, `project_id`, `agent_preset_id`, `skill_name`, `description`, and timestamps. + +Attachment does not copy the graph into the agent preset, and detach removes only the binding. The flow remains project-owned and can still be edited, scheduled, manually run, or attached to other agents. + +## Agent Design Guidance + +Specialist agents designing node flows should adapt workflows to Code UX instead of copying n8n or another tool one node at a time. + +Use these rules: + +- Model the repeatable outcome first, then choose the smallest Code UX graph that captures the inputs, provider calls, HTTP calls, transformations, and final output. +- Use the governed executable built-ins listed in the runtime table when the workflow needs to execute today. A registered custom definition is executable only when its validated immutable artifact and custom runtime are available. Treat unknown types, legacy browser-only kinds, and non-executable manifests as planned or unavailable definitions. +- Put operator-editable values in `inputSchema` or per-node `widgetSchema` fields. Do not bury frequently changed values in opaque JSON blobs. +- Use `secretRef` widgets and secret reference strings for credentials. Do not place raw API keys, bearer tokens, cookies, passwords, or private headers in graph metadata, node data, widget defaults, run input, or examples. +- Validate every node field before saving: required prompt/template/url fields, finite numeric limits, supported HTTP method, JSON object input, and select defaults that match options. +- Keep flows deterministic and rerunnable. Avoid hidden dependence on local time, ambient chat state, or one-off sprint context unless it is explicitly passed as JSON input. +- Preserve inspection value. Name nodes for the operation they perform, keep edges acyclic, and make the output node return the artifact another operator or agent will actually consume. + +## Graph v2 contract and migration + +Graph v2 is the single workflow model used by backend, MCP, runtime, and dashboard. It adds `schemaVersion: 2`, stable definition references, typed ports and flow schemas, credential-id bindings, retry and timeout policies, capability and side-effect metadata, disabled state, and optional immutable publication metadata. Plaintext credentials, secret-shaped fields, generated source, and custom code are not valid graph data. + +The executable registry contains the original deterministic/provider/HTTP nodes plus `condition`, `switch`, `foreach`, `merge`, `delay`, `approval`, `email_draft`, `email_send`, `execute_subflow`, and `webhook_trigger`. Unregistered custom types remain non-executable. + +Backend Graph v1 migration retains the exact prior version and appends deterministic v2. Invalid legacy members are carried across the migration boundary so validation can report their original paths rather than silently dropping them. Browser canvas v1 migration returns the untouched legacy snapshot separately from the normalized graph. + +## Dashboard and security prerequisites + +Outside development builds, the Nodes workspace is enabled only when `VITE_CODEUX_FEATURE_NODES`, `VITE_CODEUX_NODE_FLOW_BACKEND`, and `VITE_CODEUX_AUTOMATION_SECURITY` are true. These flags expose the surface; they do not replace runtime dependencies. Provider execution, credential resolution, outbound HTTP, approval-gated email, webhook ingress, and custom-node containers each require their corresponding configured service and security policy. + +The dashboard exposes credential binding ids, declared kinds, scopes, and status metadata only. Resolved values stay at the credential broker/runtime boundary and are redacted before invocation messages, attempt payloads, diagnostics, and route responses are persisted or rendered. Policy review must surface requested capabilities and external side effects before publication, and publication requires a valid current draft with all required bindings satisfied. diff --git a/docs-web/content/docs/architecture-custom-dashboard-foundation.mdx b/docs-web/content/docs/architecture-custom-dashboard-foundation.mdx index ae43447fa4..ba3a63d42d 100644 --- a/docs-web/content/docs/architecture-custom-dashboard-foundation.mdx +++ b/docs-web/content/docs/architecture-custom-dashboard-foundation.mdx @@ -15,6 +15,12 @@ Primary records: Dashboard status values are `draft`, `validating`, `validated`, `published`, `rejected`, and `archived`. Validation status values are `queued`, `building`, `running`, `passed`, `failed`, and `cancelled`. +### Feature baseline and bounded addition + +Repository history provides the negative baseline for this subsystem: at the pre-feature `dev` commit `716ac2c55`, `CustomDashboardManifest` had no `credentialSlots`, and mutable dashboard and immutable revision records had no `credentialBindings` or binding revision. The implemented change is intentionally limited to bounded manifest declarations, credential-ID bindings in dedicated draft/revision columns, metadata-only compatibility review, optimistic binding mutation, and validation/publication gates. It does not migrate provider secrets and it does not add custom-dashboard secret injection. + +Declarations are normalized and bounded for count, slot ID, label, phase (`build` or `runtime`), allowed kinds, and required capabilities. Bindings contain only `slotId` and `credentialId`; generic draft/revision writes cannot set them, and immutable revisions snapshot them. The phase is policy metadata for review and validation, not permission to inject a value into build or runtime artifacts. + ## Persistence SQLite tables are created in both the initial schema and startup migrations: @@ -59,7 +65,9 @@ Validation flow: - A validation session is marked `passed` only after install, build, start, and root URL health checks succeed. Build/start/health failures are recorded as failed validation reports with bounded log excerpts. - Runtime metadata persists the workspace path, log path, host port, container id/name, image, validation URL path, commands, latest error/log excerpt, and a browser-ready Vite `dist` artifact for passed revisions so the published viewer can render TSX-based drafts without a live validation container. -Validation does not publish or activate dashboards. A successful run only marks the revision validation status as `passed`; publication remains gated by `publishRevision`. REST and MCP publication re-run metadata-only binding review immediately before calling the repository, then require either a revision already marked `passed` with a valid report or an explicit passed validation session for that revision. Failed binding review returns sanitized slot-specific issues without credential IDs or values; queued/running/cancelled validation, missing state, and cross-revision sessions are rejected before the publication pointer changes. +Validation does not publish or activate dashboards. A successful run only marks the revision validation status as `passed`; publication remains gated by `publishRevision`. REST and MCP publication re-run metadata-only binding review immediately before calling the repository, then require either a revision already marked `passed` with a valid report or an explicit passed validation session for that revision. Failed binding review, queued/running/cancelled validation, missing state, and cross-revision sessions are rejected before the publication pointer changes. + +No custom-dashboard service resolves credential plaintext. Build workspaces, generated files and Vite artifacts, Docker arguments/mounts/environment, validation reports/logs, generic REST/MCP records, viewer configuration, iframe `srcdoc`, data-bridge payloads, and `postMessage` traffic receive neither credential values nor binding IDs. Only the dedicated metadata-management response may return binding IDs alongside non-secret credential metadata. ## REST and MCP Surface diff --git a/docs-web/content/docs/architecture-node-flow-builtins-and-security.mdx b/docs-web/content/docs/architecture-node-flow-builtins-and-security.mdx index 80324199ed..18174020be 100644 --- a/docs-web/content/docs/architecture-node-flow-builtins-and-security.mdx +++ b/docs-web/content/docs/architecture-node-flow-builtins-and-security.mdx @@ -1,29 +1,66 @@ # Node Flow Built-ins and External-Effect Security -The governed catalog adds deterministic branches, bounded collection processing, durable approvals, and replay-safe external effects while keeping the versioned definition registry as the executable authority. +The governed built-in catalog extends publication-based node-flow execution with deterministic control nodes and durable boundaries for external effects. The definition registry remains the executable authority; a graph can only run a node when its versioned manifest is registered and executable. -## Control and integration nodes +## Built-in catalog -- `condition` selects `true` or `false`; `switch` selects one named case or `default`. Unselected branches persist as skipped node runs. -- `foreach` rejects inputs above its configured bound (never more than 1,000), then runs downstream nodes once per deterministic logical item. Configured concurrency defaults to one and is capped at 64; zero items explicitly select `empty`. `merge` supports `object`, `array`, and `first` strategies. -- `delay` is cancellable and capped at one hour. `execute_subflow` requires same-project ownership, rejects direct self-reference, and caps depth at eight. -- `approval` persists an idempotent operator decision and continues the exact pinned run after approval. `email_draft` never sends. `email_send` requires approval and uses the idempotent outbox. -- `webhook_trigger` emits payloads accepted through secret-authenticated webhook ingress. +| Node | Contract | +| --- | --- | +| `condition` | Evaluates a bounded operator and selects exactly the `true` or `false` output port. Unselected branches persist as skipped node runs. | +| `switch` | Evaluates no more than 100 configured cases and selects one named case or `default`. | +| `foreach` | Validates an array, rejects inputs above the configured bound (at most 1,000), and executes the selected downstream branch once per logical item with bounded concurrency. | +| `merge` | Combines active upstream values with `object`, `array`, or `first` strategy. | +| `delay` | Waits for a cancellable duration from zero through one hour. | +| `approval` | Creates or reuses a durable approval keyed by run, node, and logical item. | +| `email_draft` | Produces a draft only and never contacts a provider. | +| `email_send` | Requires an approved decision, then dispatches through the idempotent outbox. | +| `execute_subflow` | Executes a published flow owned by the same project, rejects direct self-reference, and caps nesting at eight. | +| `webhook_trigger` | Emits input accepted by a secret-authenticated webhook configuration. | -## Network policy +The existing `input`, `set_fields`, `template`, `provider_prompt`, `http_request`, and `output` nodes retain their previous contracts. Typed manifest ports identify branch handles, many-valued merge inputs, and trigger outputs. Branch routing only runs a node when at least one incoming edge is active, allowing merges to join a selected path without treating an unselected sibling as a failure. -HTTP nodes and future custom nodes use the same `EgressPolicyService`. HTTPS is required unless HTTP is explicitly enabled. Private, loopback, link-local, metadata, multicast, and other non-public addresses remain blocked in both modes. Credentials in URLs and raw restricted headers are rejected. +## Credential-bound execution -Every redirect is manually revalidated. DNS is checked for private results and rebinding. Host and port allowlists, response-size and content-type limits, propagated cancellation and timeouts, capped retries, idempotency requirements for unsafe retry, normalized headers, and per-key rate windows keep requests bounded. +Versioned definition manifests declare credential slots by required state, allowed kinds, and required capabilities. Draft review and publication use metadata-only broker compatibility; the canonical graph stores only slot-to-credential-ID bindings. Required missing bindings and bindings denied for unavailable custody, configuration, status, project access, kind, or capability stop publication. Optional unbound slots remain valid. -## OAuth, approvals, and outbox +At runtime, the immutable published graph is revalidated and the broker repeats authorization immediately before resolving a value for the active attempt. A revoked, restricted, rebound, wrong-kind, insufficiently capable, or unavailable credential fails the attempt closed. Exact resolved values are redacted from built-in output, invocation/attempt records, diagnostics, retries, HTTP/provider responses, and external-effect persistence; neither publication nor MCP inspection injects or returns plaintext. -Pending approvals preserve the run, governed node, logical item, and numbered attempt. Approved decisions resume at that node boundary; rejected and expired decisions terminate durably. Repeated decisions and restart recovery do not create a second approval request, attempt, or external delivery. +Foreach assigns deterministic logical-item identities from the published node id and item index. Each downstream node run and numbered attempt persists that identity together with the item-specific input. The `concurrency` setting defaults to one and is capped at 64; `maxItems` is a rejection bound rather than a truncation rule. A zero-item input selects `empty`, while the `items` branch is persisted as skipped. Per-item failures retain their own retry history, successful siblings are not replayed during approval or restart continuation, and aggregated output preserves input order. -Foreach descendant node runs and attempts persist item-specific inputs and logical identity. Retries remain item-local, completed siblings are reconstructed rather than replayed after restart, and aggregation preserves input order. +## Governed egress -OAuth authorization uses PKCE S256 and short-lived AES-256-GCM state tied to an allowlisted callback origin. Tokens live behind the connection store, rotate on refresh, enforce scopes and expiry, and are never written into graph JSON or agent-visible output. Revocation, reconnect, and health checks expose no token values. +`EgressPolicyService` is the single request boundary for HTTP nodes and future custom-node network calls. HTTPS is required by default. A node must explicitly opt into HTTP, and even then private networking remains blocked. The service rejects credentials embedded in URLs; loopback, private, link-local, carrier-grade NAT, benchmarking, multicast, and cloud-metadata addresses; metadata hostnames; restricted raw headers; and ports or hosts outside configured allowlists. -Approvals are unique per run, node, and logical item. Outbox entries use a unique key derived from publication, run, node, and logical item, and store the provider message id after success. A restart while an entry is sending changes it to `attention_required`; Code UX does not automatically replay an unknown provider outcome. +Each redirect is handled manually and fully revalidated. DNS is resolved twice before dispatch, and a changed or newly private result is treated as rebinding. Cross-origin redirects remove credential headers. Response bodies are streamed into a bounded buffer, content types are allowlisted, timeouts and caller cancellation propagate, retry counts are capped, unsafe methods require an idempotency key before retry, and an in-process rate window bounds requests per project and host. -Webhook configuration returns a newly rotated path token and secret while persisting only their hashes. Ingress requires `x-codeux-webhook-secret` and dispatches the latest published flow version. +## OAuth boundary + +`OAuthBroker` implements authorization-code flow with PKCE S256. Authorization state is authenticated AES-256-GCM ciphertext containing a short expiry, callback origin, redirect URI, verifier, connection id, and nonce. Callback origins must be explicitly allowlisted and match the state. Token exchange and refresh results are stored behind an `OAuthConnectionStore`; access and refresh tokens are returned only to provider-bound execution code, never to graph JSON or agent-visible output. + +Refresh happens shortly before expiry and rotates the stored refresh token when the provider returns one. Required scopes are checked before access. Revocation deletes local state after provider revocation; reconnect begins from a revoked local connection; health checks refresh when necessary and expose only health, expiry, and scopes. + +## Approvals and outbox + +`automation_approvals` persists pending and terminal decisions. Repeating the same run, node, and logical item returns the existing decision, so restarts do not create a second prompt. Repeating an identical decision is also idempotent. Approval or rejection through the decision endpoint resumes or terminates the exact waiting run; approval preserves its publication, run id, logical item, and attempt number. Email sending is approval-gated by default; `email_draft` is the non-irreversible default. + +`automation_outbox` has a unique SHA-256 idempotency key derived from publication id, run id, node id, and logical item. Provider message ids are stored after success. A process restart while an entry is `sending` changes it to `attention_required`, because the provider may have accepted the operation; Code UX does not replay an unknown external outcome automatically. + +## Webhook routes + +Creating `POST /api/node-flows/:flowId/webhook` rotates and returns a path token and secret once. Only their hashes are persisted. `POST /api/webhooks/node-flows/:pathToken` requires the secret in `x-codeux-webhook-secret`, uses constant-time digest comparison, and dispatches the latest published version with `triggerType: webhook`. The response returns only run identity and status. + +Example condition edges use explicit handles: + +```json +{ + "nodes": [ + { "id": "check", "type": "condition", "title": "Check", "data": { "path": "input.enabled" } }, + { "id": "draft", "type": "email_draft", "title": "Draft", "data": { "to": "owner@example.test", "subject": "Ready", "body": "Review this draft." } }, + { "id": "done", "type": "output", "title": "Done" } + ], + "edges": [ + { "fromNodeId": "check", "fromHandle": "true", "toNodeId": "draft" }, + { "fromNodeId": "check", "fromHandle": "false", "toNodeId": "done" } + ] +} +``` diff --git a/docs-web/content/docs/architecture-node-flows.mdx b/docs-web/content/docs/architecture-node-flows.mdx index 8f1537e636..0b68846784 100644 --- a/docs-web/content/docs/architecture-node-flows.mdx +++ b/docs-web/content/docs/architecture-node-flows.mdx @@ -1,30 +1,138 @@ # Node Flows -Node flows are project-owned, versioned Graph v2 workflows. +Node flows are project-scoped, repeatable workflow graphs for turning an operator or agent-defined procedure into a saved Code UX workflow. They are not a generic n8n compatibility layer. A good flow uses Code UX concepts, project-owned records, provider settings, execution invocations, and editable widget schemas so the same workflow can be inspected, rerun, scheduled, and attached to agents. -Authenticated dashboard routes resolve persisted project ownership from flow, run, or approval ids before authorizing the request. Drafts, publications, comparisons, rollbacks, attachments, webhook configuration, debugger data, attempts, cancellation, retry, and approvals cannot be accessed by presenting a different body or query project id. Webhook ingress remains on its path-token and webhook-secret scheme, with dashboard host and browser-origin protections still enforced. +The foundation page in [Node Flow Foundation](/docs/architecture-node-flow-foundation) lists the low-level contracts. This page describes the end-to-end architecture and runtime expectations for developers and specialist agents. -## Implemented runtime nodes +## Data Model -| Type | Execution | +Node-flow persistence is owned by `NodeFlowRepository` and stored in SQLite: + +| Table | Purpose | +| --- | --- | +| `node_flows` | Current project-scoped flow record: id, project id, title, description, normalized `graph_json`, current version, and timestamps. | +| `node_flow_versions` | Immutable edit snapshots written on create and every update. | +| `node_flow_publications` | Immutable executable graph and execution-policy snapshots selected by pinned or latest-published runs. | +| `node_flow_agent_skills` | Agent attachment table keyed by flow and agent preset. It stores the skill display name and description used when exposing the flow as a repeatable agent capability. | +| `node_flow_runs` | Flow run records with status, version, trigger type, redacted trigger payload, redacted input/output, error message, timestamps, and optional `execution_invocation_id`. | +| `node_flow_node_runs` | Per-node run records with status, node id, redacted input/output, error message, timestamps, and optional `execution_invocation_id`. | +| `node_flow_node_attempts` | Numbered attempts with executor/invocation identity, artifact digest, credential ids, redacted payloads, failure class, and retry decision. | + +All graphs, widget schemas, run inputs, outputs, and trigger payloads are stored as JSON text and hydrated into typed contracts at the repository boundary. Flow, version, run, and attachment records belong to a project. Agent attachment operations verify that the target agent preset belongs to the same project as the flow. + +Authenticated dashboard requests resolve project ownership from the persisted flow, run, or approval record before role and project authorization. This applies to ID-only draft, publication, comparison, rollback, attachment, webhook-configuration, run debugger, attempt, cancellation, retry, and approval routes; a caller-supplied body or query project id is not treated as proof of ownership. Webhook ingress is the exception to dashboard bearer authentication and continues to use its path-token and webhook-secret scheme, while dashboard host and browser-origin protections still apply. + +## Graph Contract + +The shared contract lives in `src/contracts/node-flow-types.ts`. + +A `NodeFlowGraph` contains: + +- `nodes`: stable node ids, string node `type`, title, optional description, optional position, optional `widgetSchema`, and JSON `data`. +- `edges`: directed links from `fromNodeId` to `toNodeId`. +- `inputSchema`: optional graph-level widget schema for run input. +- `metadata`: optional JSON object for non-secret descriptive data. + +Validation is owned by `src/domain/node-flows/node-flow-validation.ts`. It normalizes ids, labels, positions, widget defaults, and graph shape; rejects missing node/edge arrays; rejects duplicate node ids; rejects edges that point at missing nodes; requires at least one node; and rejects cycles. Widget validation supports `text`, `textarea`, `number`, `boolean`, `select`, `json`, `secretRef`, and `keyValue` fields. + +Migration and validation treat persisted Graph v1 and canonical Graph v2 as untrusted input. Malformed collection members are rejected at their original index, such as `nodes[1].ports[0]` or `edges[2]`, while structurally valid siblings remain available to the rest of normalization. Definition references, credential bindings, capabilities, policies, port and graph schemas, and JSON metadata emit deterministic field-level issues instead of throwing. Revalidating the same graph produces the same ordered issue list. + +Validation requires every node's type/version reference to resolve through the registry and rejects unknown definitions. Runtime execution then dispatches according to the registered definition's executable state and execution kind; a planning concept is not runnable merely because it has a string type. + +The dashboard uses the same backend-owned Graph v2 record as the runtime. The selected project controls library loading; no project means no flow, credential, publication, or run requests. The registry list endpoint returns flat palette summaries, while the node-type detail endpoint returns a complete `NodeDefinitionManifest` with nested `ui`, schemas, policies, documentation, and deprecation metadata. The inspector consumes that full manifest. Draft saves use optimistic `draftRevision` checks and surface conflicts without overwriting the newer record. + +`dashboard/src/v2/lib/nodes-canvas-state.ts` remains only a compatibility and pure graph-state layer. Its legacy browser graph can be imported once into a project draft. The adapter translates `trigger`/`agent`/`task` into registered `input`/`set_fields`/`provider_prompt` nodes, remaps legacy handles to governed ports, and retains non-secret canvas metadata. Import failure is isolated from the normal library load; only a successful draft creation removes the old graph key and records the project marker. Browser storage is not the workflow source of truth. + +### Credential binding lifecycle + +Each versioned node definition is the slot-policy authority: every slot declares whether it is required, its allowed credential kinds, and all required capabilities. The picker lists project-visible metadata, then filters each candidate through secure-backend readiness, configured/active state, project access, kind, and capability compatibility. It never resolves a value. + +`NodeFlowNode.credentialBindings` is the only persisted binding source. Selecting, replacing, or unbinding a credential changes the matching `{ slot, credentialId }` entry in the complete canonical graph and saves with the current `draftRevision`. The dashboard adopts the returned graph and revision, then refreshes governed review. A `409`-style revision conflict refreshes the latest draft and requires a deliberate retry; it never replays a stale binding over sibling changes. + +Required unbound slots and any bound credential denied by backend readiness, configuration, active status, project access, allowed kind, or required capabilities block publication. Optional unbound slots do not. Runtime revalidates the immutable publication and repeats the same policy immediately before direct credential-ID resolution, so revocation, restriction, rotation/rebinding races, missing custody, or incompatible policy deny the node attempt rather than injecting stale plaintext. Graph, review, publication, MCP, and dashboard payloads contain IDs and non-secret policy metadata only. + +## Runtime + +`NodeFlowRuntimeService.runFlow(projectId, flowId, input, options)` resolves an explicit pinned or latest-published snapshot, revalidates that immutable graph, claims a durable lease, and executes nodes in topological order. See [Node Flow Durable Execution](/docs/architecture-node-flow-durable-execution) for queue, retry, lease, recovery, quota, and redaction guarantees. + +Runtime-supported node types are: + +| Node type | Behavior | | --- | --- | -| `input` | Emits run input. | -| `set_fields` | Transforms object fields. | -| `template` | Renders text templates. | -| `provider_prompt` | Invokes a configured CLI provider. | -| `http_request` | Performs a bounded HTTP/HTTPS request. | -| `condition`, `switch` | Selects one explicit output branch and persists unselected branches as skipped. | -| `foreach`, `merge` | Bounds item fan-out and combines active inputs with an explicit strategy. | -| `delay`, `approval` | Waits with cancellation or persists an operator decision gate. | -| `email_draft`, `email_send` | Produces a draft, or sends only after approval through the idempotent outbox. | -| `execute_subflow` | Executes a same-project published flow with recursion bounds. | -| `webhook_trigger` | Emits secret-authenticated webhook input. | -| `output` | Selects the result. | - -These are the executable definitions. Other custom palette concepts remain non-executable until a versioned handler is registered. Graph v1 migration preserves the legacy snapshot and appends deterministic v2. Both migrated v1 and canonical v2 graphs are validated as untrusted input: malformed nested members fail closed with stable paths at their original array indices, valid siblings are retained where safe, and repeated validation returns the same ordered issues instead of throwing. - -The separate browser-canvas compatibility bridge translates legacy `trigger`, `agent`, and `task` kinds into registered `input`, `set_fields`, and `provider_prompt` nodes and remaps their handles before draft creation. Import failure is reported without blocking the selected project's existing flow library. - -Execution uses immutable publications rather than the mutable editor row. Runs select a pinned publication or the latest published version, then use durable queue claims, leases, bounded quotas, timeout/cancellation propagation, and numbered retry attempts. Expired external attempts with unknown outcomes require operator attention and are not silently replayed. See [Node Flow Durable Execution](/docs/architecture-node-flow-durable-execution). - -HTTP and future custom-node requests share one HTTPS-first egress policy with URL-credential rejection, DNS and redirect revalidation, private-network and metadata blocking, host/port allowlists, bounded content, retries, timeouts, and rate limits. See [Built-ins and External-Effect Security](/docs/architecture-node-flow-builtins-and-security). +| `input` | Emits the run input object. | +| `set_fields` | Merges upstream object output with configured `fields` or `values`; set `replace: true` to ignore upstream output. | +| `template` | Renders `template` or `prompt` into `outputKey` (default `text`). | +| `provider_prompt` | Renders a prompt and calls an existing CLI provider configuration through `ProviderExecutionService`. | +| `http_request` | Performs bounded HTTP/HTTPS requests with method, URL, headers, query, body, timeout, and optional JSON path extraction. | +| `condition`, `switch` | Select one explicit output branch; non-selected branches are persisted as skipped. | +| `foreach` | Validates and emits a bounded item list. | +| `merge` | Combines active inputs with `object`, `array`, or `first` strategy. | +| `delay` | Waits for a bounded cancellable duration. | +| `approval` | Persists an operator decision gate. | +| `email_draft`, `email_send` | Creates a draft, or sends only after approval through the idempotent outbox. | +| `execute_subflow` | Executes a same-project published subflow with recursion bounds. | +| `webhook_trigger` | Emits authenticated webhook input. | +| `output` | Selects final output from a path, configured fields, or upstream output. | + +Template interpolation reads from `{{ input.path }}` and `{{ nodes.nodeId.path }}`. Node config is built from widget defaults, node `data`, and optional `data.values`, with later values overriding defaults. + +Provider prompt nodes require a configured CLI provider. HTTP nodes require HTTPS unless HTTP is explicitly enabled and support `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, and `HEAD`. Requests pass through the shared SSRF, redirect, DNS, response-size, content-type, retry, timeout, and rate-limit policy described in [Node Flow Built-ins and External-Effect Security](/docs/architecture-node-flow-builtins-and-security). + +## Invocation Tracking + +Node flows use `execution_invocations` as the observable runtime surface: + +- each flow run creates a parent invocation with `type: "node_flow"` +- externally observable node steps create invocation rows with `type: "node_flow_node"` +- `node_flow_runs.execution_invocation_id` links the run record to the parent invocation +- `node_flow_node_runs.execution_invocation_id` links provider and HTTP node rows to their invocation record + +Only `provider_prompt` and `http_request` nodes currently create `node_flow_node` invocation rows. Deterministic local nodes still create `node_flow_node_runs` rows, but do not create extra execution invocations. + +Provider prompt nodes pass an existing invocation id into `ProviderExecutionService` and disable prompt/assistant transcript capture for raw prompt content. HTTP nodes append a redacted request summary. Flow run inputs, outputs, node input/output payloads, trigger payloads, graph data, and MCP responses redact secret-shaped keys such as `apiKey`, `authorization`, `cookie`, `password`, `secret`, and `token`. + +## Failure Semantics + +A failed node fails the flow and persists skipped records for downstream descendants by default. If a node has `data.continueOnError = true`, the failed node records `{ "error": "" }` as output and downstream nodes may continue. + +Cancellation records cancelled node rows for the current and remaining nodes. At completion, the parent invocation is updated to `completed`, `failed`, or `cancelled` to match the flow outcome. + +## Scheduling + +Scheduler entries with `targetType: "node_flow"` persist an explicit `versionSelection`: pinned schedules continue to execute version N after N+1 is published, while latest-published schedules resolve the newest publication at dispatch time. Legacy `flowVersion` values normalize to pinned selection and are executable semantics, not audit-only metadata. Ownership is validated when entries are created or updated and again before due-run execution. + +Due runs call `NodeFlowRuntimeService.runFlow` with `triggerType = "scheduler"` and trigger payload metadata for the scheduler entry id, scheduled occurrence time, target type, and persisted flow version when present. Node-flow schedules advance only when `runFlow` returns a run status of `succeeded`. Returned `failed` or `cancelled` runs mark the scheduler entry `failed` with the run error and still count the attempted occurrence in `lastRunAt` and `runCount`; runtime startup rejections mark failure without creating a false successful schedule run. + +## Agent Skill Attachment + +`node_flow_agent_skills` exposes a saved flow to an agent preset as a repeatable skill. Attachment stores `flow_id`, `project_id`, `agent_preset_id`, `skill_name`, `description`, and timestamps. + +Attachment does not copy the graph into the agent preset, and detach removes only the binding. The flow remains project-owned and can still be edited, scheduled, manually run, or attached to other agents. + +## Agent Design Guidance + +Specialist agents designing node flows should adapt workflows to Code UX instead of copying n8n or another tool one node at a time. + +Use these rules: + +- Model the repeatable outcome first, then choose the smallest Code UX graph that captures the inputs, provider calls, HTTP calls, transformations, and final output. +- Use the governed executable built-ins listed in the runtime table when the workflow needs to execute today. A registered custom definition is executable only when its validated immutable artifact and custom runtime are available. Treat unknown types, legacy browser-only kinds, and non-executable manifests as planned or unavailable definitions. +- Put operator-editable values in `inputSchema` or per-node `widgetSchema` fields. Do not bury frequently changed values in opaque JSON blobs. +- Use `secretRef` widgets and secret reference strings for credentials. Do not place raw API keys, bearer tokens, cookies, passwords, or private headers in graph metadata, node data, widget defaults, run input, or examples. +- Validate every node field before saving: required prompt/template/url fields, finite numeric limits, supported HTTP method, JSON object input, and select defaults that match options. +- Keep flows deterministic and rerunnable. Avoid hidden dependence on local time, ambient chat state, or one-off sprint context unless it is explicitly passed as JSON input. +- Preserve inspection value. Name nodes for the operation they perform, keep edges acyclic, and make the output node return the artifact another operator or agent will actually consume. + +## Graph v2 contract and migration + +Graph v2 is the single workflow model used by backend, MCP, runtime, and dashboard. It adds `schemaVersion: 2`, stable definition references, typed ports and flow schemas, credential-id bindings, retry and timeout policies, capability and side-effect metadata, disabled state, and optional immutable publication metadata. Plaintext credentials, secret-shaped fields, generated source, and custom code are not valid graph data. + +The executable registry contains the original deterministic/provider/HTTP nodes plus `condition`, `switch`, `foreach`, `merge`, `delay`, `approval`, `email_draft`, `email_send`, `execute_subflow`, and `webhook_trigger`. Unregistered custom types remain non-executable. + +Backend Graph v1 migration retains the exact prior version and appends deterministic v2. Invalid legacy members are carried across the migration boundary so validation can report their original paths rather than silently dropping them. Browser canvas v1 migration returns the untouched legacy snapshot separately from the normalized graph. + +## Dashboard and security prerequisites + +Outside development builds, the Nodes workspace is enabled only when `VITE_CODEUX_FEATURE_NODES`, `VITE_CODEUX_NODE_FLOW_BACKEND`, and `VITE_CODEUX_AUTOMATION_SECURITY` are true. These flags expose the surface; they do not replace runtime dependencies. Provider execution, credential resolution, outbound HTTP, approval-gated email, webhook ingress, and custom-node containers each require their corresponding configured service and security policy. + +The dashboard exposes credential binding ids, declared kinds, scopes, and status metadata only. Resolved values stay at the credential broker/runtime boundary and are redacted before invocation messages, attempt payloads, diagnostics, and route responses are persisted or rendered. Policy review must surface requested capabilities and external side effects before publication, and publication requires a valid current draft with all required bindings satisfied. diff --git a/docs-web/content/docs/operations-credential-security.mdx b/docs-web/content/docs/operations-credential-security.mdx index 41fa50e335..48665eb98f 100644 --- a/docs-web/content/docs/operations-credential-security.mdx +++ b/docs-web/content/docs/operations-credential-security.mdx @@ -1,50 +1,76 @@ # Automation Credential Security -Code UX resolves canonical node credential IDs and named project binding keys through the credential broker. Stored values are not exposed to nodes, dashboard reads, MCP payloads, agent context, run inspection records, or access audits. +Code UX stores automation credentials through a broker rather than exposing secret values to node definitions, dashboard reads, MCP payloads, agent context, or run inspection records. Canonical node bindings reference credential metadata by ID; only the broker can resolve the value at execution time after project and capability checks. Named project binding keys use the same broker for other automation consumers. ## Scope and policy -- Project credentials are owned by one project. -- Global credentials require an explicit project allowlist and retain the configuring project as their management owner. Other allowlisted projects may bind and resolve the credential but cannot mutate it. -- The credential kind must be allowed, and both the binding and credential must approve every declared capability before one secret read. +- Project credentials can be managed only through their owning project. +- Global credentials are opt-in and require an explicit project allowlist containing the configuring project. The configuring project remains the credential's management owner after promotion; other allowlisted projects may bind and resolve it but cannot rotate, replace, revoke, promote, or restrict it. +- Resolution succeeds only when the credential kind is allowed and both the credential and binding approve every declared capability. Authorization is completed before the broker performs its single secret read. - Revoked, unavailable, missing, cross-project, or insufficiently capable credentials fail closed. Node-flow definition slots explicitly declare required/optional state, allowed kinds, and required capabilities. Draft review and every publication path use the broker's metadata-only compatibility assessment; runtime sends the same declaration to direct credential-id resolution immediately before execution. Graph `credentialBindings` are canonical. The legacy credential-request endpoint records no binding and identifies its result as non-persistent. -Create, rotate, and replace requests are write-only. API responses contain configuration and status metadata but never stored values. +The dashboard accepts secret values only on create, rotate, and replace requests. Responses contain configuration, scope, status, key-version, and validation metadata but never stored values. Access-event rows contain identifiers, binding keys, capabilities, outcomes, and denial reasons; they never contain secret material. -Create requests explicitly declare kind, scope, capabilities, and an allowlist (empty for project credentials). Runtime validation bounds names, identifiers, capabilities, list counts, and secret size (64 KiB UTF-8). Malformed arrays, unknown mutation fields, and control characters are rejected rather than coerced. +The Settings Integrations catalog exposes this broker as its first standard card. The card derives unavailable, ready/unconfigured, and configured states from backend health and project-visible metadata; **Manage** opens a project-aware detail view without rendering a secret, request body, or raw server error. Allowlisted non-owner projects can understand and use compatible global credentials but see management actions disabled. -Every lifecycle mutation includes `expectedVersion`. The only mutable descriptive field is the bounded name; kind and management ownership remain immutable. Restrictions may remove allowlisted projects or capabilities but cannot add them. Project-to-global promotion is the explicit scope expansion and requires managing-project authority, `confirmScopeExpansion: true`, the current version, and an allowlist of existing projects that retains the managing project. Current-version repeated revocation is idempotent; stale versions conflict. +Create controls require deliberate capability selection and explicit project or global scope. Global allowlists retain the management owner, and scope-expanding creation or promotion is confirmed. Rename, test, rotation/replacement, restriction, promotion, and revocation report typed inline status, disable overlapping actions, and refresh after stale-version conflicts. Destructive and scope-expanding actions use keyboard-operable confirmation dialogs with focus restoration. + +All create, rotate, and replacement fields are controlled write-only inputs. They are never hydrated from metadata and are cleared after every submission outcome, project change, and component teardown. Credential metadata drafts and browser stores do not receive secret values. + +Management inputs are validated at runtime rather than trusted from TypeScript types. Create requests must explicitly declare kind, scope, capabilities, and an allowlist (an empty array for project credentials). Names, kinds, binding keys, project ids, capabilities, and list counts are bounded; malformed arrays, unknown mutation fields, and control characters are rejected instead of being silently coerced. A stored value is limited to 64 KiB of UTF-8 data. Global allowlists must explicitly retain the management owner. + +Every lifecycle mutation carries `expectedVersion`. Successful name updates, validation tests, rotations/replacements, promotions, restrictions, and first-time revocations increment the version. A repeated revoke against an already-revoked credential at its current version is an idempotent no-op; stale requests return a conflict. Metadata updates may change only the bounded display name, so kind and management ownership remain immutable. + +Restriction is monotonic: it may remove allowlisted projects or capabilities but cannot add either. Project-to-global promotion is the explicit scope expansion and requires the managing project, a current version, `confirmScopeExpansion: true`, an allowlist containing the managing project, and project IDs that already exist. ## Runtime redaction boundary -Node-flow credentials exist in plaintext only for the active node attempt. Exact resolved values are replaced with `[REDACTED]` before provider responses, HTTP bodies, retry errors, external-effect payloads, diagnostics, invocation messages, attempts, node outputs, or run summaries are stored. Credential IDs and non-secret metadata remain available for auditability. +Node-flow execution resolves credential values only for the active node attempt. Before any provider response, HTTP body, retry error, external-effect payload, diagnostic, invocation message, attempt, node output, or run summary is persisted, the runtime replaces exact resolved values with `[REDACTED]` in addition to masking secret-shaped keys. Credential IDs and non-secret metadata remain available for audit and attempt correlation. -The same redactor protects provider activity and raw usage telemetry. Temporary credential references are cleared after the attempt and are never logged as redaction input. Custom-node outputs, stderr logs, and diagnostics follow the same rule. +Provider activity persistence uses the same invocation-scoped redactor, including raw usage telemetry and provider session identifiers. Temporary credential references are cleared after each attempt and are never included in redaction logs or diagnostics. Custom-node containers apply the equivalent policy to structured output, stderr logs, and diagnostics before returning control to the flow runtime. -Authorization is rechecked after decryption. Concurrent revocation, rotation, restriction, promotion, or rebinding clears the plaintext buffer and causes a retry or denial instead of returning stale access. +Resolution authorization is checked both before and after decryption. If a credential is revoked, rotated, restricted, promoted, or rebound while a read is in flight, the plaintext buffer is cleared and the broker denies or retries against the current version; stale authorization is never returned to the caller. ## Encryption and key custody -The SQLite secret store uses AES-256-GCM envelope encryption with a unique data key, payload nonce, and key-wrapping nonce for every write. Credential ownership and workspace context are authenticated. SQLite stores ciphertext, authentication tags, wrapped keys, nonces, and key identifiers/versions—not root keys. +The SQLite secret store uses AES-256-GCM envelope encryption. Each write generates a unique 256-bit data key, payload nonce, and key-wrapping nonce. Credential ownership and workspace context are authenticated as additional data. SQLite stores only ciphertext, authentication tags, wrapped keys, nonces, and key identifiers/versions. + +Root keys are never stored in SQLite or a project checkout. The normal loopback dashboard automatically provisions one raw 32-byte root key at `~/.code-ux/security/credential-root.key`. Its dedicated parent directory is `0700` and the regular file is `0600`. Creation uses an exclusive atomic install, durable filesystem synchronization where supported, and concurrent startup convergence so restarts recover the identical key. Before creation or access, every custody-path component from the Code UX home through the key parent is inspected without following symbolic links; a symbolic-link or non-directory ancestor fails closed before a redirected key can be provisioned. Existing symbolic links, non-files, malformed keys, permissive modes, or unexpected ownership are never repaired automatically; credential operations fail closed with metadata-only setup guidance. -The trusted loopback dashboard automatically provisions one raw 32-byte root key at `~/.code-ux/security/credential-root.key`, with a `0700` parent and `0600` regular file. Provisioning is exclusive, atomic, durable where filesystem synchronization is supported, and safe across concurrent startup. Every custody-path component from the Code UX home through the key parent is inspected without following symbolic links, so a symbolic-link or non-directory ancestor fails closed before a redirected key can be provisioned. Symbolic links, non-files, malformed content, permissive modes, and unexpected ownership fail closed and are not repaired automatically. +Automatic local-file custody is limited to the non-server dashboard with local authentication, loopback binding, and remote credential management disabled. Electron's process provider remains first priority and continues to use OS `safeStorage`. Explicit `CODE_UX_CREDENTIAL_KEY_PROVIDER=mounted-key-file|vault|kms` configuration takes priority over automatic custody; setting `CODE_UX_CREDENTIAL_KEY_FILE` alone remains compatible with the mounted-file provider. Unknown values and an explicit `local-file` selection are rejected. Dashboard-disabled headless operation, server mode, authenticated dashboards, non-loopback bindings, and remote credential-management deployments do not auto-provision a local key. -Automatic local-file custody is disabled for server mode, dashboard-disabled headless operation, authenticated or non-loopback dashboards, and remote credential management. Electron remains first priority and persists only an OS-protected blob. Explicit `CODE_UX_CREDENTIAL_KEY_PROVIDER=mounted-key-file|vault|kms` configuration takes priority; `CODE_UX_CREDENTIAL_KEY_FILE` alone remains a compatible mounted-file selection. Unknown values and explicit `local-file` selection are rejected. If secure key material is unavailable, credential operations fail closed; there is no plaintext fallback. +For mounted-file custody, `CODE_UX_CREDENTIAL_KEY_FILE` identifies a regular, owner-only mounted file whose contents are an exact base64 or hexadecimal encoding of 32 bytes. Oversized or permissively decodable key files are rejected. The environment variable contains a path, not key material. Keep the mount readable only by the Code UX process and outside the project workspace. + +Electron serializes first-use root-key creation, persists only the OS-protected blob through an atomic owner-only file replacement, and refuses credential operations when `safeStorage` is unavailable. Vault and KMS adapters validate 32-byte caller-owned key material and report the active key id/version in health results. No provider silently falls back to plaintext or an insecure locally derived key. + +| Deployment boundary | Root-key custody | Provisioning behavior | +| --- | --- | --- | +| Normal CLI dashboard on loopback with local authentication | Owner-only file under the user-home Code UX security directory | Automatically created on first use and reused after restart. A normal local dashboard user does not mount or configure a key file. | +| Electron desktop | Operating-system `safeStorage` | Automatically creates and persists only the OS-protected blob; unavailable `safeStorage` blocks credential operations. | +| Dashboard-disabled headless, server mode, authenticated dashboard, non-loopback binding, or remote credential management | Explicit mounted file, Vault, or KMS provider | Never auto-provisions local custody. Setup and recovery fail closed until the configured provider reports available, secure key identity and version metadata. | ## Recovery and rotation -Back up root keys separately from `app.db`; the database alone cannot recover credentials. Local dashboard backups must include `~/.code-ux/security/credential-root.key` with owner-only handling. Creation, rotation/replacement, and promotion commit ciphertext and metadata atomically. Version compare-and-swap protects every lifecycle mutation and permits only one overlapping value change to commit. Revocation also wins against an in-flight resolution while retaining audit metadata. +Back up root keys independently from `app.db`. For the normal local dashboard, back up `~/.code-ux/security/credential-root.key` while preserving owner-only handling; for external providers, retain every referenced key version. Losing a required key version makes its ciphertext unrecoverable by design. Restoring only SQLite is insufficient. + +Credential creation commits metadata and its first envelope in one SQLite transaction. Rotation/replacement and promotion likewise commit the new envelope, metadata, version, and rotation record atomically. Compare-and-swap guards apply to every lifecycle mutation so losing callers must refresh metadata and retry instead of overwriting newer state. Root-key providers must retain old key IDs and versions until envelopes are rewrapped. Revocation wins against in-flight resolutions and preserves audit metadata. -Lifecycle success and denial audits carry correlation IDs, credential IDs, and policy metadata only. Validation records `valid`, `invalid`, or `unavailable` without exposing tested values or cryptographic internals. +Lifecycle successes and denials emit correlation-aware automation audit records containing credential IDs and policy metadata only. Validation updates report `valid`, `invalid`, or `unavailable` without including tested values or low-level cryptographic errors. Custom dashboards use a stricter metadata-only consumer boundary. Dedicated slot declarations define allowed kinds and required capabilities, while separate draft and immutable-revision binding columns store credential IDs. Binding review delegates to the broker's compatibility assessment and never resolves plaintext. Required or invalid bindings stop validation before workspace creation and are rechecked before publication. Credential values and binding IDs are excluded from generated dashboard artifacts, Docker configuration, validation output, generic REST/MCP responses, and iframe messages; only the dedicated binding-management response may expose IDs with non-secret metadata. -Legacy global records use their first valid allowlisted project as the migrated management owner; verify that owner before expanding an old global allowlist. +Existing global credentials created before management ownership was stored are migrated with their first valid allowlisted project as the management owner. Operators should verify that owner before expanding a legacy global credential's allowlist. + +## API surface + +Project-scoped routes live under `/api/projects/:projectId/credentials`. Supported operations are create, bounded-name update (`PATCH /:credentialId`), bind, metadata-only compatibility assessment, test, rotate, replace, revoke, promote, and restrict. Compatibility evaluates key-backend readiness, configuration, active status, project access, allowed kinds, and all required capabilities without resolving plaintext. A backend is ready only when it is available and secure and reports both a non-empty key ID and a key version; missing key identity metadata produces the stable `backend_unavailable` compatibility issue. List, compatibility, health, and mutation responses return metadata or policy results only. Existing dashboard authentication and remote credential-management guards apply before these routes. -## Dashboard API +Runtime validation failures return `400`, project/management denials return `403`, compare-and-swap conflicts return `409`, invalid encrypted state returns `422`, and unavailable key custody returns an actionable `503` response. -Credential management uses project-scoped dashboard routes. The API includes create, bounded-name update, bind, metadata-only compatibility assessment, test, rotate, replace, revoke, confirmed promotion, and monotonic restriction. Compatibility checks backend readiness, configured/active state, project access, allowed kinds, and all required capabilities without reading plaintext. Backend readiness requires an available, secure backend with a non-empty key ID and a reported key version; missing identity metadata produces `backend_unavailable`. List, health, compatibility, and mutation responses never contain secret values; secrets are accepted only by create, rotate, and replace operations. +## Troubleshooting without disclosure -Validation failures return `400`, project/management denials return `403`, concurrent-write conflicts return `409`, invalid encrypted state returns `422`, and unavailable key custody returns `503` with a safe recovery message. +- If custody is unavailable, inspect the metadata-only credential health or readiness result and the configured provider name. For the normal loopback dashboard, verify ownership, file type, and owner-only modes on the existing Code UX security path; for Electron, restore OS `safeStorage`; for headless or remote operation, restore the configured mount, Vault, or KMS version. Never paste, print, regenerate over, or move root-key material into a repository to diagnose the failure. +- If a mutation reports a stale `expectedVersion`, refresh credential metadata and review the newer scope, capabilities, validation state, and status before retrying. Do not reuse the rejected request blindly and do not bypass the comparison check. +- If encrypted rows exist but their key version is unavailable, restore the exact retained provider version before starting runners. Replacing it with a new key does not decrypt old envelopes; restore from the independent custody backup or recover the affected credential through the supported replacement workflow after the runtime is ready. diff --git a/docs-web/content/docs/operations-server-mode.mdx b/docs-web/content/docs/operations-server-mode.mdx index 27ba8c0fa4..99331fbedd 100644 --- a/docs-web/content/docs/operations-server-mode.mdx +++ b/docs-web/content/docs/operations-server-mode.mdx @@ -1,29 +1,249 @@ -# Authenticated Headless Server Mode +# Secure Headless Server Mode -Code UX separates MCP bearer access from the authenticated dashboard administrative API. Remote dashboard/API deployments must use digest-backed service identities or terminate OIDC at a trusted reverse proxy; loopback desktop operation remains a trusted local boundary. +Server mode runs Code UX as an authenticated MCP HTTP control plane without binding the dashboard UI, dashboard REST routes, dashboard realtime websocket, terminal websocket, or static dashboard assets. Use it for headless hosts, CI-adjacent automation, and cluster worker control planes where clients connect over Streamable HTTP instead of launching Code UX over stdio. -## Identity and authorization +Server mode is different from ordinary `--headless` mode: -Set `CODE_UX_DASHBOARD_AUTH_MODE=service_token` and provide `CODE_UX_SERVICE_IDENTITIES_JSON` entries containing `id`, `displayName`, SHA-256 `tokenSha256`, `roles`, explicit `projectIds`, and `enabled`. Workers send the bearer through `CODE_UX_WORKER_AUTH_TOKEN` and may assert the matching identity with `CODE_UX_WORKER_SERVICE_ID`. +| Mode | Dashboard | MCP HTTP | Token behavior | +| --- | --- | --- | --- | +| Default dashboard mode | Enabled | Enabled by default | Uses an explicit token or the generated user token in `~/.code-ux/security.json`. | +| `--headless` / `--no-dashboard` | Disabled | Uses normal MCP HTTP enablement rules | Preserves local-development behavior and can use the generated user token when HTTP is enabled. | +| `--server-mode` / `CODE_UX_SERVER_MODE=true` | Disabled | Enabled by default | Requires an explicit MCP HTTP bearer token with at least 32 bearer-safe characters. | -Alternatively, set `CODE_UX_DASHBOARD_AUTH_MODE=trusted_proxy`, configure `CODE_UX_TRUSTED_PROXY_SECRET`, terminate/validate OIDC at the proxy, strip client identity headers, and inject trusted principal, role, and project headers. Authenticated remote traffic requires TLS (`X-Forwarded-Proto: https`) unless insecure HTTP is explicitly enabled for an isolated test. +## Threat Model -Roles are `credential_admin`, `automation_author`, `automation_publisher`, `automation_runner`, and `viewer`. Credential routes additionally require `CODE_UX_REMOTE_CREDENTIAL_MANAGEMENT=true`; enabling it without a healthy secure key provider makes readiness fail. Host/origin checks, no-store responses, and administrative rate limits remain active. +MCP bearer access remains a runtime-wide control-plane identity. The dashboard administrative API has a separate authenticated-headless boundary with project-scoped roles; do not treat an MCP bearer as a dashboard service identity. -The `credential_admin` role can still read administrative readiness, audit export, and SLO metrics while remote credential management is disabled. The feature flag gates credential-management and credential-health routes only. +## Authenticated Dashboard API -## Probes, audit, and SLOs +Remote dashboard/API operation is fail-closed. Setting a non-loopback `DASHBOARD_HOST` without an explicit authentication mode defaults the API to `service_token`, so unconfigured callers receive `401`/`403` instead of inheriting desktop access. Loopback desktop mode remains `local`. -`/health` is liveness. `/ready` also checks credential-key recovery, the audit store, and distributed-runner identities and returns `503` when required components are unavailable. If encrypted credential rows exist and their key cannot be recovered, startup aborts before listeners bind. Server mode never auto-provisions local-file custody; configure `mounted-key-file`, Vault, or KMS explicitly. +Choose one boundary: -Authenticated operators can use `/api/admin/readiness`, `/api/admin/audit/export` (redacted NDJSON), and `/api/admin/metrics/slo`. Audit covers management calls, credential access, runs, attempts, approvals, and outbox delivery with correlation ids. +- `CODE_UX_DASHBOARD_AUTH_MODE=service_token`: define `CODE_UX_SERVICE_IDENTITIES_JSON` as an array of identities with `id`, `displayName`, a lowercase SHA-256 `tokenSha256`, `roles`, `projectIds`, and `enabled`. Workers may send the matching id with `--service-identity-id` or `CODE_UX_WORKER_SERVICE_ID`; the bearer remains in `CODE_UX_WORKER_AUTH_TOKEN`. +- `CODE_UX_DASHBOARD_AUTH_MODE=trusted_proxy`: terminate OIDC at a trusted proxy, set `CODE_UX_TRUSTED_PROXY_SECRET`, and have the proxy overwrite `X-Code-UX-Proxy-Secret`, `X-Code-UX-Principal-Id`, `X-Code-UX-Roles`, `X-Code-UX-Project-Ids`, and optional name/kind headers. Never forward client-supplied copies. -Baseline alerts: readiness not ready for five minutes, management 5xx above 1% or p95 above one second for ten minutes, repeated lease expiry, credential-denial spikes, outbox failure backlog, or any secret/audit check failure. Target zero unauthorized project grants, secret disclosures, and duplicate side effects. +Roles are `credential_admin`, `automation_author`, `automation_publisher`, `automation_runner`, and `viewer`. Project ids are explicit; `*` is an operator-only all-project grant. Credential routes additionally require `CODE_UX_REMOTE_CREDENTIAL_MANAGEMENT=true`. Enabling that flag without a healthy secure key provider makes readiness fail. -## Backup and recovery +The `credential_admin` role can read `/api/admin/readiness`, `/api/admin/audit/export`, and `/api/admin/metrics/slo` even when remote credential management is disabled. The feature flag gates credential creation, binding, testing, rotation, replacement, revocation, promotion, restriction, and credential-health routes; it does not disable operational readiness, audit, or SLO inspection. -Back up SQLite with WAL consistency, settings, project `.code-ux/` state, and every referenced external key version. Restore keys before databases, keep runner admission disabled, require `/ready`, then reconcile leases, approvals, audit continuity, and outbox counts. Never back up plaintext service tokens beside their digests. +TLS is assumed at the reverse proxy. Authenticated remote requests must arrive with HTTPS or a trusted `X-Forwarded-Proto: https`; `CODE_UX_ALLOW_INSECURE_HTTP=true` is limited to isolated test networks. Same-origin browser checks, no-store headers, host validation, and a 600-request/minute administrative API limiter remain active. Webhook and provider-ingress endpoints retain their dedicated authentication schemes. -Rotate service identities by overlapping new/old digests until runners authenticate with the new token. Rotate credential values through the broker so graph bindings retain ids and resolve the next version. Retain old KMS/Vault versions until envelope rewrap and restore drills pass. +Example identity generation (the JSON stores only the digest): -Rollback creates and publishes a new draft from an earlier immutable version; in-flight runs stay pinned. Recovery requeues only known-safe pre-invocation work and leaves uncertain external outcomes for attention. OIDC validation and Vault/KMS client integration remain deployment-host responsibilities, and MCP bearer authority remains broader than dashboard roles. +```bash +token="$(openssl rand -base64 48 | tr -d '\n')" +digest="$(printf '%s' "$token" | sha256sum | cut -d' ' -f1)" +# Put $token in the runner secret manager and $digest in CODE_UX_SERVICE_IDENTITIES_JSON. +``` + +Use server mode when: + +- the dashboard must not be reachable from the host +- MCP clients or workers need a stable HTTP endpoint +- a reverse proxy or private network boundary provides TLS and network admission +- operators can treat the bearer token as a secret with full runtime authority + +Do not expose the MCP HTTP listener directly to the public internet. The Node listener is HTTP; terminate HTTPS with a trusted reverse proxy, tunnel, service mesh, or load balancer when traffic leaves the host. + +## Startup + +Generate the token in the process environment or a secret manager. Do not paste real bearer values into shell history, logs, tickets, release notes, or documentation. + +```bash +export MCP_HTTP_AUTH_TOKEN="$(openssl rand -base64 48 | tr -d '\n')" + +codeux \ + --server-mode \ + --mcp-http-host 127.0.0.1 \ + --mcp-http-port 4445 \ + --mcp-http-path /mcp +``` + +For a cluster control plane behind a reverse proxy or private network interface: + +```bash +export CODE_UX_SERVER_MODE=true +export MCP_HTTP_AUTH_TOKEN="$(openssl rand -base64 48 | tr -d '\n')" +export MCP_HTTP_HOST=0.0.0.0 +export MCP_HTTP_PORT=4445 +export MCP_HTTP_PATH=/mcp +export MCP_HTTP_MAX_SESSIONS=500 +export MCP_HTTP_SESSION_TIMEOUT_MS=3600000 + +codeux +``` + +The legacy `mcp-https` names remain supported for compatibility: + +| Purpose | Preferred | Legacy-compatible | +| --- | --- | --- | +| Gateway enablement | `MCP_HTTP_ENABLED`, `--no-mcp-http` to disable outside server mode | `MCP_HTTPS_ENABLED`, `--no-mcp-https` to disable outside server mode | +| Gateway host | `MCP_HTTP_HOST`, `--mcp-http-host` | `MCP_HTTPS_HOST`, `--mcp-https-host` | +| Gateway port | `MCP_HTTP_PORT`, `--mcp-http-port` | `MCP_HTTPS_PORT`, `--mcp-https-port` | +| Gateway path | `MCP_HTTP_PATH`, `--mcp-http-path` | `MCP_HTTPS_PATH`, `--mcp-https-path` | +| Bearer token | `MCP_HTTP_AUTH_TOKEN`, `--mcp-http-auth-token` | `MCP_HTTPS_AUTH_TOKEN`, `--mcp-https-auth-token` | +| Session cap | `MCP_HTTP_MAX_SESSIONS`, `--mcp-http-max-sessions` | `MCP_HTTPS_MAX_SESSIONS`, `--mcp-https-max-sessions` | +| Idle timeout | `MCP_HTTP_SESSION_TIMEOUT_MS`, `--mcp-http-session-timeout-ms` | `MCP_HTTPS_SESSION_TIMEOUT_MS`, `--mcp-https-session-timeout-ms` | + +Server mode rejects startup when the explicit token is missing, empty, shorter than 32 characters, or contains characters outside the bearer-safe set. It does not fall back to the generated local user token. + +If `--server-mode` is combined with an explicit MCP HTTP disable flag, server mode still restores the MCP HTTP listener on the default MCP port because the server-mode contract requires authenticated remote MCP access while the dashboard stays disabled. + +## Health And Readiness + +The MCP HTTP listener serves probes without the dashboard server: + +```bash +curl --fail http://127.0.0.1:4445/health +curl --fail http://127.0.0.1:4445/ready +``` + +Use `/health` for process liveness. It only proves that the listener is up. + +Use `/ready` for runtime readiness. It reports whether the Code UX runtime finished the required startup path and can accept work. During startup, maintenance such as Docker cleanup, preview reconciliation, branch reaping, and recovery work can continue after the listener binds, so `/health` can pass before `/ready`. + +Do not include `Authorization` headers in probe logs. The probe endpoints do not require bearer credentials. + +`/ready` also reports `credentialKey`, `auditStore`, and `distributedRunner`. `/health` remains live during a key-provider outage, while `/ready` returns `503`. Startup aborts before dashboard or MCP binding when encrypted credential rows exist but their key provider cannot recover the wrapping key. Server mode never auto-provisions local-file custody. Select a provider with `CODE_UX_CREDENTIAL_KEY_PROVIDER=mounted-key-file|vault|kms`; mounted files use `CODE_UX_CREDENTIAL_KEY_FILE` and owner-only permissions. Vault/KMS modes require their host adapter to be configured and healthy. + +The same explicit-custody requirement applies to dashboard-disabled headless operation, authenticated dashboards, non-loopback dashboard bindings, and remote credential management. Only the trusted loopback local dashboard auto-provisions its owner-only user-home key; Electron uses OS `safeStorage`. Remote setup therefore fails closed rather than borrowing the local-dashboard key, deriving a key, or falling back to plaintext. Restore the configured mount or the exact Vault/KMS key version before enabling runners; do not copy root keys into SQLite, a project checkout, deployment logs, or diagnostic bundles. + +Authenticated operators can inspect `/api/admin/readiness`, export redacted NDJSON from `/api/admin/audit/export`, and sample `/api/admin/metrics/slo`. Audit rows include the correlation id, principal, project, action, outcome, and redacted metadata for management requests, credential access, runs, attempts, approvals, and outbox delivery. + +## Backup, Restore, Rotation, And Rollback + +Back up `~/.code-ux/app.db` with a SQLite-aware snapshot that includes/checkpoints WAL state, the settings database, project `.code-ux/` directories, and the external key-provider versions needed by every encrypted envelope. Never place plaintext service tokens or root keys in the database backup. Restore into an isolated host, restore keys first, run `/ready`, then enable runners. + +Rotate service tokens by adding the new digest, deploying the new runner secret, observing successful authenticated calls, and disabling the old identity entry. Rotate credential values through the credential rotation API; existing graph bindings keep the credential id and resolve the new version. Retain old KMS/Vault key versions until every envelope has been rewrapped and a restore drill succeeds. + +To roll back an automation, create a new draft from the earlier immutable version, review it, and publish it. In-flight runs remain pinned to their original publication. Stop runner admission before database recovery; after restore, startup recovery requeues only known-safe work and leaves unknown external outcomes in `attention_required`. + +## Baseline SLOs And Alerts + +Initial operator baselines are 99.9% authenticated management availability, p95 management latency below 500 ms, zero unauthorized project grants, zero secret disclosure, and zero duplicate outbox side effects. Alert when readiness is not ready for 5 minutes, management 5xx rate exceeds 1% for 10 minutes, p95 exceeds 1 second for 10 minutes, leases repeatedly expire, denied credential access spikes, outbox failures remain pending for 5 minutes, or any audit/secret scanning check fails. + +Local mode is intentionally a trusted loopback desktop boundary. Authenticated headless mode adds API RBAC, project scope, key readiness, durable audit, and service identities, but it is not a general multi-tenant identity platform: OIDC token validation belongs at the trusted proxy, Vault/KMS require host adapters, and MCP bearer authority remains broader than dashboard roles. + +## Client Connections + +MCP HTTP clients connect to the configured path with `Authorization: Bearer `. The first JSON-RPC request on a new Streamable HTTP session must be `initialize`; the server returns an `mcp-session-id` header that the client echoes on later calls. + +For a local CLI or dashboard-adjacent session that supports MCP HTTP, configure: + +- URL: `http://:4445/mcp` +- header name: `Authorization` +- header value: `Bearer ` + +Verify without exposing the token: + +```bash +curl --fail http://127.0.0.1:4445/health +curl --fail http://127.0.0.1:4445/ready +``` + +Then verify through the MCP client by listing tools or running a read-only management action such as listing projects. Do not use `curl -v`, shell tracing, or command transcripts that print the authorization header. + +If a local dashboard app is used only as an operator console for a separate server-mode instance, configure its MCP client entry to the server-mode URL and bearer header. The dashboard UI of the server-mode process itself remains unavailable by design. + +## Settings Synchronization + +Settings synchronization uses the `manage_settings` bundle actions: + +- `export_settings_bundle` +- `apply_settings_bundle` + +Bundles can include system, project, and sprint scopes. Metadata includes `schemaVersion: 1`, `exportedAt`, `includedScopes`, a SHA-256 `fingerprint` computed from a secret-redacted representation, and `containsSecrets`. + +Approved workflow: + +1. Export a redacted bundle from the source runtime. Export defaults to the `system` scope and redacts provider API keys, git tokens, issue-tracker tokens, login credentials, and other secret-bearing fields. +2. Review the bundle before moving it to the destination. Redacted placeholders are expected and must not be replaced in shared artifacts. +3. If project or sprint settings are required, include `scopes`, `projectIds`, and `sprintIds`. Sprint exports require the owning `projectId` so imports can normalize sprint overrides against the resolved project base. +4. Apply the bundle on the destination with `apply_settings_bundle`. The importer persists through `saveSystemSettings`, `saveProjectSettings`, and `saveSprintSettings`, so values follow the same sanitizer and override normalization as dashboard saves. +5. For partial rollout or rollback, pass `scopes` on apply to limit which bundle scopes are written. + +Secret-bearing exports and imports require the stateful settings approval flow: + +- `includeSecrets: true` on export returns secrets only after the first response asks for approval and the exact same request is repeated with `approval.confirmed: true`. +- A bundle marked `containsSecrets: true`, or one whose payload contains secret-bearing fields, is applied only after the same one-use approval flow. +- Approval is bound to the exact normalized payload, expires after 15 minutes, and is consumed after one successful execution. + +Rollback is another approved apply. Export a known-good bundle before changing a destination runtime, then apply that bundle back to the affected scopes if the rollout must be reverted. Do not rely on logs or chat transcripts as backups because redaction intentionally removes sensitive values. + +## Cluster Workers + +External workers connect to the server-mode MCP HTTP endpoint as control-plane clients. The worker process also starts a local `worker-host` runtime over stdio for execution on the worker machine. + +Start a worker with the shipped bin: + +```bash +codeux-worker \ + --server-url http://SERVER_HOST:4445/mcp \ + --auth-token "$CODE_UX_WORKER_AUTH_TOKEN" \ + --connection-key worker:build-node-01 \ + --display-name "Build node 01" \ + --project-id project-id +``` + +Equivalent environment variables: + +```bash +export CODE_UX_WORKER_SERVER_URL=http://SERVER_HOST:4445/mcp +export CODE_UX_WORKER_AUTH_TOKEN="$MCP_HTTP_AUTH_TOKEN" + +codeux-worker --connection-key worker:build-node-01 --project-id project-id +``` + +Worker config supports multi-project operation: + +- repeat `--project-id` to register eligible projects +- repeat `--active-project-id` to advertise active project focus +- use a stable `--connection-key` so reconnects update the existing registered endpoint +- set `--server-command`, repeated `--server-arg`, and `--server-cwd` only when the worker-local execution runtime needs a custom command + +Cluster behavior: + +- Registered workers are not license-capped. The active Streamable HTTP session cap defaults to 100 and can be raised for large clusters. +- Project assignments live in `project_worker_assignments`. A project can have one primary worker and any number of overflow workers. +- Active-session protection prevents runaway clients from allocating unlimited Streamable HTTP sessions. Raise `MCP_HTTP_MAX_SESSIONS` only to the capacity the server can actually operate. +- Heartbeats derive endpoint status. Stale or offline workers are excluded from new claims, and stale primary workers can be bypassed by eligible overflow workers. +- Dispatch safety depends on both `task_dispatches` and `execution_leases`. A worker must not start local execution unless the server returns a claim with a lease token. Heartbeats renew the lease while the task runs; expired leases can be claimed by another eligible worker. +- Multi-project workers claim only work for projects they are assigned to and advertise as active or eligible. + +## Token Rotation + +Safe rotation is a short planned restart unless a reverse proxy or secret manager can coordinate old/new tokens externally. + +1. Generate a new token in the secret manager. +2. Update client and worker secret references, but do not restart them yet. +3. Restart the server-mode process with the new token. +4. Restart or reconnect MCP clients and workers so they initialize new sessions with the new token. +5. Confirm `/ready` passes and clients can list tools or claim work. +6. Revoke the old token from the secret manager and remove it from local shells, process managers, and deployment manifests. + +Existing HTTP sessions authenticated with the previous token should be treated as invalid after server restart because Streamable HTTP sessions are in memory. Workers should reinitialize rather than attempting to reuse old `mcp-session-id` values. + +## Troubleshooting + +| Symptom | Likely cause | Fix | +| --- | --- | --- | +| Startup fails with a server-mode token error | `--server-mode` or `CODE_UX_SERVER_MODE=true` is set without an explicit valid bearer token. | Set `MCP_HTTP_AUTH_TOKEN` or `MCP_HTTPS_AUTH_TOKEN`, or pass the matching CLI flag. Use at least 32 bearer-safe characters. | +| Startup fails when binding `0.0.0.0`, `::`, or a LAN address | MCP HTTP is reachable beyond loopback without an active token. | Configure an explicit bearer token and put TLS/auth network controls in front of the HTTP listener. | +| Dashboard URL is unavailable | Expected in server mode. | Use MCP HTTP clients and `/health` or `/ready`. Start a separate dashboard-mode process only when an operator UI is required. | +| HTTP returns `401 Unauthorized` | Missing `Authorization: Bearer `, wrong token, duplicate authorization headers, or a client still using the old token after rotation. | Reinstall or update the client secret, reconnect, and avoid printing headers in diagnostics. | +| HTTP returns `400` on a new MCP session | The first request was not JSON-RPC `initialize`, or `mcp-session-id` / `x-code-ux-agent` was malformed. | Let the MCP SDK initialize the session, or clear stale session state and reconnect. | +| Session cap errors appear | Too many active Streamable HTTP sessions, usually from leaked clients or a cluster larger than the default cap. | Stop stale clients, shorten `MCP_HTTP_SESSION_TIMEOUT_MS`, or raise `MCP_HTTP_MAX_SESSIONS` within server capacity. | +| Worker appears stale or offline | Heartbeats stopped, the worker process is down, network access failed, or the stable connection key changed unexpectedly. | Restart the worker with the same `--connection-key`, verify `/ready`, and check logs for bounded connection metadata. | +| Worker connects but does not claim work | No active project assignment, project not included in `--project-id` / `--active-project-id`, stale endpoint status, task executor mismatch, or no lease returned. | Confirm project assignment and worker status, then verify queued dispatches. Do not start local execution without a lease token. | +| `/health` passes but `/ready` fails | Listener is alive but runtime readiness has not completed or the server is degraded. | Wait for startup recovery to finish, then inspect structured logs. Use `/ready` for load balancer readiness gates. | +| `/ready` reports credential custody unavailable | The explicit mounted-file, Vault, or KMS provider is missing, insecure, unhealthy, or cannot return the required key version. | Keep runners disabled, inspect metadata-only readiness and provider configuration, and restore the exact provider/key version. Do not print key material or substitute a new key for encrypted rows. | +| A credential operation returns a version conflict | Another operator changed metadata, scope, capabilities, status, or encrypted value first. | Refresh the metadata-only record, review the new version, and intentionally retry with that version. Do not bypass optimistic concurrency. | +| Secret values appear in an exported settings bundle | The export was explicitly approved with `includeSecrets: true`. | Store the bundle only in approved secret storage, rotate exposed credentials if it was shared, and prefer redacted exports for review. | + +## Related Docs + +- [MCP Tools](/docs/developer-mcp-tools) +- [Security Hardening](/docs/operations-security-hardening) +- [Automation Credential Security](/docs/operations-credential-security) +- [Runtime Configuration](/docs/developer-configuration) diff --git a/docs-web/content/docs/registry.ts b/docs-web/content/docs/registry.ts index c346b5dcaf..c64bd719a6 100644 --- a/docs-web/content/docs/registry.ts +++ b/docs-web/content/docs/registry.ts @@ -273,7 +273,7 @@ export const docsRegistry: Record = { id: 'user-dashboard-node-flows', path: '/docs/user-dashboard-node-flows', section: 'User Guide', - title: "Node Flows", + title: "Node Flows Dashboard", description: "The Nodes page (/nodes) is the project-scoped backend authoring, publication, and operations surface for canonical node flows. No selected project means no flow library, credential metadata, publications, or durable r...", }, 'user-dashboard-scheduler': { @@ -561,7 +561,7 @@ export const docsRegistry: Record = { path: '/docs/settings-integrations', section: 'User Guide', title: "Integrations", - description: "Lists provider, git-host, issue-tracker, and read-only importer integrations and exposes manage/add actions.", + description: "Lists automation credentials, providers, git hosts, issue trackers, and read-only importer integrations and exposes manage/add actions.", }, 'settings-jules-automation': { id: 'settings-jules-automation', @@ -834,7 +834,7 @@ export const docsRegistry: Record = { path: '/docs/operations-credential-security', section: 'User Guide', title: "Automation Credential Security", - description: "Code UX resolves canonical node credential IDs and named project binding keys through the credential broker. Stored values are not exposed to nodes, dashboard reads, MCP payloads, agent context, run inspection records...", + description: "Code UX stores automation credentials through a broker rather than exposing secret values to node definitions, dashboard reads, MCP payloads, agent context, or run inspection records. Canonical node bindings reference...", }, 'operations-runbook': { id: 'operations-runbook', @@ -854,8 +854,8 @@ export const docsRegistry: Record = { id: 'operations-server-mode', path: '/docs/operations-server-mode', section: 'User Guide', - title: "Authenticated Headless Server Mode", - description: "Code UX separates MCP bearer access from the authenticated dashboard administrative API. Remote dashboard/API deployments must use digest-backed service identities or terminate OIDC at a trusted reverse proxy; loopbac...", + title: "Secure Headless Server Mode", + description: "Server mode runs Code UX as an authenticated MCP HTTP control plane without binding the dashboard UI, dashboard REST routes, dashboard realtime websocket, terminal websocket, or static dashboard assets. Use it for hea...", }, 'settings-google-drive-mount': { id: 'settings-google-drive-mount', @@ -897,7 +897,7 @@ export const docsRegistry: Record = { path: '/docs/architecture-node-flow-builtins-and-security', section: 'Architecture', title: "Node Flow Built-ins and External-Effect Security", - description: "The governed catalog adds deterministic branches, bounded collection processing, durable approvals, and replay-safe external effects while keeping the versioned definition registry as the executable authority.", + description: "The governed built-in catalog extends publication-based node-flow execution with deterministic control nodes and durable boundaries for external effects. The definition registry remains the executable authority; a gra...", }, 'architecture-node-flow-durable-execution': { id: 'architecture-node-flow-durable-execution', @@ -918,7 +918,7 @@ export const docsRegistry: Record = { path: '/docs/architecture-node-flows', section: 'Architecture', title: "Node Flows", - description: "Node flows are project-owned, versioned Graph v2 workflows.", + description: "Node flows are project-scoped, repeatable workflow graphs for turning an operator or agent-defined procedure into a saved Code UX workflow. They are not a generic n8n compatibility layer. A good flow uses Code UX conc...", }, 'architecture-speech-input': { id: 'architecture-speech-input', diff --git a/docs-web/content/docs/settings-integrations.mdx b/docs-web/content/docs/settings-integrations.mdx index 67e686660b..47ab855fa7 100644 --- a/docs-web/content/docs/settings-integrations.mdx +++ b/docs-web/content/docs/settings-integrations.mdx @@ -1,19 +1,19 @@ # Integrations -Lists provider, git-host, issue-tracker, and read-only importer integrations and exposes manage/add actions. +Lists automation credentials, providers, git hosts, issue trackers, and read-only importer integrations and exposes manage/add actions. > Settings area: `integrations` > Dashboard documentation route: `/docs/settings-integrations` ## What This Area Is For -Lists provider, git-host, issue-tracker, and read-only importer integrations and exposes manage/add actions. This page expands the short Settings-page help text into an operator reference for deciding when to change this area, what behavior the controls affect, and what to verify after saving. +Lists automation credentials, providers, git hosts, issue trackers, and read-only importer integrations and exposes manage/add actions. This page expands the short Settings-page help text into an operator reference for deciding when to change this area, what behavior the controls affect, and what to verify after saving. Use it when you are configuring a new project, auditing inherited settings, or debugging behavior that changed after a system, project, or sprint override was saved. ## Controls And Runtime Effect -Cards show connection state, auth hints, active/configured importer status, and management entry points; host hints can import detected local settings. +Cards show connection state, auth hints, active/configured importer status, and management entry points; host hints can import detected local settings. Automation Credentials is the first catalog entry and reports secure-storage unavailable, ready but unconfigured, or configured state for the selected project. Its **Manage** action uses the same detail and back-navigation behavior as every other integration. | Control Surface | Runtime Effect | Review Before Saving | | --- | --- | --- | @@ -21,9 +21,30 @@ Cards show connection state, auth hints, active/configured importer status, and | Inherited values | Values can flow from system defaults into project and sprint behavior. | Check the source badge before assuming a value is project-specific. | | Related runtime paths | The affected service reads the saved settings during planning, dispatch, dashboard rendering, or maintenance work. | Re-run the affected workflow after changing operational settings. | +## Automation Credential Management + +Credential management is project-aware even when Settings is displaying system scope. Select a project before opening the detail view so Code UX can list only metadata visible to that project and determine whether the project has management authority. + +The create form requires an explicit name, kind, project or global scope, capability selection, and—when global scope is selected—an allowlist that retains the managing project. No capability is granted implicitly. Global creation and project-to-global promotion require confirmation because they expand access. + +Each project-managed credential supports bounded rename, metadata-only validation test, value rotation, encrypted-state replacement, monotonic access restriction, confirmed promotion, and confirmed revocation. Revocation requires typing `REVOKE` exactly; each lifecycle confirmation starts with cleared confirmation state and returns focus to the credential controls when it closes. Every lifecycle request uses the metadata version shown by the service. If another session wins the compare-and-swap update, the detail view refreshes metadata and asks the operator to review and retry instead of overwriting the newer state. + +| Workflow | What the operator supplies | What remains readable afterward | +| --- | --- | --- | +| Create | Name, kind, write-only value, explicit capabilities, and project/global policy | Metadata, configured state, validation state, scope, capabilities, and version only. | +| Update metadata | A bounded display name and current version | Updated metadata; kind and management ownership cannot be changed. | +| Rotate / replace | A new write-only value and current version | New key/version and validation metadata, never either the old or new value. | +| Test | The current version | `valid`, `invalid`, or `unavailable` plus timestamps; no tested value or low-level custody error. | +| Restrict / promote | A monotonic restriction, or a confirmed global allowlist expansion owned by the managing project | Updated non-secret policy metadata. | +| Revoke | Exact confirmation and current version | Revoked status and audit metadata; the stored value cannot be read back. | + +Secret inputs are write-only. Create, rotate, and replace fields are never populated from responses, are cleared after successful or failed submissions and project changes, and are removed with the detail view. Notices, metadata cards, browser storage, and reusable drafts contain no secret value. An allowlisted project that is not the management owner sees a **Use only** state and cannot invoke management actions. + +Unavailable key custody leaves non-secret metadata visible and disables secret-bearing changes and tests. Follow the inline custody guidance, restore secure storage, then use **Refresh**. See [Automation Credential Security](/docs/operations-credential-security) for encryption, authority, recovery, and API behavior. + ## Recommended Configuration -Configure provider and importer credentials at system scope and use project overrides only for repository-specific git hosts or importer defaults. +Configure provider and importer credentials at system scope and use project overrides only for repository-specific git hosts or importer defaults. Automation credentials follow their own project-aware ownership and allowlist policy rather than Settings inheritance. For Google Drive, link an existing host-side sync or mount directory and enable the opt-in Docker mount only for projects that need it. The mount defaults to read-only; see [Google Drive Project Mount](/docs/settings-google-drive-mount) for access, inheritance, security, and troubleshooting details. This integration does not configure Google Drive API synchronization or credentials. @@ -51,11 +72,14 @@ If the saved setting does not appear to take effect: - Check for a project or sprint override that takes precedence over the system value. - Refresh the affected dashboard page if the setting controls a rendered surface. - Restart the local runtime only when the setting explicitly controls startup, listener, or process-level behavior. +- If secure custody is unavailable, keep the metadata view open, restore the deployment's supported custody provider, and use **Refresh**. Local loopback CLI/dashboard mode provisions its owner-only user-home key automatically; do not add mounted-key configuration for a normal local user. +- If a save reports stale metadata, review the refreshed record before retrying with its new version. Never copy secret fields into notes, browser storage, logs, or a repository as a workaround. ## Related Documentation - [Settings overview](/docs/settings-overview) +- [Automation Credential Security](/docs/operations-credential-security) - [Google Drive Project Mount](/docs/settings-google-drive-mount) - [Dashboard Settings](/docs/user-dashboard-settings) -- [Configuration and Storage](/docs/developer-settings-reference) -- [Security Hardening](/docs/user-troubleshooting) +- [Runtime Configuration](/docs/developer-configuration) +- [Security Hardening](/docs/operations-security-hardening) diff --git a/docs-web/content/docs/user-dashboard-custom-dashboards.mdx b/docs-web/content/docs/user-dashboard-custom-dashboards.mdx index 9bc9006971..5ba4672c86 100644 --- a/docs-web/content/docs/user-dashboard-custom-dashboards.mdx +++ b/docs-web/content/docs/user-dashboard-custom-dashboards.mdx @@ -2,59 +2,140 @@ Custom dashboards are project-scoped dashboard apps generated and revised by agents, then validated in a detached Docker runtime before publication. Use them when the built-in dashboard pages do not match the operational view a team needs, such as a project-specific release panel, sprint-health cockpit, or integration-status board. -## Workflow +The source of truth is the Code UX database. Drafts stay mutable, revisions are immutable snapshots, validation sessions record build/runtime results, and publication is a single active pointer to one validated revision. -1. Ask the Project Manager for the dashboard you want. Include the purpose, target audience, data sources, layout preferences, review criteria, and whether it should be published after validation. -2. Review the draft at `/custom-dashboards`. Drafts expose manifest JSON, generated file bundle content, source-node graph JSON, styleguide JSON, and data catalog selections. +## User Workflow + +1. Ask the Project Manager for the dashboard you want. Include the purpose, target audience, data sources, layout preferences, review criteria, and whether the dashboard should be published after validation. +2. Review the draft in the dashboard workspace at `/custom-dashboards`. The draft includes editable manifest JSON, generated file bundle content, source-node graph JSON, styleguide JSON, and data catalog selections. 3. Ask for changes or edit the draft before creating a revision. Draft edits do not change previous revisions or the currently published dashboard. -4. If the manifest declares credential slots, open the editor's **Credentials** tab. It shows bounded declarations and current non-secret metadata, and offers only active, configured, project-authorized credentials that satisfy the declared kinds and capabilities. Bind every required slot; no secret value is entered into the dashboard draft or generated code. -5. Create a revision when the draft is ready. A revision snapshots the current manifest, files, source graph, styleguide, runtime metadata, and credential-ID bindings. -6. Run detached validation. Code UX reviews bindings before it builds the revision in Docker, captures the browser-ready Vite artifact, starts a detached preview container, and health-checks the root URL. -7. Inspect validation status, logs, and the proxied preview link. Validation passes only after credential policy, install, build, artifact capture, container start, and root health checks succeed. -8. Publish the validated revision. Publication rechecks credential metadata and remains blocked unless the revision has a passed validation report. -9. Roll back by publishing an earlier passed revision, or archive the dashboard to clear its active publication while preserving history. +4. If the manifest declares credential slots, open the editor's **Credentials** tab. It shows each bounded declaration and current non-secret credential metadata, and offers only active, configured, project-authorized credentials that satisfy the allowed kinds and required capabilities. Bind every required slot; no secret value is entered into the dashboard draft or generated code. +5. Create a revision when the draft is ready. A revision snapshots the current manifest, file bundle, source graph, styleguide, runtime metadata, and credential-ID bindings. +6. Run detached validation for the revision. Code UX reviews bindings before it materializes the bundle, builds it in Docker, starts a detached preview container, and health-checks the root URL. +7. Inspect validation status, logs, and the proxied preview link. Validation passes only after credential policy, install, build, browser artifact capture, container start, and root health checks succeed. A passed validation does not publish by itself. +8. Publish the validated revision. Publication rechecks credential metadata and requires `validationStatus: "passed"` with a valid validation report. Publishing another passed revision is the rollback path. +9. Archive dashboards you no longer want active. Archiving clears the active publication and marks the dashboard archived while preserving revision and validation history. + +If validation fails, use the report and logs to create a new revision. Do not publish around the failure; the repository rejects failed, queued, running, cancelled, missing, or mismatched validation sessions before publication state changes. When a dashboard is already published, validating later drafts keeps the active published dashboard open, and validation sessions for the active published revision do not replace its published validation snapshot. + +The Credentials tab appears only when the saved manifest declares slots. Secure-backend failures and empty compatible lists link to credential management in Settings. Binding, replacement, and unbinding use the current optimistic binding revision; a concurrent edit refreshes the dashboard and requires an explicit retry instead of overwriting the other operator. Required unbinding immediately shows the draft as not ready for its next revision, while optional unbound slots remain valid. Every successful binding change refreshes validation and publication readiness. + +Credential selection and actions are keyboard accessible, restore focus after completion, and announce saving or error state. Credential IDs remain confined to the dedicated metadata-management request state and never enter manifest, generated-file, source-graph, styleguide, runtime-text, or secret-value fields. + +If secure custody is unavailable, the Credentials tab keeps existing bindings unchanged, reports metadata-only readiness, and links to Settings. Restore the supported custody provider and refresh the review; do not put a key or credential value in manifest JSON, generated files, validation logs, or project files. If a bind/unbind returns a stale binding revision, the editor refreshes declarations, candidates, bindings, and readiness, then requires an explicit retry. If validation or publication denies a formerly compatible binding, refresh review because revocation, restriction, project access, capabilities, kind policy, or custody health may have changed. + +## Agent Workflow -If validation fails, use the report and logs to create a new revision. Code UX rejects failed, queued, running, cancelled, missing, or mismatched validation sessions before publication state changes. When a dashboard is already published, validating later drafts keeps the active published dashboard open, and validation sessions for the active published revision do not replace its published validation snapshot. +Project Manager agents should use the `manage_custom_dashboards` MCP surface rather than writing generated code into `dashboard/src`. -The Credentials tab appears only for manifests with declared slots. Unavailable secure custody and empty compatible lists link to credential management in Settings. Binding changes use the current optimistic revision; a conflict refreshes the dashboard and asks for an explicit retry. Required unbinding marks the next revision as not ready, while optional unbound slots remain valid. Successful changes refresh validation and publication readiness. The controls support keyboard selection, visible focus, focus restoration, and live save/error announcements. Credential IDs stay out of manifest, file, source, styleguide, and runtime text editors, and the editor never requests or displays secret values. +Recommended sequence: -## Data Sources +1. Gather missing requirements for purpose, audience, source data, style, accessibility, and publication intent. +2. Call `data_catalog` for the project when reusing existing custom-dashboard source declarations. +3. Call `create` or `update` with a complete manifest, file bundle, source-node graph, styleguide, and runtime metadata. +4. Call `list_credential_slots` when the manifest declares slots. Select only candidate credential IDs reported compatible, then call `bind_credential` with the current `expectedBindingRevision` and complete the human-approval flow. Use `unbind_credential` before changing a bound slot's policy. +5. Call `create_revision` to snapshot the draft and binding IDs. +6. Call `validate_revision`, then poll `validation_status` and read `validation_logs` when the session is not passed. +7. Repair failures by updating the draft or binding metadata and creating a new revision. +8. Call `publish_revision` only after validation passed. Include `validationSessionId` when publishing from the session just reviewed. +9. Use `archive` only after human approval; the action follows the standard destructive-action approval flow. -Custom dashboards declare a `sourceNodeGraph` with nodes, edges, and optional metadata. Nodes have `id`, `type`, `title`, and optional JSON `config`. +## Data-Source Node Graph + +Each dashboard draft and revision can declare a `sourceNodeGraph`: + +```json +{ + "nodes": [ + { "id": "execution", "type": "project_dashboard_data", "title": "Project execution" }, + { "id": "stats", "type": "stats", "title": "Seven-day stats", "config": { "window": "7d" } } + ], + "edges": [], + "metadata": {} +} +``` + +Nodes have `id`, `type`, `title`, and optional JSON `config`. Edges have `fromNodeId`, `toNodeId`, and an optional `id`. The graph records the data the generated dashboard expects; it is also used by the in-app viewer to decide which source requests are allowed. + +Supported user-level source types: | Source type | Runtime behavior | | --- | --- | -| `project_dashboard_data`, `project_dashboard`, `dashboard_data` | Reads project execution data. | -| `stats`, `project_stats` | Reads project stats. `config.window` selects the stats window when present. | -| `telemetry`, `overview_telemetry` | Reads overview telemetry. | -| `integrations_metadata`, `integrations` | Returns only non-secret metadata declared on the source node. | -| `external_api` | Placeholder only. Arbitrary external calls are not proxied and return an unavailable-source error. | +| `project_dashboard_data`, `project_dashboard`, `dashboard_data` | Reads project execution data from `GET /api/projects/:projectId/execution`. | +| `stats`, `project_stats` | Reads project stats from `GET /api/projects/:projectId/stats`; `config.window` selects the stats window when present, otherwise `7d` is used. | +| `telemetry`, `overview_telemetry` | Reads overview telemetry from `GET /api/telemetry/overview`. | +| `integrations_metadata`, `integrations` | Returns only the non-secret metadata declared on the source node. It does not expose provider credentials or effective settings secrets. | +| `external_api` | Placeholder only in the in-app viewer. It is declared in the graph and validation bridge, but arbitrary external calls are not proxied and return an unavailable-source error. | + +Unsupported source types return an explicit unavailable-source error. Generated dashboards should handle these errors visibly instead of assuming all declared data is available. + +## REST API Surface + +Custom dashboard routes are registered with the dashboard server: + +| Method | Route | Purpose | +| --- | --- | --- | +| `GET` | `/api/projects/:projectId/custom-dashboards` | List dashboards for a project. | +| `POST` | `/api/projects/:projectId/custom-dashboards` | Create a mutable draft. | +| `GET` | `/api/projects/:projectId/custom-dashboards/data-catalog` | Return project dashboard summaries and declared source nodes. | +| `GET` | `/api/custom-dashboards/:dashboardId` | Return a dashboard plus revisions. | +| `PATCH` | `/api/custom-dashboards/:dashboardId` | Update mutable draft fields. | +| `DELETE` | `/api/custom-dashboards/:dashboardId` | Archive the dashboard and clear active publication. | +| `POST` | `/api/custom-dashboards/:dashboardId/revisions` | Create an immutable revision from the draft or supplied overrides. | +| `POST` | `/api/custom-dashboards/:dashboardId/revisions/:revisionId/validate` | Start a detached validation session. Body may include `projectId`; otherwise the server resolves it from the revision. | +| `POST` | `/api/custom-dashboards/:dashboardId/revisions/:revisionId/publish` | Publish a validated revision, optionally with `validationSessionId`. | +| `GET` | `/api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings?revisionId=...` | Review draft or revision slots, current bindings, backend health, and bounded compatible credential metadata. | +| `PUT` | `/api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings` | Bind or replace one slot using `slotId`, `credentialId`, and `expectedBindingRevision`. | +| `DELETE` | `/api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings/:slotId` | Unbind one slot using `expectedBindingRevision`. | +| `GET` | `/api/custom-dashboard-validations/:sessionId` | Read validation session status and runtime metadata. | +| `GET` | `/api/custom-dashboard-validations/:sessionId/logs?tail=200` | Read bounded validation and container logs. | +| `POST` | `/api/custom-dashboard-validations/:sessionId/stop` | Stop the detached validation container. | +| `DELETE` | `/api/custom-dashboard-validations/:sessionId` | Remove a validation session after cleanup. | +| `ALL` | `/api/custom-dashboard-validations/:sessionId/proxy{*rest}` | Same-origin proxy to the detached validation runtime. | +| `ALL` | `/api/custom-dashboards/validation-sessions/:sessionId/proxy{*rest}` | Backward-compatible validation proxy route. | + +The binding routes are credential-management routes: authenticated remote callers require `credential_admin`, project access, and enabled remote credential management. Stale binding revisions return `409`; incompatible credential selection returns `403`. Publication first repeats metadata-only binding review, then applies the repository validation gate. REST and MCP denials preserve a sanitized `issues` array with slot-specific `field`, `code`, and `message` values while omitting credential IDs and values. Active publications remain the opening source of truth while later validation sessions run. + +## MCP Surface + +The dedicated MCP tool is `manage_custom_dashboards` and is available to the project-manager runtime role. It supports: + +- `list`, `get`, `create`, `update` +- `create_revision` +- `validate_revision`, `validation_status`, `validation_logs` +- `publish_revision` +- `archive` +- `data_catalog` +- `list_credential_slots`, `bind_credential`, `unbind_credential` + +Credential actions use `projectId`, `dashboardId`, `slotId`, `credentialId`, and `expectedBindingRevision`; an optional `revisionId` reviews an immutable snapshot. Bind and unbind require the normal stateful human-approval handshake. Before creating any approval fingerprint, Code UX rejects secret, header, environment, malformed approval, and other undeclared fields, then rebuilds the approval payload from only the allowed metadata. Other important fields include `sessionId`, `validationSessionId`, `title`, `description`, `manifest`, `fileBundle`, `sourceNodeGraph`, `styleguide`, `runtimeMetadata`, `tail`, and `approval`. + +The dashboard chat JSON-action bridge also understands the legacy `custom_dashboards` management domain, but agents should prefer the dedicated MCP tool when it is available. + +## Validation Runtime + +Validation sessions move through `queued`, `building`, `running`, `passed`, `failed`, or `cancelled`. -Generated dashboards should handle unavailable-source errors visibly. External API connectors are not fully available through the in-app viewer yet. +During validation, Code UX: -## Agent and API Notes +- performs metadata-only compatibility review for every bound slot and every required slot +- creates a validation session row and runtime directory under the selected project +- writes the generated bundle plus a known Vite/Preact harness +- injects a read-only `codeUxDataBridge` / `CodeUXCustomDashboard` object +- runs install and build in Docker using the resolved CLI workflow image +- persists the built Vite `dist` files on the validated revision as the published-viewer artifact +- starts a detached preview container on an allocated localhost port +- health-checks the root URL before marking the session passed +- records workspace path, log path, container id/name, host port, validation proxy path, commands, and log excerpts in runtime metadata -Project Manager agents use the `manage_custom_dashboards` MCP tool to create drafts, list credential slots, bind or unbind credential IDs, create revisions, validate revisions, inspect logs, publish passed revisions, archive dashboards, and read the data catalog. Credential mutations require the normal stateful human-approval handshake and an optimistic `expectedBindingRevision`; unsupported or secret-bearing fields are rejected before approval state is created. +Required missing bindings and bound credentials that are missing, revoked, inaccessible, unconfigured, wrong-kind, missing capabilities, or blocked by unavailable/insecure key custody fail with slot-specific issues before workspace creation. Optional unbound slots remain valid. No custom-dashboard path resolves secret plaintext: credential values and binding IDs stay out of generated source, file bundles, bridge files, Docker arguments and mounts, validation reports and logs, viewer records, iframe configuration, and browser messages. Generic response and viewer boundaries recursively redact known binding IDs from nested manifests, file content and metadata, source graphs, styleguides, runtime metadata, validation reports, and persisted viewer artifacts. Dedicated credential-binding management responses may return credential IDs and non-secret metadata so operators and agents can select them. -The same workflow is available through the dashboard REST API: +The `build` and `runtime` slot phases are bounded declarations used for review and policy only. They do not inject a secret into the build container, published artifact, iframe, MCP result, or runtime data bridge. This feature does not migrate or expose broader provider secrets. -- `GET/POST /api/projects/:projectId/custom-dashboards` -- `GET /api/projects/:projectId/custom-dashboards/data-catalog` -- `GET/PATCH/DELETE /api/custom-dashboards/:dashboardId` -- `POST /api/custom-dashboards/:dashboardId/revisions` -- `POST /api/custom-dashboards/:dashboardId/revisions/:revisionId/validate` -- `POST /api/custom-dashboards/:dashboardId/revisions/:revisionId/publish` -- `GET /api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings` -- `PUT /api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings` -- `DELETE /api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings/:slotId` -- `GET /api/custom-dashboard-validations/:sessionId` -- `GET /api/custom-dashboard-validations/:sessionId/logs` -- `POST /api/custom-dashboard-validations/:sessionId/stop` -- `DELETE /api/custom-dashboard-validations/:sessionId` -- `ALL /api/custom-dashboard-validations/:sessionId/proxy{*rest}` +Stopping a validation session removes the detached container. It does not invalidate a passed revision report. Removing a validation session deletes the session row after cleanup; the revision's validation metadata remains the publication gate. -Remote callers to credential-binding routes require the credential-administrator role, project access, and enabled remote credential management. Required missing bindings and bound credentials that are revoked, inaccessible, unconfigured, wrong-kind, missing capabilities, or blocked by unavailable key custody fail before workspace creation and are rechecked before publication. REST and MCP publication denials include sanitized slot-specific issues without credential IDs or values. Optional unbound slots remain valid. +## Published Viewer and Rollback -Custom-dashboard binding is metadata-only: no secret is resolved, and credential values and binding IDs are excluded from generated files, bridges, Docker configuration, validation output, generic REST/MCP responses, iframe configuration, and browser messages. Generic response and viewer boundaries recursively redact known IDs from nested manifests, files, source graphs, runtime metadata, validation reports, and viewer artifacts. Dedicated binding-management responses may return credential IDs and non-secret metadata so operators and agents can select them. +The in-app viewer renders only published dashboards whose active `publishedRevisionId` points to a revision with a passed validation report. For the default `src/dashboard.tsx` draft and other TSX/Preact revisions validated through the harness, the viewer uses the persisted Vite `dist` artifact instead of the source entry file, so publication does not depend on the detached validation container still running. Generated code runs inside a sandboxed iframe document and talks to the parent app through a constrained `postMessage` bridge. The parent serves only declared source-node requests. -Published dashboards render inside a sandboxed iframe. For TSX/Preact drafts such as the default `src/dashboard.tsx` bundle, the viewer uses the persisted validation artifact instead of the source entry file, so it can open after publication even when the detached validation preview is gone. The frame can request only declared source nodes through the Code UX bridge, parent and frame handlers verify the expected window source, and the parent dashboard returns data through same-origin API calls. +Rollback is publish-based: select an earlier passed revision and publish it again. The publication pointer moves back to that immutable revision. Archive is the safe removal path when no dashboard should be active; it clears the publication pointer while preserving history. diff --git a/docs-web/content/docs/user-dashboard-node-flows.mdx b/docs-web/content/docs/user-dashboard-node-flows.mdx index c767364e31..f77885f78b 100644 --- a/docs-web/content/docs/user-dashboard-node-flows.mdx +++ b/docs-web/content/docs/user-dashboard-node-flows.mdx @@ -1,42 +1,44 @@ -# Node Flows +# Node Flows Dashboard The **Nodes** page (`/nodes`) is the project-scoped backend authoring, publication, and operations surface for canonical node flows. No selected project means no flow library, credential metadata, publications, or durable run history are requested. -## Library, Drafts, And Migration +## Library, drafts, and migration -The flow library contains backend drafts and publications owned by the active project. Saves include the loaded draft revision, so a concurrent edit produces a visible conflict and never overwrites newer work. +The library loads through `GET /api/projects/:projectId/node-flows`. Drafts are created through `POST /api/projects/:projectId/node-flow-drafts` and saved through revision-checked `PATCH /api/node-flow-drafts/:flowId`. A stale revision produces a visible conflict and never overwrites newer work. -The former browser graph at `codeux:nodes-canvas:v1` is eligible for one import into the selected project. The bridge maps `trigger`, `agent`, and `task` to registered `input`, `set_fields`, and `provider_prompt` definitions, retains `condition` and `output`, and remaps their ports before creating an **Imported Nodes Canvas** draft. A failed import remains retryable and does not block the normal library load; only success removes the old value and records the marker. +The former browser graph at `codeux:nodes-canvas:v1` is eligible for one import into the selected project. The bridge maps `trigger` to `input`, `agent` to `set_fields`, and `task` to `provider_prompt`; `condition` and `output` remain governed definitions, ports are remapped, and legacy configuration is retained as non-secret metadata. Code UX creates an **Imported Nodes Canvas** backend draft and only then removes the legacy value and records a project-specific marker. A failed import remains retryable and is isolated from normal library loading, while a successful marker prevents duplicates. -## Registry-Driven Editing And Credentials +## Registry-driven editing and credentials -The registry list returns flat versioned palette summaries. Selecting a definition loads the full manifest from the node-type detail endpoint, including nested `ui.widgetSchema`, configuration schema, policies, documentation, and deprecation metadata. The inspector renders from that full contract. Graphs reference a definition version and store non-secret configuration and credential ids; they do not contain custom-node source or resolved credentials. +`GET /api/node-flow-catalog` returns flat versioned palette summaries. `GET /api/node-flow-catalog/:nodeType` returns the full `NodeDefinitionManifest`, including nested `ui.widgetSchema`, configuration schema, policies, documentation, and deprecation metadata. The inspector renders from that full contract. Graphs reference a definition version and store non-secret configuration and credential ids; they do not contain custom-node source or resolved credentials. Credential slots use the versioned definition's allowed kinds and required capabilities to offer project-visible credential metadata. Only active, configured credentials with project access and a healthy secure backend are selectable; unavailable entries explain the operator-facing reason without exposing secret or key-custody details, and an empty compatible set links directly to **Settings → Integrations**. -Selecting, replacing, or removing a credential updates only that slot in the node's canonical `credentialBindings` and immediately saves the complete draft through the current optimistic revision. The dashboard then adopts the canonical flow revision and refreshes governed review. Saving, saved, policy-denial, and error states are announced. A revision conflict loads the latest draft, preserves the selected slot workflow and sibling edits, and requires the operator to choose again rather than replaying the stale mutation. Credential plaintext remains behind the broker and is excluded from graph data, component state, and browser output. +Selecting, replacing, or removing a credential updates only that slot in the node's canonical `credentialBindings` and immediately saves the complete draft through the current optimistic revision. The dashboard then adopts the canonical flow revision and refreshes governed review. Saving, saved, policy-denial, and error states are announced. A revision conflict loads the latest draft, preserves the selected slot workflow and sibling edits, and requires the operator to choose again rather than replaying the stale mutation. Credential plaintext remains behind the broker and is excluded from graph data, component state, browser output, logs, and documentation examples. + +Removing a required binding is allowed as a draft edit but immediately changes review and publication readiness to blocked; removing an optional binding remains valid. Publication is denied for required missing bindings and for credentials that become unavailable, unconfigured, revoked, inaccessible to the project, wrong-kind, or short of a required capability. Runtime repeats compatibility against the immutable publication, so a later custody outage, restriction, revocation, or rebinding denies execution instead of using a stale dashboard decision. The complete governed built-in set currently registered with executable handlers is `input`, `set_fields`, `template`, `provider_prompt`, `http_request`, `condition`, `switch`, `foreach`, `merge`, `delay`, `approval`, `email_draft`, `email_send`, `execute_subflow`, `webhook_trigger`, and `output`. -Registered custom definitions can execute only when their validated versioned manifest, immutable artifact, and custom-node runtime are available. Raw legacy kinds are translated during import rather than executed directly. Unknown or unregistered types, mockup entries, and definitions marked non-executable remain planned or unavailable. +Registered custom definitions can execute only when their validated versioned manifest, immutable artifact, and custom-node runtime are available. Raw legacy `trigger`/`agent`/`task` kinds are translated by the browser import bridge rather than executed directly. Unknown or unregistered types, mockup entries, and definitions marked non-executable are planned or unavailable definitions. -## Governance And Publication +## Governance and publication Draft review provides structural validation, policy findings, requested permissions, side-effect review, and a non-executing dry run. Publication requires the current draft revision, a valid governed review, and all required credentials. Each publication is an immutable snapshot; comparison and rollback operate on versioned history, and only a pinned or latest-published version can execute. -## Durable Debugger And Scheduling +## Durable debugger and scheduling + +The debugger reads persisted flow runs, node runs, attempt history, retry classifications and decisions, approval records, invocation links, timing, and redacted input and output. Pending approvals expose keyboard-accessible **Approve & continue** and **Reject** actions. A decision continues or terminates the same pinned run, and repeated decisions return its current durable state without duplicating a governed attempt or external send. The debugger also supports cancellation and safe retry. -The debugger reads persisted flow runs, node runs, attempt history, retry classifications and decisions, approval records, invocation links, timing, and redacted input and output. Pending approvals offer **Approve & continue** and **Reject** actions. A decision continues or terminates the same pinned run, and repeated decisions return its current durable state without duplicating a governed attempt or external send. The debugger also supports cancellation and safe retry. +Foreach runs persist one downstream node run and attempt sequence per deterministic logical item. Item inputs, retries, cancellation, approvals, and side-effect identity survive restart; concurrency is bounded by the node configuration. Empty collections select the explicit `empty` branch and persist the item branch as skipped, while oversized collections fail instead of being truncated. -Use the [Scheduler](/docs/user-dashboard-scheduler) to target a pinned or latest-published version. A flow can also be attached to a project agent preset as a reusable skill; removing the attachment does not remove the flow, publications, schedules, or run history. +The layout stacks on small screens, preserves keyboard-visible focus, labels loading/error/empty states, and bounds long histories and JSON output with scrolling. Rendered run payloads redact secret-shaped keys such as `apiKey`, `authorization`, `cookie`, `password`, `secret`, and `token`. The run debugger lists durable approvals beside node attempts. A pending item offers **Approve & continue** and **Reject** actions. The decision applies to the same pinned run, and repeated clicks return its current state without sending an approved external effect twice. -Foreach executes the selected downstream branch once per deterministic logical item. The node's `concurrency` setting bounds active items, `maxItems` rejects oversized inputs, and zero items explicitly select the `empty` branch. Item-specific inputs, retries, cancellation, approvals, and external-effect identity are persisted so restart continuation does not replay completed items or duplicate sends. - -## Agent Attachment +## Agent attachment A selected project loads its agent presets, and selecting a flow loads that flow's current bindings. The inspector exposes only agent names and attachment skill metadata; it never renders agent instructions, custom source, credential values, or decrypted material. @@ -44,9 +46,9 @@ Attaching and detaching use the governed node-flow attachment routes, then refre A flow can be attached to a project agent preset as a repeatable skill with a name and description. Detaching removes only that binding; the flow, its graph, schedules, and run history remain in the project. -## Scheduling +Scheduling is entered through `/scheduler` and targets a pinned or latest-published version. A flow can also be attached to a project agent preset as a reusable skill; removing the attachment does not remove the flow, publications, schedules, or run history. -Use the [Scheduler](/docs/user-dashboard-scheduler) page to run a saved node flow once or on a recurrence. Scheduled node-flow entries select a project-owned flow and may include optional JSON object input. Pause, resume, failure handling, and due-run behavior match the normal scheduler model. +Scheduled node-flow entries select a project-owned flow and may include optional JSON object input. Pause, resume, failure handling, and due-run behavior match the normal scheduler model. ## Graph v2 boundary diff --git a/docs-web/operations/credential-security.md b/docs-web/operations/credential-security.md index 41fa50e335..48665eb98f 100644 --- a/docs-web/operations/credential-security.md +++ b/docs-web/operations/credential-security.md @@ -1,50 +1,76 @@ # Automation Credential Security -Code UX resolves canonical node credential IDs and named project binding keys through the credential broker. Stored values are not exposed to nodes, dashboard reads, MCP payloads, agent context, run inspection records, or access audits. +Code UX stores automation credentials through a broker rather than exposing secret values to node definitions, dashboard reads, MCP payloads, agent context, or run inspection records. Canonical node bindings reference credential metadata by ID; only the broker can resolve the value at execution time after project and capability checks. Named project binding keys use the same broker for other automation consumers. ## Scope and policy -- Project credentials are owned by one project. -- Global credentials require an explicit project allowlist and retain the configuring project as their management owner. Other allowlisted projects may bind and resolve the credential but cannot mutate it. -- The credential kind must be allowed, and both the binding and credential must approve every declared capability before one secret read. +- Project credentials can be managed only through their owning project. +- Global credentials are opt-in and require an explicit project allowlist containing the configuring project. The configuring project remains the credential's management owner after promotion; other allowlisted projects may bind and resolve it but cannot rotate, replace, revoke, promote, or restrict it. +- Resolution succeeds only when the credential kind is allowed and both the credential and binding approve every declared capability. Authorization is completed before the broker performs its single secret read. - Revoked, unavailable, missing, cross-project, or insufficiently capable credentials fail closed. Node-flow definition slots explicitly declare required/optional state, allowed kinds, and required capabilities. Draft review and every publication path use the broker's metadata-only compatibility assessment; runtime sends the same declaration to direct credential-id resolution immediately before execution. Graph `credentialBindings` are canonical. The legacy credential-request endpoint records no binding and identifies its result as non-persistent. -Create, rotate, and replace requests are write-only. API responses contain configuration and status metadata but never stored values. +The dashboard accepts secret values only on create, rotate, and replace requests. Responses contain configuration, scope, status, key-version, and validation metadata but never stored values. Access-event rows contain identifiers, binding keys, capabilities, outcomes, and denial reasons; they never contain secret material. -Create requests explicitly declare kind, scope, capabilities, and an allowlist (empty for project credentials). Runtime validation bounds names, identifiers, capabilities, list counts, and secret size (64 KiB UTF-8). Malformed arrays, unknown mutation fields, and control characters are rejected rather than coerced. +The Settings Integrations catalog exposes this broker as its first standard card. The card derives unavailable, ready/unconfigured, and configured states from backend health and project-visible metadata; **Manage** opens a project-aware detail view without rendering a secret, request body, or raw server error. Allowlisted non-owner projects can understand and use compatible global credentials but see management actions disabled. -Every lifecycle mutation includes `expectedVersion`. The only mutable descriptive field is the bounded name; kind and management ownership remain immutable. Restrictions may remove allowlisted projects or capabilities but cannot add them. Project-to-global promotion is the explicit scope expansion and requires managing-project authority, `confirmScopeExpansion: true`, the current version, and an allowlist of existing projects that retains the managing project. Current-version repeated revocation is idempotent; stale versions conflict. +Create controls require deliberate capability selection and explicit project or global scope. Global allowlists retain the management owner, and scope-expanding creation or promotion is confirmed. Rename, test, rotation/replacement, restriction, promotion, and revocation report typed inline status, disable overlapping actions, and refresh after stale-version conflicts. Destructive and scope-expanding actions use keyboard-operable confirmation dialogs with focus restoration. + +All create, rotate, and replacement fields are controlled write-only inputs. They are never hydrated from metadata and are cleared after every submission outcome, project change, and component teardown. Credential metadata drafts and browser stores do not receive secret values. + +Management inputs are validated at runtime rather than trusted from TypeScript types. Create requests must explicitly declare kind, scope, capabilities, and an allowlist (an empty array for project credentials). Names, kinds, binding keys, project ids, capabilities, and list counts are bounded; malformed arrays, unknown mutation fields, and control characters are rejected instead of being silently coerced. A stored value is limited to 64 KiB of UTF-8 data. Global allowlists must explicitly retain the management owner. + +Every lifecycle mutation carries `expectedVersion`. Successful name updates, validation tests, rotations/replacements, promotions, restrictions, and first-time revocations increment the version. A repeated revoke against an already-revoked credential at its current version is an idempotent no-op; stale requests return a conflict. Metadata updates may change only the bounded display name, so kind and management ownership remain immutable. + +Restriction is monotonic: it may remove allowlisted projects or capabilities but cannot add either. Project-to-global promotion is the explicit scope expansion and requires the managing project, a current version, `confirmScopeExpansion: true`, an allowlist containing the managing project, and project IDs that already exist. ## Runtime redaction boundary -Node-flow credentials exist in plaintext only for the active node attempt. Exact resolved values are replaced with `[REDACTED]` before provider responses, HTTP bodies, retry errors, external-effect payloads, diagnostics, invocation messages, attempts, node outputs, or run summaries are stored. Credential IDs and non-secret metadata remain available for auditability. +Node-flow execution resolves credential values only for the active node attempt. Before any provider response, HTTP body, retry error, external-effect payload, diagnostic, invocation message, attempt, node output, or run summary is persisted, the runtime replaces exact resolved values with `[REDACTED]` in addition to masking secret-shaped keys. Credential IDs and non-secret metadata remain available for audit and attempt correlation. -The same redactor protects provider activity and raw usage telemetry. Temporary credential references are cleared after the attempt and are never logged as redaction input. Custom-node outputs, stderr logs, and diagnostics follow the same rule. +Provider activity persistence uses the same invocation-scoped redactor, including raw usage telemetry and provider session identifiers. Temporary credential references are cleared after each attempt and are never included in redaction logs or diagnostics. Custom-node containers apply the equivalent policy to structured output, stderr logs, and diagnostics before returning control to the flow runtime. -Authorization is rechecked after decryption. Concurrent revocation, rotation, restriction, promotion, or rebinding clears the plaintext buffer and causes a retry or denial instead of returning stale access. +Resolution authorization is checked both before and after decryption. If a credential is revoked, rotated, restricted, promoted, or rebound while a read is in flight, the plaintext buffer is cleared and the broker denies or retries against the current version; stale authorization is never returned to the caller. ## Encryption and key custody -The SQLite secret store uses AES-256-GCM envelope encryption with a unique data key, payload nonce, and key-wrapping nonce for every write. Credential ownership and workspace context are authenticated. SQLite stores ciphertext, authentication tags, wrapped keys, nonces, and key identifiers/versions—not root keys. +The SQLite secret store uses AES-256-GCM envelope encryption. Each write generates a unique 256-bit data key, payload nonce, and key-wrapping nonce. Credential ownership and workspace context are authenticated as additional data. SQLite stores only ciphertext, authentication tags, wrapped keys, nonces, and key identifiers/versions. + +Root keys are never stored in SQLite or a project checkout. The normal loopback dashboard automatically provisions one raw 32-byte root key at `~/.code-ux/security/credential-root.key`. Its dedicated parent directory is `0700` and the regular file is `0600`. Creation uses an exclusive atomic install, durable filesystem synchronization where supported, and concurrent startup convergence so restarts recover the identical key. Before creation or access, every custody-path component from the Code UX home through the key parent is inspected without following symbolic links; a symbolic-link or non-directory ancestor fails closed before a redirected key can be provisioned. Existing symbolic links, non-files, malformed keys, permissive modes, or unexpected ownership are never repaired automatically; credential operations fail closed with metadata-only setup guidance. -The trusted loopback dashboard automatically provisions one raw 32-byte root key at `~/.code-ux/security/credential-root.key`, with a `0700` parent and `0600` regular file. Provisioning is exclusive, atomic, durable where filesystem synchronization is supported, and safe across concurrent startup. Every custody-path component from the Code UX home through the key parent is inspected without following symbolic links, so a symbolic-link or non-directory ancestor fails closed before a redirected key can be provisioned. Symbolic links, non-files, malformed content, permissive modes, and unexpected ownership fail closed and are not repaired automatically. +Automatic local-file custody is limited to the non-server dashboard with local authentication, loopback binding, and remote credential management disabled. Electron's process provider remains first priority and continues to use OS `safeStorage`. Explicit `CODE_UX_CREDENTIAL_KEY_PROVIDER=mounted-key-file|vault|kms` configuration takes priority over automatic custody; setting `CODE_UX_CREDENTIAL_KEY_FILE` alone remains compatible with the mounted-file provider. Unknown values and an explicit `local-file` selection are rejected. Dashboard-disabled headless operation, server mode, authenticated dashboards, non-loopback bindings, and remote credential-management deployments do not auto-provision a local key. -Automatic local-file custody is disabled for server mode, dashboard-disabled headless operation, authenticated or non-loopback dashboards, and remote credential management. Electron remains first priority and persists only an OS-protected blob. Explicit `CODE_UX_CREDENTIAL_KEY_PROVIDER=mounted-key-file|vault|kms` configuration takes priority; `CODE_UX_CREDENTIAL_KEY_FILE` alone remains a compatible mounted-file selection. Unknown values and explicit `local-file` selection are rejected. If secure key material is unavailable, credential operations fail closed; there is no plaintext fallback. +For mounted-file custody, `CODE_UX_CREDENTIAL_KEY_FILE` identifies a regular, owner-only mounted file whose contents are an exact base64 or hexadecimal encoding of 32 bytes. Oversized or permissively decodable key files are rejected. The environment variable contains a path, not key material. Keep the mount readable only by the Code UX process and outside the project workspace. + +Electron serializes first-use root-key creation, persists only the OS-protected blob through an atomic owner-only file replacement, and refuses credential operations when `safeStorage` is unavailable. Vault and KMS adapters validate 32-byte caller-owned key material and report the active key id/version in health results. No provider silently falls back to plaintext or an insecure locally derived key. + +| Deployment boundary | Root-key custody | Provisioning behavior | +| --- | --- | --- | +| Normal CLI dashboard on loopback with local authentication | Owner-only file under the user-home Code UX security directory | Automatically created on first use and reused after restart. A normal local dashboard user does not mount or configure a key file. | +| Electron desktop | Operating-system `safeStorage` | Automatically creates and persists only the OS-protected blob; unavailable `safeStorage` blocks credential operations. | +| Dashboard-disabled headless, server mode, authenticated dashboard, non-loopback binding, or remote credential management | Explicit mounted file, Vault, or KMS provider | Never auto-provisions local custody. Setup and recovery fail closed until the configured provider reports available, secure key identity and version metadata. | ## Recovery and rotation -Back up root keys separately from `app.db`; the database alone cannot recover credentials. Local dashboard backups must include `~/.code-ux/security/credential-root.key` with owner-only handling. Creation, rotation/replacement, and promotion commit ciphertext and metadata atomically. Version compare-and-swap protects every lifecycle mutation and permits only one overlapping value change to commit. Revocation also wins against an in-flight resolution while retaining audit metadata. +Back up root keys independently from `app.db`. For the normal local dashboard, back up `~/.code-ux/security/credential-root.key` while preserving owner-only handling; for external providers, retain every referenced key version. Losing a required key version makes its ciphertext unrecoverable by design. Restoring only SQLite is insufficient. + +Credential creation commits metadata and its first envelope in one SQLite transaction. Rotation/replacement and promotion likewise commit the new envelope, metadata, version, and rotation record atomically. Compare-and-swap guards apply to every lifecycle mutation so losing callers must refresh metadata and retry instead of overwriting newer state. Root-key providers must retain old key IDs and versions until envelopes are rewrapped. Revocation wins against in-flight resolutions and preserves audit metadata. -Lifecycle success and denial audits carry correlation IDs, credential IDs, and policy metadata only. Validation records `valid`, `invalid`, or `unavailable` without exposing tested values or cryptographic internals. +Lifecycle successes and denials emit correlation-aware automation audit records containing credential IDs and policy metadata only. Validation updates report `valid`, `invalid`, or `unavailable` without including tested values or low-level cryptographic errors. Custom dashboards use a stricter metadata-only consumer boundary. Dedicated slot declarations define allowed kinds and required capabilities, while separate draft and immutable-revision binding columns store credential IDs. Binding review delegates to the broker's compatibility assessment and never resolves plaintext. Required or invalid bindings stop validation before workspace creation and are rechecked before publication. Credential values and binding IDs are excluded from generated dashboard artifacts, Docker configuration, validation output, generic REST/MCP responses, and iframe messages; only the dedicated binding-management response may expose IDs with non-secret metadata. -Legacy global records use their first valid allowlisted project as the migrated management owner; verify that owner before expanding an old global allowlist. +Existing global credentials created before management ownership was stored are migrated with their first valid allowlisted project as the management owner. Operators should verify that owner before expanding a legacy global credential's allowlist. + +## API surface + +Project-scoped routes live under `/api/projects/:projectId/credentials`. Supported operations are create, bounded-name update (`PATCH /:credentialId`), bind, metadata-only compatibility assessment, test, rotate, replace, revoke, promote, and restrict. Compatibility evaluates key-backend readiness, configuration, active status, project access, allowed kinds, and all required capabilities without resolving plaintext. A backend is ready only when it is available and secure and reports both a non-empty key ID and a key version; missing key identity metadata produces the stable `backend_unavailable` compatibility issue. List, compatibility, health, and mutation responses return metadata or policy results only. Existing dashboard authentication and remote credential-management guards apply before these routes. -## Dashboard API +Runtime validation failures return `400`, project/management denials return `403`, compare-and-swap conflicts return `409`, invalid encrypted state returns `422`, and unavailable key custody returns an actionable `503` response. -Credential management uses project-scoped dashboard routes. The API includes create, bounded-name update, bind, metadata-only compatibility assessment, test, rotate, replace, revoke, confirmed promotion, and monotonic restriction. Compatibility checks backend readiness, configured/active state, project access, allowed kinds, and all required capabilities without reading plaintext. Backend readiness requires an available, secure backend with a non-empty key ID and a reported key version; missing identity metadata produces `backend_unavailable`. List, health, compatibility, and mutation responses never contain secret values; secrets are accepted only by create, rotate, and replace operations. +## Troubleshooting without disclosure -Validation failures return `400`, project/management denials return `403`, concurrent-write conflicts return `409`, invalid encrypted state returns `422`, and unavailable key custody returns `503` with a safe recovery message. +- If custody is unavailable, inspect the metadata-only credential health or readiness result and the configured provider name. For the normal loopback dashboard, verify ownership, file type, and owner-only modes on the existing Code UX security path; for Electron, restore OS `safeStorage`; for headless or remote operation, restore the configured mount, Vault, or KMS version. Never paste, print, regenerate over, or move root-key material into a repository to diagnose the failure. +- If a mutation reports a stale `expectedVersion`, refresh credential metadata and review the newer scope, capabilities, validation state, and status before retrying. Do not reuse the rejected request blindly and do not bypass the comparison check. +- If encrypted rows exist but their key version is unavailable, restore the exact retained provider version before starting runners. Replacing it with a new key does not decrypt old envelopes; restore from the independent custody backup or recover the affected credential through the supported replacement workflow after the runtime is ready. diff --git a/docs-web/operations/server-mode.md b/docs-web/operations/server-mode.md index 27ba8c0fa4..c02e9eb262 100644 --- a/docs-web/operations/server-mode.md +++ b/docs-web/operations/server-mode.md @@ -1,29 +1,249 @@ -# Authenticated Headless Server Mode +# Secure Headless Server Mode -Code UX separates MCP bearer access from the authenticated dashboard administrative API. Remote dashboard/API deployments must use digest-backed service identities or terminate OIDC at a trusted reverse proxy; loopback desktop operation remains a trusted local boundary. +Server mode runs Code UX as an authenticated MCP HTTP control plane without binding the dashboard UI, dashboard REST routes, dashboard realtime websocket, terminal websocket, or static dashboard assets. Use it for headless hosts, CI-adjacent automation, and cluster worker control planes where clients connect over Streamable HTTP instead of launching Code UX over stdio. -## Identity and authorization +Server mode is different from ordinary `--headless` mode: -Set `CODE_UX_DASHBOARD_AUTH_MODE=service_token` and provide `CODE_UX_SERVICE_IDENTITIES_JSON` entries containing `id`, `displayName`, SHA-256 `tokenSha256`, `roles`, explicit `projectIds`, and `enabled`. Workers send the bearer through `CODE_UX_WORKER_AUTH_TOKEN` and may assert the matching identity with `CODE_UX_WORKER_SERVICE_ID`. +| Mode | Dashboard | MCP HTTP | Token behavior | +| --- | --- | --- | --- | +| Default dashboard mode | Enabled | Enabled by default | Uses an explicit token or the generated user token in `~/.code-ux/security.json`. | +| `--headless` / `--no-dashboard` | Disabled | Uses normal MCP HTTP enablement rules | Preserves local-development behavior and can use the generated user token when HTTP is enabled. | +| `--server-mode` / `CODE_UX_SERVER_MODE=true` | Disabled | Enabled by default | Requires an explicit MCP HTTP bearer token with at least 32 bearer-safe characters. | -Alternatively, set `CODE_UX_DASHBOARD_AUTH_MODE=trusted_proxy`, configure `CODE_UX_TRUSTED_PROXY_SECRET`, terminate/validate OIDC at the proxy, strip client identity headers, and inject trusted principal, role, and project headers. Authenticated remote traffic requires TLS (`X-Forwarded-Proto: https`) unless insecure HTTP is explicitly enabled for an isolated test. +## Threat Model -Roles are `credential_admin`, `automation_author`, `automation_publisher`, `automation_runner`, and `viewer`. Credential routes additionally require `CODE_UX_REMOTE_CREDENTIAL_MANAGEMENT=true`; enabling it without a healthy secure key provider makes readiness fail. Host/origin checks, no-store responses, and administrative rate limits remain active. +MCP bearer access remains a runtime-wide control-plane identity. The dashboard administrative API has a separate authenticated-headless boundary with project-scoped roles; do not treat an MCP bearer as a dashboard service identity. -The `credential_admin` role can still read administrative readiness, audit export, and SLO metrics while remote credential management is disabled. The feature flag gates credential-management and credential-health routes only. +## Authenticated Dashboard API -## Probes, audit, and SLOs +Remote dashboard/API operation is fail-closed. Setting a non-loopback `DASHBOARD_HOST` without an explicit authentication mode defaults the API to `service_token`, so unconfigured callers receive `401`/`403` instead of inheriting desktop access. Loopback desktop mode remains `local`. -`/health` is liveness. `/ready` also checks credential-key recovery, the audit store, and distributed-runner identities and returns `503` when required components are unavailable. If encrypted credential rows exist and their key cannot be recovered, startup aborts before listeners bind. Server mode never auto-provisions local-file custody; configure `mounted-key-file`, Vault, or KMS explicitly. +Choose one boundary: -Authenticated operators can use `/api/admin/readiness`, `/api/admin/audit/export` (redacted NDJSON), and `/api/admin/metrics/slo`. Audit covers management calls, credential access, runs, attempts, approvals, and outbox delivery with correlation ids. +- `CODE_UX_DASHBOARD_AUTH_MODE=service_token`: define `CODE_UX_SERVICE_IDENTITIES_JSON` as an array of identities with `id`, `displayName`, a lowercase SHA-256 `tokenSha256`, `roles`, `projectIds`, and `enabled`. Workers may send the matching id with `--service-identity-id` or `CODE_UX_WORKER_SERVICE_ID`; the bearer remains in `CODE_UX_WORKER_AUTH_TOKEN`. +- `CODE_UX_DASHBOARD_AUTH_MODE=trusted_proxy`: terminate OIDC at a trusted proxy, set `CODE_UX_TRUSTED_PROXY_SECRET`, and have the proxy overwrite `X-Code-UX-Proxy-Secret`, `X-Code-UX-Principal-Id`, `X-Code-UX-Roles`, `X-Code-UX-Project-Ids`, and optional name/kind headers. Never forward client-supplied copies. -Baseline alerts: readiness not ready for five minutes, management 5xx above 1% or p95 above one second for ten minutes, repeated lease expiry, credential-denial spikes, outbox failure backlog, or any secret/audit check failure. Target zero unauthorized project grants, secret disclosures, and duplicate side effects. +Roles are `credential_admin`, `automation_author`, `automation_publisher`, `automation_runner`, and `viewer`. Project ids are explicit; `*` is an operator-only all-project grant. Credential routes additionally require `CODE_UX_REMOTE_CREDENTIAL_MANAGEMENT=true`. Enabling that flag without a healthy secure key provider makes readiness fail. -## Backup and recovery +The `credential_admin` role can read `/api/admin/readiness`, `/api/admin/audit/export`, and `/api/admin/metrics/slo` even when remote credential management is disabled. The feature flag gates credential creation, binding, testing, rotation, replacement, revocation, promotion, restriction, and credential-health routes; it does not disable operational readiness, audit, or SLO inspection. -Back up SQLite with WAL consistency, settings, project `.code-ux/` state, and every referenced external key version. Restore keys before databases, keep runner admission disabled, require `/ready`, then reconcile leases, approvals, audit continuity, and outbox counts. Never back up plaintext service tokens beside their digests. +TLS is assumed at the reverse proxy. Authenticated remote requests must arrive with HTTPS or a trusted `X-Forwarded-Proto: https`; `CODE_UX_ALLOW_INSECURE_HTTP=true` is limited to isolated test networks. Same-origin browser checks, no-store headers, host validation, and a 600-request/minute administrative API limiter remain active. Webhook and provider-ingress endpoints retain their dedicated authentication schemes. -Rotate service identities by overlapping new/old digests until runners authenticate with the new token. Rotate credential values through the broker so graph bindings retain ids and resolve the next version. Retain old KMS/Vault versions until envelope rewrap and restore drills pass. +Example identity generation (the JSON stores only the digest): -Rollback creates and publishes a new draft from an earlier immutable version; in-flight runs stay pinned. Recovery requeues only known-safe pre-invocation work and leaves uncertain external outcomes for attention. OIDC validation and Vault/KMS client integration remain deployment-host responsibilities, and MCP bearer authority remains broader than dashboard roles. +```bash +token="$(openssl rand -base64 48 | tr -d '\n')" +digest="$(printf '%s' "$token" | sha256sum | cut -d' ' -f1)" +# Put $token in the runner secret manager and $digest in CODE_UX_SERVICE_IDENTITIES_JSON. +``` + +Use server mode when: + +- the dashboard must not be reachable from the host +- MCP clients or workers need a stable HTTP endpoint +- a reverse proxy or private network boundary provides TLS and network admission +- operators can treat the bearer token as a secret with full runtime authority + +Do not expose the MCP HTTP listener directly to the public internet. The Node listener is HTTP; terminate HTTPS with a trusted reverse proxy, tunnel, service mesh, or load balancer when traffic leaves the host. + +## Startup + +Generate the token in the process environment or a secret manager. Do not paste real bearer values into shell history, logs, tickets, release notes, or documentation. + +```bash +export MCP_HTTP_AUTH_TOKEN="$(openssl rand -base64 48 | tr -d '\n')" + +codeux \ + --server-mode \ + --mcp-http-host 127.0.0.1 \ + --mcp-http-port 4445 \ + --mcp-http-path /mcp +``` + +For a cluster control plane behind a reverse proxy or private network interface: + +```bash +export CODE_UX_SERVER_MODE=true +export MCP_HTTP_AUTH_TOKEN="$(openssl rand -base64 48 | tr -d '\n')" +export MCP_HTTP_HOST=0.0.0.0 +export MCP_HTTP_PORT=4445 +export MCP_HTTP_PATH=/mcp +export MCP_HTTP_MAX_SESSIONS=500 +export MCP_HTTP_SESSION_TIMEOUT_MS=3600000 + +codeux +``` + +The legacy `mcp-https` names remain supported for compatibility: + +| Purpose | Preferred | Legacy-compatible | +| --- | --- | --- | +| Gateway enablement | `MCP_HTTP_ENABLED`, `--no-mcp-http` to disable outside server mode | `MCP_HTTPS_ENABLED`, `--no-mcp-https` to disable outside server mode | +| Gateway host | `MCP_HTTP_HOST`, `--mcp-http-host` | `MCP_HTTPS_HOST`, `--mcp-https-host` | +| Gateway port | `MCP_HTTP_PORT`, `--mcp-http-port` | `MCP_HTTPS_PORT`, `--mcp-https-port` | +| Gateway path | `MCP_HTTP_PATH`, `--mcp-http-path` | `MCP_HTTPS_PATH`, `--mcp-https-path` | +| Bearer token | `MCP_HTTP_AUTH_TOKEN`, `--mcp-http-auth-token` | `MCP_HTTPS_AUTH_TOKEN`, `--mcp-https-auth-token` | +| Session cap | `MCP_HTTP_MAX_SESSIONS`, `--mcp-http-max-sessions` | `MCP_HTTPS_MAX_SESSIONS`, `--mcp-https-max-sessions` | +| Idle timeout | `MCP_HTTP_SESSION_TIMEOUT_MS`, `--mcp-http-session-timeout-ms` | `MCP_HTTPS_SESSION_TIMEOUT_MS`, `--mcp-https-session-timeout-ms` | + +Server mode rejects startup when the explicit token is missing, empty, shorter than 32 characters, or contains characters outside the bearer-safe set. It does not fall back to the generated local user token. + +If `--server-mode` is combined with an explicit MCP HTTP disable flag, server mode still restores the MCP HTTP listener on the default MCP port because the server-mode contract requires authenticated remote MCP access while the dashboard stays disabled. + +## Health And Readiness + +The MCP HTTP listener serves probes without the dashboard server: + +```bash +curl --fail http://127.0.0.1:4445/health +curl --fail http://127.0.0.1:4445/ready +``` + +Use `/health` for process liveness. It only proves that the listener is up. + +Use `/ready` for runtime readiness. It reports whether the Code UX runtime finished the required startup path and can accept work. During startup, maintenance such as Docker cleanup, preview reconciliation, branch reaping, and recovery work can continue after the listener binds, so `/health` can pass before `/ready`. + +Do not include `Authorization` headers in probe logs. The probe endpoints do not require bearer credentials. + +`/ready` also reports `credentialKey`, `auditStore`, and `distributedRunner`. `/health` remains live during a key-provider outage, while `/ready` returns `503`. Startup aborts before dashboard or MCP binding when encrypted credential rows exist but their key provider cannot recover the wrapping key. Server mode never auto-provisions local-file custody. Select a provider with `CODE_UX_CREDENTIAL_KEY_PROVIDER=mounted-key-file|vault|kms`; mounted files use `CODE_UX_CREDENTIAL_KEY_FILE` and owner-only permissions. Vault/KMS modes require their host adapter to be configured and healthy. + +The same explicit-custody requirement applies to dashboard-disabled headless operation, authenticated dashboards, non-loopback dashboard bindings, and remote credential management. Only the trusted loopback local dashboard auto-provisions its owner-only user-home key; Electron uses OS `safeStorage`. Remote setup therefore fails closed rather than borrowing the local-dashboard key, deriving a key, or falling back to plaintext. Restore the configured mount or the exact Vault/KMS key version before enabling runners; do not copy root keys into SQLite, a project checkout, deployment logs, or diagnostic bundles. + +Authenticated operators can inspect `/api/admin/readiness`, export redacted NDJSON from `/api/admin/audit/export`, and sample `/api/admin/metrics/slo`. Audit rows include the correlation id, principal, project, action, outcome, and redacted metadata for management requests, credential access, runs, attempts, approvals, and outbox delivery. + +## Backup, Restore, Rotation, And Rollback + +Back up `~/.code-ux/app.db` with a SQLite-aware snapshot that includes/checkpoints WAL state, the settings database, project `.code-ux/` directories, and the external key-provider versions needed by every encrypted envelope. Never place plaintext service tokens or root keys in the database backup. Restore into an isolated host, restore keys first, run `/ready`, then enable runners. + +Rotate service tokens by adding the new digest, deploying the new runner secret, observing successful authenticated calls, and disabling the old identity entry. Rotate credential values through the credential rotation API; existing graph bindings keep the credential id and resolve the new version. Retain old KMS/Vault key versions until every envelope has been rewrapped and a restore drill succeeds. + +To roll back an automation, create a new draft from the earlier immutable version, review it, and publish it. In-flight runs remain pinned to their original publication. Stop runner admission before database recovery; after restore, startup recovery requeues only known-safe work and leaves unknown external outcomes in `attention_required`. + +## Baseline SLOs And Alerts + +Initial operator baselines are 99.9% authenticated management availability, p95 management latency below 500 ms, zero unauthorized project grants, zero secret disclosure, and zero duplicate outbox side effects. Alert when readiness is not ready for 5 minutes, management 5xx rate exceeds 1% for 10 minutes, p95 exceeds 1 second for 10 minutes, leases repeatedly expire, denied credential access spikes, outbox failures remain pending for 5 minutes, or any audit/secret scanning check fails. + +Local mode is intentionally a trusted loopback desktop boundary. Authenticated headless mode adds API RBAC, project scope, key readiness, durable audit, and service identities, but it is not a general multi-tenant identity platform: OIDC token validation belongs at the trusted proxy, Vault/KMS require host adapters, and MCP bearer authority remains broader than dashboard roles. + +## Client Connections + +MCP HTTP clients connect to the configured path with `Authorization: Bearer `. The first JSON-RPC request on a new Streamable HTTP session must be `initialize`; the server returns an `mcp-session-id` header that the client echoes on later calls. + +For a local CLI or dashboard-adjacent session that supports MCP HTTP, configure: + +- URL: `http://:4445/mcp` +- header name: `Authorization` +- header value: `Bearer ` + +Verify without exposing the token: + +```bash +curl --fail http://127.0.0.1:4445/health +curl --fail http://127.0.0.1:4445/ready +``` + +Then verify through the MCP client by listing tools or running a read-only management action such as listing projects. Do not use `curl -v`, shell tracing, or command transcripts that print the authorization header. + +If a local dashboard app is used only as an operator console for a separate server-mode instance, configure its MCP client entry to the server-mode URL and bearer header. The dashboard UI of the server-mode process itself remains unavailable by design. + +## Settings Synchronization + +Settings synchronization uses the `manage_settings` bundle actions: + +- `export_settings_bundle` +- `apply_settings_bundle` + +Bundles can include system, project, and sprint scopes. Metadata includes `schemaVersion: 1`, `exportedAt`, `includedScopes`, a SHA-256 `fingerprint` computed from a secret-redacted representation, and `containsSecrets`. + +Approved workflow: + +1. Export a redacted bundle from the source runtime. Export defaults to the `system` scope and redacts provider API keys, git tokens, issue-tracker tokens, login credentials, and other secret-bearing fields. +2. Review the bundle before moving it to the destination. Redacted placeholders are expected and must not be replaced in shared artifacts. +3. If project or sprint settings are required, include `scopes`, `projectIds`, and `sprintIds`. Sprint exports require the owning `projectId` so imports can normalize sprint overrides against the resolved project base. +4. Apply the bundle on the destination with `apply_settings_bundle`. The importer persists through `saveSystemSettings`, `saveProjectSettings`, and `saveSprintSettings`, so values follow the same sanitizer and override normalization as dashboard saves. +5. For partial rollout or rollback, pass `scopes` on apply to limit which bundle scopes are written. + +Secret-bearing exports and imports require the stateful settings approval flow: + +- `includeSecrets: true` on export returns secrets only after the first response asks for approval and the exact same request is repeated with `approval.confirmed: true`. +- A bundle marked `containsSecrets: true`, or one whose payload contains secret-bearing fields, is applied only after the same one-use approval flow. +- Approval is bound to the exact normalized payload, expires after 15 minutes, and is consumed after one successful execution. + +Rollback is another approved apply. Export a known-good bundle before changing a destination runtime, then apply that bundle back to the affected scopes if the rollout must be reverted. Do not rely on logs or chat transcripts as backups because redaction intentionally removes sensitive values. + +## Cluster Workers + +External workers connect to the server-mode MCP HTTP endpoint as control-plane clients. The worker process also starts a local `worker-host` runtime over stdio for execution on the worker machine. + +Start a worker with the shipped bin: + +```bash +codeux-worker \ + --server-url http://SERVER_HOST:4445/mcp \ + --auth-token "$CODE_UX_WORKER_AUTH_TOKEN" \ + --connection-key worker:build-node-01 \ + --display-name "Build node 01" \ + --project-id project-id +``` + +Equivalent environment variables: + +```bash +export CODE_UX_WORKER_SERVER_URL=http://SERVER_HOST:4445/mcp +export CODE_UX_WORKER_AUTH_TOKEN="$MCP_HTTP_AUTH_TOKEN" + +codeux-worker --connection-key worker:build-node-01 --project-id project-id +``` + +Worker config supports multi-project operation: + +- repeat `--project-id` to register eligible projects +- repeat `--active-project-id` to advertise active project focus +- use a stable `--connection-key` so reconnects update the existing registered endpoint +- set `--server-command`, repeated `--server-arg`, and `--server-cwd` only when the worker-local execution runtime needs a custom command + +Cluster behavior: + +- Registered workers are not license-capped. The active Streamable HTTP session cap defaults to 100 and can be raised for large clusters. +- Project assignments live in `project_worker_assignments`. A project can have one primary worker and any number of overflow workers. +- Active-session protection prevents runaway clients from allocating unlimited Streamable HTTP sessions. Raise `MCP_HTTP_MAX_SESSIONS` only to the capacity the server can actually operate. +- Heartbeats derive endpoint status. Stale or offline workers are excluded from new claims, and stale primary workers can be bypassed by eligible overflow workers. +- Dispatch safety depends on both `task_dispatches` and `execution_leases`. A worker must not start local execution unless the server returns a claim with a lease token. Heartbeats renew the lease while the task runs; expired leases can be claimed by another eligible worker. +- Multi-project workers claim only work for projects they are assigned to and advertise as active or eligible. + +## Token Rotation + +Safe rotation is a short planned restart unless a reverse proxy or secret manager can coordinate old/new tokens externally. + +1. Generate a new token in the secret manager. +2. Update client and worker secret references, but do not restart them yet. +3. Restart the server-mode process with the new token. +4. Restart or reconnect MCP clients and workers so they initialize new sessions with the new token. +5. Confirm `/ready` passes and clients can list tools or claim work. +6. Revoke the old token from the secret manager and remove it from local shells, process managers, and deployment manifests. + +Existing HTTP sessions authenticated with the previous token should be treated as invalid after server restart because Streamable HTTP sessions are in memory. Workers should reinitialize rather than attempting to reuse old `mcp-session-id` values. + +## Troubleshooting + +| Symptom | Likely cause | Fix | +| --- | --- | --- | +| Startup fails with a server-mode token error | `--server-mode` or `CODE_UX_SERVER_MODE=true` is set without an explicit valid bearer token. | Set `MCP_HTTP_AUTH_TOKEN` or `MCP_HTTPS_AUTH_TOKEN`, or pass the matching CLI flag. Use at least 32 bearer-safe characters. | +| Startup fails when binding `0.0.0.0`, `::`, or a LAN address | MCP HTTP is reachable beyond loopback without an active token. | Configure an explicit bearer token and put TLS/auth network controls in front of the HTTP listener. | +| Dashboard URL is unavailable | Expected in server mode. | Use MCP HTTP clients and `/health` or `/ready`. Start a separate dashboard-mode process only when an operator UI is required. | +| HTTP returns `401 Unauthorized` | Missing `Authorization: Bearer `, wrong token, duplicate authorization headers, or a client still using the old token after rotation. | Reinstall or update the client secret, reconnect, and avoid printing headers in diagnostics. | +| HTTP returns `400` on a new MCP session | The first request was not JSON-RPC `initialize`, or `mcp-session-id` / `x-code-ux-agent` was malformed. | Let the MCP SDK initialize the session, or clear stale session state and reconnect. | +| Session cap errors appear | Too many active Streamable HTTP sessions, usually from leaked clients or a cluster larger than the default cap. | Stop stale clients, shorten `MCP_HTTP_SESSION_TIMEOUT_MS`, or raise `MCP_HTTP_MAX_SESSIONS` within server capacity. | +| Worker appears stale or offline | Heartbeats stopped, the worker process is down, network access failed, or the stable connection key changed unexpectedly. | Restart the worker with the same `--connection-key`, verify `/ready`, and check logs for bounded connection metadata. | +| Worker connects but does not claim work | No active project assignment, project not included in `--project-id` / `--active-project-id`, stale endpoint status, task executor mismatch, or no lease returned. | Confirm project assignment and worker status, then verify queued dispatches. Do not start local execution without a lease token. | +| `/health` passes but `/ready` fails | Listener is alive but runtime readiness has not completed or the server is degraded. | Wait for startup recovery to finish, then inspect structured logs. Use `/ready` for load balancer readiness gates. | +| `/ready` reports credential custody unavailable | The explicit mounted-file, Vault, or KMS provider is missing, insecure, unhealthy, or cannot return the required key version. | Keep runners disabled, inspect metadata-only readiness and provider configuration, and restore the exact provider/key version. Do not print key material or substitute a new key for encrypted rows. | +| A credential operation returns a version conflict | Another operator changed metadata, scope, capabilities, status, or encrypted value first. | Refresh the metadata-only record, review the new version, and intentionally retry with that version. Do not bypass optimistic concurrency. | +| Secret values appear in an exported settings bundle | The export was explicitly approved with `includeSecrets: true`. | Store the bundle only in approved secret storage, rotate exposed credentials if it was shared, and prefer redacted exports for review. | + +## Related Docs + +- [MCP Tools](../developer/mcp-tools.md) +- [Security Hardening](./security-hardening.md) +- [Automation Credential Security](./credential-security.md) +- [Runtime Configuration](../developer/configuration.md) diff --git a/docs-web/settings/integrations.md b/docs-web/settings/integrations.md index 67e686660b..5c43622f8b 100644 --- a/docs-web/settings/integrations.md +++ b/docs-web/settings/integrations.md @@ -1,19 +1,19 @@ # Integrations -Lists provider, git-host, issue-tracker, and read-only importer integrations and exposes manage/add actions. +Lists automation credentials, providers, git hosts, issue trackers, and read-only importer integrations and exposes manage/add actions. > Settings area: `integrations` > Dashboard documentation route: `/docs/settings-integrations` ## What This Area Is For -Lists provider, git-host, issue-tracker, and read-only importer integrations and exposes manage/add actions. This page expands the short Settings-page help text into an operator reference for deciding when to change this area, what behavior the controls affect, and what to verify after saving. +Lists automation credentials, providers, git hosts, issue trackers, and read-only importer integrations and exposes manage/add actions. This page expands the short Settings-page help text into an operator reference for deciding when to change this area, what behavior the controls affect, and what to verify after saving. Use it when you are configuring a new project, auditing inherited settings, or debugging behavior that changed after a system, project, or sprint override was saved. ## Controls And Runtime Effect -Cards show connection state, auth hints, active/configured importer status, and management entry points; host hints can import detected local settings. +Cards show connection state, auth hints, active/configured importer status, and management entry points; host hints can import detected local settings. Automation Credentials is the first catalog entry and reports secure-storage unavailable, ready but unconfigured, or configured state for the selected project. Its **Manage** action uses the same detail and back-navigation behavior as every other integration. | Control Surface | Runtime Effect | Review Before Saving | | --- | --- | --- | @@ -21,11 +21,32 @@ Cards show connection state, auth hints, active/configured importer status, and | Inherited values | Values can flow from system defaults into project and sprint behavior. | Check the source badge before assuming a value is project-specific. | | Related runtime paths | The affected service reads the saved settings during planning, dispatch, dashboard rendering, or maintenance work. | Re-run the affected workflow after changing operational settings. | +## Automation Credential Management + +Credential management is project-aware even when Settings is displaying system scope. Select a project before opening the detail view so Code UX can list only metadata visible to that project and determine whether the project has management authority. + +The create form requires an explicit name, kind, project or global scope, capability selection, and—when global scope is selected—an allowlist that retains the managing project. No capability is granted implicitly. Global creation and project-to-global promotion require confirmation because they expand access. + +Each project-managed credential supports bounded rename, metadata-only validation test, value rotation, encrypted-state replacement, monotonic access restriction, confirmed promotion, and confirmed revocation. Revocation requires typing `REVOKE` exactly; each lifecycle confirmation starts with cleared confirmation state and returns focus to the credential controls when it closes. Every lifecycle request uses the metadata version shown by the service. If another session wins the compare-and-swap update, the detail view refreshes metadata and asks the operator to review and retry instead of overwriting the newer state. + +| Workflow | What the operator supplies | What remains readable afterward | +| --- | --- | --- | +| Create | Name, kind, write-only value, explicit capabilities, and project/global policy | Metadata, configured state, validation state, scope, capabilities, and version only. | +| Update metadata | A bounded display name and current version | Updated metadata; kind and management ownership cannot be changed. | +| Rotate / replace | A new write-only value and current version | New key/version and validation metadata, never either the old or new value. | +| Test | The current version | `valid`, `invalid`, or `unavailable` plus timestamps; no tested value or low-level custody error. | +| Restrict / promote | A monotonic restriction, or a confirmed global allowlist expansion owned by the managing project | Updated non-secret policy metadata. | +| Revoke | Exact confirmation and current version | Revoked status and audit metadata; the stored value cannot be read back. | + +Secret inputs are write-only. Create, rotate, and replace fields are never populated from responses, are cleared after successful or failed submissions and project changes, and are removed with the detail view. Notices, metadata cards, browser storage, and reusable drafts contain no secret value. An allowlisted project that is not the management owner sees a **Use only** state and cannot invoke management actions. + +Unavailable key custody leaves non-secret metadata visible and disables secret-bearing changes and tests. Follow the inline custody guidance, restore secure storage, then use **Refresh**. See [Automation Credential Security](../operations/credential-security.md) for encryption, authority, recovery, and API behavior. + ## Recommended Configuration -Configure provider and importer credentials at system scope and use project overrides only for repository-specific git hosts or importer defaults. +Configure provider and importer credentials at system scope and use project overrides only for repository-specific git hosts or importer defaults. Automation credentials follow their own project-aware ownership and allowlist policy rather than Settings inheritance. -For Google Drive, link an existing host-side sync or mount directory and enable the opt-in Docker mount only for projects that need it. The mount defaults to read-only; see [Google Drive Project Mount](/docs/settings-google-drive-mount) for access, inheritance, security, and troubleshooting details. This integration does not configure Google Drive API synchronization or credentials. +For Google Drive, link an existing host-side sync or mount directory and enable the opt-in Docker mount only for projects that need it. The mount defaults to read-only; see [Google Drive Project Mount](./google-drive-mount.md) for access, inheritance, security, and troubleshooting details. This integration does not configure Google Drive API synchronization or credentials. A practical review flow is: @@ -51,11 +72,14 @@ If the saved setting does not appear to take effect: - Check for a project or sprint override that takes precedence over the system value. - Refresh the affected dashboard page if the setting controls a rendered surface. - Restart the local runtime only when the setting explicitly controls startup, listener, or process-level behavior. +- If secure custody is unavailable, keep the metadata view open, restore the deployment's supported custody provider, and use **Refresh**. Local loopback CLI/dashboard mode provisions its owner-only user-home key automatically; do not add mounted-key configuration for a normal local user. +- If a save reports stale metadata, review the refreshed record before retrying with its new version. Never copy secret fields into notes, browser storage, logs, or a repository as a workaround. ## Related Documentation -- [Settings overview](/docs/settings-overview) -- [Google Drive Project Mount](/docs/settings-google-drive-mount) -- [Dashboard Settings](/docs/user-dashboard-settings) -- [Configuration and Storage](/docs/developer-settings-reference) -- [Security Hardening](/docs/user-troubleshooting) +- [Settings overview](./index.md) +- [Automation Credential Security](../operations/credential-security.md) +- [Google Drive Project Mount](./google-drive-mount.md) +- [Dashboard Settings](../user/dashboard/settings.md) +- [Runtime Configuration](../developer/configuration.md) +- [Security Hardening](../operations/security-hardening.md) diff --git a/docs-web/user/dashboard/custom-dashboards.md b/docs-web/user/dashboard/custom-dashboards.md index 9bc9006971..5ba4672c86 100644 --- a/docs-web/user/dashboard/custom-dashboards.md +++ b/docs-web/user/dashboard/custom-dashboards.md @@ -2,59 +2,140 @@ Custom dashboards are project-scoped dashboard apps generated and revised by agents, then validated in a detached Docker runtime before publication. Use them when the built-in dashboard pages do not match the operational view a team needs, such as a project-specific release panel, sprint-health cockpit, or integration-status board. -## Workflow +The source of truth is the Code UX database. Drafts stay mutable, revisions are immutable snapshots, validation sessions record build/runtime results, and publication is a single active pointer to one validated revision. -1. Ask the Project Manager for the dashboard you want. Include the purpose, target audience, data sources, layout preferences, review criteria, and whether it should be published after validation. -2. Review the draft at `/custom-dashboards`. Drafts expose manifest JSON, generated file bundle content, source-node graph JSON, styleguide JSON, and data catalog selections. +## User Workflow + +1. Ask the Project Manager for the dashboard you want. Include the purpose, target audience, data sources, layout preferences, review criteria, and whether the dashboard should be published after validation. +2. Review the draft in the dashboard workspace at `/custom-dashboards`. The draft includes editable manifest JSON, generated file bundle content, source-node graph JSON, styleguide JSON, and data catalog selections. 3. Ask for changes or edit the draft before creating a revision. Draft edits do not change previous revisions or the currently published dashboard. -4. If the manifest declares credential slots, open the editor's **Credentials** tab. It shows bounded declarations and current non-secret metadata, and offers only active, configured, project-authorized credentials that satisfy the declared kinds and capabilities. Bind every required slot; no secret value is entered into the dashboard draft or generated code. -5. Create a revision when the draft is ready. A revision snapshots the current manifest, files, source graph, styleguide, runtime metadata, and credential-ID bindings. -6. Run detached validation. Code UX reviews bindings before it builds the revision in Docker, captures the browser-ready Vite artifact, starts a detached preview container, and health-checks the root URL. -7. Inspect validation status, logs, and the proxied preview link. Validation passes only after credential policy, install, build, artifact capture, container start, and root health checks succeed. -8. Publish the validated revision. Publication rechecks credential metadata and remains blocked unless the revision has a passed validation report. -9. Roll back by publishing an earlier passed revision, or archive the dashboard to clear its active publication while preserving history. +4. If the manifest declares credential slots, open the editor's **Credentials** tab. It shows each bounded declaration and current non-secret credential metadata, and offers only active, configured, project-authorized credentials that satisfy the allowed kinds and required capabilities. Bind every required slot; no secret value is entered into the dashboard draft or generated code. +5. Create a revision when the draft is ready. A revision snapshots the current manifest, file bundle, source graph, styleguide, runtime metadata, and credential-ID bindings. +6. Run detached validation for the revision. Code UX reviews bindings before it materializes the bundle, builds it in Docker, starts a detached preview container, and health-checks the root URL. +7. Inspect validation status, logs, and the proxied preview link. Validation passes only after credential policy, install, build, browser artifact capture, container start, and root health checks succeed. A passed validation does not publish by itself. +8. Publish the validated revision. Publication rechecks credential metadata and requires `validationStatus: "passed"` with a valid validation report. Publishing another passed revision is the rollback path. +9. Archive dashboards you no longer want active. Archiving clears the active publication and marks the dashboard archived while preserving revision and validation history. + +If validation fails, use the report and logs to create a new revision. Do not publish around the failure; the repository rejects failed, queued, running, cancelled, missing, or mismatched validation sessions before publication state changes. When a dashboard is already published, validating later drafts keeps the active published dashboard open, and validation sessions for the active published revision do not replace its published validation snapshot. + +The Credentials tab appears only when the saved manifest declares slots. Secure-backend failures and empty compatible lists link to credential management in Settings. Binding, replacement, and unbinding use the current optimistic binding revision; a concurrent edit refreshes the dashboard and requires an explicit retry instead of overwriting the other operator. Required unbinding immediately shows the draft as not ready for its next revision, while optional unbound slots remain valid. Every successful binding change refreshes validation and publication readiness. + +Credential selection and actions are keyboard accessible, restore focus after completion, and announce saving or error state. Credential IDs remain confined to the dedicated metadata-management request state and never enter manifest, generated-file, source-graph, styleguide, runtime-text, or secret-value fields. + +If secure custody is unavailable, the Credentials tab keeps existing bindings unchanged, reports metadata-only readiness, and links to Settings. Restore the supported custody provider and refresh the review; do not put a key or credential value in manifest JSON, generated files, validation logs, or project files. If a bind/unbind returns a stale binding revision, the editor refreshes declarations, candidates, bindings, and readiness, then requires an explicit retry. If validation or publication denies a formerly compatible binding, refresh review because revocation, restriction, project access, capabilities, kind policy, or custody health may have changed. + +## Agent Workflow -If validation fails, use the report and logs to create a new revision. Code UX rejects failed, queued, running, cancelled, missing, or mismatched validation sessions before publication state changes. When a dashboard is already published, validating later drafts keeps the active published dashboard open, and validation sessions for the active published revision do not replace its published validation snapshot. +Project Manager agents should use the `manage_custom_dashboards` MCP surface rather than writing generated code into `dashboard/src`. -The Credentials tab appears only for manifests with declared slots. Unavailable secure custody and empty compatible lists link to credential management in Settings. Binding changes use the current optimistic revision; a conflict refreshes the dashboard and asks for an explicit retry. Required unbinding marks the next revision as not ready, while optional unbound slots remain valid. Successful changes refresh validation and publication readiness. The controls support keyboard selection, visible focus, focus restoration, and live save/error announcements. Credential IDs stay out of manifest, file, source, styleguide, and runtime text editors, and the editor never requests or displays secret values. +Recommended sequence: -## Data Sources +1. Gather missing requirements for purpose, audience, source data, style, accessibility, and publication intent. +2. Call `data_catalog` for the project when reusing existing custom-dashboard source declarations. +3. Call `create` or `update` with a complete manifest, file bundle, source-node graph, styleguide, and runtime metadata. +4. Call `list_credential_slots` when the manifest declares slots. Select only candidate credential IDs reported compatible, then call `bind_credential` with the current `expectedBindingRevision` and complete the human-approval flow. Use `unbind_credential` before changing a bound slot's policy. +5. Call `create_revision` to snapshot the draft and binding IDs. +6. Call `validate_revision`, then poll `validation_status` and read `validation_logs` when the session is not passed. +7. Repair failures by updating the draft or binding metadata and creating a new revision. +8. Call `publish_revision` only after validation passed. Include `validationSessionId` when publishing from the session just reviewed. +9. Use `archive` only after human approval; the action follows the standard destructive-action approval flow. -Custom dashboards declare a `sourceNodeGraph` with nodes, edges, and optional metadata. Nodes have `id`, `type`, `title`, and optional JSON `config`. +## Data-Source Node Graph + +Each dashboard draft and revision can declare a `sourceNodeGraph`: + +```json +{ + "nodes": [ + { "id": "execution", "type": "project_dashboard_data", "title": "Project execution" }, + { "id": "stats", "type": "stats", "title": "Seven-day stats", "config": { "window": "7d" } } + ], + "edges": [], + "metadata": {} +} +``` + +Nodes have `id`, `type`, `title`, and optional JSON `config`. Edges have `fromNodeId`, `toNodeId`, and an optional `id`. The graph records the data the generated dashboard expects; it is also used by the in-app viewer to decide which source requests are allowed. + +Supported user-level source types: | Source type | Runtime behavior | | --- | --- | -| `project_dashboard_data`, `project_dashboard`, `dashboard_data` | Reads project execution data. | -| `stats`, `project_stats` | Reads project stats. `config.window` selects the stats window when present. | -| `telemetry`, `overview_telemetry` | Reads overview telemetry. | -| `integrations_metadata`, `integrations` | Returns only non-secret metadata declared on the source node. | -| `external_api` | Placeholder only. Arbitrary external calls are not proxied and return an unavailable-source error. | +| `project_dashboard_data`, `project_dashboard`, `dashboard_data` | Reads project execution data from `GET /api/projects/:projectId/execution`. | +| `stats`, `project_stats` | Reads project stats from `GET /api/projects/:projectId/stats`; `config.window` selects the stats window when present, otherwise `7d` is used. | +| `telemetry`, `overview_telemetry` | Reads overview telemetry from `GET /api/telemetry/overview`. | +| `integrations_metadata`, `integrations` | Returns only the non-secret metadata declared on the source node. It does not expose provider credentials or effective settings secrets. | +| `external_api` | Placeholder only in the in-app viewer. It is declared in the graph and validation bridge, but arbitrary external calls are not proxied and return an unavailable-source error. | + +Unsupported source types return an explicit unavailable-source error. Generated dashboards should handle these errors visibly instead of assuming all declared data is available. + +## REST API Surface + +Custom dashboard routes are registered with the dashboard server: + +| Method | Route | Purpose | +| --- | --- | --- | +| `GET` | `/api/projects/:projectId/custom-dashboards` | List dashboards for a project. | +| `POST` | `/api/projects/:projectId/custom-dashboards` | Create a mutable draft. | +| `GET` | `/api/projects/:projectId/custom-dashboards/data-catalog` | Return project dashboard summaries and declared source nodes. | +| `GET` | `/api/custom-dashboards/:dashboardId` | Return a dashboard plus revisions. | +| `PATCH` | `/api/custom-dashboards/:dashboardId` | Update mutable draft fields. | +| `DELETE` | `/api/custom-dashboards/:dashboardId` | Archive the dashboard and clear active publication. | +| `POST` | `/api/custom-dashboards/:dashboardId/revisions` | Create an immutable revision from the draft or supplied overrides. | +| `POST` | `/api/custom-dashboards/:dashboardId/revisions/:revisionId/validate` | Start a detached validation session. Body may include `projectId`; otherwise the server resolves it from the revision. | +| `POST` | `/api/custom-dashboards/:dashboardId/revisions/:revisionId/publish` | Publish a validated revision, optionally with `validationSessionId`. | +| `GET` | `/api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings?revisionId=...` | Review draft or revision slots, current bindings, backend health, and bounded compatible credential metadata. | +| `PUT` | `/api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings` | Bind or replace one slot using `slotId`, `credentialId`, and `expectedBindingRevision`. | +| `DELETE` | `/api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings/:slotId` | Unbind one slot using `expectedBindingRevision`. | +| `GET` | `/api/custom-dashboard-validations/:sessionId` | Read validation session status and runtime metadata. | +| `GET` | `/api/custom-dashboard-validations/:sessionId/logs?tail=200` | Read bounded validation and container logs. | +| `POST` | `/api/custom-dashboard-validations/:sessionId/stop` | Stop the detached validation container. | +| `DELETE` | `/api/custom-dashboard-validations/:sessionId` | Remove a validation session after cleanup. | +| `ALL` | `/api/custom-dashboard-validations/:sessionId/proxy{*rest}` | Same-origin proxy to the detached validation runtime. | +| `ALL` | `/api/custom-dashboards/validation-sessions/:sessionId/proxy{*rest}` | Backward-compatible validation proxy route. | + +The binding routes are credential-management routes: authenticated remote callers require `credential_admin`, project access, and enabled remote credential management. Stale binding revisions return `409`; incompatible credential selection returns `403`. Publication first repeats metadata-only binding review, then applies the repository validation gate. REST and MCP denials preserve a sanitized `issues` array with slot-specific `field`, `code`, and `message` values while omitting credential IDs and values. Active publications remain the opening source of truth while later validation sessions run. + +## MCP Surface + +The dedicated MCP tool is `manage_custom_dashboards` and is available to the project-manager runtime role. It supports: + +- `list`, `get`, `create`, `update` +- `create_revision` +- `validate_revision`, `validation_status`, `validation_logs` +- `publish_revision` +- `archive` +- `data_catalog` +- `list_credential_slots`, `bind_credential`, `unbind_credential` + +Credential actions use `projectId`, `dashboardId`, `slotId`, `credentialId`, and `expectedBindingRevision`; an optional `revisionId` reviews an immutable snapshot. Bind and unbind require the normal stateful human-approval handshake. Before creating any approval fingerprint, Code UX rejects secret, header, environment, malformed approval, and other undeclared fields, then rebuilds the approval payload from only the allowed metadata. Other important fields include `sessionId`, `validationSessionId`, `title`, `description`, `manifest`, `fileBundle`, `sourceNodeGraph`, `styleguide`, `runtimeMetadata`, `tail`, and `approval`. + +The dashboard chat JSON-action bridge also understands the legacy `custom_dashboards` management domain, but agents should prefer the dedicated MCP tool when it is available. + +## Validation Runtime + +Validation sessions move through `queued`, `building`, `running`, `passed`, `failed`, or `cancelled`. -Generated dashboards should handle unavailable-source errors visibly. External API connectors are not fully available through the in-app viewer yet. +During validation, Code UX: -## Agent and API Notes +- performs metadata-only compatibility review for every bound slot and every required slot +- creates a validation session row and runtime directory under the selected project +- writes the generated bundle plus a known Vite/Preact harness +- injects a read-only `codeUxDataBridge` / `CodeUXCustomDashboard` object +- runs install and build in Docker using the resolved CLI workflow image +- persists the built Vite `dist` files on the validated revision as the published-viewer artifact +- starts a detached preview container on an allocated localhost port +- health-checks the root URL before marking the session passed +- records workspace path, log path, container id/name, host port, validation proxy path, commands, and log excerpts in runtime metadata -Project Manager agents use the `manage_custom_dashboards` MCP tool to create drafts, list credential slots, bind or unbind credential IDs, create revisions, validate revisions, inspect logs, publish passed revisions, archive dashboards, and read the data catalog. Credential mutations require the normal stateful human-approval handshake and an optimistic `expectedBindingRevision`; unsupported or secret-bearing fields are rejected before approval state is created. +Required missing bindings and bound credentials that are missing, revoked, inaccessible, unconfigured, wrong-kind, missing capabilities, or blocked by unavailable/insecure key custody fail with slot-specific issues before workspace creation. Optional unbound slots remain valid. No custom-dashboard path resolves secret plaintext: credential values and binding IDs stay out of generated source, file bundles, bridge files, Docker arguments and mounts, validation reports and logs, viewer records, iframe configuration, and browser messages. Generic response and viewer boundaries recursively redact known binding IDs from nested manifests, file content and metadata, source graphs, styleguides, runtime metadata, validation reports, and persisted viewer artifacts. Dedicated credential-binding management responses may return credential IDs and non-secret metadata so operators and agents can select them. -The same workflow is available through the dashboard REST API: +The `build` and `runtime` slot phases are bounded declarations used for review and policy only. They do not inject a secret into the build container, published artifact, iframe, MCP result, or runtime data bridge. This feature does not migrate or expose broader provider secrets. -- `GET/POST /api/projects/:projectId/custom-dashboards` -- `GET /api/projects/:projectId/custom-dashboards/data-catalog` -- `GET/PATCH/DELETE /api/custom-dashboards/:dashboardId` -- `POST /api/custom-dashboards/:dashboardId/revisions` -- `POST /api/custom-dashboards/:dashboardId/revisions/:revisionId/validate` -- `POST /api/custom-dashboards/:dashboardId/revisions/:revisionId/publish` -- `GET /api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings` -- `PUT /api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings` -- `DELETE /api/projects/:projectId/custom-dashboards/:dashboardId/credential-bindings/:slotId` -- `GET /api/custom-dashboard-validations/:sessionId` -- `GET /api/custom-dashboard-validations/:sessionId/logs` -- `POST /api/custom-dashboard-validations/:sessionId/stop` -- `DELETE /api/custom-dashboard-validations/:sessionId` -- `ALL /api/custom-dashboard-validations/:sessionId/proxy{*rest}` +Stopping a validation session removes the detached container. It does not invalidate a passed revision report. Removing a validation session deletes the session row after cleanup; the revision's validation metadata remains the publication gate. -Remote callers to credential-binding routes require the credential-administrator role, project access, and enabled remote credential management. Required missing bindings and bound credentials that are revoked, inaccessible, unconfigured, wrong-kind, missing capabilities, or blocked by unavailable key custody fail before workspace creation and are rechecked before publication. REST and MCP publication denials include sanitized slot-specific issues without credential IDs or values. Optional unbound slots remain valid. +## Published Viewer and Rollback -Custom-dashboard binding is metadata-only: no secret is resolved, and credential values and binding IDs are excluded from generated files, bridges, Docker configuration, validation output, generic REST/MCP responses, iframe configuration, and browser messages. Generic response and viewer boundaries recursively redact known IDs from nested manifests, files, source graphs, runtime metadata, validation reports, and viewer artifacts. Dedicated binding-management responses may return credential IDs and non-secret metadata so operators and agents can select them. +The in-app viewer renders only published dashboards whose active `publishedRevisionId` points to a revision with a passed validation report. For the default `src/dashboard.tsx` draft and other TSX/Preact revisions validated through the harness, the viewer uses the persisted Vite `dist` artifact instead of the source entry file, so publication does not depend on the detached validation container still running. Generated code runs inside a sandboxed iframe document and talks to the parent app through a constrained `postMessage` bridge. The parent serves only declared source-node requests. -Published dashboards render inside a sandboxed iframe. For TSX/Preact drafts such as the default `src/dashboard.tsx` bundle, the viewer uses the persisted validation artifact instead of the source entry file, so it can open after publication even when the detached validation preview is gone. The frame can request only declared source nodes through the Code UX bridge, parent and frame handlers verify the expected window source, and the parent dashboard returns data through same-origin API calls. +Rollback is publish-based: select an earlier passed revision and publish it again. The publication pointer moves back to that immutable revision. Archive is the safe removal path when no dashboard should be active; it clears the publication pointer while preserving history. diff --git a/docs-web/user/dashboard/node-flows.md b/docs-web/user/dashboard/node-flows.md index 5668b99750..f77885f78b 100644 --- a/docs-web/user/dashboard/node-flows.md +++ b/docs-web/user/dashboard/node-flows.md @@ -1,42 +1,44 @@ -# Node Flows +# Node Flows Dashboard The **Nodes** page (`/nodes`) is the project-scoped backend authoring, publication, and operations surface for canonical node flows. No selected project means no flow library, credential metadata, publications, or durable run history are requested. -## Library, Drafts, And Migration +## Library, drafts, and migration -The flow library contains backend drafts and publications owned by the active project. Saves include the loaded draft revision, so a concurrent edit produces a visible conflict and never overwrites newer work. +The library loads through `GET /api/projects/:projectId/node-flows`. Drafts are created through `POST /api/projects/:projectId/node-flow-drafts` and saved through revision-checked `PATCH /api/node-flow-drafts/:flowId`. A stale revision produces a visible conflict and never overwrites newer work. -The former browser graph at `codeux:nodes-canvas:v1` is eligible for one import into the selected project. The bridge maps `trigger`, `agent`, and `task` to registered `input`, `set_fields`, and `provider_prompt` definitions, retains `condition` and `output`, and remaps their ports before creating an **Imported Nodes Canvas** draft. A failed import remains retryable and does not block the normal library load; only success removes the old value and records the marker. +The former browser graph at `codeux:nodes-canvas:v1` is eligible for one import into the selected project. The bridge maps `trigger` to `input`, `agent` to `set_fields`, and `task` to `provider_prompt`; `condition` and `output` remain governed definitions, ports are remapped, and legacy configuration is retained as non-secret metadata. Code UX creates an **Imported Nodes Canvas** backend draft and only then removes the legacy value and records a project-specific marker. A failed import remains retryable and is isolated from normal library loading, while a successful marker prevents duplicates. -## Registry-Driven Editing And Credentials +## Registry-driven editing and credentials -The registry list returns flat versioned palette summaries. Selecting a definition loads the full manifest from the node-type detail endpoint, including nested `ui.widgetSchema`, configuration schema, policies, documentation, and deprecation metadata. The inspector renders from that full contract. Graphs reference a definition version and store non-secret configuration and credential ids; they do not contain custom-node source or resolved credentials. +`GET /api/node-flow-catalog` returns flat versioned palette summaries. `GET /api/node-flow-catalog/:nodeType` returns the full `NodeDefinitionManifest`, including nested `ui.widgetSchema`, configuration schema, policies, documentation, and deprecation metadata. The inspector renders from that full contract. Graphs reference a definition version and store non-secret configuration and credential ids; they do not contain custom-node source or resolved credentials. Credential slots use the versioned definition's allowed kinds and required capabilities to offer project-visible credential metadata. Only active, configured credentials with project access and a healthy secure backend are selectable; unavailable entries explain the operator-facing reason without exposing secret or key-custody details, and an empty compatible set links directly to **Settings → Integrations**. -Selecting, replacing, or removing a credential updates only that slot in the node's canonical `credentialBindings` and immediately saves the complete draft through the current optimistic revision. The dashboard then adopts the canonical flow revision and refreshes governed review. Saving, saved, policy-denial, and error states are announced. A revision conflict loads the latest draft, preserves the selected slot workflow and sibling edits, and requires the operator to choose again rather than replaying the stale mutation. Credential plaintext remains behind the broker and is excluded from graph data, component state, and browser output. +Selecting, replacing, or removing a credential updates only that slot in the node's canonical `credentialBindings` and immediately saves the complete draft through the current optimistic revision. The dashboard then adopts the canonical flow revision and refreshes governed review. Saving, saved, policy-denial, and error states are announced. A revision conflict loads the latest draft, preserves the selected slot workflow and sibling edits, and requires the operator to choose again rather than replaying the stale mutation. Credential plaintext remains behind the broker and is excluded from graph data, component state, browser output, logs, and documentation examples. + +Removing a required binding is allowed as a draft edit but immediately changes review and publication readiness to blocked; removing an optional binding remains valid. Publication is denied for required missing bindings and for credentials that become unavailable, unconfigured, revoked, inaccessible to the project, wrong-kind, or short of a required capability. Runtime repeats compatibility against the immutable publication, so a later custody outage, restriction, revocation, or rebinding denies execution instead of using a stale dashboard decision. The complete governed built-in set currently registered with executable handlers is `input`, `set_fields`, `template`, `provider_prompt`, `http_request`, `condition`, `switch`, `foreach`, `merge`, `delay`, `approval`, `email_draft`, `email_send`, `execute_subflow`, `webhook_trigger`, and `output`. -Registered custom definitions can execute only when their validated versioned manifest, immutable artifact, and custom-node runtime are available. Raw legacy kinds are translated during import rather than executed directly. Unknown or unregistered types, mockup entries, and definitions marked non-executable remain planned or unavailable. +Registered custom definitions can execute only when their validated versioned manifest, immutable artifact, and custom-node runtime are available. Raw legacy `trigger`/`agent`/`task` kinds are translated by the browser import bridge rather than executed directly. Unknown or unregistered types, mockup entries, and definitions marked non-executable are planned or unavailable definitions. -## Governance And Publication +## Governance and publication Draft review provides structural validation, policy findings, requested permissions, side-effect review, and a non-executing dry run. Publication requires the current draft revision, a valid governed review, and all required credentials. Each publication is an immutable snapshot; comparison and rollback operate on versioned history, and only a pinned or latest-published version can execute. -## Durable Debugger And Scheduling +## Durable debugger and scheduling + +The debugger reads persisted flow runs, node runs, attempt history, retry classifications and decisions, approval records, invocation links, timing, and redacted input and output. Pending approvals expose keyboard-accessible **Approve & continue** and **Reject** actions. A decision continues or terminates the same pinned run, and repeated decisions return its current durable state without duplicating a governed attempt or external send. The debugger also supports cancellation and safe retry. -The debugger reads persisted flow runs, node runs, attempt history, retry classifications and decisions, approval records, invocation links, timing, and redacted input and output. Pending approvals offer **Approve & continue** and **Reject** actions. A decision continues or terminates the same pinned run, and repeated decisions return its current durable state without duplicating a governed attempt or external send. The debugger also supports cancellation and safe retry. +Foreach runs persist one downstream node run and attempt sequence per deterministic logical item. Item inputs, retries, cancellation, approvals, and side-effect identity survive restart; concurrency is bounded by the node configuration. Empty collections select the explicit `empty` branch and persist the item branch as skipped, while oversized collections fail instead of being truncated. -Use the [Scheduler](./scheduler.md) to target a pinned or latest-published version. A flow can also be attached to a project agent preset as a reusable skill; removing the attachment does not remove the flow, publications, schedules, or run history. +The layout stacks on small screens, preserves keyboard-visible focus, labels loading/error/empty states, and bounds long histories and JSON output with scrolling. Rendered run payloads redact secret-shaped keys such as `apiKey`, `authorization`, `cookie`, `password`, `secret`, and `token`. The run debugger lists durable approvals beside node attempts. A pending item offers **Approve & continue** and **Reject** actions. The decision applies to the same pinned run, and repeated clicks return its current state without sending an approved external effect twice. -Foreach executes the selected downstream branch once per deterministic logical item. The node's `concurrency` setting bounds active items, `maxItems` rejects oversized inputs, and zero items explicitly select the `empty` branch. Item-specific inputs, retries, cancellation, approvals, and external-effect identity are persisted so restart continuation does not replay completed items or duplicate sends. - -## Agent Attachment +## Agent attachment A selected project loads its agent presets, and selecting a flow loads that flow's current bindings. The inspector exposes only agent names and attachment skill metadata; it never renders agent instructions, custom source, credential values, or decrypted material. @@ -44,9 +46,9 @@ Attaching and detaching use the governed node-flow attachment routes, then refre A flow can be attached to a project agent preset as a repeatable skill with a name and description. Detaching removes only that binding; the flow, its graph, schedules, and run history remain in the project. -## Scheduling +Scheduling is entered through `/scheduler` and targets a pinned or latest-published version. A flow can also be attached to a project agent preset as a reusable skill; removing the attachment does not remove the flow, publications, schedules, or run history. -Use the [Scheduler](./scheduler.md) page to run a saved node flow once or on a recurrence. Scheduled node-flow entries select a project-owned flow and may include optional JSON object input. Pause, resume, failure handling, and due-run behavior match the normal scheduler model. +Scheduled node-flow entries select a project-owned flow and may include optional JSON object input. Pause, resume, failure handling, and due-run behavior match the normal scheduler model. ## Graph v2 boundary diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index cfd54d82db..5581313c90 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -92,6 +92,7 @@ - [Chat Provider Integrations](./settings/chat-provider-integrations.md) - [Settings Reference](./settings/index.md) - [Google Drive Project Mount](./settings/google-drive-mount.md) + - [Integrations](./settings/integrations.md) - [Project Context](./settings/project-context.md) - [System Runtime](./settings/system-runtime.md) - [Provider Credentials](./settings/provider-credentials.md) diff --git a/docs/architecture/custom-dashboard-foundation.md b/docs/architecture/custom-dashboard-foundation.md index 57aeb569b4..ba3a63d42d 100644 --- a/docs/architecture/custom-dashboard-foundation.md +++ b/docs/architecture/custom-dashboard-foundation.md @@ -15,6 +15,12 @@ Primary records: Dashboard status values are `draft`, `validating`, `validated`, `published`, `rejected`, and `archived`. Validation status values are `queued`, `building`, `running`, `passed`, `failed`, and `cancelled`. +### Feature baseline and bounded addition + +Repository history provides the negative baseline for this subsystem: at the pre-feature `dev` commit `716ac2c55`, `CustomDashboardManifest` had no `credentialSlots`, and mutable dashboard and immutable revision records had no `credentialBindings` or binding revision. The implemented change is intentionally limited to bounded manifest declarations, credential-ID bindings in dedicated draft/revision columns, metadata-only compatibility review, optimistic binding mutation, and validation/publication gates. It does not migrate provider secrets and it does not add custom-dashboard secret injection. + +Declarations are normalized and bounded for count, slot ID, label, phase (`build` or `runtime`), allowed kinds, and required capabilities. Bindings contain only `slotId` and `credentialId`; generic draft/revision writes cannot set them, and immutable revisions snapshot them. The phase is policy metadata for review and validation, not permission to inject a value into build or runtime artifacts. + ## Persistence SQLite tables are created in both the initial schema and startup migrations: @@ -61,6 +67,8 @@ Validation flow: Validation does not publish or activate dashboards. A successful run only marks the revision validation status as `passed`; publication remains gated by `publishRevision`. REST and MCP publication re-run metadata-only binding review immediately before calling the repository, then require either a revision already marked `passed` with a valid report or an explicit passed validation session for that revision. Failed binding review, queued/running/cancelled validation, missing state, and cross-revision sessions are rejected before the publication pointer changes. +No custom-dashboard service resolves credential plaintext. Build workspaces, generated files and Vite artifacts, Docker arguments/mounts/environment, validation reports/logs, generic REST/MCP records, viewer configuration, iframe `srcdoc`, data-bridge payloads, and `postMessage` traffic receive neither credential values nor binding IDs. Only the dedicated metadata-management response may return binding IDs alongside non-secret credential metadata. + ## REST and MCP Surface Dashboard HTTP routes live in `src/server/custom-dashboard-routes.ts` and are registered with the existing dashboard route groups. They are thin adapters over `CustomDashboardRepository` and `CustomDashboardValidationService`: diff --git a/docs/architecture/node-flow-builtins-and-security.md b/docs/architecture/node-flow-builtins-and-security.md index 4cb15afeba..18174020be 100644 --- a/docs/architecture/node-flow-builtins-and-security.md +++ b/docs/architecture/node-flow-builtins-and-security.md @@ -19,6 +19,12 @@ The governed built-in catalog extends publication-based node-flow execution with The existing `input`, `set_fields`, `template`, `provider_prompt`, `http_request`, and `output` nodes retain their previous contracts. Typed manifest ports identify branch handles, many-valued merge inputs, and trigger outputs. Branch routing only runs a node when at least one incoming edge is active, allowing merges to join a selected path without treating an unselected sibling as a failure. +## Credential-bound execution + +Versioned definition manifests declare credential slots by required state, allowed kinds, and required capabilities. Draft review and publication use metadata-only broker compatibility; the canonical graph stores only slot-to-credential-ID bindings. Required missing bindings and bindings denied for unavailable custody, configuration, status, project access, kind, or capability stop publication. Optional unbound slots remain valid. + +At runtime, the immutable published graph is revalidated and the broker repeats authorization immediately before resolving a value for the active attempt. A revoked, restricted, rebound, wrong-kind, insufficiently capable, or unavailable credential fails the attempt closed. Exact resolved values are redacted from built-in output, invocation/attempt records, diagnostics, retries, HTTP/provider responses, and external-effect persistence; neither publication nor MCP inspection injects or returns plaintext. + Foreach assigns deterministic logical-item identities from the published node id and item index. Each downstream node run and numbered attempt persists that identity together with the item-specific input. The `concurrency` setting defaults to one and is capped at 64; `maxItems` is a rejection bound rather than a truncation rule. A zero-item input selects `empty`, while the `items` branch is persisted as skipped. Per-item failures retain their own retry history, successful siblings are not replayed during approval or restart continuation, and aggregated output preserves input order. ## Governed egress diff --git a/docs/architecture/node-flows.md b/docs/architecture/node-flows.md index d55a665f41..4a20a15ffe 100644 --- a/docs/architecture/node-flows.md +++ b/docs/architecture/node-flows.md @@ -43,6 +43,14 @@ The dashboard uses the same backend-owned Graph v2 record as the runtime. The se `dashboard/src/v2/lib/nodes-canvas-state.ts` remains only a compatibility and pure graph-state layer. Its legacy browser graph can be imported once into a project draft. The adapter translates `trigger`/`agent`/`task` into registered `input`/`set_fields`/`provider_prompt` nodes, remaps legacy handles to governed ports, and retains non-secret canvas metadata. Import failure is isolated from the normal library load; only a successful draft creation removes the old graph key and records the project marker. Browser storage is not the workflow source of truth. +### Credential binding lifecycle + +Each versioned node definition is the slot-policy authority: every slot declares whether it is required, its allowed credential kinds, and all required capabilities. The picker lists project-visible metadata, then filters each candidate through secure-backend readiness, configured/active state, project access, kind, and capability compatibility. It never resolves a value. + +`NodeFlowNode.credentialBindings` is the only persisted binding source. Selecting, replacing, or unbinding a credential changes the matching `{ slot, credentialId }` entry in the complete canonical graph and saves with the current `draftRevision`. The dashboard adopts the returned graph and revision, then refreshes governed review. A `409`-style revision conflict refreshes the latest draft and requires a deliberate retry; it never replays a stale binding over sibling changes. + +Required unbound slots and any bound credential denied by backend readiness, configuration, active status, project access, allowed kind, or required capabilities block publication. Optional unbound slots do not. Runtime revalidates the immutable publication and repeats the same policy immediately before direct credential-ID resolution, so revocation, restriction, rotation/rebinding races, missing custody, or incompatible policy deny the node attempt rather than injecting stale plaintext. Graph, review, publication, MCP, and dashboard payloads contain IDs and non-secret policy metadata only. + ## Runtime `NodeFlowRuntimeService.runFlow(projectId, flowId, input, options)` resolves an explicit pinned or latest-published snapshot, revalidates that immutable graph, claims a durable lease, and executes nodes in topological order. See [Node Flow Durable Execution](./node-flow-durable-execution.md) for queue, retry, lease, recovery, quota, and redaction guarantees. diff --git a/docs/dashboard/custom-dashboards.md b/docs/dashboard/custom-dashboards.md index 91e340f6d6..5ba4672c86 100644 --- a/docs/dashboard/custom-dashboards.md +++ b/docs/dashboard/custom-dashboards.md @@ -22,6 +22,8 @@ The Credentials tab appears only when the saved manifest declares slots. Secure- Credential selection and actions are keyboard accessible, restore focus after completion, and announce saving or error state. Credential IDs remain confined to the dedicated metadata-management request state and never enter manifest, generated-file, source-graph, styleguide, runtime-text, or secret-value fields. +If secure custody is unavailable, the Credentials tab keeps existing bindings unchanged, reports metadata-only readiness, and links to Settings. Restore the supported custody provider and refresh the review; do not put a key or credential value in manifest JSON, generated files, validation logs, or project files. If a bind/unbind returns a stale binding revision, the editor refreshes declarations, candidates, bindings, and readiness, then requires an explicit retry. If validation or publication denies a formerly compatible binding, refresh review because revocation, restriction, project access, capabilities, kind policy, or custody health may have changed. + ## Agent Workflow Project Manager agents should use the `manage_custom_dashboards` MCP surface rather than writing generated code into `dashboard/src`. @@ -128,6 +130,8 @@ During validation, Code UX: Required missing bindings and bound credentials that are missing, revoked, inaccessible, unconfigured, wrong-kind, missing capabilities, or blocked by unavailable/insecure key custody fail with slot-specific issues before workspace creation. Optional unbound slots remain valid. No custom-dashboard path resolves secret plaintext: credential values and binding IDs stay out of generated source, file bundles, bridge files, Docker arguments and mounts, validation reports and logs, viewer records, iframe configuration, and browser messages. Generic response and viewer boundaries recursively redact known binding IDs from nested manifests, file content and metadata, source graphs, styleguides, runtime metadata, validation reports, and persisted viewer artifacts. Dedicated credential-binding management responses may return credential IDs and non-secret metadata so operators and agents can select them. +The `build` and `runtime` slot phases are bounded declarations used for review and policy only. They do not inject a secret into the build container, published artifact, iframe, MCP result, or runtime data bridge. This feature does not migrate or expose broader provider secrets. + Stopping a validation session removes the detached container. It does not invalidate a passed revision report. Removing a validation session deletes the session row after cleanup; the revision's validation metadata remains the publication gate. ## Published Viewer and Rollback diff --git a/docs/dashboard/node-flows.md b/docs/dashboard/node-flows.md index 66ad383700..f77885f78b 100644 --- a/docs/dashboard/node-flows.md +++ b/docs/dashboard/node-flows.md @@ -16,6 +16,8 @@ Credential slots use the versioned definition's allowed kinds and required capab Selecting, replacing, or removing a credential updates only that slot in the node's canonical `credentialBindings` and immediately saves the complete draft through the current optimistic revision. The dashboard then adopts the canonical flow revision and refreshes governed review. Saving, saved, policy-denial, and error states are announced. A revision conflict loads the latest draft, preserves the selected slot workflow and sibling edits, and requires the operator to choose again rather than replaying the stale mutation. Credential plaintext remains behind the broker and is excluded from graph data, component state, browser output, logs, and documentation examples. +Removing a required binding is allowed as a draft edit but immediately changes review and publication readiness to blocked; removing an optional binding remains valid. Publication is denied for required missing bindings and for credentials that become unavailable, unconfigured, revoked, inaccessible to the project, wrong-kind, or short of a required capability. Runtime repeats compatibility against the immutable publication, so a later custody outage, restriction, revocation, or rebinding denies execution instead of using a stale dashboard decision. + The complete governed built-in set currently registered with executable handlers is `input`, `set_fields`, `template`, `provider_prompt`, `http_request`, `condition`, `switch`, `foreach`, `merge`, `delay`, `approval`, `email_draft`, `email_send`, `execute_subflow`, `webhook_trigger`, and `output`. Registered custom definitions can execute only when their validated versioned manifest, immutable artifact, and custom-node runtime are available. Raw legacy `trigger`/`agent`/`task` kinds are translated by the browser import bridge rather than executed directly. Unknown or unregistered types, mockup entries, and definitions marked non-executable are planned or unavailable definitions. diff --git a/docs/index.md b/docs/index.md index f482d30009..955f071a30 100644 --- a/docs/index.md +++ b/docs/index.md @@ -215,6 +215,7 @@ Use this page as the main entrypoint. - [Chat Provider Integrations](./settings/chat-provider-integrations.md) - [Settings Reference](./settings/index.md) - [Google Drive Project Mount](./settings/google-drive-mount.md) + - [Integrations](./settings/integrations.md) - [Qwen Code Integration](./settings/qwen-code-integration.md) - [OpenCode Integration](./settings/opencode-integration.md) - [Operations Runbook](./operations/runbook.md) diff --git a/docs/operations/credential-security.md b/docs/operations/credential-security.md index 90f2a4eaa1..48665eb98f 100644 --- a/docs/operations/credential-security.md +++ b/docs/operations/credential-security.md @@ -45,6 +45,12 @@ For mounted-file custody, `CODE_UX_CREDENTIAL_KEY_FILE` identifies a regular, ow Electron serializes first-use root-key creation, persists only the OS-protected blob through an atomic owner-only file replacement, and refuses credential operations when `safeStorage` is unavailable. Vault and KMS adapters validate 32-byte caller-owned key material and report the active key id/version in health results. No provider silently falls back to plaintext or an insecure locally derived key. +| Deployment boundary | Root-key custody | Provisioning behavior | +| --- | --- | --- | +| Normal CLI dashboard on loopback with local authentication | Owner-only file under the user-home Code UX security directory | Automatically created on first use and reused after restart. A normal local dashboard user does not mount or configure a key file. | +| Electron desktop | Operating-system `safeStorage` | Automatically creates and persists only the OS-protected blob; unavailable `safeStorage` blocks credential operations. | +| Dashboard-disabled headless, server mode, authenticated dashboard, non-loopback binding, or remote credential management | Explicit mounted file, Vault, or KMS provider | Never auto-provisions local custody. Setup and recovery fail closed until the configured provider reports available, secure key identity and version metadata. | + ## Recovery and rotation Back up root keys independently from `app.db`. For the normal local dashboard, back up `~/.code-ux/security/credential-root.key` while preserving owner-only handling; for external providers, retain every referenced key version. Losing a required key version makes its ciphertext unrecoverable by design. Restoring only SQLite is insufficient. @@ -62,3 +68,9 @@ Existing global credentials created before management ownership was stored are m Project-scoped routes live under `/api/projects/:projectId/credentials`. Supported operations are create, bounded-name update (`PATCH /:credentialId`), bind, metadata-only compatibility assessment, test, rotate, replace, revoke, promote, and restrict. Compatibility evaluates key-backend readiness, configuration, active status, project access, allowed kinds, and all required capabilities without resolving plaintext. A backend is ready only when it is available and secure and reports both a non-empty key ID and a key version; missing key identity metadata produces the stable `backend_unavailable` compatibility issue. List, compatibility, health, and mutation responses return metadata or policy results only. Existing dashboard authentication and remote credential-management guards apply before these routes. Runtime validation failures return `400`, project/management denials return `403`, compare-and-swap conflicts return `409`, invalid encrypted state returns `422`, and unavailable key custody returns an actionable `503` response. + +## Troubleshooting without disclosure + +- If custody is unavailable, inspect the metadata-only credential health or readiness result and the configured provider name. For the normal loopback dashboard, verify ownership, file type, and owner-only modes on the existing Code UX security path; for Electron, restore OS `safeStorage`; for headless or remote operation, restore the configured mount, Vault, or KMS version. Never paste, print, regenerate over, or move root-key material into a repository to diagnose the failure. +- If a mutation reports a stale `expectedVersion`, refresh credential metadata and review the newer scope, capabilities, validation state, and status before retrying. Do not reuse the rejected request blindly and do not bypass the comparison check. +- If encrypted rows exist but their key version is unavailable, restore the exact retained provider version before starting runners. Replacing it with a new key does not decrypt old envelopes; restore from the independent custody backup or recover the affected credential through the supported replacement workflow after the runtime is ready. diff --git a/docs/operations/server-mode.md b/docs/operations/server-mode.md index e6a8e19cf5..a46d3b104d 100644 --- a/docs/operations/server-mode.md +++ b/docs/operations/server-mode.md @@ -107,6 +107,8 @@ Do not include `Authorization` headers in probe logs. The probe endpoints do not `/ready` also reports `credentialKey`, `auditStore`, and `distributedRunner`. `/health` remains live during a key-provider outage, while `/ready` returns `503`. Startup aborts before dashboard or MCP binding when encrypted credential rows exist but their key provider cannot recover the wrapping key. Server mode never auto-provisions local-file custody. Select a provider with `CODE_UX_CREDENTIAL_KEY_PROVIDER=mounted-key-file|vault|kms`; mounted files use `CODE_UX_CREDENTIAL_KEY_FILE` and owner-only permissions. Vault/KMS modes require their host adapter to be configured and healthy. +The same explicit-custody requirement applies to dashboard-disabled headless operation, authenticated dashboards, non-loopback dashboard bindings, and remote credential management. Only the trusted loopback local dashboard auto-provisions its owner-only user-home key; Electron uses OS `safeStorage`. Remote setup therefore fails closed rather than borrowing the local-dashboard key, deriving a key, or falling back to plaintext. Restore the configured mount or the exact Vault/KMS key version before enabling runners; do not copy root keys into SQLite, a project checkout, deployment logs, or diagnostic bundles. + Authenticated operators can inspect `/api/admin/readiness`, export redacted NDJSON from `/api/admin/audit/export`, and sample `/api/admin/metrics/slo`. Audit rows include the correlation id, principal, project, action, outcome, and redacted metadata for management requests, credential access, runs, attempts, approvals, and outbox delivery. ## Backup, Restore, Rotation, And Rollback @@ -235,6 +237,8 @@ Existing HTTP sessions authenticated with the previous token should be treated a | Worker appears stale or offline | Heartbeats stopped, the worker process is down, network access failed, or the stable connection key changed unexpectedly. | Restart the worker with the same `--connection-key`, verify `/ready`, and check logs for bounded connection metadata. | | Worker connects but does not claim work | No active project assignment, project not included in `--project-id` / `--active-project-id`, stale endpoint status, task executor mismatch, or no lease returned. | Confirm project assignment and worker status, then verify queued dispatches. Do not start local execution without a lease token. | | `/health` passes but `/ready` fails | Listener is alive but runtime readiness has not completed or the server is degraded. | Wait for startup recovery to finish, then inspect structured logs. Use `/ready` for load balancer readiness gates. | +| `/ready` reports credential custody unavailable | The explicit mounted-file, Vault, or KMS provider is missing, insecure, unhealthy, or cannot return the required key version. | Keep runners disabled, inspect metadata-only readiness and provider configuration, and restore the exact provider/key version. Do not print key material or substitute a new key for encrypted rows. | +| A credential operation returns a version conflict | Another operator changed metadata, scope, capabilities, status, or encrypted value first. | Refresh the metadata-only record, review the new version, and intentionally retry with that version. Do not bypass optimistic concurrency. | | Secret values appear in an exported settings bundle | The export was explicitly approved with `includeSecrets: true`. | Store the bundle only in approved secret storage, rotate exposed credentials if it was shared, and prefer redacted exports for review. | ## Related Docs @@ -242,4 +246,5 @@ Existing HTTP sessions authenticated with the previous token should be treated a - [MCP Runtime and Dispatch](../mcp/runtime-and-dispatch.md) - [Streamable HTTP Worker Gateway](../architecture/streamable-http-worker-gateway.md) - [Security Hardening](./security-hardening.md) +- [Automation Credential Security](./credential-security.md) - [CLI Commands Reference](../reference/cli-commands.md) diff --git a/docs/settings/integrations.md b/docs/settings/integrations.md index c19a83155a..f7d5a2c5a9 100644 --- a/docs/settings/integrations.md +++ b/docs/settings/integrations.md @@ -29,6 +29,15 @@ The create form requires an explicit name, kind, project or global scope, capabi Each project-managed credential supports bounded rename, metadata-only validation test, value rotation, encrypted-state replacement, monotonic access restriction, confirmed promotion, and confirmed revocation. Revocation requires typing `REVOKE` exactly; each lifecycle confirmation starts with cleared confirmation state and returns focus to the credential controls when it closes. Every lifecycle request uses the metadata version shown by the service. If another session wins the compare-and-swap update, the detail view refreshes metadata and asks the operator to review and retry instead of overwriting the newer state. +| Workflow | What the operator supplies | What remains readable afterward | +| --- | --- | --- | +| Create | Name, kind, write-only value, explicit capabilities, and project/global policy | Metadata, configured state, validation state, scope, capabilities, and version only. | +| Update metadata | A bounded display name and current version | Updated metadata; kind and management ownership cannot be changed. | +| Rotate / replace | A new write-only value and current version | New key/version and validation metadata, never either the old or new value. | +| Test | The current version | `valid`, `invalid`, or `unavailable` plus timestamps; no tested value or low-level custody error. | +| Restrict / promote | A monotonic restriction, or a confirmed global allowlist expansion owned by the managing project | Updated non-secret policy metadata. | +| Revoke | Exact confirmation and current version | Revoked status and audit metadata; the stored value cannot be read back. | + Secret inputs are write-only. Create, rotate, and replace fields are never populated from responses, are cleared after successful or failed submissions and project changes, and are removed with the detail view. Notices, metadata cards, browser storage, and reusable drafts contain no secret value. An allowlisted project that is not the management owner sees a **Use only** state and cannot invoke management actions. Unavailable key custody leaves non-secret metadata visible and disables secret-bearing changes and tests. Follow the inline custody guidance, restore secure storage, then use **Refresh**. See [Automation Credential Security](../operations/credential-security.md) for encryption, authority, recovery, and API behavior. @@ -63,12 +72,14 @@ If the saved setting does not appear to take effect: - Check for a project or sprint override that takes precedence over the system value. - Refresh the affected dashboard page if the setting controls a rendered surface. - Restart the local runtime only when the setting explicitly controls startup, listener, or process-level behavior. +- If secure custody is unavailable, keep the metadata view open, restore the deployment's supported custody provider, and use **Refresh**. Local loopback CLI/dashboard mode provisions its owner-only user-home key automatically; do not add mounted-key configuration for a normal local user. +- If a save reports stale metadata, review the refreshed record before retrying with its new version. Never copy secret fields into notes, browser storage, logs, or a repository as a workaround. ## Related Documentation - [Settings overview](./index.md) - [Automation Credential Security](../operations/credential-security.md) - [Google Drive Project Mount](./google-drive-mount.md) -- [Dashboard Settings](../../dashboard/design-system-settings.md) -- [Configuration and Storage](../configuration-and-storage.md) -- [Security Hardening](../../operations/security-hardening.md) +- [Dashboard Settings](../dashboard/design-system-settings.md) +- [Configuration and Storage](./configuration-and-storage.md) +- [Security Hardening](../operations/security-hardening.md) From 06f4cf9f77217f327b3f2126fb86099a204ad409 Mon Sep 17 00:00:00 2001 From: Code UX Date: Mon, 13 Jul 2026 23:50:21 +0000 Subject: [PATCH 2/2] fix(task T09): address qa review via codex --- docs-web/content/docs/user-dashboard-nodes-canvas.mdx | 6 +++++- docs-web/user/dashboard/nodes-canvas.md | 6 +++++- docs/dashboard/nodes-canvas.md | 6 +++++- 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/docs-web/content/docs/user-dashboard-nodes-canvas.mdx b/docs-web/content/docs/user-dashboard-nodes-canvas.mdx index 8aad7c5f0f..dc3c198a2b 100644 --- a/docs-web/content/docs/user-dashboard-nodes-canvas.mdx +++ b/docs-web/content/docs/user-dashboard-nodes-canvas.mdx @@ -26,7 +26,11 @@ Validated custom definitions can also execute after their immutable artifact and ## Credentials, review, and publication -Credential slots display metadata-only status such as bound, missing, or denied and can submit a binding request. Secret values remain behind the credential broker and are not returned to the graph or browser. +Credential slots display metadata-only status such as bound, missing, or denied. Opening a slot picker loads project-visible credential metadata and secure-backend health, then assesses every candidate against the versioned definition's allowed kinds and required capabilities. Only active, configured, project-authorized candidates with compatible kind/capabilities and ready secure custody are selectable; incompatible candidates remain non-selectable with a safe policy reason. The picker never resolves credential plaintext. + +Bind, replace, and remove actions persist directly from the inspector. Code UX changes only the selected slot's `{ slot, credentialId }` entry in the node's canonical `credentialBindings`, preserves sibling bindings and node data, and saves the complete graph with the loaded `draftRevision`. Removing a required binding is allowed as a draft edit, but the refreshed review immediately marks that requirement missing and blocks publication until it is satisfied. + +After a successful mutation, the page refetches the canonical flow, adopts its new revision, and refreshes governed review before reporting success. If another editor advanced the draft, the optimistic conflict path loads the latest flow and review, keeps the selected node/slot workflow available, and requires the operator to choose again; it never replays the stale binding over newer edits. Authorization or compatibility denial leaves the prior binding state intact or reports the saved binding as currently denied. Graphs, requests, component state, notices, browser storage, logs, and rendered review contain credential IDs and non-secret metadata only—never stored values. Agent attachments are also metadata-only. The selected project supplies the available preset names, and the selected flow supplies its current skill names and descriptions. Attach and detach refresh that governed backend state; project or flow transitions clear previous bindings and ignore obsolete requests. Agent instructions, custom source, credentials, and decrypted values are never rendered by the attachment controls. diff --git a/docs-web/user/dashboard/nodes-canvas.md b/docs-web/user/dashboard/nodes-canvas.md index 5d213ce7b9..26f837a8c2 100644 --- a/docs-web/user/dashboard/nodes-canvas.md +++ b/docs-web/user/dashboard/nodes-canvas.md @@ -26,7 +26,11 @@ Validated custom definitions can also execute after their immutable artifact and ## Credentials, review, and publication -Credential slots display metadata-only status such as bound, missing, or denied and can submit a binding request. Secret values remain behind the credential broker and are not returned to the graph or browser. +Credential slots display metadata-only status such as bound, missing, or denied. Opening a slot picker loads project-visible credential metadata and secure-backend health, then assesses every candidate against the versioned definition's allowed kinds and required capabilities. Only active, configured, project-authorized candidates with compatible kind/capabilities and ready secure custody are selectable; incompatible candidates remain non-selectable with a safe policy reason. The picker never resolves credential plaintext. + +Bind, replace, and remove actions persist directly from the inspector. Code UX changes only the selected slot's `{ slot, credentialId }` entry in the node's canonical `credentialBindings`, preserves sibling bindings and node data, and saves the complete graph with the loaded `draftRevision`. Removing a required binding is allowed as a draft edit, but the refreshed review immediately marks that requirement missing and blocks publication until it is satisfied. + +After a successful mutation, the page refetches the canonical flow, adopts its new revision, and refreshes governed review before reporting success. If another editor advanced the draft, the optimistic conflict path loads the latest flow and review, keeps the selected node/slot workflow available, and requires the operator to choose again; it never replays the stale binding over newer edits. Authorization or compatibility denial leaves the prior binding state intact or reports the saved binding as currently denied. Graphs, requests, component state, notices, browser storage, logs, and rendered review contain credential IDs and non-secret metadata only—never stored values. Agent attachments are also metadata-only. The selected project supplies the available preset names, and the selected flow supplies its current skill names and descriptions. Attach and detach refresh that governed backend state; project or flow transitions clear previous bindings and ignore obsolete requests. Agent instructions, custom source, credentials, and decrypted values are never rendered by the attachment controls. diff --git a/docs/dashboard/nodes-canvas.md b/docs/dashboard/nodes-canvas.md index 1181d90371..a30cc8372a 100644 --- a/docs/dashboard/nodes-canvas.md +++ b/docs/dashboard/nodes-canvas.md @@ -12,7 +12,11 @@ On the first load for a selected project, the dashboard checks the former `codeu The versioned definition registry supplies the palette, executable state, typed ports, configuration and widget schemas, capabilities, credential slots, side-effect classification, and default retry/timeout policy. The inspector is rendered from the selected definition rather than a hard-coded node form. The graph stores a type/version reference, non-secret configuration, and credential ids; it never stores custom source or credential values. -Credential slots show metadata-only states such as bound, missing, or denied and can submit a binding request. Secret material stays behind the credential broker and is excluded from graphs, browser output, logs, and examples. +Credential slots show metadata-only states such as bound, missing, or denied. Opening a slot picker loads project-visible credential metadata and secure-backend health, then assesses every candidate against the versioned definition's allowed kinds and required capabilities. Only active, configured, project-authorized candidates with compatible kind/capabilities and ready secure custody are selectable; incompatible candidates remain non-selectable with a safe policy reason. The picker never resolves credential plaintext. + +Bind, replace, and remove actions persist directly from the inspector. Code UX changes only the selected slot's `{ slot, credentialId }` entry in the node's canonical `credentialBindings`, preserves sibling bindings and node data, and saves the complete graph with the loaded `draftRevision`. Removing a required binding is allowed as a draft edit, but the refreshed review immediately marks that requirement missing and blocks publication until it is satisfied. + +After a successful mutation, the page refetches the canonical flow, adopts its new revision, and refreshes governed review before reporting success. If another editor advanced the draft, the optimistic conflict path loads the latest flow and review, keeps the selected node/slot workflow available, and requires the operator to choose again; it never replays the stale binding over newer edits. Authorization or compatibility denial leaves the prior binding state intact or reports the saved binding as currently denied. Graphs, requests, component state, notices, browser storage, logs, and rendered review contain credential IDs and non-secret metadata only—never stored values. Pointer dragging uses local preview state inside the canvas and persists the final position only on pointer release. The workspace also suspends the global animated WebGL background while `/nodes` is active, which keeps canvas interaction on a bounded compositor path without changing the configured appearance on other visible routes.