Skip to content

Integrations

Jump to bottom
npond edited this page Sep 2, 2026 · 1 revision

Integrations

Provenance. Generated by /n8-map from the commit named below, and not regenerated since. Individual statements may have been overtaken by later work — most of the issues this map cites have since been fixed and closed. Read it as a snapshot of the codebase's shape, not as a current defect list; the GitHub issue register is authoritative for what is actually open. (The generating commit hash predates a history rewrite and no longer resolves.)

Every external service and sidecar AutoNate.Web talks to — where the client lives, which config keys drive it, how it authenticates, what happens when it is down, and how to run it locally. Read Stack.md §8–§11 first for the commands and config keys.

Generated from commit 01f0f174 on 2026-08-31 by /n8-map.

Topology at a glance

browser ──5173 Vite──▶ AutoNate.Web :5108 ──HTTP Basic──▶ Flowable REST :8080 (JVM + flowable-extension jar)
   │                       │  ▲                                  │ daprd (flowable-dapr, netns-shared) ──▶ NATS JetStream :4222
   │ ws://:1234            │  │ POST /api/workflow-behaviors/{key}/execute  (X-AutoNate-Internal-Token)
   ▼                       │  │
hocuspocus ──pg──▶ Postgres:5432 ◀── EF Core / Npgsql ── AutoNate.Web
   │ POST /internal/yjs-auth, /internal/yjs-webhook (X-AutoNate-Internal-Token + HMAC)
   ▼
AutoNate.Web ──HTTP :3500──▶ daprd (autonate-web, host process) ──▶ NATS JetStream (pubsub) / Redis :6379 (statestore)
AutoNate.Web ──NATS request/reply──▶ services/executor (core queue group "executor"; no JetStream stream captures this subject — archived-141)
AutoNate.Web ──HTTPS──▶ api.anthropic.com / api.openai.com / api.tavily.com / REST data connectors

Rule of thumb enforced across the codebase: every outbound HttpClient is a named IHttpClientFactory client with an explicit Timeout, registered in Program.cs ("data-connector" 30 s at :490, "agent.anthropic"/"agent.openai" 5 min at :618-619, "agent.webfetch" 10 s + no cookies + 5 redirects at :645-654, "agent.websearch" 15 s + no redirects at :660-668, FlowableClient typed client 30 s at :935 via FlowableClient.ConfigureHttpClient). Probes that use the unnamed CreateClient() set httpClient.Timeout inline (2–3 s). Follow that pattern for any new integration; never new HttpClient().

1. PostgreSQL (primary store)

  • Role: single source of truth for the app (AutoNate DB), plus Flowable's own DB (flowable, the image's POSTGRES_DB), plus the optional autonate_datastores DB, plus per-plugin schemas.
  • Client: EF Core 9 via IDbContextFactory<AutoNateDbContext> (src/AutoNate.Web/Persistence/AutoNateDbContext.cs, .DataStores.cs, .ProjectionCaches.cs; scaffolded entities in Persistence/Scaffolded/). Registered at Program.cs:265-280 with DbConnectionFailureLoggingInterceptor and RelationalEventId.ConnectionError suppressed (cancelled SPA requests otherwise spam fail: 20004). Stores are EfCore*Store classes under Services/<Domain>/; singletons that need DB access take the factory and create a context per call (e.g. Services/SystemIssues/EfCoreSystemIssueStore).
  • Config: ConnectionStrings:Default (dev: Host=localhost;Port=5432;Database=AutoNate;Username=autonate;Password=Your_password123!;Keepalive=30;Tcp Keepalive=true;Connection Idle Lifetime=60;Connection Pruning Interval=10). Keep the keepalive/pruning suffix when you copy the string — it exists to stop idle-pool drops.
  • Schema init — no EF migrations: Persistence/IDatabaseInitializer.cs implementations run in Order at boot (Program.cs:1039-1048): PrimaryDatabaseInitializer (Order 0) → DatabaseSchemaInitializer.EnsureAsync (Persistence/DatabaseSchemaInitializer.cs:3746, ~70 sequential ExecuteSqlRawAsync calls over idempotent SQL constants, 82 CREATE TABLE IF NOT EXISTS), then DatastoresDatabaseInitializer (Order 10). Base tables and the seeded admin user come from infra/postgres/init/02-create-autonate-app-schema.sql (compose runs it once; the E2E fixture replays it). To add schema: append a private const string XyzSql = """…"""; and an ExecuteSqlRawAsync(XyzSql, …) line at the end of EnsureAsync; write it idempotently (IF NOT EXISTS, ADD COLUMN IF NOT EXISTS, ON CONFLICT DO NOTHING).
  • Datastores DB (Services/DataStores/Sql/DatastoresDatabaseInitializer.cs): probes pg_database, CREATE DATABASE via the maintenance DB postgres, then ensures the DataStores:Sql:WriterRole (autonate_datastore_writer) with a password from DataStores:Sql:WriterRolePassword or a generated one persisted at {Data:Root}/datastores-writer.secret. Absent ConnectionStrings:Datastores ⇒ Info log, feature off, SqlType endpoints return 503. SqlDataStoreProvisioner creates one schema + read-only role per data store; CsvIngestor streams via NpgsqlBinaryImporter (COPY).
  • Per-plugin isolation (Plugins/PluginSchemaProvisioner.cs): 8-char code, LOGIN role with DataProtection-encrypted password (purpose AutoNate.Plugins.RolePassword.v1), schema plg_<code> owned by that role, group role plg_readers for cross-plugin SELECT. Plugins/PluginMigrationRunner.cs applies <plugin>/migrations/*.sql in lexical order, tracking in <schema>.__plugin_migrations, one transaction per file; a failure disables the plugin with last_error.
  • Failure behaviour: boot aborts if ConnectionStrings:Default is missing (Program.cs:269) or the initializer SQL throws. Runtime connection faults are logged at Warning by the interceptor; request-cancellation errors are silenced.
  • Local: make infra-ensure (compose postgres:16-alpine, data in infra/mounts/postgres/data, superuser autonate). Inspect: docker exec autonate-postgres psql -U autonate -d AutoNate. Tests create autonate_test_<guid> DBs (tests/AutoNate.Web.Tests/PostgresTestDatabase.cs); E2E recreates AutoNate_E2E.

2. Flowable REST (workflow engine) + flowable-extension

.NET → Flowable

  • Client: Services/Flowable/FlowableClient.cs (1560 lines) implementing IFlowableClient; typed HttpClient registered at Program.cs:935-940. FlowableClient.ConfigureHttpClient (:1534-1549) sets BaseAddress = Flowable:BaseUrl + "/", Timeout = 30 s, and Authorization: Basic base64(Username:Password). Every call does await EnsureSuccessAsync(response, "verb the thing") (:1515-1525) which throws InvalidOperationException("Flowable could not <op>. HTTP <code> <reason>. <body>") — copy that wording when adding calls. Paths are relative (service/repository/deployments, service/runtime/process-instances, service/history/..., service/runtime/signals, …).
  • Config: Flowable:BaseUrl (http://localhost:8080/flowable-rest), Flowable:Username/Password (image default rest-admin/test). FlowableOptions in Configuration/InfrastructureOptions.cs.
  • Script-task guard: before deploying BPMN containing a scriptTask, EnsureJavaScriptScriptTaskSupportAsync (:1395-1435) probes actuator/scriptTaskSupport then service/autonate/script-task-support (both served by the extension) and refuses the deploy if the JVM has no JS engine or the probe endpoint is absent ("rebuild the Flowable image").
  • Projection cache: Services/Flowable/Cache/Flowable{Execution,Task,Variable,History}PollingFeed.cs poll Flowable on FlowableCache:*PollInterval timers into Postgres cache tables through the projection framework (Program.cs:942-1003); FlowableReadThrough serves reads with ReadThroughFreshness 30 s; WorkflowCacheRetentionService + ColdTier/ColdTierArchiverService (Parquet + DuckDB, FlowableCache:ColdTier:Enabled=false by default) age data out.
  • Failure behaviour: exceptions propagate to endpoints (500 / problem details); polling feeds log and retry on the next tick. SystemHealthService.CheckFlowableAsync surfaces reachability on GET /api/health/system. Tests replace the client with StubFlowableClient (AutoNateWebApplicationFactory.cs:100-104).

Flowable → .NET (the extension, flowable-extension/src/main/java/com/autonate/flowableevents/)

  • Registration: FlowableExecutionEventAutoConfiguration (Spring Boot auto-config via META-INF/spring/…AutoConfiguration.imports) registers WorkflowExecutionEventListener + WorkflowFailureEventListener as Flowable engine event listeners, the autonateBehaviorDelegate bean (name is load-bearing — BPMN service tasks reference ${autonateBehaviorDelegate}), dueDateHelper, and the script-task probe controller (GET /service/autonate/script-task-support) + actuator endpoint.
  • Config (FlowableExecutionEventProperties, prefix autonate.flowable-events, injected by compose as AUTONATE_FLOWABLE_EVENTS_* env vars): dapr-publish-base-url (http://127.0.0.1:3500 — the flowable-dapr sidecar sharing the container's network namespace), pubsub-name (pubsub), topic-root (workflow.execution.events), source-app-id (flowable), callback-base-url (http://host.docker.internal:5108), callback-shared-secret, behavior-timeout-seconds (30).
  • Event loop: engine events (PROCESS_STARTED, ACTIVITY_, TASK_, PROCESS_COMPLETED/CANCELLED, failures) → WorkflowExecutionEventMapper → DaprWorkflowEventPublisher.publish → POST {daprPublishBaseUrl}/v1.0/publish/{pubsub}/{topic}?metadata.rawPayload=true. Failures are logged at WARN and dropped (no retry). On the .NET side DaprStreamingSubscriber always subscribes to workflow.execution.events and fans out through BusWatcherStreamService to WorkflowSignalDispatcher, WorkflowExecutionErrorRecorder, WorkflowTaskNotificationListener, the /ws/bus-watcher WebSocket, and the SystemIssue detectors.
  • Behavior callback loop: AutoNateBehaviorDelegate.execute reads flowable:autonateServiceKind="behavior" + flowable:behaviorKey off the service task, POSTs execution id/variables to {callback-base-url}/api/workflow-behaviors/{key}/execute with headers X-AutoNate-Internal-Token: <secret> and X-Correlation-Id, timeout = behavior-timeout-seconds. Non-2xx / IO / timeout / missing config ⇒ FlowableException (engine job retry); a JSON reply with failed: true is applied (variableUpdates) and not thrown so the workflow author branches on it. Server side: Endpoints/WorkflowBehaviorEndpoints.cs:22 MapPost("/{key}/execute") on an .AllowAnonymous() group with .DisableAntiforgery().AddEndpointFilter<SharedSecretEndpointFilter>(); the filter (Endpoints/SharedSecretEndpointFilter.cs) does a constant-time compare against WorkflowBehaviors:CallbackSharedSecret and returns 401 for missing header, mismatch, or unconfigured secret alike. Behaviors implement IWorkflowBehavior (Services/Workflow/Behaviors/, built-in UnlockAccountBehavior; plugins add more via IPluginContext.Behaviors).
  • Secret pairing: .env AUTONATE_BEHAVIOR_CALLBACK_SECRET (compose → JVM) must equal WorkflowBehaviors:CallbackSharedSecret (.NET). Dev default on both sides: dev-only-workflow-behavior-secret-change-me. Non-Development .NET refuses to start without it (Program.cs:776-781).
  • Local: image built by infra/flowable/Dockerfile (Maven stage runs mvn --batch-mode test package); infra/ensure-up.sh rebuilds on source change and force-recreates flowable + flowable-dapr. FLOWABLE_PROCESS_HISTORY_LEVEL=full is required for the variable-history UI. Health: curl http://127.0.0.1:8080/flowable-rest.

3. Dapr (pub/sub + state + control plane)

  • Sidecar: autonate-web daprd runs on the host, started by make app (dapr run …) or infra/start-autonate-web-sidecar.sh (daprd --app-id autonate-web --app-port 5108 --dapr-http-port 3500 --dapr-grpc-port 50001 --placement-host-address 127.0.0.1:50006 --scheduler-host-address 127.0.0.1:50007 --resources-path infra/mounts/dapr-dashboard/components). Components: infra/dapr/components/pubsub.yaml (pubsub.jetstream, natsURL nats://localhost:4222, streamName workflow-execution, deliverPolicy new, scopes autonate-web,flowable) and statestore.yaml (state.redis, localhost:6379, actorStateStore true, scope autonate-web).

  • Config: Dapr:* (Configuration/InfrastructureOptions.cs DaprOptions): HttpEndpoint http://127.0.0.1:3500, GrpcEndpoint http://127.0.0.1:50001, PubSubName pubsub, StateStoreName statestore, placement/scheduler addresses (used only by the health page).

  • Startup probe: Services/Dapr/DaprSidecarProbe.cs — GET {HttpEndpoint}/v1.0/metadata, 2 s timeout, false on any error. In Development Program.cs:1022-1037 throws at boot unless it succeeds or AUTONATE_ALLOW_RUNNING_WITHOUT_DAPR=true. Outside Development nothing checks; publishes just fail.

  • Publishing (outbound): never DaprClient. All domain publishers (Services/Events/AuditEventPublisher.cs DaprAuditEventPublisher, Services/Records/RecordEventPublisher.cs, Services/ApplicationEvents/ApplicationEventPublisher.cs, Services/Notifications/NotificationEventPublisher.cs) serialize an envelope and call IAuditEventOutbox.EnqueueAsync(topic, eventType, json). With AuditOutbox:Enabled=true (default) that is EfCoreAuditEventOutbox → audit_outbox row → Services/Events/AuditOutboxDispatcher.cs (BackgroundService, 2 s poll, FOR UPDATE SKIP LOCKED, exponential backoff 5 s→10 m, 50 attempts then parked; dead letters handled by AuditOutboxDeadLetterParkRemediator). With it false, DirectPublishAuditEventOutbox (Services/Events/AuditEventOutbox.cs:128) posts immediately. Both do:

    var publishUri = new Uri(endpoint, $"/v1.0/publish/{pubsub}/{topicEscaped}?metadata.rawPayload=true");
using var content = new ByteArrayContent(Encoding.UTF8.GetBytes(payloadJson)) { Headers = { ContentType = new("application/json") } };
await httpClientFactory.CreateClient().PostAsync(publishUri, content, ct);

    Topics are <root>.events constants next to each publisher (record.events, application.events, notification.events, auth.events, iam.events, record-schema.events, site.events, workflow-admin.events, system.issues, agent.events, external-connections.events, content.events, dashboards.events, query.events, datastore.events, workflow.execution.events). A new top-level root must also be added to NatsStreamProvisioner.DesiredStreams (§4) or Dapr answers HTTP 500 "no response from stream" and the outbox loops. Use the add-audit-event / add-record-event-type skills.

  • Subscribing (inbound): Services/Signals/DaprStreamingSubscriber.cs (IHostedService) uses Dapr.Messaging DaprPublishSubscribeClient.SubscribeAsync (gRPC/HTTP endpoints from Dapr:*, Program.cs:247-258). It always subscribes workflow.execution.events and dynamically subscribes every signal-start topic registered by published workflows (IWorkflowSignalRegistry, EfCoreWorkflowSignalRegistry). There is no /dapr/subscribe manifest and no HTTP push endpoint.

  • Watchdog: every 15 s the subscriber publishes {} to application.healthprobe (DaprStreamingSubscriber.cs:312-338, 3 s timeout). Down→Up edge ⇒ dispose + resubscribe all handles. Unhealthy ≥ 45 s with ≥ 120 s since the last restart ⇒ it executes infra/restart-autonate-web-sidecar.sh via /bin/bash (found by walking up ≤ 8 parents from AppContext.BaseDirectory, 45 s wait) and re-syncs. This only works on a dev box with the repo layout; in containers the script is absent and the watchdog just logs.

  • State store (Redis): only touched by SystemHealthService.CheckRedisStateAsync (GET /v1.0/state/{StateStoreName}/health-probe, 3 s). No feature persists state through Dapr today.

  • Health: GET /api/health/dapr (probe) and GET /api/health/system (Endpoints/HealthEndpoints.cs) → Services/SystemHealth/SystemHealthService.cs graph: sidecar metadata, pub/sub publish probe, NATS stream stats, Redis via state API, Flowable, placement/scheduler TCP.

  • Local: make infra-ensure brings up dapr-placement :50006, dapr-scheduler :50007 (etcd data in infra/mounts/dapr-scheduler/data), flowable-dapr; dapr init --slim once so daprd exists on the host (start-autonate-web-sidecar.sh looks in PATH then ~/.dapr/bin/daprd). Dashboard: make infra-up-dashboard → :8081.

4. NATS / JetStream

  • Client: NATS.Client.JetStream 2.5.10. Three usage patterns:
    1. Services/Nats/NatsStreamProvisioner.cs — short-lived connection at boot (Program.cs:1054-1058): deletes legacy stream autonate-records, then CreateOrUpdateStreamAsync for workflow-execution (every <root>.> subject listed above, MaxAge 24 h) and pipeline-code-runs (removed; listed in LegacyStreamsToRemove) (pipeline-code-run.>, 24 h). Skipped with an Info log when Nats:Url is empty.
    2. Services/Nats/INatsConnectionProvider.cs NatsConnectionProvider — one lazily-opened shared connection for hot request/reply callers.
    3. SystemHealthService.CheckNatsAsync — short-lived connection reading workflow-execution stream info (messages/consumers/lastSeq; zero consumers with messages = the silent Dapr disconnect signature).
  • Config: Nats:Url (nats://127.0.0.1:4222 in dev). No auth (compose runs nats --jetstream --store_dir /data --http_port 8222, data in infra/mounts/nats/data).
  • Executor request/reply: Services/Pipelines/Execution/JetStreamCodeNodeRunner.cs builds a CodeNodeRequest (CodeNodeWireFormat.cs, wire version: 1, language js|python, kind transformer|analyzer, timeoutMs 30000, memoryMb 128) and calls an explicit inbox (NewInbox + SubscribeCoreAsync + PublishAsync(replyTo:)), skipping any inbox message that is not shaped like a CodeNodeReply (a JetStream PubAck would otherwise be parsed as a failed reply — archived-141) with a 30 s linked cancellation; timeout ⇒ InvalidOperationException("Executor sidecar did not reply within 30s."). services/executor/src/index.ts is a core NATS subscriber (nc.subscribe("pipeline-code-run.>", { queue: "executor" }), not a JetStream durable consumer despite the comments) and replies { success, errorMessage, output } on the reply subject; JS runs in isolated-vm (jsRunner.ts); Python runs in Pyodide inside a single-use worker_threads Worker per request (pythonRunner.ts spawns/pools, pythonWorker.ts loads Pyodide with an empty jsglobals, unregisters pyodide_js, disables fetch/WebSocket; the parent enforces timeoutMs via SIGINT-in-interrupt-buffer then terminate(), and memoryMb via a WebAssembly.Memory.prototype.grow cap — archived-58/archived-161; knobs EXECUTOR_PY_WARM_WORKERS, EXECUTOR_PY_MAX_CONCURRENCY, EXECUTOR_PY_JS_HEAP_MB); isUnsafe is received but ignored. The executor runs as the executor service in infra/docker-compose.yml (started by make infra-ensure; no ports, restart: unless-stopped); its compose healthcheck is a NATS request to executor.health (services/executor/src/healthcheck.ts), which infra/ensure-up.sh waits on via container health.
  • Stream bootstrap outside the app: infra/scripts/bootstrap-jetstream.sh (run by the nats-init container on every compose up) creates/edits workflow-execution with subjects workflow.execution.> record.> application.> content.> so the Flowable extension can publish before .NET boots. infra/ensure-nats-stream.sh (called at the end of infra/ensure-up.sh and by make infra-prepare) edits the same stream with --force to only workflow.execution.> — the app's provisioner re-widens it on the next boot, but between make infra-ensure and app start, publishes to other roots fail. Keep NatsStreamProvisioner.DesiredStreams and bootstrap-jetstream.sh in sync, and prefer fixing ensure-nats-stream.sh to match if you touch it.
  • Local: make infra-ensure; monitor at http://localhost:8222; docker run --rm --network container:autonate-nats natsio/nats-box:0.16.0 nats --server nats://127.0.0.1:4222 stream info workflow-execution.

5. Redis

  • Used exclusively as the Dapr statestore component (infra/dapr/components/statestore.yaml, redis:7.4-alpine, --appendonly yes, data in infra/mounts/redis/data, no password). No .NET code references Redis directly (grep -ri redis src/AutoNate.Web --include=*.cs hits only the health service). The README lists it under Dapr pub/sub, but pub/sub is JetStream.
  • Failure: the app keeps working; GET /api/health/system reports the redis component Down via the Dapr state probe.
  • Local: compose service redis :6379; docker exec autonate-redis redis-cli ping.

6. Hocuspocus / Yjs (collaborative editing sidecar)

  • Sidecar: services/hocuspocus/src/index.ts — @hocuspocus/server on HOCUSPOCUS_PORT (1234) with three extensions: persistence.ts (Y.Doc binary state in the AutoNate DB table yjs_documents, created by DatabaseSchemaInitializer YjsDocumentsSchemaSql at :2952; seeds first-open docs from pages.body_jsonb/notes.content_jsonb), auth.ts (calls .NET), webhook.ts (mirrors snapshots to .NET). Document names are <prefix>:<guid> with prefixes page, note, napkin, diagram, documents (materializers.ts:287-293). Required env: YJS_INTERNAL_SHARED_SECRET, AUTONATE_WEB_URL, POSTGRES_DB/USER/PASSWORD (process.exit(1) if missing); unhandledRejection/uncaughtException are logged, not fatal.
  • .NET side (Endpoints/YjsEndpoints.cs, options Services/Yjs/YjsServerOptions.cs):
    • POST /api/yjs/ticket (cookie-authenticated, :44-46): authorizes the caller on the document via IContentAuthorizer, mints an HMAC-SHA256 ticket (MintTicket :629, payload = document, actor, display name, role editor|commenter|viewer, jti, exp = TicketTtlSeconds 60) and returns { ticket, wsUrl: HocuspocusWsUrl, expiresInSeconds, role }. The SPA passes the ticket as the HocuspocusProvider token.
    • POST /internal/yjs-auth (:219, filter YjsInternalSecretEndpointFilter on header X-AutoNate-Internal-Token): verifies the HMAC, document-name match, single-use jti via IMemoryCache (yjs-jti:<jti>, TTL+60 s — single-instance only), re-runs the authorizer, returns { userId, displayName, role }. auth.ts sets connectionConfig.readOnly = true for any role other than editor.
    • POST /internal/yjs-webhook (:344, same filter plus X-AutoNate-Yjs-Signature: sha256=<hex HMAC of raw body> check): stores the materialized bodyJsonb verbatim into the page/note/document row. YjsManagedContentGuard (Endpoints/YjsManagedContentGuard.cs) rejects REST patches to Yjs-managed bodies — write through Yjs, never through the REST body.
  • Secret pairing: .env YJS_INTERNAL_SHARED_SECRET (compose → sidecar) must equal YjsServer:InternalSharedSecret (.NET). Dev default dev-only-yjs-internal-secret-change-me; non-Development .NET refuses to start without it (Program.cs:790-795).
  • Failure behaviour: sidecar down ⇒ browsers cannot connect (ticket still issues); .NET down ⇒ every WebSocket connect is refused (auth hook throws) and webhook change posts throw (Hocuspocus logs, next edit retries — Y.Doc state in Postgres is authoritative, the JSON mirror lags); disconnect webhook failures are swallowed. No /health endpoint — ensure-up.sh treats an open TCP port as ready.
  • Local: compose service hocuspocus (restart: unless-stopped, AUTONATE_WEB_URL=http://host.docker.internal:5108, extra_hosts host-gateway for Linux). Rebuild is automatic on source change via ensure-up.sh; --force Vite deps after heavy installs.

7. LLM providers and web search (External Connections)

  • Storage: admin-managed rows in external_connections via Services/ExternalConnections/EfCoreExternalConnectionStore.cs; secrets encrypted with ASP.NET DataProtection (DataProtectionConnectionSecretProtector.cs, purpose AutoNate.ExternalConnections.v1; keyring under the content root by default — mount/persist it or rotated keys make stored secrets unreadable). RevealForResolverAsync is the only decrypt path. Kinds are strings: LlmProvider:Anthropic, LlmProvider:OpenAI, WebSearchProvider:Tavily. Metadata JSON may carry model and baseUrl overrides. baseUrl is allowlisted per kind by Services/ExternalConnections/ProviderBaseUrlPolicy.cs (IProviderBaseUrlPolicy) at the three places untrusted metadata becomes a destination — ChatProviderResolver, WebSearchProviderResolver, ConnectionModelLister — because the decrypted key rides on every request built from it (archived-61). Built-ins api.anthropic.com / api.openai.com / api.tavily.com; extend via ExternalConnections:AllowedProviderHosts:<kind> (host-only, *. wildcard allowed); https required.
  • Chat: Services/Agent/Providers/ChatProviderResolver.cs picks the connection (explicit id or default-enabled for the kind), resolves the model (metadata.model → IAgentModelCatalog default/first-available → hard-coded claude-sonnet-4-6 / gpt-4.1) and builds AnthropicChatProvider (POST {baseUrl|https://api.anthropic.com}/v1/messages, headers x-api-key, anthropic-version: 2023-06-01, SSE streaming) or OpenAIChatProvider ({baseUrl|https://api.openai.com}, Authorization: Bearer) over the 5-minute named clients. Non-2xx becomes a ChatStreamChunk.Error (retryable when ≥ 500) rather than an exception. ConnectionModelLister.cs lists models from /v1/models on both. Model catalog seed + default live in agent_model (DatabaseSchemaInitializer AgentModelCatalogSeedSql), broadcast to SPAs over /ws/agent-model-default.
  • Search / fetch: Services/Agent/Search/WebSearchProviderResolver.cs → TavilyWebSearchProvider (https://api.tavily.com, "agent.websearch" client). WebFetchSkill uses "agent.webfetch" with IDnsResolver for SSRF checks. Both are filtered out of the tool list when the site setting chatbot.internetAccessEnabled is off.
  • Loop bounds: Agent:MaxIterations 25, Agent:ToolTimeoutSeconds 30, Agent:DefaultMaxTokens 4096 (Services/Agent/Loop/AgentOptions.cs). Streams to the browser via SSE on /api/agent/... — proxies must not buffer.
  • Failure behaviour: no enabled connection ⇒ ResolveDefaultForKindAsync returns null and the chat endpoint reports a configuration error; provider errors surface as error chunks in the stream. Tests stub HTTP (AnthropicChatProviderTests.cs, OpenAIChatProviderTests.cs).
  • Local: create a connection at the External Connections admin page (or POST /api/external-connections) with a real API key; nothing in compose.

8. Data connectors (REST / SMB / plugin)

  • Contract: Services/DataConnectors/IDataConnectorHandler.cs — Kind, TestAsync(DataConnector), FetchAsync(connector, state, IConnectorFetchSink). Registry DataConnectorHandlerRegistry composes host handlers + plugin ones (Plugins/PluginConnectorRegistry → PluginDataConnectorAdapter.cs).
  • REST (Builtin/RestDataConnectorHandler.cs): GET only, auth modes bearer / basic / api-key from RestConnectorConfig, {lastFetchDate} URL interpolation, JSON RowsPath selection, uses the "data-connector" client (30 s). The interpolated URL goes through Services/Http/OutboundUrlGuard.cs (IOutboundUrlGuard) before any socket opens — scheme must be http/https (https required outside Development) and every resolved address must be public, so loopback / RFC1918 / link-local 169.254.169.254 / CGNAT / IPv6 ULA destinations are refused (archived-60). No host allowlist here on purpose: calling arbitrary third-party APIs is the feature. Note ParseConfig binds case-sensitively while the SPA writes camelCase, so UI-authored configs currently do not bind (archived-165). TestAsync returns ConnectorTestResult.Fail(...) on HTTP/network/config errors instead of throwing.
  • SMB (Builtin/SmbDataConnectorHandler.cs): registered so the kind appears in the UI, but TestAsync fails with an explanatory message and FetchAsync throws NotSupportedException — not implemented.
  • Consumers: Datasets (Services/Datasets/*, DatasetRefreshScheduler 1-minute cron loop, CachedDatasetMaterializer) and Pipelines (Services/Pipelines/Orchestration/PipelineRunWorker 5 s poll). Files-backed datasets parse via Services/Datasets/Files/{CsvFileParser,RawFileParser} from {Data:Root}/datastores.

9. Plugin host

  • Load path: Plugins/PluginHostedService.cs at boot sweeps orphan folders under PluginRuntime.PluginRoot (Plugins:Folder or {Data:Root}/plugins), hard-deletes DeletedPending rows, then loads every Enabled row through Plugins/PluginRuntime.cs EnableAsync: extract zip → validate plugin.json (PluginManifest.cs: name, version, entryAssembly, optional entryType, templates) → provision schema/role (PluginSchemaProvisioner) → run migrations/*.sql (PluginMigrationRunner) → new collectible PluginAssemblyLoadContext (Plugins/PluginAssemblyLoadContext.cs; SharedAssemblies = Abstractions, DI/Logging Abstractions, Npgsql, Dapper resolve to the host ALC) → instantiate IAutoNatePlugin → Configure(IPluginContext) (hooks via IHookRegistrar, menus, behaviors, connectors, transformers, analyzers, agent skills, scheduled projections, page templates). Failure flips the row to Disabled with last_error; with Plugins:FailFastOnStartup=true the host exits instead. DisableAsync/CleanupAsync call Cleanup(context) then alc.Unload().
  • Upload: POST /api/admin/plugins (Endpoints/AdminPluginsEndpoints.cs) validates size (Plugins:MaxUploadBytes) and zip contents (PluginUploadValidator.cs). Plugins run in-process with the host's identity; there is no sandbox beyond the ALC and the per-plugin DB role.
  • Hooks: Hooks/ (HookRegistrar, IActionHub, IFilterHub; HookPoints constants in the abstractions project). Host services dispatch through IActionHub/IFilterHub; plugins receive IHookRegistrar. Data access: IPluginDataAccess returns host NpgsqlConnections scoped to the plugin role.
  • Local: build any project under plugins/ → dist/<Name>.zip → upload + enable at /admin/plugins. Sample plugins: plugins/HelloPlugin (minimal), plugins/Auditor (migrations + page templates + audit firehose). The test plugin tests/AutoNate.Web.Tests.SamplePlugin is loaded from disk by PluginLoaderTests.

10. Dev tooling integrations

  • Vite dev proxy (src/AutoNate.Spa/vite.config.ts): forwards /api, /account, /dapr, /bus-watcher, /files and WebSockets /ws/bus-watcher, /ws/agent-model-default to ASPNETCORE_URL ?? http://localhost:5108; swallows ECONNRESET/EPIPE WS-proxy noise. SpaProxy (Microsoft.AspNetCore.SpaProxy 10.0.7) spawns it from the http launch profile.
  • MCP servers (.mcp.json): mantine (npx -y @mantine/mcp-server) for component docs, playwright (npx @playwright/mcp@latest --output-dir ./temp).
  • Dependabot (.github/dependabot.yml) tracks nuget (root), npm (/src/AutoNate.Spa, /services/hocuspocus, /services/executor, root), maven (/flowable-extension), and github-actions. CI lives in .github/workflows/ci.yml (added after this map was generated).

Home

Getting started

Using Auton8

Building Auton8

Repository

Clone this wiki locally