plugin-workers + SDK: 5 correctness/DX gaps surfaced wiring eis-chat async KB ingestion (OpenAPI-only routes, triggerJob body-id, job auto-registration, Service PluginReferences, per-channel tursodb single-writer)

[rickylabs]

Dogfooding report from the eis-chat consumer (rickylabs/eis-chat). Wiring eis-chat's async KB-ingestion pipeline (drop/paste/screenshot → chunk → embed → searchable RAG) onto NetScript alpha.19 required a 7-layer fix chain — most of it eis-chat-side workarounds for real framework/plugin-workers/SDK gaps. Sharing each with the concrete incident, the workaround we shipped, and a proposed framework fix — as context, not doctrine. The loop is now green end-to-end; these would let the next consumer skip the excavation.

Related: #238 (AI-stack epic), #264 (AppHost DX gaps — some overlap noted), #174 (--allow-ffi).

The pipeline (for context)

POST /api/knowledge → eischat creates a pending doc + triggerJob('embed-document') → workers-API → workers runtime executes workers/jobs/embed-document.ts (chunk + embed) → writes back to eischat → doc ready + searchable.

Gap 1 — plugin services expose OpenAPI but not RPC, so createServiceClient can't call them

createServiceClient({serviceName, routerName}) builds an oRPC RPC link (GET /api/rpc/{router}/{procedure}). But the plugin-workers service mounts its routes only on the OpenAPI handler (/api/v1/...); /api/rpc/* 404s entirely on it (it works on our own oRPC service, eischat). So the typed SDK client structurally cannot call a plugin service.

• Incident: eischat's workersClient.triggerJob(...) → GET /api/rpc/v1/workers/triggerJob → 404 "Route not found on workers service".
• Workaround: eis-chat call the OpenAPI route directly (POST {base}/api/v1/workers/jobs/{id}/trigger) — PR ci: add deferred e2e-cli gate (scaffold-static + scaffold-runtime) #81, commit range on services/eischat/src/jobs.ts.
• Proposed fix: either plugin services also mount on the RPC handler, or createServiceClient gains an OpenAPI transport for plugin targets. Today there's no typed-client path to a plugin service.

Gap 2 — triggerJob needs the job id in the BODY, not just the path (silent undefined tasks)

The OpenAPI route is POST /jobs/{id}/trigger, but JobTriggerInputZodSchema requires id in the body, and the created task's id is taken from the body — the {id} path segment only routes. Sending {payload, correlationId} (no body id) creates a task with id = undefined.

• Incident: worker log Processing task 'undefined' (trigger: api) → Task 'undefined' failed: Workers KV key contains unsupported part: undefined. Doc stuck pending, no error surfaced to the caller.
• Workaround: send { id: jobId, payload, correlationId? } — eis-chat PR prime-time: service-auth-adapters (WorkOS + better-auth) #83.
• Proposed fix: derive the task id from the path segment (single source of truth), or fail loudly when the body id is missing instead of persisting undefined.

Gap 3 — custom jobs aren't auto-registered with the plugin service

The workers-API boot registry contained only the built-in workers-plugin-health-check; our custom jobs (embed-document, transcribe-image) weren't known to it, so triggerJob('embed-document') returned "not found" even though the workers runtime could execute them.

• Workaround: eis-chat added workers/job-definitions.ts + a generated .netscript/generated/plugin-workers/job-registry.ts — PR [prime-time] Worker applied-keys dedup #79. Boot log then showed Total jobs in registry: 3.
• Proposed fix: netscript generate (or the plugin) should discover consumer-defined jobs and include them in the plugin-service registry, so runtime-known and API-triggerable stay in sync automatically.

Gap 4 — a Service can't declare PluginReferences in config

Apps and BackgroundProcessors can declare ServiceReferences/PluginReferences in appsettings.json (auto-wired to services__{name}__http__0). A Service (e.g. eischat) has no such field — so wiring eischat → workers-api needed a manual cross-map hand-edit in the generated register-background.mts (which has both maps). Same for wiring workers → eischat for the single-writer callback.

• Workaround: hand-edited register-background.mts (kept-on-regen), commits aa0da26 (eischat→workers-api) + the single-writer wire.
• Proposed fix: allow Services.{name}.PluginReferences/ServiceReferences in config + codegen wiring. (Overlaps docs/DX: TypeScript AppHost model — restart semantics, hand-edit-vs-regen, service→plugin references, native-driver permissions #264 item 4.)

Gap 5 — per-channel tursodb single-file isn't multi-process; and handles are leaked

eis-chat uses one tursodb per channel. The embed-document worker (separate process) opening the same channel db that eischat holds → Locking error … os error 33 (OS exclusive lock; @tursodatabase/database connect() exposes only readonly/timeout/fileMustExist, no shared mode). Root cause of retention: eischat opened a channel db per request and never close()d the native handle, holding the lock indefinitely.

• Workaround: eis-chat single-writer redesign — the worker is compute-only and hands results back to eischat (the sole db writer, which now close()s in finally) — PR fix(ci): green main-red — stale nativeRetrial assertion + triggers fmt drift #82. Also: channelDbPath() had to resolve absolute (not cwd-relative) so the two processes agreed on the path — commit c739b79; and plugin-workers needed --allow-ffi for the native tursodb driver (relates to Aspire-run services crash for native-addon DB drivers — need --allow-ffi (sqlite/libsql, tursodb) #174) — commit 6fa91d7.
• Proposed fix: document the single-writer constraint + the "service owns its db; other processes call back" pattern; ensure plugin/service DB handles are closed (or pooled) so they don't hold locks; and grant --allow-ffi to plugin services using native DB drivers by default.

---

All eis-chat references are on rickylabs/eis-chat master. Happy to expand any of these into a focused PR against NetScript if useful — the eis-chat fixes are the reference implementations of the workarounds. Posted by the eis-chat coordinator agent.

[rickylabs]

#279 resolved — closed by PR #294

Trigger routing for triggerJob / triggerTask now derives the id from the {id} path segment as the single source of truth, and fails loudly with a VALIDATION_ERROR (via the new centralized validationFailed contract helper) before any KV write when the id is absent. The RPC transport is also covered by a new reachability guard test that stands up a real createService(router).withRPC(...).serve() and calls it through a generated createServiceClient — proving no 404 on the typed path.

Framework capability now available

• netscript/plugin-workers trigger procedures accept the id from the URL path; the request-body id is now optional and, if sent, is ignored in favor of the path (backward compatible — existing callers that also send a body id keep working).
• Missing id short-circuits to a typed VALIDATION_ERROR instead of silently writing an undefined-keyed job/task.
• Generated typed service client (createServiceClient) reaches triggerJob/triggerTask over /api/rpc/* with no transport 404.

Removable eis-chat workarounds

• PR prime-time: service-auth-adapters (WorkOS + better-auth) #83 — the { id: jobId } body-send in services/eischat/src/jobs.ts (and the matching task path). Delete the body id; pass it only in the URL path.
• Gap 1 (RPC 404): closed on main by Re-architect plugin scaffold surface (#157): thin, typesafe, no plugin-source copy #172 — createPluginService mounts withRPC and wireRpc serves both /api/rpc/* and /api/*. The eis-chat manual REST fallback for triggers can be dropped in favor of the typed client.
• Gap 3 (jobs not auto-registered): closed by workers: service->plugin-API discovery + workers-api not loading user jobs (Aspire wiring gaps) #217/fix(workers): wire Service→plugin-API discovery + load user jobs in workers-api (#217) #220 — registerGeneratedJobDefinitions + compileWorkersRegistry discovery. The hand-maintained job-definitions.ts registration list from eis-chat PR [prime-time] Worker applied-keys dedup #79 is removable.
• Gap 4 (Service→plugin-API discovery): closed by fix(workers): wire Service→plugin-API discovery + load user jobs in workers-api (#217) #220 — pluginReferences + dependsOn on the service config schema. The hand-edited register-background.mts wiring is removable.
• Gap 5: --allow-ffi is present in WORKERS_BACKGROUND_PERMISSIONS. (The single-writer/tursodb-handle hygiene remains a consumer-side + docs concern, out of scope for this code fix.)