feat(appkit): supervisor API adapter for agents plugin

[hubertzub-db]

Adds a fourth AgentAdapter that targets the Databricks AI Gateway Responses API, so AppKit apps can host server-side-orchestrated agents (Genie, Knowledge Assistants, UC functions, custom apps, UC connections) without managing tool execution locally.

import { fromSupervisorApi, supervisorTools } from "@databricks/appkit/agents/supervisor-api";

const supervisor = createAgent({
  instructions: "You are an assistant powered by the Databricks Supervisor API.",
  model: await fromSupervisorApi({
    model: "databricks-claude-sonnet-4-5",
    tools: [
      supervisorTools.genieSpace("01ABCDEF12345678", "NYC taxi trip records and zones"),
      supervisorTools.ucFunction("main.default.add", "Adds two integers and returns the sum."),
    ],
  }),
});

What's new

• fromSupervisorApi({...}) — factory that returns an AgentAdapter. Uses the SDK's default credential chain (env, profiles, OAuth, OIDC) just like DatabricksAdapter.fromModelServing.
• supervisorTools.* — concise factories for the five hosted tool types (genieSpace, ucFunction, knowledgeAssistant, app, ucConnection). description is required on every one — it's both validation and the routing hint the LLM uses to choose between tools.
• @databricks/appkit/agents/supervisor-api — new subpath export so consumers pick only the adapter they want; mirrors the existing ./agents/{databricks,vercel-ai,langchain} pattern.

Reference app

Demo supervisor agent in apps/dev-playground/server/index.ts. Tools are commented out by default — uncomment any supervisorTools.* entry to give the model real powers.

Test plan

• Manual tests
• 39 new tests (13 SSE reader + 26 adapter / factory).
• Full appkit vitest suite passes; typecheck clean.

[MarioCadenas]

Structural feedback on the supervisor API adapter

First off — the substance of the adapter (recovery paths, terminal-event handling, structured tests) is solid. The feedback below is mostly about the public API shape and a few correctness / security items. The implementation can largely stay as-is; the changes are concentrated at the boundary between the adapter, createAgent, and the agents plugin.

This is agent/v2/sa/1-adapter — pre-release — so the API-shape changes are cheap now and expensive once users learn the current form. I'd push to land them in this PR rather than as a follow-up.

---

1. Discoverability — make fromSupervisorApi a static on DatabricksAdapter

Today users would import fromSupervisorApi from @databricks/appkit/beta and DatabricksAdapter.fromModelServing from the same place. Same vendor, two import shapes.

Proposal: keep SupervisorApiAdapter as its own class in its own file (no implementation merge), but expose the factory as a static on DatabricksAdapter that delegates to the free function.

// packages/appkit/src/agents/databricks.ts
export class DatabricksAdapter implements AgentAdapter {
  // ...existing chat-completions implementation, unchanged...

  static async fromServingEndpoint(opts: ServingEndpointOptions): Promise<DatabricksAdapter> {
    /* unchanged */
  }

  static async fromModelServing(name?: string, opts?: ModelServingOptions): Promise<DatabricksAdapter> {
    /* unchanged */
  }

  /**
   * Builds an adapter that runs the agent loop server-side on the
   * Databricks AI Gateway Responses API. Returns a {@link SupervisorApiAdapter}.
   */
  static async fromSupervisorApi(
    opts: SupervisorApiAdapterOptions,
  ): Promise<SupervisorApiAdapter> {
    // Dynamic import so databricks.ts ↔ supervisor-api.ts don't form a
    // load-time cycle (both share connectors/serving/client.ts).
    const { fromSupervisorApi } = await import("./supervisor-api");
    return fromSupervisorApi(opts);
  }
}

Users get a single discovery root (DatabricksAdapter. autocomplete shows three factories side by side). Implementations stay split.

Keep the free fromSupervisorApi export in beta.ts for users who prefer the bare function — it's the same code path. No breakage.

---

2. Tools — move them off the adapter constructor onto createAgent

This is the biggest structural change and the one I'd push hardest on.

Today

fromSupervisorApi({
  model: "databricks-claude-sonnet-4-5",
  tools: [supervisorTools.genieSpace("01ABC", "NYC taxi data")], // on the adapter
});

Target

createAgent({
  model: DatabricksAdapter.fromSupervisorApi({ model: "databricks-claude-sonnet-4-5" }),
  tools: () => ({
    nyc: supervisorTools.genieSpace({ id: "01ABC", description: "NYC taxi data" }),
  }),
});

Why

Every other adapter takes tools via createAgent({ tools }). Today supervisor agents have a separate mental model — tools go on the adapter constructor, not on the agent. One tool-declaration site removes that asymmetry.

This isn't free: SA hosted tools aren't callable functions (no JSON schema, no executor callback), so they can't pretend to be AgentToolDefinitions the way MCP hosted tools can. The path through to the adapter needs a small side channel.

Mechanism

2a. New extensions map on AgentInput (adapter-agnostic)

// packages/shared/src/agent.ts
export interface AgentInput {
  messages: Message[];
  tools: AgentToolDefinition[];
  threadId: string;
  signal?: AbortSignal;
  /**
   * Adapter-specific opaque payloads, keyed by adapter namespace.
   * The shared contract intentionally does not enumerate keys — see each
   * adapter's docs for what it reads.
   */
  extensions?: Readonly<Record<string, unknown>>;
}

The shared contract gains one optional field. SupervisorTool wire format stays in the adapter file, not in shared.

2b. Adapter declares the extension key + helper

// packages/appkit/src/agents/supervisor-api.ts
export const SUPERVISOR_EXTENSION_KEY = "databricks.supervisor" as const;

export interface SupervisorExtension {
  hostedTools?: SupervisorTool[];
}

function readSupervisorExtension(input: AgentInput): SupervisorExtension {
  const raw = input.extensions?.[SUPERVISOR_EXTENSION_KEY];
  if (!raw || typeof raw !== "object") return {};
  return raw as SupervisorExtension; // single cast at the boundary
}

Inside the adapter, this.tools goes away; the adapter reads from the extension on every run(). Constructor surface shrinks to { model, workspaceClient? }.

2c. Tool record returns a tagged record (not wire format)

// packages/appkit/src/agents/supervisor-api.ts
export interface HostedSupervisorTool {
  readonly __kind: "hosted-supervisor";
  readonly spec: SupervisorTool;
}

export const supervisorTools = {
  genieSpace: ({ id, description }: { id: string; description: string }): HostedSupervisorTool => ({
    __kind: "hosted-supervisor",
    spec: { type: "genie_space", genie_space: { id, description } },
  }),
  ucFunction: ({ name, description }: { name: string; description: string }): HostedSupervisorTool => ({
    __kind: "hosted-supervisor",
    spec: { type: "uc_function", uc_function: { name, description } },
  }),
  knowledgeAssistant: ({ knowledgeAssistantId, description }: { knowledgeAssistantId: string; description: string }): HostedSupervisorTool => ({
    __kind: "hosted-supervisor",
    spec: { type: "knowledge_assistant", knowledge_assistant: { knowledge_assistant_id: knowledgeAssistantId, description } },
  }),
  app: ({ name, description }: { name: string; description: string }): HostedSupervisorTool => ({
    __kind: "hosted-supervisor",
    spec: { type: "app", app: { name, description } },
  }),
  ucConnection: ({ name, description }: { name: string; description: string }): HostedSupervisorTool => ({
    __kind: "hosted-supervisor",
    spec: { type: "uc_connection", uc_connection: { name, description } },
  }),
};

export function isSupervisorTool(value: unknown): value is HostedSupervisorTool {
  return typeof value === "object"
    && value !== null
    && (value as Record<string, unknown>).__kind === "hosted-supervisor";
}

Two changes vs current PR:

• Each factory takes a single named options object (instead of two positional strings). Routing-critical strings deserve labels at the call site; description is the only signal the LLM uses to disambiguate two Genie spaces — two adjacent positional strings is the kind of pattern that produces "we swapped the args and didn't notice for two weeks" bugs.
• Return value is a tagged record, not raw wire format. Single-field classifier, no namespace overlap with MCP's isHostedTool (which uses type: "genie-space" hyphenated for its own Genie tool). Future-proofs against SA wire-format evolution.

Note: This leaves the MCP factories (mcpServer, etc.) on the old pattern. A follow-up PR could migrate them to __kind: "hosted-mcp" for codebase uniformity; out of scope here.

2d. Agents plugin classifies and routes

In packages/appkit/src/plugins/agents/agents.ts buildToolIndex (around L539), add a branch alongside isFunctionTool / isHostedTool:

if (isSupervisorTool(tool)) {
  index.set(key, { source: "hosted-supervisor", spec: tool.spec, def: /* synthetic def */ });
  continue;
}

When invoking the adapter (around L1044), aggregate hosted-supervisor entries into the extensions payload:

const supervisorSpecs = [...index.values()]
  .filter((e): e is HostedSupervisorEntry => e.source === "hosted-supervisor")
  .map((e) => e.spec);

const extensions: Record<string, unknown> = {};
if (supervisorSpecs.length > 0) {
  extensions[SUPERVISOR_EXTENSION_KEY] = { hostedTools: supervisorSpecs };
}

const stream = registered.adapter.run(
  { messages: messagesWithSystem, tools, threadId: thread.id, signal, extensions },
  { executeTool, signal },
);

Same logic in runAgent for standalone mode (see point 4).

---

3. Capability negotiation — two optional adapter fields

The mismatch-detection problem cuts both ways and needs symmetric treatment.

3a. acceptsExtensions — for "supervisor tool on a non-supervisor adapter"

// packages/shared/src/agent.ts
export interface AgentAdapter {
  run(input: AgentInput, ctx: AgentRunContext): AsyncGenerator<AgentEvent, void, unknown>;

  /**
   * Extension keys this adapter consumes from {@link AgentInput.extensions}.
   * The agents plugin warns at registration if the tool index produces
   * extensions whose keys aren't listed here.
   */
  readonly acceptsExtensions?: readonly string[];

  /**
   * Whether the adapter consumes tools from `input.tools`. Defaults to true.
   * Adapters whose tool execution happens elsewhere (e.g. Supervisor API,
   * where SA owns the tool loop) declare false; the agents plugin warns at
   * registration if the agent declares function tools or local sub-agents.
   */
  readonly consumesInputTools?: boolean;
}

class SupervisorApiAdapter implements AgentAdapter {
  readonly acceptsExtensions = [SUPERVISOR_EXTENSION_KEY] as const;
  readonly consumesInputTools = false;
  // ...
}

3b. Warning behaviour

At agent registration, the agents plugin (and runAgent) checks both directions and logs a warning — does not throw. Sample message:

Agent 'analytics' declares tools requiring extension 'databricks.supervisor'
(nyc_taxi, add_fn) but its model adapter does not accept it. Pair these
tools with DatabricksAdapter.fromSupervisorApi({ model: ... }), or remove
them.

Agent 'analytics' declares function tools / sub-agents (summarize, agent-foo)
but its model adapter (Supervisor API) does not consume input.tools. These
tools will not be exposed to the model. See docs/plugins/agents.md for
details.

Warn-not-throw was a deliberate choice: it keeps the app starting, and the warning is loud enough that anyone watching logs will notice. Throw would be defensible too — tell me if you'd prefer fail-fast.

---

4. Adapter constructor surface shrinks

After (2) and (3), the factory signature is:

fromSupervisorApi({
  model: string;
  workspaceClient?: WorkspaceClientLike;
}): Promise<SupervisorApiAdapter>;

tools is removed entirely from the factory. The B2b/G2 detection mechanism only works if the tool index is the single source of truth — factory-supplied tools bypass it and create a validation hole. Since the PR is pre-release, there's no compatibility to break.

Users calling adapter.run(...) directly outside createAgent/runAgent (rare) populate input.extensions[SUPERVISOR_EXTENSION_KEY] = { hostedTools: [...] } themselves. The key is exported.

---

5. Standalone runAgent supports hosted-supervisor natively

MCP rejects in standalone because it needs a live MCP client. Supervisor doesn't — the adapter has everything it needs (workspace client + extension payload). Rejecting "to match precedent" would be cargo-culting; the precedent's reason doesn't apply.

In packages/appkit/src/core/agent/run-agent.ts buildStandaloneToolIndex / classifyTool, add a hosted-supervisor branch:

if (isSupervisorTool(tool)) {
  return { kind: "hosted-supervisor", spec: tool.spec, def: /* synthetic */ };
}
if (isHostedTool(tool)) {
  throw new Error(`runAgent: tool "${key}" is a hosted tool (type="${tool.type}") which is only supported via createApp({ plugins: [..., agents(...)] }). Standalone runAgent has no MCP client.`);
}

In runAgentInternal, populate input.extensions the same way the agents plugin does. executeTool errors if it's ever invoked on a hosted-supervisor entry (defense-in-depth — should never fire since execution is server-side).

This enables batch-eval / CI use of supervisor agents — explicitly cited as a use case for standalone in the PR description.

---

6. Markdown agents — code-declared, frontmatter-referenced

No new frontmatter syntax. Supervisor tools are declared in code (via agents({ tools: { ... } })) and referenced by name in markdown frontmatter:

// code
agents({
  agents: { ... },
  tools: {
    nyc_taxi: supervisorTools.genieSpace({ id: "01ABC", description: "NYC taxi" }),
  },
});

# markdown frontmatter
endpoint: supervisor
tools:
  - nyc_taxi

This composes for free with the existing availableTools resolution in load-agents.ts:404 once supervisorTools.* returns the tagged record. Frontmatter stays portable; YAML-position validation of required description strings is avoided.

---

7. Cross-adapter sub-agent composition

• Chat-completions parent → supervisor sub-agent: works for free under the design. Each agent has its own adapter + tool index + extensions; the parent dispatches to the child via agent-{key} as a regular function tool, and the child runs entirely on SA.
• Supervisor parent → function tools / local sub-agents: detected by consumesInputTools: false, warned at registration (point 3b). Documents existing v1 limit. Future PR can wire SA's response.function_call events through context.executeTool to lift the restriction.

Worth a one-liner in docs/plugins/agents.md documenting which direction works.

---

8. End-to-end after the refactor

User code:

// apps/dev-playground/server/index.ts
import { createAgent, createApp } from "@databricks/appkit";
import { agents, DatabricksAdapter, supervisorTools } from "@databricks/appkit/beta";

const supervisor = createAgent({
  instructions: "You are an assistant powered by the Databricks Supervisor API.",
  model: DatabricksAdapter.fromSupervisorApi({ model: "databricks-claude-sonnet-4-5" }),
  tools: () => ({
    // nyc: supervisorTools.genieSpace({ id: "01ABC", description: "NYC taxi data" }),
    // add: supervisorTools.ucFunction({ name: "main.default.add", description: "Adds two integers" }),
  }),
});

await createApp({ plugins: [agents({ agents: { supervisor } })] });

What this buys:

• One mental model for "where do tools live": always on the agent, never on the adapter.
• Discoverability via DatabricksAdapter.from*.
• Standalone runAgent works with supervisor agents.
• Cross-adapter sub-agent composition (one direction).
• Capability-negotiated mismatch warnings, no concrete-class coupling in the agents plugin.

What this costs:

• Two optional fields on AgentAdapter (acceptsExtensions, consumesInputTools). Both opt-in.
• One optional field on AgentInput (extensions). Opt-in.
• One new tool classification kind (hosted-supervisor) in buildToolIndex and buildStandaloneToolIndex.

---

9. Independent of the above — review findings to address in this PR

These don't depend on the structural refactor. They came out of the review of the current code and should be fixed regardless of whether the API redesign lands.

High

• Verbatim upstream error text leaks to clients via three SSE error paths in supervisor-api.ts: catch on streamBody (~L416), mapEvent("error") (~L641), and output_item.done with id === "error" (~L658). CWE-209. Recurring pattern across PRs fix: unblock SP calls for UC in production #310, feat(files): per-volume in-app policy enforcement #197, fix: log execution errors server-side (stream, Lakebase, SQL Warehouse) #255, feat: decode inline Arrow IPC + warehouse-compat fallback #329 v1/v2/v3. Route the three through one helper that returns a stable code to clients and logs detail server-side.

Medium

• response.completed with nested status: "failed" still yields complete (supervisor-api.ts:522). The final complete is gated on eventCounts.has("response.completed") only; doesn't check lastCompleted.status or lastCompleted.error. Silent success-on-failure. Add a regression test.
• Mid-flight aborts surface as terminal error (supervisor-api.ts:413-425). Pre-call and post-loop branches return silently on signal.aborted; the during-streamBody window yields status: error. Add if (signal?.aborted) return; to the catch + regression test.
• output_item.done id === "error" is a literal-string magic-match (supervisor-api.ts:658). 5-char collision possibility on an SA-controlled identifier. Tighten to require item.type === "error" (or pair with response.failed), or comment with the SA contract that guarantees the reserved id.
• readSseEvents has no max-buffer / max-line cap (stream/sse-reader.ts). CWE-770. DatabricksAdapter's inline parser has these caps (maxSseLineChars, maxStreamTextChars, maxToolArgumentsChars); the new shared parser does not. Add them as options (with sane defaults) before either adapter relies on this in production.
• Hosted-tool description is a prompt-injection vector and docs don't warn (docs/plugins/agents.md). CWE-1427. Add a callout: "do not derive description from untrusted input".
• No-delta logger.warn JSON-stringifies lastCompleted.error / incomplete_details (supervisor-api.ts:530-543). CWE-532. Redact or summarise; don't dump raw upstream payload to logs.
• buffer.replace(/\r\n/g, "\n") on every chunk in readSseEvents (stream/sse-reader.ts:~49). Gate with buffer.indexOf("\r") !== -1 to skip on LF-only steady state.

Low

• mapEvent("error") defaults "Unknown error" then JSON.stringifys it, surfacing '"Unknown error"' with literal quotes (supervisor-api.ts:642-647). Branch on typeof data.error === "string" first.
• dev-playground comment claims await at module init but code doesn't await (apps/dev-playground/server/index.ts:28-35). Resolves fine at first use (resolveAdapter awaits lazily), but the "fails fast at startup" rationale is false. Either actually await or drop the comment. PR description has the same drift.
• JSDoc @example imports from @databricks/appkit/agents/supervisor-api but the public re-export is @databricks/appkit/beta (supervisor-api.ts:335, 704). Copy-pasted examples won't resolve.
• sse-reader uses trimStart() where the SSE spec says strip a single U+0020. Invisible while consumers feed JSON, footgun otherwise.
• Per-line \r$ strip is dead code after the buffer-level CRLF normaliser in sse-reader. Pick one normalisation site.
• streamPath accepts arbitrary workspace paths with no allowlist. CWE-918 (defense-in-depth). Currently internal-only, correctly not re-exported in beta.ts. Flag @internal so a future caller doesn't turn it into workspace-credentialled SSRF.
• workspaceClient captured at construction (supervisor-api.ts:298-300, 720-736). CWE-664. Passing a per-request OBO client would silently leak identity. Add a JSDoc warning.

---

10. Suggested follow-ups (not blocking this PR)

• Migrate MCP classification from isHostedTool-by-wire-discriminator to __kind-tagged record for codebase uniformity.
• Port DatabricksAdapter onto readSseEvents, lifting its DoS caps into the shared parser's options. ~80 lines come out of databricks.ts; both adapters benefit from the same protections.
• Wire SA's response.function_call events through context.executeTool to lift the "supervisor parent can't have local sub-agents" limitation. Flip consumesInputTools back to true once done.
• Rename internal streamResponse → decodeResponsesApiStream (or co-locate with mapEvent under a "// --- protocol decoder ---" boundary). Cosmetic, but clarifies the transport/decoder split that prompted some of the design questions.

---

Happy to talk through any of these on a call — most of the structural items are connected (the tools refactor pulls in capability negotiation, which pulls in the runAgent path, etc.) so it might be faster to align on the direction than to discuss each in isolation.