Declarative Extension (Middleware/Hook) Mounting for Managed Agents

[shushuiu]

Type: Feature / Discussion
Component: agentscope-service (control plane + dataplane), agentscope-harness (middleware contract)
Versions affected: v2.1.0-BETA1 (2026-10-08), v2.0.4
Author: Zhao Li, STO Express AI Operations Platform Team
Status: Draft for discussion

---

1. Summary

Add a named, tenant-scoped extension mounting mechanism to the Managed Agent definition and to
session agentOverrides, so that business teams can attach their own middleware to platform-managed
agents by writing configuration, not by changing the host application.

We propose generalizing the mechanism the platform already uses for JEV — overrides → List<MiddlewareBase>
at build time — into a public SPI, instead of inventing a new concept.

2. Motivation / Problem

We are operating an internal Agent platform on AgentScope Java 2.x (HarnessAgent, Redis AgentStateStore,
sandbox filesystem, custom middlewares + toolkit injected from application code). The Service work in
2.0.4 / 2.1.0-BETA1 lands exactly where we need it to go: one shared, self-hosted control plane where
business teams create and tune their own agents.

The remaining blocker is the extension point story. Our roadmap requires every business line to own its
cross-cutting behaviour — model-routing policy, tool-result budget and redaction, per-scene context injection,
guardrail validation, latency budget enforcement — without any of them landing code in the host application.
Today that is not possible for a Managed agent.

What we verified on main (2026-10-16)

1. The managed agent definition has no middleware/hook field. Documented fields in
   docs/v2/zh/service/managed-agent-configuration.md are: name, description, system, model, maxIters,
   workspaceId, workspaceBinding{version, overrides, instructions}, workspacePath, defaultEnvironmentId,
   defaultMemoryStoreIds, defaultVaultIds, tools, mcpServers, skills, multiagent, version.
   The corresponding records in
   agentscope-service/service-common/.../catalog/spec/AgentSpecTypes.java
   (PermissionPolicy, ToolDefaultConfig, ToolConfigEntry, AgentToolset, McpServerSpec, SkillRef,
   MultiagentAgentRef, MultiagentSpec, ModelConfig) likewise contain no extension type.

2. The middleware chain of a managed agent is fixed at build time by the platform, with exactly one
   override-driven contributor. In
   agentscope-service/service-dataplane/.../catalog/HarnessAgentBuildService.java (~L488):

   b.middleware(new ToolNotificationMiddleware(toolEventBus));
   b.middleware(toolConfirmationMiddleware);
   if (jevServiceSupport != null) {
       jevServiceSupport.middlewares(spec.overridesJson()).forEach(b::middleware);
       jevServiceSupport.compaction(spec.overridesJson(), ...).ifPresent(b::compaction);
   }

   JevServiceSupport.middlewares(String overrides) returns List<MiddlewareBase> and internally fans out
   per purpose (addSupervision, addEvaluation, addPhaseRouting, …). This is already the right shape —
   it is just hard-wired to one consumer. A business team today can neither add an entry here nor replace it
   without forking the dataplane.

3. DataSessionService.mergeAgentOverrides(...) currently logs
   "mergeAgentOverrides is not persisted via control plane yet", so even the JEV-style per-session override
   path is not fully wired for reuse.

What already exists (and is not enough on its own)

• Middleware contract is strong. Six extension points (onAgent, onReasoning, onActing, onModelCall,
  onSystemPrompt, onAgentStateReady — see docs/v2/zh/docs/others/release-notes.md), MiddlewareBase.order()
  for deterministic ordering (feat(core): add middleware execution ordering #2532), activePoints() for declarative participation (feat(middleware): add activePoints() extension-point participation declaration #3371, merged 2026-09-30).
• PR #2488 adds AgentBuilderCustomizer SPI plus
  auto-assembly of MiddlewareBase / PermissionContextState / Hook beans. Its own description states the
  problem precisely: "Everything else — middleware, permission context, hooks — must be wired by hand, and a
  third-party extension that wants to add one of these has no option but to replace the whole ReActAgent bean."
  This solves the SDK path. It does not solve the Service/Managed path, because in a shared multi-tenant
  dataplane, "every MiddlewareBase bean in the JVM is attached to every agent" is the wrong semantics: it is
  global, not tenant-scoped, not agent-scoped, and cannot be enabled per definition version.
• External Agent (agentscope-extensions-controlplane, ControlPlane.instrument(...), extension points
  SessionHistorySource / AgentRuntimeSource / AgentTaskStarter) keeps full control in the business process,
  including its own framework hooks (attach / detach). This works, but it forces every business line to run
  and operate its own agent process — which cancels the "shared self-hosted platform" value we are trying to adopt,
  and the docs also note the registration endpoint does not yet verify caller identity, so it must sit behind a
  controlled network.
• docs/v2/zh/docs/others/going-to-production.md tells users to write their own MiddlewareBase for rate
  limiting. Correct for SDK users; for platform-managed agents there is nowhere to put it.

3. Goals / Non-goals

Goals

• G1: A business team can attach its own middleware to its own managed agents by editing agent definition +
  session override only; zero changes in the host/dataplane application.
• G2: Platform-mandatory middleware (tracing, tool confirmation, budget/quota, redaction) cannot be dropped,
  reordered below business extensions, or bypassed by a definition.
• G3: Failures in a business extension degrade the run, they do not take down the dataplane.
• G4: Backward compatible: default behaviour of existing deployments is byte-for-byte unchanged.

Non-goals (this RFC)

• In-process scripting hooks (Lua/JS/CEL evaluated inside the JVM). We believe this is the natural next step, but
  it needs its own sandboxing design; see §8.
• Any change to JEV semantics; JEV should become one implementation of the proposed SPI.
• Substituting tool-level extension (@Tool / MCP) — orthogonal and already sufficient for capability, not policy.

4. Proposal

4.1 Definition surface (additive)

// POST /api/v1/agents  → definition
{
  "name": "dispatch-planner",
  "system": "...",
  "model": "dashscope:qwen-max",
  "extensions": [
    {
      "provider": "sto-model-router",
      "version": "3",
      "mode": "ENFORCE",
      "order": 2000,
      "config": { "prefer": "local-small", "escalateAtContextRatio": 0.8 },
      "limits": { "timeoutMs": 50, "failureMode": "PASS_THROUGH" }
    },
    {
      "provider": "sto-tool-result-redactor",
      "mode": "SHADOW",
      "order": 3000,
      "points": ["onActing", "onModelCall"]
    }
  ]
}

// POST/PATCH /api/sessions → per-session narrowing (same shape as agentOverrides.jev)
{ "agentOverrides": { "extensions": { "sto-model-router": { "mode": "OFF" } } } }

Rules we assume: unknown provider fails at definition write time, not at runtime; mode semantics reuse
OFF | SHADOW | ENFORCE (already established by JEV); points maps onto activePoints(); a session may only
narrow what the definition allows, never widen.

4.2 Dataplane SPI (generalizes what JEV already does)

public interface ManagedExtensionProvider {
    /** Stable, tenant-addressable name used in `extensions[].provider`. */
    String name();

    /** Extension contract version, for compatibility gating (§4.5). */
    int apiVersion();

    /** Build middleware for one agent run. Return empty to be a no-op. */
    List<MiddlewareBase> middlewares(ExtensionContext ctx);

    default Optional<CompactionSpec> compaction(ExtensionContext ctx) { return Optional.empty(); }
}

public record ExtensionContext(
        String tenant, String namespace, String agentKey,
        int definitionVersion, String sessionId, String ownerId,
        JsonNode config, JsonNode sessionOverride,
        ModelAllowList modelAllowList, Budget budget, WorkspaceRef workspace) {}

HarnessAgentBuildService changes from "one hard-coded JEV call" to a registry loop, keeping the mandatory chain
untouchable:

// mandatory: platform-owned, outermost, never removable
b.middleware(tracingMiddleware);            // highest order() band
b.middleware(new ToolNotificationMiddleware(toolEventBus));
b.middleware(toolConfirmationMiddleware);

for (ExtensionSpec ext : spec.extensions()) {
    ManagedExtensionProvider p = registry.resolveRequired(ext.provider(), ctx);   // fail fast at build
    p.middlewares(ctx.with(ext)).forEach(b::middleware);
}

b.middleware(budgetGuardMiddleware);        // lowest order() band, closest to the loop

JevServiceSupport is refactored behind the same interface with name() = "jev" and
config = agentOverrides.jev, so existing deployments keep working and the new path is proven by an existing
consumer rather than by a hypothetical one. Discovery via Spring ObjectProvider<ManagedExtensionProvider> is
consistent with the direction of #2488.

4.3 Ordering bands (uses existing MiddlewareBase.order())

Band	Order	Owner	Purpose
Platform-mandatory	10000+	Service	tracing, tenant guard, tool confirmation, SSE/protocol
Business extension	1000–9999	Business team	routing, redaction, budget, policy validation
Platform-fallback	1–999	Service	budget/quota enforcement, audit sink closest to the loop

Requested addition: reject, at build(), any agent definition whose business extension sits outside
1000–9999, and expose an assertion that the mandatory set is present (build() throws if a business extension
removed it). This makes G2 enforceable rather than documentary.

4.4 Failure and isolation semantics (G3)

• Per-extension limits.timeoutMs is required; a business extension cannot opt out of a deadline.
• failureMode: PASS_THROUGH (default) | REJECT (fail the turn) | FALLBACK (use cached/last-known config).
  Timeout always resolves to PASS_THROUGH for observation-only modes (SHADOW), never blocks the run.
• Per-agent concurrency and memory quota; extension exceptions are caught, attributed to
  {agentKey, provider, definitionVersion} and surfaced as metrics — mirroring the existing "bounded queue never
  blocks the agent" stance of jev.decision.
• An extension that exceeds its error budget is auto-quarantined for that agent and reported, without affecting
  other tenants.

4.5 Contract evolution and governance

• apiVersion on the provider, declared in the definition; the platform refuses to build an agent whose required
  version it no longer serves, and reports it to the owner — the blast radius is one agent, not the dataplane.
• Definition + extensions list is included in the session's fixed definition snapshot (already the behaviour:
  "创建会话时 Service 会保存当时的配置… 后续 Turn 沿用"), so replay and audit see the same extension set.
• Emit extensions.decision / extensions.error events on the same session event stream used by
  types=jev.decision, so consoles and evaluation pipelines get one place to look.
• Server-side allow-list for providers per tenant, symmetric to the existing
  agentscope.jev.allowed-models: an unapproved provider must not be attachable by definition edit alone.

4.6 Phase 2 (explicitly separable)

Remote extensions: the platform invokes a business-owned HTTP/gRPC endpoint at a declared point
({phase, ctx, payload} → {action: PASS|REPLACE|SHORT_CIRCUIT|REJECT, payload}), so non-Java teams participate
without contributing a JVM bean. Same ManagedExtensionProvider surface, with a
RemoteExtensionProvider adapter. We deliberately do not ask for this in phase 1; the contract above is
designed so it can be added without changing definitions.

5. Alternatives considered

Alternative	Why insufficient
Everything as External Agent	Keeps the hook, but re-imposes per-team deployment/ops; contradicts the shared-service model; registration trust is still maturing.
Tools / MCP only	Good for capability, wrong shape for cross-cutting policy: cannot rewrite the system prompt, throttle a model call, or bound tool-result size.
Wait for #2488 and add per-team Spring profiles to the host app	Global JVM-wide bean semantics, no tenant/agent scoping, and reintroduces "everyone merges code into one host app" — the exact cost we are trying to remove.
Enforce policy outside the agent loop (Workflow/gateway)	Loses per-turn context (tool results, reasoning state) that these policies need; also cannot cover onSystemPrompt/onModelCall.

6. Compatibility

• Fully additive: no field changes meaning, no default behaviour change; extensions absent ⇒ today's chain.
• activePoints() (feat(middleware): add activePoints() extension-point participation declaration #3371) is used rather than duplicated, so unused points cost no wrapper layer.
• JEV remains the reference implementation; we volunteer to port JevServiceSupport onto the SPI as the first PR
  if the shape is acceptable.
• agentOverrides handling should be persisted via the control plane before per-session extension narrowing ships
  (see the current mergeAgentOverrides stub).

7. Milestone & acceptance

Phase 1 (suggested to ride the 2.1 GA train, small diff):
ManagedExtensionProvider SPI + registry + extensions field in AgentDefinition/AgentSpecTypes/AgentSpecCodec

• order-band validation + timeout/failureMode enforcement + JEV refactor behind the SPI + control-plane validation
  of unknown providers at write time.

Acceptance:

1. Existing e2e suites unchanged; a golden-definition fixture asserts the mandatory chain is present and ordered.
2. A no-op test provider proves: attach, narrow per session, OFF, timeout ⇒ PASS_THROUGH, quarantine.
3. A definition with an unregistered provider is rejected at write time with a typed error.
4. Removing/reordering mandatory middleware from a business extension fails at build() with a clear message.
5. Events visible on the session event stream, attributed per {agentKey, provider, version}.

8. Open questions

1. Do you want a generic extension registry in agentscope-service, or is the intended direction "managed agents
   stay policy-free and teams use External Agents for anything needing hooks"? A clear answer here is enough for us
   to plan the next two quarters.
2. Is per-session narrowing of extensions in scope, or definition-level only for v1?
3. Should mode: SHADOW be a platform-level convention shared with JEV, or per-provider?
4. Interest in phase 2 remote hooks as an RFC of its own? Related demand we see: [Feature]: 什么时候支持从Nacos 动态读取Mcp #3107 (dynamic MCP from Nacos),
   [Bug]:skill热加载问题 #2419/[Bug]:skill热加载问题 #2420 (skill hot reload), A2aAgent 不支持 Middleware 机制, 仅能通过 deprecated Hook API 拦截事件流 — 请求支持 Middleware #2612 (A2aAgent lacks middleware support) — all three are "change capability
   without redeploying the host" in different disguises.
5. Longer term: would a sandboxed script hook (bounded JS/Lua expression evaluated against a typed context) be
   accepted upstream, or is that out of scope for the project's philosophy of "file-as-configuration + typed Java
   contracts"?

9. Prior art referenced

docs/v2/zh/service/managed-agent-configuration.md, docs/v2/zh/service/external-agent.md,
docs/v2/zh/service/index.md, docs/v2/zh/jev/guides/service-api.md,
docs/v2/zh/docs/others/release-notes.md, docs/v2/zh/docs/others/going-to-production.md;
HarnessAgentBuildService.java, JevServiceSupport.java, AgentSpecTypes.java, DataSessionService.java;
PRs #2488,
#3371,
#2532,
#3080;
issues #3107,
#2612,
#2419.