Tool surfaces need policy and provenance, not a hard-coded list (data + a proposal for dsh-superagent)

[kmyy350234-dz]

Tool surfaces need policy and provenance, not a hard-coded list

We build a small DSH plugin that hides unused tool groups (lazy-tools). While studying
dsh-superagent-prototype we measured a few things about tool surfaces and ran an
experiment. This post shares the measurements, a proposal, and the traps we hit.
Claims are tagged [measured], [source-read] or [inferred]; anything we did
not verify says so.

1. A full profile, measured through our own layer

On one Windows machine (DSH 2.0.21-next desktop, a profile with about 20 bundles),
using our own bench — that is, the plugin being argued for, so the "after" is a layer,
not a neutral baseline:

• 163 registered tools, 89 visible, tool schemas 190,902 chars total, 53,340 after
  [measured — one profile, one machine; per-layer attribution of the 74 names not
  visible is not shown here];
• a filter-layer probe reports a 56-name deny layer whose exclusions match the
  cua_driver_native__* family (37 + 19) [measured via our probe];
• the shipped desktop bundle registers a Computer Use deny through
  agent.ctx.tools.restrict({ deny: cuaToolNames }) and defaults commandMode to
  true [source-read]. We did not run /computer-use on, so we have not
  observed that gate lifting [inferred].

The general lesson: a large registry is normal; what matters is how many schemas go
on the wire, and who decided that.

The painful part is attribution: tools.schemas(agent) omits a name without saying
who omitted it. Our own probe gives layer index and deny counts, not an owner, so a
desktop permission gate and a third-party plugin's restrict still look alike. A
small "why is this name missing" probe with per-layer counts (ours is
layersFor) helps; it did not, by itself, name the owner.

2. The prototype's role surface is a list ∩ mount

src/app/sessions.ts defines [source-read]:

• WORK_TOOLS — 20 names for worker Sessions;
• a router allow-list — 7 names, same file;
• the presets in cordis.patch.yml decide what is mounted at all.

The allow-list is the only place a role's surface is expressed; the effective
surface is that list ∩ what the preset mounts. Two drift examples from reading
the source:

• web_fetch is in WORK_TOOLS, but the worker preset mounts tool-web with
  fetch: false, so the list expects a tool the mount does not provide
  [source-read; the mount set is our model — we have not run a live instance];
• the router allow-list names bash, while on Windows the preset mounts pwsh and
  disables bash. The allow-list is applied; if that is all that happens, the main
  Session loses its shell on Windows [source-read allow-list; inferred mount
  behaviour; not verified on a live Windows instance].

The structural issue is not the 20 names. It is that a role's surface is expressed
as names in code, so any new tool (MCP server, browser, PDF reader, document
converter) needs a source change, and every fork repeats it.

3. Proposal: role policy, same defaults

Add a settings object, not a new subsystem:

executor.roles:
  worker:   { allowGroups: [fs, shell, jobs, skill, todo, delegation, web], unknown: hide }
  router:   { allowGroups: [fs, jobs, skill, shell], unknown: hide }
  subagent: { allowGroups: [fs, shell, jobs, skill, todo, delegation, web], unknown: hide }

• Defaults are today's lists. A profile that never touches it behaves identically.
• Groups are name patterns (read, web_*, mcp__github__*). The policy is
  static: it is evaluated when a Session is created (and again when its preset
  changes), not per turn.
• Closed by default: a tool that matches no group is hidden for a managed role
  (unknown: hide).
• Admin-owned: allowGroups are settings-only. Group bindings must be
  admin-owned too — a model-callable group editor would let a tool move itself into an
  already-allowed group.
• Versioned: this is the proposal, not our prototype. Our prototype implements a
  diff reporter (policy vs the current lists), not settings history or an apply path.
• No permission change: visibility only; guards, approval and sandboxing stay.

4. What we verified (logic level, not a live instance)

We wrote a logic simulation (no DSH, no live superagent) that resolves the proposed
policy over a modelled worker mount set. It runs six checks; two of them were
rewritten after an independent review because they restated their own premises
(web_fetch was in no mount list; the router check compared two hardcoded arrays).
All six now pass:

• the resolved worker set equals WORK_TOOLS ∩ mounted exactly (14 distractor tools
  included);
• a name that the list expects but the mount lacks (web_fetch) is absent, and the
  check now asserts both sides of that drift;
• a new tool (PDF / MCP / browser / desktop) is hidden by the closed default;
• a misclassification is reported by the diff (a tool that disappears against the
  current lists);
• on the same mounted set, the name list and the group policy differ by exactly
  pwsh (the platform drift);
• adding a group (e.g. acme_*) makes third-party tools visible with two admin
  edits — a match rule plus the role authorising that group — and no code change.

Not verified: a real dsh-superagent instance, the real mounted set, token/cache
numbers, and model behaviour.

5. Traps for anyone building this

1. Preset race. A Session can be created with the default preset and switched to
   a role preset before the first turn [source-read in sessions.ts]. Resolving
   the role once at agent/created can therefore see the wrong preset. We added an
   agent-preset/selected listener to our plugin and a test that reproduces the
   timing (smoke [62]); the listener lives in lib/index.js, not in the policy
   module.
2. Cache semantics. Tool schema changes block prefix reuse from the first changed
   token [inferred from the DSH adapter's documented behaviour; not measured here].
   A policy resolved once per Session is safe; a mid-session release is not. Per-ticket
   Sessions already have different prompts, so cross-ticket reuse is limited
   [inferred].
3. Closed default is a decision, not a detail. If an unclassified tool defaults to
   visible, a newly installed MCP server silently gains access. Default hidden, and
   report it.
4. Visibility is not permission. restrict decides what the model sees; guards
   and approvals decide what may run. Keep them separate.

6. Question

Is this direction useful for the prototype? Would a PR in this shape (a settings
schema + a resolver + an equivalence test that pins the default to today's lists) be
considered — or is the intended answer that the tool surface stays a constant until
the project is out of research? We can share the test and the 4-role prototype; they
are small and dependency-free (165 and 139 lines, no imports).

[kmyy350234-dz]

Source-level follow-up from the same review (no live dsh-superagent run). Environment: DSH 2.0.21-next desktop, profile desktop, Windows 11, 2026-11-01.

In cordis.patch.yml the router preset disables tool-bash on win32 and mounts tool-pwsh instead. In src/app/sessions.ts the main session's restrict({ allow: [...] }) names bash but not pwsh. Host docs say a mask applies to global tools inherited from an ancestor scope (tools registered in the caller's own scope are exempt), and preset tools are inherited — so on Windows the main session's shell is masked out by its own allow list.

Two related notes:

1. This host also has an independent deny of pwsh from another plugin, so do not attribute "no shell" to a single layer; the point here is the allow list missing pwsh, not the cause of any given machine's absence.
2. The same file's noWait guard checks name === 'bash' only, while pwsh exposes the same timeoutMs / run_in_background fields — a policy gap if the shell is ever restored.

Worker sessions build their allow list from the parent scope's visible names (WORK_TOOLS.filter(...)), which includes pwsh, so they keep their shell. That last point is conditional on the parent being the preset scope; a log line at worker construction would settle it.