Skip to content

Add request-side tool mediation controls#74

Merged
bglusman merged 4 commits into
release/docs-and-version-prepfrom
tool-mediation-control-plane
May 25, 2026
Merged

Add request-side tool mediation controls#74
bglusman merged 4 commits into
release/docs-and-version-prepfrom
tool-mediation-control-plane

Conversation

@bglusman
Copy link
Copy Markdown
Owner

@bglusman bglusman commented May 25, 2026

Summary

Adds the first Wardwright tool-mediation control-plane slice for issue #47.

  • documents the bidirectional tool visibility and execution model
  • adds OpenAPI config schema for tool_mediation
  • implements non-streaming OpenAI-compatible request-side patch rules for hide, augment, and replace
  • records original and provider-visible tool descriptors in receipt metadata
  • expands integration coverage for dynamic provider-visible function schemas and arbitrary Dune-backed Wardwright server tools
  • documents the 0.1.0 review pass coverage for canonical schema hashes, no-op mediation receipts, and replacement-name collision handling

Closes #47.

Validation

  • mise exec --command 'cd app && mix test'
  • mise exec --command 'cd app && mix format --check-formatted'
  • gitleaks protect --staged --config .gitleaks.toml --verbose
  • mise run check:docs
  • targeted Muex: MIX_ENV=test mix muex --files "lib/wardwright/tool_mediation.ex" --test-paths "test/stream_provider_transport_test.exs" --max-mutations 40 --fail-at 80 -> 40 mutants, 36 killed, 4 invalid, 0 survived, 100.0% mutation score
  • pre-commit app/docs checks and staged gitleaks
  • pre-push mise check

Limits

This PR does not claim native provider-hosted tool normalization or streaming tool-call mediation. observe, mutate, and execute remain non-patching mediation modes until extension execution and provider-specific normalizers land.

Copilot AI review requested due to automatic review settings May 25, 2026 00:22
@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented May 25, 2026

Reviewer's Guide

Implements the first slice of Wardwright request-side tool mediation for OpenAI-compatible Chat Completions: normalizing tool_mediation config, applying non-streaming patch rules to request tool catalogs, recording mediation metadata in receipts, expanding server-tool handling to dynamic Dune tools, and documenting the new control plane and OpenAPI schema surface.

Sequence diagram for request-side tool mediation in complete_selected_model

sequenceDiagram
  actor Agent
  participant ServerTools as Wardwright.ServerTools
  participant ToolMediation as Wardwright.ToolMediation
  participant Core as Wardwright

  Agent->>ServerTools: complete_selected_model(selected_model, request, config)
  alt request.stream == true
    ServerTools->>Core: complete_selected_model(selected_model, request, config)
    Core-->>ServerTools: outcome
    ServerTools-->>Agent: outcome
  else tools == []
    ServerTools->>ToolMediation: apply(request, config)
    ToolMediation-->>ServerTools: {patched_request, mediation}
    ServerTools->>Core: complete_selected_model(selected_model, patched_request, config)
    Core-->>ServerTools: outcome
    ServerTools->>ServerTools: add_tool_mediation_metadata(outcome, mediation)
    ServerTools-->>Agent: outcome_with_mediation
  else tools != []
    ServerTools->>ServerTools: inject_tools(request, tools)
    ServerTools->>ToolMediation: apply(request_with_tools, config)
    ToolMediation-->>ServerTools: {patched_request, mediation}
    ServerTools->>ServerTools: complete_with_tool_loop(selected_model, patched_request, config, tools)
    ServerTools-->>ServerTools: outcome
    ServerTools->>ServerTools: add_tool_mediation_metadata(outcome, mediation)
    ServerTools-->>Agent: outcome_with_mediation
  end
Loading

File-Level Changes

Change Details Files
Introduce Wardwright.ToolMediation to normalize and patch provider-visible tools based on configuration rules.
  • Add a ToolMediation module that inspects request tools and tool_mediation config and returns a patched request plus mediation metadata.
  • Support hide, augment, and replace actions on tools, including description appends and parameter/tool replacement.
  • Compute schema hashes for original and provider-visible tools, and track who declared each tool (agent vs Wardwright).
app/lib/wardwright/tool_mediation.ex
Wire tool mediation into the server tool pipeline and default configuration normalization.
  • Change ServerTools.complete_selected_model to run mediation in non-streaming flows, before tool-loop execution or plain completion, and attach mediation metadata into provider_metadata.
  • Add add_tool_mediation_metadata helper to inject wardwright_tool_mediation into outcome provider metadata when present.
  • Extend Wardwright.default_config and normalize_config to include normalized tool_mediation, with helper normalize_tool_mediation handling various config shapes.
app/lib/wardwright/server_tools.ex
app/lib/wardwright.ex
Extend OpenAPI schemas and configuration surfaces to expose tool_mediation on routes and test targets.
  • Add ToolMediation and ToolMediationRule schemas describing mode and rules (observe, patch, mutate, execute with hide/augment/replace actions) to contracts.
  • Wire tool_mediation into RouteTarget and TestRouteTarget configuration schemas so clients can configure mediation via policy config.
contracts/openapi.yaml
Update the streaming provider test harness to support mediated and dynamic server-tool flows, including dynamic Dune tools.
  • Make forwarded_tool_request handling derive the tool name and arguments dynamically from request tools, so replacement tools are respected.
  • Extend server_tool_request? and server_tool_result_request? to recognize dynamic Wardwright server tool names and call IDs based on prefixes.
  • Add helpers for first_tool_name, dynamic server tool name/call ID detection, argument building, and result-message detection, and extend upstream_result to emit dynamic server tool result messages.
app/test_support/router_case.ex
Add integration tests covering tool_mediation behavior and dynamic Dune-backed server tools for OpenAI-compatible targets.
  • Add tests that hide agent-declared tools under patch mode and verify provider-visible catalog and mediation metadata in receipts.
  • Add tests ensuring observe and mutate modes remain non-mutating and do not emit mediation metadata yet.
  • Add tests for replace rules that swap agent tools with alternative provider-visible schemas and verify provider behavior and receipt descriptors.
  • Add tests for augment rules that modify Wardwright-hosted server tools before provider choice, including schema hash changes and server_tool receipts.
  • Add an integration test that executes an arbitrary dynamic Dune server tool in execute mode, recording execution metadata but without tool_mediation metadata.
app/test/stream_provider_transport_test.exs
Document the tool-mediation control plane and link it into existing docs and landing page.
  • Describe the bidirectional tool mediation model, extension modes (observe, patch, mutate, execute), and backlog for provider tool normalization in a new Tool Mediation doc.
  • Update the tool-context-policy doc to reference tool mediation receipts and the new doc.
  • Update the index landing page to surface Tool Mediation as a primary concept and mention it in the status blurb.
docs/tool-mediation.md
docs/tool-context-policy.md
docs/index.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="docs/tool-context-policy.md" line_range="134-138" />
<code_context>
 output. Streaming, side-effecting tools, remote MCP passthrough, and hidden
 provider tools remain deferred.

+Tool mediation is the broader control plane around this first server-tool
+surface. Request-side mediation can inspect agent-declared and Wardwright-added
+tool declarations, patch the provider-visible catalog, and receipt original vs
+final tool schema hashes under
+`final.provider_metadata.wardwright_tool_mediation`. See
</code_context>
<issue_to_address>
**issue (typo):** Consider adjusting phrasing around "and receipt original vs final" for grammatical clarity.

The phrase "and receipt original vs final tool schema hashes" reads as if it’s missing a preposition. Consider "and receipt of original vs final tool schema hashes" or "and record original vs final tool schema hashes" to make it grammatically complete.

```suggestion
Tool mediation is the broader control plane around this first server-tool
surface. Request-side mediation can inspect agent-declared and Wardwright-added
tool declarations, patch the provider-visible catalog, and record original vs
final tool schema hashes under
`final.provider_metadata.wardwright_tool_mediation`. See
```
</issue_to_address>

### Comment 2
<location path="docs/tool-mediation.md" line_range="133" />
<code_context>
+}
+```
+
+The hash records schema change without storing the full schema in every receipt.
+Future slices should add argument hashes, result hashes, tool-specific timing,
+approval evidence, and provider-hosted event normalization.
</code_context>
<issue_to_address>
**nitpick (typo):** The phrase "records schema change" is slightly awkward and could be smoothed out.

Very minor: consider rephrasing to "records the schema change" or "records schema changes" for more natural grammar.

```suggestion
The hash records the schema change without storing the full schema in every receipt.
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread docs/tool-context-policy.md
Comment thread docs/tool-mediation.md Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an initial “request-side tool mediation” control plane slice to Wardwright: operators can configure rule-based patching of provider-visible tool catalogs (hide/augment/replace) for non-streaming OpenAI-compatible chat completions, with receipt evidence capturing original vs provider-visible tool descriptors.

Changes:

  • Documented the tool mediation model (visibility/execution taxonomy, extension modes, patch rule shapes, receipt evidence).
  • Extended OpenAPI config schema and runtime config normalization to include tool_mediation.
  • Implemented Wardwright.ToolMediation and integrated it into the server-tool completion pipeline; expanded integration tests and the streaming-provider test harness for dynamic tool names.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
docs/tool-mediation.md New documentation for mediation model, patch rules, and receipt evidence.
docs/tool-context-policy.md Links tool mediation into existing tool-context policy docs.
docs/index.md Adds tool mediation to docs index navigation and status blurb.
contracts/openapi.yaml Adds ToolMediation / ToolMediationRule schemas and wires into model config shapes.
app/lib/wardwright/tool_mediation.ex Implements request-side tool catalog patching + receipt metadata emission.
app/lib/wardwright/server_tools.ex Applies tool mediation before provider calls and stores mediation metadata in provider metadata.
app/lib/wardwright.ex Adds tool_mediation defaults and normalization into config normalization pipeline.
app/test/stream_provider_transport_test.exs Integration coverage for hide/augment/replace behavior + execute-mode dynamic Dune tool coverage.
app/test_support/router_case.ex Updates test provider to support dynamic forwarded tool names and dynamic server-tool IDs.

Comment thread docs/tool-mediation.md
Comment thread docs/tool-mediation.md
Comment thread app/lib/wardwright/tool_mediation.ex
Comment thread app/lib/wardwright/tool_mediation.ex Outdated
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a tool mediation control plane to Wardwright, enabling the inspection, patching, and replacement of tool declarations before they reach providers. It includes the new Wardwright.ToolMediation module, integration into the model completion flow, and updated API contracts and documentation. Review feedback highlighted several improvement opportunities: optimizing list appends to avoid O(N^2) performance, ensuring that mediation rules are only recorded in receipts when changes are actually applied, and addressing potential non-determinism in tool schema hashing due to map key ordering.

Comment thread app/lib/wardwright/tool_mediation.ex Outdated
Comment thread app/lib/wardwright/tool_mediation.ex
Comment thread app/lib/wardwright/tool_mediation.ex
Comment thread app/lib/wardwright/tool_mediation.ex Outdated
Comment thread app/lib/wardwright/tool_mediation.ex Outdated
@bglusman bglusman merged commit c3966c4 into release/docs-and-version-prep May 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants