Skip to content

test(tools): enable tool_schema_filter in testing.toml for live coverage of #2020 #2040

@bug-ops

Description

@bug-ops

Problem

The dynamic tool schema filtering feature (#2020, PR #2026) added 752 lines across 17 files with a new ToolSchemaFilter struct, embedding-based tool selection, and config-driven always-on list. Despite thorough unit tests, the feature has zero live agent session coverage.

The feature is disabled in .local/config/testing.toml (no [agent.tool_filter] section with enabled = true).

Evidence

  • CI-16 session (2026-03-20): grep 'tool schema filter' .local/testing/debug/session-ci16.log — empty
  • Code path in assembly.rs: only fires when self.tool_schema_filter.is_some(), which requires config opt-in
  • maybe_init_tool_schema_filter() returns early when config.enabled == false

Impact

Untested aspects:

  1. Graceful degradation when embed() fails at startup (all tools fallback)
  2. send_status("filtering tools...") spinner visibility in CLI/TUI
  3. NoEmbedding fallback for late MCP tool registration
  4. Actual token reduction in real sessions (the stated goal of research(tools): dynamic tool schema filtering to reduce context waste and improve selection accuracy #2020)
  5. Interaction with MCP tool registration after startup
  6. Top-K selection behavior with real tool descriptions

Proposed Fix

Add a live test session with [agent.tool_filter] enabled = true to validate the path. This requires an embedding-capable provider (OpenAI is available in testing config).

Optionally, enable the filter in .local/config/testing.toml:

[agent.tool_filter]
enabled = true
top_k = 5
always_on = ["memory_save", "memory_search", "shell"]

Note: enabling this in testing.toml will add latency at startup (embeds all tool descriptions).

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2High value, medium complexitytestingTests and qualitytoolsTool execution and MCP integration

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions