fix(search): fall back to local search when semantic results don't match MCP tools

[shashi-stackone]

Summary

When semantic search returns results but none match MCP tool names (e.g. due to composite ID format differences), the SDK now falls back to local BM25+TF-IDF search instead of returning empty results.

Previously, searchTools() in auto mode only fell back to local search when the semantic API call failed. Now it also falls back when
the API succeeds but the returned action IDs don't match any fetched tools.

Test plan

• Existing tests pass
• Auto mode returns local results when semantic matches are empty
• Semantic mode still raises on failure (no fallback)
• Local mode unaffected
• Happy path (semantic results match tools) unaffected

---

Summary by cubic

Fallback to local BM25+TF‑IDF search when semantic search returns results that don’t match any MCP tool names. Prevents empty results in auto mode and makes tool discovery more reliable.

• Bug Fixes
  • Auto mode now falls back to local search if semantic results contain zero matching tools.
  • Adds a warning log before falling back; uses the same top_k and similarity thresholds.
  • In semantic mode, no fallback: returns empty on no match and still raises on API failure; local mode unchanged.

^{Written for commit 53dc30a. Summary will update on new commits.}

[Copilot]

Pull request overview

Updates StackOneToolSet.search_tools() auto-search behavior to avoid returning empty tool lists when semantic search returns results that don’t map to fetched MCP tool names, by falling back to the local BM25+TF‑IDF search.

Changes:

• Add a local-search fallback when semantic search returns non-empty results but none match fetched MCP tools.
• Log a warning when this “semantic results but no MCP matches” fallback path is taken.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This fallback runs regardless of effective_search. In search="semantic" mode the docstring says to use only the semantic API; falling back to local search here changes that contract and can return tools that weren’t in semantic results. Consider gating this branch to effective_search == "auto" (and in semantic mode just return Tools([]) / the empty matched_tools result).

[Copilot]

New behavior (fallback when semantic returns results but none match fetched MCP tools) isn’t covered by existing tests. Add a unit test that simulates semantic results whose normalized IDs don’t exist in all_tools and asserts: (1) search="auto" falls back to _local_search, and (2) search="semantic" does not fall back (once the mode gating is fixed).

Suggested change

	# If semantic returned results but none matched MCP tools, fall back to local search
	if len(all_results) > 0 and len(matched_tools) == 0:
	logger.warning(
	"Semantic search returned %d results but none matched MCP tools, "
	"falling back to local search",
	len(all_results),
	)
	return self._local_search(
	query,
	all_tools,
	connector=connector,
	top_k=effective_top_k,
	min_similarity=effective_min_sim,
	)
	# If semantic returned results but none matched MCP tools, optionally fall back to local search
	if len(all_results) > 0 and len(matched_tools) == 0:
	if effective_search == "auto":
	logger.warning(
	"Semantic search returned %d results but none matched MCP tools, "
	"falling back to local search",
	len(all_results),
	)
	return self._local_search(
	query,
	all_tools,
	connector=connector,
	top_k=effective_top_k,
	min_similarity=effective_min_sim,
	)
	logger.warning(
	"Semantic search returned %d results but none matched MCP tools; "
	"not falling back in '%s' search mode",
	len(all_results),
	effective_search,
	)
	return Tools([])

[cubic-dev-ai]

1 issue found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="stackone_ai/toolset.py">

<violation number="1" location="stackone_ai/toolset.py:937">
P2: Guard this new fallback so it only applies in `search="auto"`; currently `search="semantic"` can also fall back to local results.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}