fix(chat): app tools not discoverable in agentic chat#6413
Conversation
…discovery Claude was not using deferred app tools (GitHub, Twitter, etc.) because the system prompt never mentioned they existed or how to discover them via tool_search_tool_regex. This adds an <available_app_tools> section listing all loaded app tool names so Claude searches for and uses them. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Greptile SummaryThis PR fixes a root-cause bug where Claude declined requests like "list my GitHub issues" despite many app tools being loaded. It appends an Confidence Score: 5/5Safe to merge — the fix is correct and directly addresses the described bug. The only finding is a P2 style issue (misleading comment and a redundant intermediate set + for loop on lines 537–541). The core logic is sound: app_tools is correctly sourced from load_app_tools(uid), the XML section is well-formed, and the instruction to use tool_search_tool_regex with keyword examples accurately describes the discovery mechanism. No blocking issues. No files require special attention.
|
| Filename | Overview |
|---|---|
| backend/utils/retrieval/agentic.py | Appends <available_app_tools> to the system prompt so Claude knows to search for deferred app tools via tool_search_tool_regex; one misleading comment and redundant loop in the new block (lines 537–541) are the only issues. |
Sequence Diagram
sequenceDiagram
participant Client
participant Agentic as agentic.py
participant Anthropic
participant App as App Integration
Client->>Agentic: POST /v2/messages
Agentic->>Agentic: load_app_tools(uid)
Note over Agentic: 82 app tool objects loaded
Agentic->>Agentic: build system_prompt<br/>+ append <available_app_tools><br/>(sorted tool names)
Agentic->>Anthropic: _run_anthropic_agent_stream(system_prompt, tool_schemas)
Anthropic->>Anthropic: Claude sees available tools,<br/>calls tool_search_tool_regex("github")
Anthropic->>Anthropic: Resolves deferred tool (list_issues)
Anthropic->>App: Calls list_issues
App-->>Anthropic: Issues result
Anthropic-->>Agentic: Streamed response chunks
Agentic-->>Client: Streamed response
Reviews (1): Last reviewed commit: "fix(chat): add app tool awareness to sys..." | Re-trigger Greptile
| app_names = set() | ||
| for t in app_tools: | ||
| # Tool names are prefixed with app_id; extract the human-readable app name from description | ||
| app_names.add(t.name) | ||
| app_tool_names = ", ".join(sorted(app_names)) |
There was a problem hiding this comment.
Misleading comment and redundant loop
The comment on line 539 says "extract the human-readable app name from description" but the loop body never reads t.description — it only uses t.name. The intermediate set and for loop are also redundant since a generator expression achieves the same result.
| app_names = set() | |
| for t in app_tools: | |
| # Tool names are prefixed with app_id; extract the human-readable app name from description | |
| app_names.add(t.name) | |
| app_tool_names = ", ".join(sorted(app_names)) | |
| app_tool_names = ", ".join(sorted(t.name for t in app_tools)) |
…6413) ## Summary - Claude was ignoring deferred app tools (GitHub, Twitter, Calendar, Notion, etc.) because the system prompt never told it about them or how to use `tool_search_tool_regex` to discover them - Added `<available_app_tools>` section to the system prompt that lists all loaded app tool names and instructs Claude to search for them when relevant - Fixes the issue where asking "List five latest gh issues" returned "I don't have GitHub access" despite 82 app tools being loaded ## Test plan - [x] Unit tests pass (`test_prompt_cache_integration.py`, `test_prompt_cache_optimization.py`) - [ ] Verify in mobile app chat: ask "List five latest gh issues" and confirm it calls `list_issues` tool 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Summary
tool_search_tool_regexto discover them<available_app_tools>section to the system prompt that lists all loaded app tool names and instructs Claude to search for them when relevantTest plan
test_prompt_cache_integration.py,test_prompt_cache_optimization.py)list_issuestool🤖 Generated with Claude Code