fix: reduce system prompt 78% to fix Qwen3.5 timeouts + MCP runtime status (#609)

[itomek]

Summary

• Perf: Reduce ChatAgent system prompt from ~17,600 → ~3,853 tokens (78%) to fix Qwen3.5-35B timeouts on local hardware
• Feat: MCP server runtime status exposed in Agent UI Settings modal (issue Agent UI: MCP client/server support and configuration (v0.17.1) #609)
• Fix: Increase chat timeouts 120/180s → 600s; fix stale log message; fix MCPTool.get() AttributeError in tools endpoint

Changes

System prompt optimization (perf: reduce system prompt...)

• Restructured _get_system_prompt() from 733 → 280 lines using two-tier RAG gating:
  • Tier 1 (always present): SMART DISCOVERY, FILE SEARCH — needed because RAG tools are always registered
  • Tier 2 (gated on has_indexed): FACTUAL ACCURACY, DOCUMENT SILENCE, POST-INDEX QUERY, etc.
• Merged duplicate platform blocks (current OS only)
• Removed: merge conflict marker <<<<<<< HEAD, hardcoded AVAILABLE TOOLS REFERENCE, both UNSUPPORTED FEATURES blocks, duplicate rules
• Increased streaming/non-streaming timeouts to 600s
• Added "Sending to model..." SSE status events for better UX

MCP runtime status (feat: MCP runtime status in Agent UI)

• MCPClientManager._failed: tracks connection errors from load_from_config()
• MCPClientManager.get_status_report(): returns {name, connected, tool_count, error} for all servers
• GET /api/mcp/status: REST endpoint returning cached runtime status
• SSE mcp_status event emitted after each agent setup
• Settings modal: MCP Servers section showing ✓ connected (N tools) or ✗ failed status
• 6 new unit tests for _failed tracking and get_status_report()

Bug fixes

• list_mcp_server_tools was calling .get() on MCPTool dataclass (pre-existing AttributeError)
• remove_server() now clears _failed to prevent ghost entries in status report
• has_docs → has_indexed for Tier 2 gating (library docs aren't indexed yet)

Tests

• 6 new tests: MCPClientManager._failed tracking and get_status_report()
• 2 new tests: Tier 1/Tier 2 prompt gating correctness

Test plan

• uv run python -m pytest tests/unit/mcp/ tests/test_chat_agent.py -x -q — 136 pass, 1 pre-existing network failure
• gaia chat --ui → send "hi" with no documents → responds without timeout (was timing out with Qwen3.5-35B)
• Settings modal → MCP Servers section shows connected/failed servers after first chat
• GET /api/mcp/status returns {"servers": []} before any chat, populated after

Closes #609

[kovtcharov]

Recommend creating your own MCP-focused agent.

[kovtcharov]

@claude review this PR in detail. Identify if there's any technical debt or bugs.

[github-actions]

Claude finished @kovtcharov's task —— View job

---

PR Review: Technical Debt and Bug Analysis

Reviewing PR #617 in detail for technical debt and bugs...

• Read PR diff and changed files
• Review system prompt optimization in ChatAgent
• Review MCP runtime status implementation
• Review bug fixes and new tests
• Identify technical debt and potential issues

---

Summary

This PR makes significant improvements to system prompt size (78% reduction), MCP runtime status visibility, and timeout handling. The implementation is generally solid with good test coverage. However, I've identified several issues ranging from minor to important.

Issues Found

🟡 Important Issues

1. Potential Race Condition in MCP Status Cache (src/gaia/ui/_chat_helpers.py:721-726)

The _mcp_status_cache is updated inside the agent thread while get_cached_mcp_status() reads from the main thread. Although there's a lock, the update pattern could lead to stale reads:

# Inside _run_agent() thread
if hasattr(agent, "get_mcp_status_report"):
    mcp_report = agent.get_mcp_status_report()
    with _mcp_status_lock:
        _mcp_status_cache[:] = mcp_report  # Modifies in-place

Recommendation: Consider using copy.deepcopy() when returning from get_cached_mcp_status() to prevent potential mutation issues, or ensure the lock is held during the entire read operation.

2. MCPClientManager ThreadPoolExecutor Lifecycle (src/gaia/mcp/client/mcp_client_manager.py:220-250)

The parallel connection code creates a new ThreadPoolExecutor per call to load_from_config(). While shutdown(wait=False, cancel_futures=True) is called, threads stuck in readline() become daemon threads that exit with the process. This is fine for normal operation, but could leave resources hanging if load_from_config() is called repeatedly.

pool = ThreadPoolExecutor(max_workers=len(to_connect))
# ... later ...
pool.shutdown(wait=False, cancel_futures=True)

Recommendation: Consider reusing a single thread pool or adding a comment documenting this behavior is intentional.

3. Agent Cache Eviction Policy (src/gaia/ui/_chat_helpers.py:94-110)

The agent cache uses a simple FIFO eviction when full:

if session_id not in _agent_cache and len(_agent_cache) >= _MAX_CACHED_AGENTS:
    oldest = next(iter(_agent_cache))  # FIFO eviction
    del _agent_cache[oldest]

Technical Debt: This evicts the oldest entry regardless of activity. A recently active session could be evicted if it was the oldest, while inactive sessions remain cached.

Recommendation: Consider LRU eviction or track last-access timestamps.

🟢 Minor Issues

4. Missing Type Annotations (src/gaia/ui/_chat_helpers.py:53-62)

Several module-level variables lack type annotations:

_agent_cache: dict = {}  # Could be: Dict[str, Dict[str, Any]]
_mcp_status_cache: list = []  # Could be: List[Dict[str, Any]]

5. Hardcoded Magic Numbers

• _CONNECT_TIMEOUT = 10.0 in mcp_client_manager.py:209 - should be configurable
• _MAX_CACHED_AGENTS = 10 in _chat_helpers.py:57 - should be configurable
• _STREAM_TIMEOUT = 600 in _chat_helpers.py:796 - duplicated with non-streaming timeout

Recommendation: Consider extracting these to a config module or environment variables.

6. Incomplete Error Message in _maybe_load_expected_model (src/gaia/ui/_chat_helpers.py:355-364)

The warning message "Could not auto-load LLM. Check that Lemonade is running." is generic. The actual exception is logged but not shown to the user.

Recommendation: Include more specific error info in the SSE status event.

✅ Good Practices Observed

1. Comprehensive Test Coverage: New tests for _failed tracking and get_status_report() (6 tests), plus Tier 1/Tier 2 prompt gating tests (2 tests).

2. Proper Lock Usage: The _mcp_status_lock and _agent_cache_lock are used correctly for thread synchronization.

3. Defensive Coding: Good use of pop(name, None) in remove_server() to clear _failed entries without raising KeyError.

4. Clear Code Organization: The two-tier RAG gating is well-documented and the separation between discovery rules (always present) and query rules (gated on has_indexed) is logical.

5. SSE Event Emission: The mcp_status SSE event is emitted after agent setup, allowing the frontend to update MCP status without a separate API call.

Addressing Reviewer Concern

@kovtcharov raised a concern about system prompt reduction potentially reverting agent performance.

Analysis: The prompt reduction is well-structured:

• Tier 1 (always present): SMART DISCOVERY, FILE SEARCH - needed because RAG tools are always registered
• Tier 2 (gated on has_indexed): FACTUAL ACCURACY, DOCUMENT SILENCE, POST-INDEX QUERY, etc.

The key insight is that Tier 2 rules only make sense when documents are indexed. Including them when no documents are indexed adds ~14,000 tokens of irrelevant instructions that can confuse the model and cause timeouts.

Tests Added: test_tier2_rag_rules_absent_without_indexed_docs and test_tier2_rag_rules_present_after_indexing verify the gating logic is correct.

Recommendation: This change should improve agent performance for the common case (no docs indexed yet) while maintaining full capability when documents are present. The concern is valid but appears to be addressed by the two-tier approach.

No Critical Bugs Found

The PR is well-implemented overall. The issues identified are technical debt and optimization opportunities rather than bugs that would cause incorrect behavior.

---