test(installer): add custom agent MCP harness

[ZardLi1115]

Closes #993

Installer validation now covers the post-install custom-agent path that was previously untested: importing a custom agent bundle, loading MCP tools from a deterministic in-tree server, surfacing MCP connection failures, and proving a no-MCP agent path emits no MCP traffic. The installer workflow runs that harness on Ubuntu and Windows and uploads the dummy MCP logs when it fails.

Test plan:

• PYTHONPATH=src python -m pytest tests/unit/test_export_import.py tests/fixtures/mcp/dummy_server/test_dummy_server.py tests/installer/test_custom_agent_mcp_harness.py -q
• python -m black --check tests/fixtures/mcp/dummy_server tests/fixtures/custom_agents/installer_mcp tests/fixtures/custom_agents/installer_no_mcp tests/installer/test_custom_agent_mcp_harness.py
• PYTHONPATH=src python - <<'PY' ... yaml.safe_load(.github/workflows/build-installers.yml) ... PY

[kovtcharov]

@claude review PR.

[kovtcharov]

Great contribution, thanks!

[github-actions]

Solid harness — fills a real gap. The deterministic in-tree MCP server is a nice substitute for npx/network, and routing through gaia agent import (instead of staging the agent dir directly) gives this the bang-for-buck of a true end-to-end check. APIs all line up with the codebase (MCPClient.from_config, AgentRegistry.discover, _snapshot_tools, get_mcp_status_report, the bundle.json schema). A few notes:

🟡 Job depends on build but doesn't use its artifacts. The workflow comment calls this #993's "installed-runtime guard," but custom-agent-mcp-harness does pip install -e ".[dev,mcp]" from source — it never opens the AppImage/DMG produced by build. Worth tightening the comment so reviewers don't think the harness is exercising the packaged binary (.github/workflows/build-installers.yml:1115).

🟢 Brittle error-string match. test_custom_agent_with_mcp_reports_diagnosable_connection_failure asserts the full transport error verbatim. Any rewording in src/gaia/mcp/client/transports/stdio.py:201 flakes this. Consider matching on "exit code: 17" substring + connected=False.

🟢 "No MCP traffic" assertion is vacuous. assert not mcp_log.exists() checks a path the no-MCP agent never references — it's true by construction. Stronger: assert the agent has no _mcp_manager, or that no mcp_* tool name appears in its tool registry.

🟢 Minor cleanup. Second test skips the try/finally disconnect that the first test uses; tests/fixtures/mcp/dummy_server/test_dummy_server.py:21 uses a cwd-relative path while the harness uses REPO_ROOT — pick one.

None blocking. Nice contribution.

[asim48-ctrl]

Small CI note from the current run: the installer harness jobs are green, and the remaining failing check appears to be formatting only. Run Code Quality Checks reports Black + isort failures in tests/installer/test_custom_agent_mcp_harness.py; the log suggests python util/lint.py --black --fix and python util/lint.py --isort --fix.

[ZardLi1115]

Lint issues from the previous CI run have been addressed in d210474. Current installer harness jobs are green now.
Would appreciate a re-review/approval when you have a moment. Thanks!

[asim48-ctrl]

Thanks for the update. The formatting blocker I pointed at is addressed from what I can see: the current visible checks no longer show the Black/isort failure. I can’t approve as a maintainer, so leaving final review to the GAIA maintainers.

[ZardLi1115]

Thanks for the update. The formatting blocker I pointed at is addressed from what I can see: the current visible checks no longer show the Black/isort failure. I can’t approve as a maintainer, so leaving final review to the GAIA maintainers.

Thanks for checking and confirming the fixes — appreciate it.