feat: add LISA MCP server with AI-native developer tools#4508
Draft
johnsongeorge-w wants to merge 2 commits into
Draft
feat: add LISA MCP server with AI-native developer tools#4508johnsongeorge-w wants to merge 2 commits into
johnsongeorge-w wants to merge 2 commits into
Conversation
Add a Model Context Protocol (MCP) server that gives AI assistants
deep knowledge of LISA conventions for test authoring, log analysis,
runbook management, debugging, and framework exploration.
Package structure (lisa_mcp/):
- server.py — FastMCP server with host header validation,
CLI with stdio/SSE transport
- tools/test_writer.py — 5 tools: write_test (with existing test
detection), scaffold suite/case, guidelines,
list requirements
- tools/runbook.py — 3 tools: generate, validate, fix runbooks
- tools/log_analysis.py — log download with Azure AD auth, blob
prefix downloads, analyze/summarize logs,
explain failures, diagnose bugs, file
search/read/list
- tools/knowledge.py — 6 tools: explain concepts/errors, API
reference, find examples, list tools/features
- tools/execution.py — 1 tool: lisa_run placeholder
- tools/_repo.py — repo discovery, doc/context loading
- docs_index.yaml — manifest mapping tools to .rst/.md docs
- context/*.md — curated knowledge base
lisa_write_test detects existing test suites and methods matching the
query, displaying method names, line numbers, and descriptions so the
caller LLM can add to an existing suite instead of creating duplicates.
Also includes:
- Dockerfile using local source with host header validation
- pyproject.toml with lisa-mcp entry point and azure dependencies
- tests (unit + integration)
- README.md with setup, configuration, and architecture docs
- .gitignore entries for MCP local files
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new mcp/ subpackage providing a Model Context Protocol (MCP) server, lisa-mcp, that exposes LISA-specific developer tools (test authoring, runbook generate/validate/fix, log analysis & download, framework knowledge lookup, and a lisa_run placeholder) to AI assistants. It ships a FastMCP-based server with stdio and SSE transports, a Dockerfile for hosted deployment, a YAML doc-mapping manifest, curated context markdown, and unit + protocol-level integration tests.
Changes:
- New
lisa_mcppackage registering ~25lisa_*-prefixed MCP tools across 5 modules, with a CLI entry point and SSE/stdio transports. - Docs/context assets:
docs_index.yaml,context/*.md, plus a Dockerfile, README, andpyproject.toml. - Test infrastructure: fixtures, unit tests, comprehensive functional tests against
mcp._tool_manager, MCP protocol integration tests, arun_tests.pyrunner, and.gitignoreupdates.
Reviewed changes
Copilot reviewed 27 out of 30 changed files in this pull request and generated 17 comments.
Show a summary per file
| File | Description |
|---|---|
| mcp/lisa_mcp/server.py | FastMCP server, tool registration, stdio/SSE CLI |
| mcp/lisa_mcp/tools/test_writer.py | Test scaffolding/authoring tools |
| mcp/lisa_mcp/tools/runbook.py | Runbook generate/validate/fix tools |
| mcp/lisa_mcp/tools/log_analysis.py | Log parsing, classification, download (Azure Blob, archive extraction) |
| mcp/lisa_mcp/tools/knowledge.py | Concept/API/example/error lookup |
| mcp/lisa_mcp/tools/execution.py | lisa_run placeholder |
| mcp/lisa_mcp/tools/_repo.py | Repo/doc/manifest helpers |
| mcp/lisa_mcp/docs_index.yaml | Tool→docs mapping (has duplicate keys) |
| mcp/lisa_mcp/context/*.md | Curated knowledge base |
| mcp/server.py, mcp/lisa_mcp/main.py | Convenience entrypoints |
| mcp/Dockerfile | Container image for SSE deployment |
| mcp/pyproject.toml | Package config & entry point |
| mcp/README.md | Setup, configuration, architecture docs |
| mcp/run_tests.py | Unit/integration/smoke test runner |
| mcp/tests/test_all_tools.py | Functional tests for all registered tools |
| mcp/tests/test_mcp_integration.py | Stdio MCP protocol integration tests |
| mcp/tests/test_authoring.py, test_log_analysis.py | Targeted unit tests |
| mcp/tests/fixtures/* | Sample logs and runbook |
| .gitignore | MCP local/temp file ignores |
Key Test Cases:
smoke_test|verify_reboot_in_platform|verify_stop_start_in_platform
Impacted LISA Features:
StartStop, SerialConsole
Tested Azure Marketplace Images:
- canonical 0001-com-ubuntu-server-jammy 22_04-lts-gen2 latest
- redhat rhel 9_5 latest
Comment on lines
+775
to
+777
| def test_tool_count(self) -> None: | ||
| count = len(mcp._tool_manager.list_tools()) | ||
| self.assertEqual(count, 25, f"Expected 25 tools, got {count}") |
Comment on lines
+1394
to
+1421
| def _extract_archive(download_path: str, download_dir: str) -> str: | ||
| """Extract tar.gz/zip archives, return the result directory path.""" | ||
| extract_dir = os.path.join(download_dir, "extracted") | ||
|
|
||
| if tarfile.is_tarfile(download_path): | ||
| os.makedirs(extract_dir, exist_ok=True) | ||
| with tarfile.open(download_path) as tf: | ||
| safe_members = [ | ||
| m | ||
| for m in tf.getmembers() | ||
| if not m.name.startswith(("/", "..")) and ".." not in m.name | ||
| ] | ||
| tf.extractall(extract_dir, members=safe_members) | ||
| os.remove(download_path) | ||
| return extract_dir | ||
|
|
||
| if zipfile.is_zipfile(download_path): | ||
| os.makedirs(extract_dir, exist_ok=True) | ||
| with zipfile.ZipFile(download_path) as zf: | ||
| safe_names = [ | ||
| n | ||
| for n in zf.namelist() | ||
| if not n.startswith(("/", "..")) and ".." not in n | ||
| ] | ||
| for name in safe_names: | ||
| zf.extract(name, extract_dir) | ||
| os.remove(download_path) | ||
| return extract_dir |
Comment on lines
+109
to
+135
|
|
||
| # Trusted hosts for Host header validation behind a reverse proxy. | ||
| # Set ALLOWED_HOSTS="host1,host2" in your deployment environment. | ||
| # Defaults to localhost only (for local development). | ||
| default_hosts = "localhost,127.0.0.1" | ||
| allowed_hosts = os.environ.get("ALLOWED_HOSTS", default_hosts).split(",") | ||
|
|
||
| app = Starlette( | ||
| routes=[ | ||
| Route("/sse", endpoint=handle_sse), | ||
| Mount("/messages/", app=sse.handle_post_message), | ||
| ], | ||
| middleware=[ | ||
| Middleware( | ||
| TrustedHostMiddleware, | ||
| allowed_hosts=allowed_hosts, | ||
| ), | ||
| ], | ||
| ) | ||
|
|
||
| uvicorn.run( | ||
| app, | ||
| host=args.host, | ||
| port=args.port, | ||
| log_level="info", | ||
| forwarded_allow_ips="*", | ||
| proxy_headers=True, |
Comment on lines
+1
to
+26
| FROM python:3.12-slim | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| # Install git (needed for repo clone) | ||
| RUN apt-get update && apt-get install -y --no-install-recommends git \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Clone the LISA repo | ||
| ARG LISA_BRANCH=main | ||
| RUN git clone --depth 1 --branch ${LISA_BRANCH} \ | ||
| https://github.com/microsoft/lisa.git /app/lisa | ||
|
|
||
| # Overlay local MCP source so unpushed changes are included in the image | ||
| COPY . /app/lisa/mcp | ||
|
|
||
| # Install the MCP server package (with Azure blob download support) | ||
| RUN pip install --no-cache-dir "/app/lisa/mcp[azure]" | ||
|
|
||
| # Point the MCP server at the cloned repo | ||
| ENV LISA_REPO_ROOT=/app/lisa | ||
|
|
||
| EXPOSE 8080 | ||
|
|
||
| ENTRYPOINT ["lisa-mcp"] | ||
| CMD ["--transport", "sse", "--port", "8080"] |
Comment on lines
+1031
to
+1069
| patterns = [ | ||
| re.compile( | ||
| r"(\w+)\s*\|\s*(PASSED|FAILED|SKIPPED|ATTEMPTED)\s*(?:\|\s*(.*))?", | ||
| re.IGNORECASE, | ||
| ), | ||
| re.compile( | ||
| r"\[?(PASSED|FAILED|SKIPPED|ATTEMPTED)\]?\s+(?:test\s+)?(\w+)" | ||
| r"(?:\s*[:\-]\s*(.*))?", | ||
| re.IGNORECASE, | ||
| ), | ||
| re.compile( | ||
| r"(?:test|case)\s+(\S+)\s+.*?(PASSED|FAILED|SKIPPED|ATTEMPTED)" | ||
| r"(?:\s*[:\-]\s*(.*))?", | ||
| re.IGNORECASE, | ||
| ), | ||
| ] | ||
|
|
||
| seen = set() | ||
| for pattern in patterns: | ||
| for m in pattern.finditer(text): | ||
| groups = m.groups() | ||
| if groups[0].upper() in ("PASSED", "FAILED", "SKIPPED", "ATTEMPTED"): | ||
| status, name = groups[0].upper(), groups[1] | ||
| message = groups[2] if len(groups) > 2 else "" | ||
| else: | ||
| name, status = groups[0], groups[1].upper() | ||
| message = groups[2] if len(groups) > 2 else "" | ||
|
|
||
| if name not in seen: | ||
| seen.add(name) | ||
| results.append( | ||
| { | ||
| "name": name, | ||
| "status": status, | ||
| "message": (message or "").strip(), | ||
| } | ||
| ) | ||
|
|
||
| return results |
johnsongeorge-w
pushed a commit
that referenced
this pull request
Jun 1, 2026
- log_analysis: exact-match portal hostname (CodeQL); safer tar/zip extraction; honor SAS URL instead of DefaultAzureCredential. - server.py: FORWARDED_ALLOW_IPS env var (default 127.0.0.1). - Dockerfile: run as non-root mcp user; add .dockerignore. - docs_index.yaml: remove duplicate keys. - tests: reconcile tool count to 25; fix trivial assertions; --xml help.
Comment on lines
+14
to
+20
| def test_generates_valid_class(self) -> None: | ||
| from lisa_mcp.tools.test_writer import _to_snake_case | ||
|
|
||
| assert _to_snake_case("MyNewFeature") == "my_new_feature" | ||
| assert _to_snake_case("GPUValidation") == "gpu_validation" | ||
| assert _to_snake_case("SRIOVTest") == "sriov_test" | ||
| assert _to_snake_case("Simple") == "simple" |
Comment on lines
+32
to
+40
| def _call(tool_name: str, **kwargs: object) -> str: | ||
| """Invoke a registered MCP tool by name and return its string result.""" | ||
| tools = {t.name: t for t in mcp._tool_manager.list_tools()} | ||
| assert ( | ||
| tool_name in tools | ||
| ), f"Tool '{tool_name}' not found. Available: {sorted(tools)}" | ||
| # Access the underlying function | ||
| fn = tools[tool_name].fn | ||
| return fn(**kwargs) |
johnsongeorge-w
force-pushed
the
johgeorg/feat_lisa_mcp_support
branch
from
cb80089 to
300225d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a Model Context Protocol (MCP) server that gives AI assistants deep knowledge of LISA conventions for test authoring, log analysis, runbook management, debugging, and framework exploration.
Package structure (lisa_mcp/):
CLI with stdio/SSE transport
lisa_write_test detects existing test suites and methods matching the query, displaying method names, line numbers, and descriptions so the caller LLM can add to an existing suite instead of creating duplicates.
Also includes:
Description
Related Issue
Type of Change
Checklist
Test Validation
Key Test Cases:
Impacted LISA Features:
Tested Azure Marketplace Images:
Test Results