Skip to content

fbratten/arbiter

MCP Server Checker

Validate MCP servers for protocol compliance, code quality, best practices, and LLM ergonomics.

Features

  • Protocol Compliance: Validates JSON-RPC 2.0 format, handshake sequence, capability negotiation
  • Functional Testing: Tool discovery, tool execution, error handling
  • Maturity Scoring: Two-score system (Compliance + Maturity) with badge levels
  • Multiple Profiles: Lenient, Standard, Strict, Paranoid validation levels

Installation

cd /path/to/mcp-server-checker
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -e ".[dev]"

Quick Start

As MCP Server

Add to your .mcp.json:

{
  "mcp-server-checker": {
    "command": "bash",
    "args": [
      "-c",
      "cd /path/to/mcp-server-checker && .venv/bin/python -m mcp_server_checker.server"
    ]
  }
}

Using the Tools

# Check a server
result = await check_server(
    server_path="python -m my_mcp.server",
    profile="standard"
)

# List available checks
checks = list_checks()

# Get validation profiles
profiles = get_profiles()

Validation Profiles

Profile Maturity Minimum Use Case
Lenient 50% Early development
Standard 70% Production servers
Strict 85% High-quality servers
Paranoid 95% Maximum compliance

Badge Levels

Badge Requirements
CERTIFIED Compliance PASS, Maturity 95%+
PRODUCTION_READY Compliance PASS, Maturity 85%+
FUNCTIONAL Compliance PASS, Maturity 70%+
DEVELOPMENT Compliance PASS, Maturity 50%+
NON_COMPLIANT Compliance FAIL

MVP Phases - ALL COMPLETE

All 9 MVPs are implemented:

  • MVP-1: Protocol + functional (handshake, tool calls, basic errors)
  • MVP-2: Transport + lifecycle (stdout corruption, shutdown, cancel, ping)
  • MVP-3: Utilities (logging, progress, pagination)
  • MVP-4: Content + contract (schema-runtime tests)
  • MVP-5: Code analysis (structure, docs, patterns)
  • MVP-6: Security (secrets, injection, path traversal)
  • MVP-7: Host compatibility (Claude Desktop, env startup)
  • MVP-8: LLM ergonomics (naming, descriptions, schema)
  • MVP-9: Scoring + fix packs (weighted scoring, remediation)

Fix Packs

Remediation suggestions grouped by effort:

Pack Effort Focus
docs_baseline ~30 min Documentation
schema_hardening ~1 hr Tool schemas
error_hygiene ~2 hr Error handling
security_baseline ~2 hr Security
lifecycle_compliance ~3 hr MCP lifecycle
llm_polish ~1 hr LLM ergonomics
transport_hardening ~1 hr Transport layer

Running Tests

pytest tests/ -v  # 173 tests

Specification

Based on MCP Protocol Specification 2025-11-25.

See KB/project-background-specificationV2.md for the full specification document.

Showcase

See the arbiter showcase site for interactive demos and documentation.

License

MIT

About

MCP Server Validator - protocol compliance, quality, and LLM ergonomics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages