Ship .mcp.json in every public repo to auto-connect coding agents to the EvalOps control plane

[haasonsaas]

Problem

EvalOps has 31 public repos. Each one is a potential funnel into the control plane. But when a coding agent (Claude Code, Codex, Cursor) opens any of these repos today, there is no signal pointing it to the EvalOps MCP server.

Context: How production MCP servers distribute

Sentry, Linear, GitHub, and Stripe all ship as a single URL. The auth happens automatically via OAuth auto-discovery (401 → DCR → browser → one click). The .mcp.json in the repo just needs the URL. Claude Code handles everything else.

Proposed implementation

1. Claude Code .mcp.json (primary)

Zero-config OAuth (recommended — matches Sentry/Linear/GitHub pattern):

{
  "mcpServers": {
    "evalops": {
      "type": "http",
      "url": "https://mcp.evalops.dev/mcp"
    }
  }
}

No auth config needed. Claude Code will:

1. Hit 401 on first connection
2. Auto-discover OAuth via Protected Resource Metadata
3. Auto-register via Dynamic Client Registration
4. Open browser → "Sign in to EvalOps with Google"
5. Store token in system keychain, refresh automatically

API key fallback (for users who already have a key):

{
  "mcpServers": {
    "evalops": {
      "type": "http",
      "url": "https://mcp.evalops.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${EVALOPS_API_KEY}"
      }
    }
  }
}

2. Codex .codex/config.toml equivalent

[mcp_servers.evalops]
url = "https://mcp.evalops.dev/mcp"

Or with API key:

[mcp_servers.evalops]
url = "https://mcp.evalops.dev/mcp"
bearer_token_env_var = "EVALOPS_API_KEY"

3. Cursor .cursor/mcp.json equivalent

Same format as Claude Code's .mcp.json.

4. Fallback for clients without HTTP transport support

For older clients that only support stdio:

{
  "mcpServers": {
    "evalops": {
      "command": "npx",
      "args": ["-y", "mcp-remote@latest", "https://mcp.evalops.dev/mcp"]
    }
  }
}

5. Agent instructions file

Each repo should include a CLAUDE.md (Claude Code) and/or AGENT.md (generic) that references the MCP server:

## EvalOps Integration

This project uses EvalOps for agent governance, metering, and memory.
The MCP server is configured in `.mcp.json` and connects automatically.

On first use, you'll be prompted to sign in with Google via your browser.
After that, authentication is automatic.

Keep it minimal — the .mcp.json does the work, the docs just explain what happened if the developer sees a browser popup.

6. Rollout strategy

Phase 1 — High-traffic repos (highest funnel value):

• cognitive-dissonance-dspy (276 stars)
• dspy-0to1-guide (211 stars)
• deep-code-reasoning-mcp (105 stars)
• dspy-micro-agent (71 stars)
• dspy-advanced-prompting (53 stars)

Phase 2 — Product repos:

• fermata, ensemble, maestro, gate, chat, cerebro, conductor
• diffscope, mocktopus, eval2otel, agent-harness

Phase 3 — All remaining public repos

7. Sync mechanism

GitHub Action in this repo that:

1. Maintains a canonical .mcp.json template
2. On template change, opens PRs against all public repos to update their .mcp.json
3. Also generates the Codex .codex/config.toml and Cursor .cursor/mcp.json variants

8. .gitignore considerations

The .mcp.json itself is safe to commit (it contains no secrets — just a URL). But repos should ensure .env files with EVALOPS_API_KEY are gitignored. Add to the sync action:

# API keys
.env
.env.local

The funnel this creates

Developer clones repo (or agent is pointed at repo)
  → Claude Code / Codex reads .mcp.json / config.toml
  → Connects to mcp.evalops.dev
  → 401 → OAuth auto-discovery → browser opens
  → Developer clicks "Sign in with Google" (one click, one time)
  → Org auto-provisioned (identity#107), token issued
  → Token stored in system keychain, auto-refreshes forever
  → Agent is registered, governed, metered, auditable
  → Developer is an EvalOps user

31 public repos × community traction = 31 always-on funnels.

Depends on

• evalops/identity#108 — OAuth AS metadata + Dynamic Client Registration (REQUIRED — Claude Code silently fails without DCR)
• evalops/identity#107 — Org auto-provisioning on first login
• Return 401 + Protected Resource Metadata for unauthenticated MCP requests agent-mcp#37 — 401 + Protected Resource Metadata response

References

• evalops/identity#110 — Epic: Zero-friction registration
• Claude Code MCP docs — .mcp.json schema, OAuth flow, headersHelper
• Codex MCP docs — TOML config, bearer_token_env_var
• Production examples: Sentry (mcp.sentry.dev/mcp), Linear (mcp.linear.app/mcp), GitHub (api.githubcopilot.com/mcp)