mcp-probe

Audit an MCP server before you connect an agent to it.

Point mcp-probe at a server command. It launches the server over stdio, speaks the MCP handshake, reads tools/list, and reports what an agent could do with the tools it found. It never calls a tool.

Summary
  tools          8
  write tools    4
  gated          1 of 4
  findings       13 (4 high, 7 medium, 2 low)
  verdict        Do not hand this to an unattended agent until the 4 high
                 severity findings are fixed.

Why this exists

Connecting an agent to an MCP server hands a model a set of buttons. The protocol has a place to say which of those buttons are safe to press without asking anyone: the readOnlyHint, destructiveHint, idempotentHint and openWorldHint annotations. Servers are not required to fill them in, and the ones that do are not required to be right.

What a client sees, and what a model chooses from, is a list of names, descriptions and JSON schemas. That interface is where the decision to call delete_note gets made. mcp-probe reads the same interface the model reads, and reports the places where it does not say enough, or says the wrong thing.

The checks are not a security scanner. They answer a narrower question: if an agent were handed these tools and left alone with them, what could it do before anyone noticed.

Install

git clone https://github.com/keivanmalhani/mcp-probe
cd mcp-probe
python3 -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"

Python 3.11 or newer. No runtime dependencies. pipx install . from a clone works if you want the command without a virtual environment in your shell.

Use

mcp-probe run -- python3 examples/unsafe_notes_server.py
mcp-probe run -- npx -y @modelcontextprotocol/server-filesystem /tmp
mcp-probe run --config ~/.config/mcp/servers.json
mcp-probe check captured/tools.json

Everything after a bare -- is the server command. Write it that way so the server's own flags are not read by mcp-probe.

Flag	What it does
--json	One JSON object on stdout instead of the text report
--fail-on high|medium|low|none	Exit 1 when a finding reaches this level. Default high
--config PATH	Read a standard MCP client config and probe every stdio server in it
--name NAME	Label for the server in the report
--cwd PATH	Working directory for the server process
--timeout SECONDS	How long to wait for each reply. Default 20
--version	Print the version

Exit codes:

Code	Meaning
0	No findings at or above --fail-on
1	Findings at or above --fail-on
2	A server could not be reached, or the arguments were wrong

check reads a tools/list response that was captured earlier, so the audit runs offline and in CI with no server process. It accepts a full JSON-RPC response, an object with a tools key, or a bare list of tools. Pass several files to check them together, which is how name collisions are found without launching anything.

In a pipeline:

- run: mcp-probe check tools.json --fail-on high

What a run looks like

Real output, from mcp-probe run -- python3 examples/unsafe_notes_server.py against the example server in this repository. That server is a working stdio MCP server whose interface carries the problems the rules look for, plus two tools that are fine.

mcp-probe 0.1.0

Server: unsafe_notes_server
  command: python3 examples/unsafe_notes_server.py
  reports itself as: unsafe-notes 0.1.0
  8 tools

  [HIGH] annotation-contradiction  delete_note
    what: The tool is annotated readOnlyHint: true but its name contains
          "delete", which names a change to state.
    why:  A wrong annotation is worse than a missing one, because a client
          that respects annotations will let this tool run without asking
          anyone.
    fix:  Either set readOnlyHint to false and add destructiveHint, or rename
          the tool so the name matches what it actually does.

  [HIGH] unguarded-write  run_shell
    what: The tool name contains "run", which implies it changes state, and no
          parameter in its input schema looks like a confirmation.
    why:  An agent can call this tool and change something on the first turn
          it decides to, with nothing in the interface asking a person first.
    fix:  Add a required parameter such as "confirm" that the caller has to
          fill with the exact identifier being changed, and reject the call
          server-side when it does not match. If the tool really only reads,
          set readOnlyHint: true.

  [HIGH] unguarded-write  send_digest
    what: The tool name contains "send", which implies it changes state, and
          no parameter in its input schema looks like a confirmation.
    why:  An agent can call this tool and change something on the first turn
          it decides to, with nothing in the interface asking a person first.
    fix:  Add a required parameter such as "confirm" that the caller has to
          fill with the exact identifier being changed, and reject the call
          server-side when it does not match. If the tool really only reads,
          set readOnlyHint: true.

  [HIGH] unguarded-write  write_note
    what: The tool name contains "write", which implies it changes state, and
          no parameter in its input schema looks like a confirmation.
    why:  An agent can call this tool and change something on the first turn
          it decides to, with nothing in the interface asking a person first.
    fix:  Add a required parameter such as "confirm" that the caller has to
          fill with the exact identifier being changed, and reject the call
          server-side when it does not match. If the tool really only reads,
          set readOnlyHint: true.

  [MED ] boolean-confirmation  delete_note
    what: The confirmation parameter "confirm" is a boolean, so the whole gate
          is one token the model writes for itself.
    why:  A boolean proves nothing, because true is the cheapest token a model
          can emit and it is the only value that does anything, so the
          parameter filters out no call that the model already intended to
          make.
    fix:  Make "confirm" a string that has to carry information the model
          cannot invent, such as the exact name or id of the target, and
          compare it server-side. Where the risk is real, move the gate out of
          the arguments and into a human approval step in the client.

  [MED ] missing-annotations  export_all
    what: The tool carries none of the four annotation hints: readOnlyHint,
          destructiveHint, idempotentHint, openWorldHint.
    why:  The agent has no machine-readable signal of what this tool does, so
          a client cannot route it to a confirmation prompt or hold it back
          from an unattended run.
    fix:  Add the hints that apply. A pure reader needs readOnlyHint: true.
          Anything that changes state needs destructiveHint and idempotentHint
          set honestly, plus openWorldHint if it reaches outside the local
          system.

  [MED ] unbounded-path  read_note
    what: The parameter "path" takes a free-form string location with no enum,
          no pattern and no description naming a root or an allowlist.
    why:  Nothing in the interface tells the agent where it is allowed to
          point this, so a path assembled from a web page or a file it just
          read is as acceptable as one you meant.
    fix:  Document the allowed root in the description of "path", or constrain
          it with a pattern or an enum, and enforce the same boundary in the
          handler.

  [MED ] missing-annotations  run_shell
    what: The tool carries none of the four annotation hints: readOnlyHint,
          destructiveHint, idempotentHint, openWorldHint.
    why:  The agent has no machine-readable signal of what this tool does, so
          a client cannot route it to a confirmation prompt or hold it back
          from an unattended run.
    fix:  Add the hints that apply. A pure reader needs readOnlyHint: true.
          Anything that changes state needs destructiveHint and idempotentHint
          set honestly, plus openWorldHint if it reaches outside the local
          system.

  [MED ] unbounded-path  run_shell
    what: The parameter "cwd" takes a free-form string location with no enum,
          no pattern and no description naming a root or an allowlist.
    why:  Nothing in the interface tells the agent where it is allowed to
          point this, so a path assembled from a web page or a file it just
          read is as acceptable as one you meant.
    fix:  Document the allowed root in the description of "cwd", or constrain
          it with a pattern or an enum, and enforce the same boundary in the
          handler.

  [MED ] missing-annotations  write_note
    what: The tool carries none of the four annotation hints: readOnlyHint,
          destructiveHint, idempotentHint, openWorldHint.
    why:  The agent has no machine-readable signal of what this tool does, so
          a client cannot route it to a confirmation prompt or hold it back
          from an unattended run.
    fix:  Add the hints that apply. A pure reader needs readOnlyHint: true.
          Anything that changes state needs destructiveHint and idempotentHint
          set honestly, plus openWorldHint if it reaches outside the local
          system.

  [MED ] unbounded-path  write_note
    what: The parameter "path" takes a free-form string location with no enum,
          no pattern and no description naming a root or an allowlist.
    why:  Nothing in the interface tells the agent where it is allowed to
          point this, so a path assembled from a web page or a file it just
          read is as acceptable as one you meant.
    fix:  Document the allowed root in the description of "path", or constrain
          it with a pattern or an enum, and enforce the same boundary in the
          handler.

  [LOW ] thin-description  export_all
    what: The description is 7 characters long: "Export.".
    why:  An agent picks tools by reading descriptions, so a tool that does
          not describe itself gets called by guesswork off its name.
    fix:  Write a description that states what the tool changes, what it
          returns, and the case where it should not be used.

  [LOW ] thin-description  write_note
    what: The description is 13 characters long: "Write a note.".
    why:  An agent picks tools by reading descriptions, so a tool that does
          not describe itself gets called by guesswork off its name.
    fix:  Write a description that states what the tool changes, what it
          returns, and the case where it should not be used.

Summary
  tools          8
  write tools    4
  gated          1 of 4
  findings       13 (4 high, 7 medium, 2 low)
  verdict        Do not hand this to an unattended agent until the 4 high
                 severity findings are fixed.

Exit code 1.

The same repository has a server with the same feature set and an interface that passes, so you can see what a clean report looks like. From mcp-probe run --fail-on low -- python3 examples/clean_notes_server.py:

mcp-probe 0.1.0

Server: clean_notes_server
  command: python3 examples/clean_notes_server.py
  reports itself as: clean-notes 0.1.0
  6 tools
  no findings

Summary
  tools          6
  write tools    3
  gated          3 of 3
  findings       0 (0 high, 0 medium, 0 low)
  verdict        Clean: every tool is annotated and described, and each of the
                 3 write tools carries a confirmation parameter.

Exit code 0.

With --json, one object on stdout, trimmed here:

{
  "tool": "mcp-probe",
  "version": "0.1.0",
  "servers": [
    {
      "name": "unsafe_notes_server",
      "command": ["python3", "examples/unsafe_notes_server.py"],
      "identity": "unsafe-notes 0.1.0",
      "ok": true,
      "error": null,
      "tool_count": 8,
      "tools": ["list_notes", "read_note", "write_note"]
    }
  ],
  "findings": [
    {
      "rule": "annotation-contradiction",
      "severity": "high",
      "server": "unsafe_notes_server",
      "tool": "delete_note",
      "problem": "The tool is annotated readOnlyHint: true but its name contains \"delete\", which names a change to state.",
      "impact": "A wrong annotation is worse than a missing one, because a client that respects annotations will let this tool run without asking anyone.",
      "fix": "Either set readOnlyHint to false and add destructiveHint, or rename the tool so the name matches what it actually does."
    }
  ],
  "summary": {
    "tools": 8,
    "write_tools": 4,
    "gated_write_tools": 1,
    "findings": 13,
    "high": 4,
    "medium": 7,
    "low": 2,
    "verdict": "Do not hand this to an unattended agent until the 4 high severity findings are fixed."
  },
  "fail_on": "high",
  "exit_code": 1
}

The rules

Every finding carries a severity, the tool it belongs to, what is wrong, why it matters in one sentence, and a concrete fix.

unguarded-write (high)

A tool whose name or description contains one of write, delete, remove, apply, update, send, create, move, rename, execute or run, and whose input schema has no parameter that reads like a confirmation. A parameter counts as a confirmation when its name contains confirm, approve, approval, acknowledge or consent.

The reasoning: a model decides to call a tool and calls it in the same step. Where a person would pause between deciding to delete something and deleting it, there is no pause. The only place a pause can be expressed to the model is in the schema of the call itself, so a mutating tool with nothing in its schema has no pause at all.

The name is the stronger signal and is checked first. Descriptions are read only when the name does not lead with a reading verb such as get, list, read, search or query, because descriptions talk about results ("returns the files that were created today") as often as about effects.

A tool that declares readOnlyHint: true is taken at its word and skipped, unless one of the contradiction checks below fires. That is what keeps run_query from being flagged. A run_query with no annotation at all is flagged, because nothing in the interface separates it from a run_command.

missing-annotations (medium)

None of readOnlyHint, destructiveHint, idempotentHint or openWorldHint is present.

The reasoning: annotations are the only machine-readable statement a server makes about what a tool does. A client that wants to auto-approve reads and prompt on writes has nothing to sort on when they are absent, so either every call gets a prompt, which trains people to click through, or none does. A hint set to false counts as present. Saying "this is not destructive" is a statement; saying nothing is not.

annotation-contradiction (high)

Any of these:

• readOnlyHint: true on a tool whose name contains a verb that cannot be read-only: write, delete, remove, update, send, create, move, rename, drop, destroy, purge, overwrite, truncate, upload, publish, revoke.
• readOnlyHint: true together with destructiveHint: true.
• destructiveHint: false on a tool whose name contains delete, remove, drop, destroy, purge, truncate, wipe or erase.

The reasoning: this is worse than saying nothing. A missing annotation makes a careful client fall back to asking. A wrong one makes it skip the question. The verb list here is deliberately narrower than the one above, because run, execute and apply are ambiguous enough that a server calling them read-only may be telling the truth.

unbounded-path (medium)

A string parameter whose name ends in a location word (path, dir, directory, folder, file, filename, dest, destination, cwd and their plurals) with no enum, no pattern, and no description that names a root, an allowlist, a sandbox, a workspace or a base directory. An array of strings is judged by its items.

The reasoning: an agent assembles arguments from whatever is in its context, including text it read from a file or a web page a moment earlier. If the interface never states where the tool is allowed to point, every path is as acceptable as every other one, and the boundary exists only in the handler where the model cannot see it. Naming the root in the description is enough to clear this rule, because it puts the boundary where the model reads.

The check looks at the trailing word, so file_path and dest are locations while file_content and file_id are not.

boolean-confirmation (medium)

A confirmation-shaped parameter typed boolean.

The reasoning: a boolean confirmation proves nothing. true is the cheapest token a model can emit, it is the only value that does anything, and the model supplies it in the same breath as the decision to call the tool. The parameter filters out no call that the model already intended to make. It also reads as a safety feature in a code review, which is the part that does damage.

A confirmation is worth something when it carries information the model has to go and get: the exact path, the row id, the branch name, compared server-side against the target. Where the risk is real, the gate belongs outside the arguments entirely, in a human approval step in the client.

This rule sits behind the write rule on purpose. A boolean confirm satisfies unguarded-write, because the interface does ask for something, and then this rule says what that something is worth.

thin-description (low)

No description, or a description under 20 characters after whitespace is stripped.

The reasoning: a model picks a tool by reading descriptions. A tool that does not describe itself gets chosen by guesswork from its name, which is how a sync gets called when a fetch was meant. The threshold is arbitrary and deliberately low. It catches "Export.", not thin prose.

name-collision (high or medium)

The same tool name offered by more than one server in a single run. High when one of the colliding tools can write, medium otherwise.

The reasoning: an agent selects a tool by name. When two servers claim the same name the call lands wherever the client's merge order puts it, which is not something either server controls or documents. Two delete_note tools backed by different stores is the case worth catching before it happens.

Checked with --config, or by passing several captured files to check.

What it will not do

This is static analysis of a declared interface. It reads names, schemas and annotations. It never calls a tool, and it cannot tell you whether the implementation honours its own annotations. A tool that declares readOnlyHint: true and then writes to disk gets a clean report from mcp-probe, and the clean report is wrong. Verifying that would mean running the tool and watching what it touches, which is a different program with different risks.

Everything else it does not do:

• A clean report is not a safety certificate. It means the interface describes itself well enough for a client and a model to act on it.
• The write detection is a list of English verbs. A tool named stage_batch that deletes rows is missed. Rename detection is not semantic analysis.
• A confirmation parameter is counted as present, not as enforced. The server is free to ignore the value it receives, and mcp-probe cannot see that.
• Only the top-level properties of inputSchema are inspected. A path nested inside an object, or behind a $ref, is not seen.
• Only the stdio transport is supported. HTTP and SSE servers in a config file are skipped with a note on stderr.
• Only tools are audited. Prompts, resources, sampling and roots are not.
• Tool descriptions are not checked for prompt injection. A description that instructs the model to ignore its operator is a real attack and a different kind of check.
• Exit code 2 means the audit did not finish. It is not a verdict about the server.

How it works

Three pieces, kept apart on purpose.

src/mcp_probe/rules.py is the engine. It is a pure function from a list of tool definitions to a list of findings. Nothing in it starts a process, opens a socket or reads a file. That is why the rule tests need no server, why check works offline, and why a captured tools.json and a live probe produce the same findings. There is a test that asserts exactly that.

src/mcp_probe/transport.py is a small MCP client: launch the process, send initialize, send notifications/initialized, page through tools/list, kill the process. Newline-delimited JSON-RPC over the child's stdin and stdout, written on subprocess and json.

There is no dependency on the mcp package, and that is deliberate. An auditor that imports the same library as the thing it audits can be broken by that library. A server that ships a conflicting pin, or an SDK version with a bug in its client, would stop the audit from running at all, on exactly the servers where the audit is most worth running. A few hundred lines of subprocess and json removes the coupling, and the wire format is documented.

The client is tolerant where a strict client would give up. A banner printed on stdout is skipped and reported. A notification or a server-initiated request arriving before the reply is handled rather than mistaken for the reply. Neither of those is legal, both of them happen, and an auditor that cannot read a slightly broken server is not much of an auditor.

src/mcp_probe/report.py renders. Plain ASCII, no escape codes, so a run redirected into a CI log reads the same as a run in a terminal.

Development

python3 -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
pytest -q

220 tests. Every rule has a test for the case that must fire and a test for the neighbouring case that must not, because a checker that flags everything is as useless as one that flags nothing. The transport tests launch real subprocesses from tests/fixtures: a well behaved server, one that paginates, one that dies on startup, one that never answers, one that prints a banner before speaking protocol, one that returns a JSON-RPC error, and one that talks over its own reply.

CI runs the suite on Python 3.11 and 3.12, audits both example servers, and fails if any file in the repository contains a byte outside printable ASCII.

License

MIT. Copyright (c) 2026 Keivan Malhani.