Runtime behavior verification for curated servers — fit for ToolHive? #5765
Unanswered
BhaveshThapar
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi all, I've been following ToolHive, and the curated-trusted-registry model (letting teams decide which MCP servers to trust and running them as containers) feels like exactly the right shape for the problem. So this is partly a feedback question, partly a possible integration idea.
I built an open-source harness that runs MCP servers in a disposable sandbox under strace and records what they do at runtime, outbound network destinations, file access, env reads, with a canary credential seeded in to catch exfiltration. It produces a per-server, per-version behavior manifest, the point being you can diff them across releases to catch a server that changes behavior after it's been approved (the rug-pull case). I ran it across 70 servers (top 20 on npm + 50 long-tail): 67 clean, 3 make outbound calls at startup (all consistent with their function), 1 reads /etc/passwd (verified benign, a standard glibc user lookup, contents never leave the process). Repo + manifests: https://github.com/BhaveshThapar/mcp-audit.
Why I'm posting here specifically: ToolHive answers "which servers should we trust and run," and this seems to answer the adjacent question, "what do those trusted servers actually do at runtime, and has their behavior drifted since we approved them." Static curation and runtime behavioral verification feel complementary rather than overlapping.
Two questions for the maintainers/community:
Cross-version diffing for rug-pull detection is the next piece I'm building. Happy to be told this is the wrong direction.
All reactions