Add anonymous usage telemetry to the MCP server - #22
Merged
Conversation
Member
Author
|
/canary |
KetanReddy
approved these changes
Aug 7, 2026
The MCP server reported nothing about its own usage, so the only signal for "is anyone running this?" was npm download counts — which conflate CI, mirrors and repeat installs, and say nothing about whether the server ever started or whether the devtools transport actually connected. Instrument it with @posthog/mcp, which reports server start, tool calls, latency and errors, plus which MCP client is driving it. Privacy is enforced by an allowlist rather than a denylist. The SDK captures tool arguments and responses unconditionally — there is no option to disable it, and its built-in sanitizer only matches sensitive key *names*, so a field named `flowContent` would be transmitted verbatim. Devtools tool arguments carry player/plugin ids and invoke_action payloads, i.e. potentially customer flow data. Every event is therefore filtered through an explicit set of known-safe property names, installed as a posthog-node client-level `before_send`: a chokepoint the SDK cannot route around, since we own the client and it does not. A denylist would silently start leaking the first time a 0.x release added a new payload property. The SDK's `context` option is disabled. It defaults to ON and would inject a required `context` parameter into every tool schema, mutating our public API to ask the calling agent to narrate user intent. Identity is a random UUID in ~/.player-ui-devtools/install.json — deliberately not beside the flipper refcount in the OS temp dir, which is periodically reaped and would turn "unique installs" into "unique boots". When the home directory is unwritable the id is null and telemetry is skipped, rather than minting a per-run id that would inflate install counts. Telemetry is on by default and opts out via PLAYER_DEVTOOLS_TELEMETRY_DISABLED or the cross-vendor DO_NOT_TRACK convention. The shipped project key is a placeholder, so this is inert until one is provisioned. @posthog/mcp requires Node ^20.20.0 || >=22.22.0, which excludes the previous 22.15.1 toolchain pin; rules_nodejs is bumped to 6.7.5 as 6.4.0 does not know any Node newer than 22.15.1. Also replaces the hardcoded, stale server version "0.0.1" with the stamped __VERSION__ global, and flushes buffered events in stop() before bin/run calls process.exit(0).
sugarmanz
force-pushed
the
feat/mcp-telemetry
branch
from
August 7, 2026 02:28
d894223 to
85e440a
Compare
Member
Author
|
/canary |
Mirrors how __VERSION__ is stamped: workspace-status.sh emits STABLE_POSTHOG_KEY from the POSTHOG_PROJECT_KEY environment variable, and tsup substitutes it into the __POSTHOG_KEY__ global. CI supplies the value, so no key lives in the repo. Unstamped builds (local, tests, non-release CI) resolve the global to an empty string and telemetry stays disabled — verified in the emitted bundle: a release build inlines `POSTHOG_PROJECT_KEY = "phc_..."` while a normal build keeps the runtime typeof guard. Also refuses any key that is not a `phc_` public project token. A `phx_` personal or `phs_` project-secret key is a real credential, and stamped values land in build artifacts and the shared BuildBuddy cache; if one is ever wired into CI by mistake the build ships inert rather than leaking it.
The ingestion key is a build/CI concern, so the user-facing section now says so explicitly — no account, no key, no configuration — and notes that non-release builds send nothing, which is the likely first question when someone builds locally and sees no events. Moves the key plumbing into a collapsed maintainer note: which environment variable CI sets, how it reaches the bundle, and why only public `phc_` keys are accepted. Also corrects the event list to the names actually emitted ($mcp_initialize, $mcp_tool_call, $mcp_tools_list, $exception), verified against a captured session rather than the SDK's constants.
Member
Author
|
/canary |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The MCP server reported nothing about its own usage, so the only signal for "is anyone running this?" was npm download counts — which conflate CI, mirrors and repeat installs, and say nothing about whether the server ever started or whether the devtools transport connected.
Instruments it with
@posthog/mcp, reporting server start, tool calls, latency, errors, and which MCP client is driving it.Privacy is enforced by an allowlist, not a denylist
The SDK captures tool arguments and responses unconditionally — there is no option to disable it (verified against 0.10.8 source:
buildCapturedMcpParametersis called at three sites with no gate), and its built-in sanitizer only matches sensitive key names, so a field namedflowContentwould be transmitted verbatim. Devtools tool arguments carry player/plugin ids andinvoke_actionpayloads, i.e. potentially customer flow data.Every event is therefore filtered through an explicit set of known-safe property names, installed as a
posthog-nodeclient-levelbefore_send— a chokepoint the SDK cannot route around, since we construct thePostHogclient and it explicitly does not own one. A denylist would silently start leaking the first time a 0.x release added a new payload property; the allowlist drops unknown keys by default.The SDK's
contextoption is disabled. It defaults to ON and would inject a requiredcontextparameter into every tool schema, mutating our public API to ask the calling agent to narrate user intent in prose.Identity
A random UUID at
~/.player-ui-devtools/install.json— deliberately not beside the flipper refcount in the OS temp dir, which is periodically reaped and would turn "unique installs" into "unique boots". When the home directory is unwritable the id isnulland telemetry is skipped, rather than minting a per-run id that would inflate counts.Toolchain
@posthog/mcprequires Node^20.20.0 || >=22.22.0, which excludes the previous22.15.1pin.rules_nodejsis bumped 6.4.0 → 6.7.5 because 6.4.0 knows no Node newer than 22.15.1.Also replaces the stale hardcoded server version
"0.0.1"with the stamped__VERSION__, and flushes buffered events instop()beforebin/runcallsprocess.exit(0).Verification
42 tests across 7 files;
//devtools/mcp:all(eslint + typecheck + vitest) all PASSED. The redaction suite asserts on the serialized payload — so a nested leak fails the test — and includes a case proving an unknown future property ($mcp_raw_request) is dropped.Not yet done: the PostHog project key is still a placeholder, so telemetry is inert until one is provisioned. Once it is, verify redaction against a local listener before real use.
Release Notes
The MCP server now reports anonymous usage analytics — server starts, tool names, latency, and errors — so we can see how widely devtools is used and whether it works in the field.
Tool arguments and tool responses are never transmitted. Outgoing events are filtered through an allowlist of known-safe properties, so flow content cannot leave your machine.
Identity is a random UUID stored at
~/.player-ui-devtools/install.json; delete the file to reset it. Opt out withPLAYER_DEVTOOLS_TELEMETRY_DISABLED=1or the cross-vendorDO_NOT_TRACK=1.Breaking:
@player-devtools/mcpnow requires Node^20.20.0 || >=22.22.0.📦 Published PR as canary version:
0.14.2--canary.22.1098Try this version out locally by upgrading relevant packages to 0.14.2--canary.22.1098