Skip to content

Spec: usage-telemetry instrumentation strategy for MCP (PostHog) #6228

Description

@JSONbored

Problem

Neither packages/loopover-mcp/bin/loopover-mcp.js nor src/mcp/server.ts records which tools actually get called, how often, or by whom (local vs. remote caller) — confirmed via this session's audit (no recordEvent/trackEvent/incr( calls, no PostHog or equivalent anywhere in either file). MCP tool prioritization — which of the ~150 combined tools across both servers are worth polishing, which are dead weight — is currently pure guesswork.

Area

packages/loopover-mcp/bin/loopover-mcp.js, src/mcp/server.ts, a new telemetry sink (PostHog, per the maintainer's stated preference, or this repo's existing Sentry/Analytics-Engine patterns if a lighter-weight option fits better — see Open Questions).

Proposal (open questions a design pass needs to answer — not yet decided)

  • What gets tracked, precisely: tool name, caller type (local stdio vs. remote HTTP), success/failure, and coarse latency are the obvious safe fields. Explicitly must NOT include: tool call arguments (may contain repo names a user hasn't consented to associate with telemetry, PR content, or anything from the "never uploaded" list in this repo's own Privacy Boundary section of README.md), source content, wallet/hotkey/trust-score data.
  • Where it's wired: a single instrumentation point per server (wrapping tool dispatch) is far preferable to per-tool instrumentation — check whether both servers already have one common dispatch chokepoint to hook, or whether one/both need a small refactor first to create one.
  • Opt-out: does a self-hoster get a way to disable this (matching this repo's existing pattern of every LOOPOVER_REVIEW_* flag shipping OFF by default, per README.md's Review Capabilities section)? Usage telemetry for a locally-run tool is a different trust question than for the hosted remote server — the local CLI probably needs an explicit opt-in, not opt-out, given it runs on a user's own machine.
  • Vendor: confirm PostHog specifically (vs. Cloudflare Analytics Engine, which this repo already uses elsewhere per src/services/) — check whether a PostHog account/project already exists for this org, and what data-residency/retention implications it has, before wiring code.

Deliverables

  • A decided design: tracked-field allowlist, instrumentation chokepoint(s), opt-in/opt-out model for the local CLI, and confirmed vendor.
  • A written spec that a future contributor-open implementation issue can be scoped from.

Resources

  • README.md's "Privacy Boundary" section (the existing commitments this must not violate)
  • packages/loopover-mcp/bin/loopover-mcp.js, src/mcp/server.ts (where instrumentation would be wired)

Boundaries

  • maintainer-only until the tracked-field allowlist and opt-in/opt-out model are decided — this is a privacy-boundary decision, not mechanical instrumentation.
  • Must never track tool-call arguments, source content, or anything already excluded from public/telemetry surfaces elsewhere in this repo.
  • The local CLI's telemetry must default to a stance at least as privacy-preserving as this repo's existing LOOPOVER_REVIEW_* flag convention (off by default) unless the design explicitly and deliberately decides otherwise, with a stated reason.

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions