chore(mcp-analytics): refresh mcp analytics skills - #70743
Merged
Conversation
The exploring-mcp-sessions skill still told agents to hand-write HogQL for listing sessions and reading a session's tool calls. Both have had typed tools since #67315 and #68169, so the skill was a version behind the tools it documents. Lead with mcp-analytics-sessions-list / -sessions-tool-calls, and keep SQL for the cases the typed tools genuinely don't cover: errored-session filtering (the session list has no error filter or error count), effective tool names inside a session (the tool-calls endpoint returns the raw $mcp_tool_name), cross-session aggregation, and projects without the mcp-analytics flag, since execute-sql is ungated while the typed tools are not. Also fixes claims in the sibling skills that no longer held: models-mcp.md said the session REST tools were disabled, and exploring-mcp-tool-quality said the frontend was the source of truth for harness bucketing when it is resolved server-side in mcp_harness.py. Tool names are now spelled out in full rather than elided, so an agent can paste them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gesh
added
the
skip-agent-review
gesh
added
the
skip-agent-review
gesh
marked this pull request as ready for review
July 14, 2026 14:16
Contributor
|
Reviews (1): Last reviewed commit: "chore(mcp-analytics): refresh mcp analyt..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…d_count Two corrections from review, both verified against the backend. query-mcp-harness-breakdown's optional toolName goes through tool_scope_exprs, which adds a $mcp_source = 'posthog_mcp_analytics' predicate alongside the effective-tool one. Scoping to a tool therefore drops old-SDK and third-party calls, so a harness can be undercounted. Say so, and point one-tool cuts across all SDK sources at execute-sql. The session list returns distinct_id_count but logic.py hardcodes it to 0, so an agent reading it would report zero distinct ids for every session. Drop it from the documented row and call out that it is never populated, rather than staying silent about a field that is still in the response. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-analytics-skills # Conflicts: # products/posthog_ai/skills/querying-posthog-data/references/models-mcp.md
lucasheriques
approved these changes
Jul 16, 2026
rnegron
pushed a commit
that referenced
this pull request
Jul 16, 2026
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
exploring-mcp-sessionsskill was a version behind the tools it documents. It told agents to hand-write HogQL for listing sessions and reading a session's tool calls, but both have had typed tools since #67315 (mcp-analytics-sessions-list) and #68169 (mcp-analytics-sessions-tool-calls). The skill was last touched before either landed.Reviewing the sibling skills turned up more that no longer held.
models-mcp.md, the shared reference all four skills point at, still claimed the session REST tools were disabled. Andexploring-mcp-tool-qualitytold agents the frontend was the source of truth for harness bucketing, which would send them toharnessRegistry.ts(a label to logo lookup) instead ofmcp_harness.py, where it actually happens.Changes
exploring-mcp-sessionsnow leads with the typed tools and keeps SQL only for what they genuinely don't cover:mcp-analytics-sessions-tool-callsreturns the raw$mcp_tool_nameand does not resolve the inner tool of a single-exec wrapper callmcp-analyticsflag, sinceexecute-sqlis ungated while the typed tools are notNote
Two behaviors worth knowing, now documented in the skill. Both detail tools default to a 7-day lookback, so a session older than that returns empty unless you pass its
session_startasdate_from- the failure looks like missing data but is not. And the typed tools report the raw tool name, which contradicts the "always use the effective tool name" rule the tool-quality skill leads with. That inconsistency may be worth fixing in the endpoint rather than documenting around.The rest is corrections: the disabled-tools claim in
models-mcp.md, the harness-bucketing source of truth inexploring-mcp-tool-quality, a sessions table in the shared reference to match the per-tool one above it, and tool names spelled out in full everywhere instead of elided as...-tool-calls, so an agent can paste them.Docs only. No production code touched.
How did you test this code?
hogli lint:skillspasses, 117 skills. It validates that every tool a skill references exists in the catalog, which is what catches the stale names. The two warnings it prints are pre-existing and inengineering_analytics, unrelated to this PR.Worth flagging that this linter only catches references to tools that don't exist. It cannot catch a skill failing to mention tools that do - which is exactly the rot this PR fixes, and why it went unnoticed for two releases.
I did not run the MCP eval harness. It would not exercise these skills anyway: evals score whether an agent picks the right tool against a live server, and they never read skills.
Automatic notifications
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
I asked Claude to update
exploring-mcp-sessionsand then review whether the other MCP skills and tools needed the same treatment. It read the tools.yaml, the generated tool definitions, and the query runners rather than trusting the skills' own descriptions, which is how the staleness got pinned to specific PRs and how the raw-vs-effective tool name gap turned up.A few things were deliberately left out of this PR:
sessions-listtool description says "one row per$mcp_session_id" while the SQL groups on$session_id. Same id, so this is a spelling inconsistency rather than a bug, but fixing it means regenerating OpenAPI and the MCP tools. Separate PR.query-mcp-tool-statsreturns a field namedconversationswhilequery-mcp-tool-daily-statsreturnssessionsfor the same thing, and the tool description promises "sessions". Renaming touches the schema, the runner, and the frontend. Separate PR.improving-mcp-toolsguardrail it bumpsversionand goes on its own.Claude initially drew a sessions-vs-conversations distinction from the
_CONVERSATION_IDcoalesce in the query runners. I corrected it: same id, and "conversation" means something else in this product. That's now consistent across the skills.