Skip to content

chore(llma): clarify file path param naming in skill-store skills#60807

Merged
pauldambra merged 2 commits into
masterfrom
posthog-code/llma-skills-path-param-clarity
Jun 1, 2026
Merged

chore(llma): clarify file path param naming in skill-store skills#60807
pauldambra merged 2 commits into
masterfrom
posthog-code/llma-skills-path-param-clarity

Conversation

@andrewm4894
Copy link
Copy Markdown
Member

Problem

Agents authoring and maintaining PostHog LLM analytics skills via the llma-skill-* MCP tools kept getting the bundled-file path parameter wrong — passing path to llma-skill-file-get yields a /files/undefined/ 404. The two skills that document this surface (skills-store, working-with-skills) didn't explain why the name differs between tools, so agents guessed from memory and got it wrong repeatedly.

The split is real but it's a REST artifact, not a bug:

  • file_path — on llma-skill-file-get / llma-skill-file-delete, where the path is a URL segment.
  • path — on llma-skill-file-create, the files=[{path, …}] array, and file_edits=[{path, …}], where it's a body field on a file object (sitting next to content).
  • old_path / new_path — body fields on llma-skill-file-rename.

Separately, llma-skill-archive is an enabled MCP tool that neither skill documented.

Changes

Documentation-only, no API change:

  • Added an explicit "file-path parameter naming" section to both skills stating the path vs file_path rule, with the mnemonic (anything carrying a file object uses path; the two tools that address a file by URL use file_path) and the /files/undefined/ 404 failure mode.
  • Documented llma-skill-archive — added it to the skills-store tool table and the working-with-skills decision tree, plus a short "Archiving a skill" section in each noting it hides all versions and cannot be undone.

The underlying API was left unchanged: the param names are already explicit in each tool's generated input schema, the names follow correct REST conventions, and path is already the dominant name (file object, manifest, create, file_edits). Unifying the API on path (renaming the URL capture group on get/delete) remains a possible follow-up but is a generated-client behavior change, so it was deliberately scoped out of this docs fix.

How did you test this code?

I'm an agent. No automated tests apply to these markdown skill files (the only checked-in copies are the source SKILL.md files edited here; there is no generated artifact to resync, and the services/mcp generated-tools test is unaffected). Verified the param-name claims against the source of truth — the LLMSkillViewSet URL patterns and serializers in products/ai_observability/backend/api/ and the generated MCP tool in services/mcp/src/tools/generated/ai_observability.ts.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

🤖 Agent context

Authored by PostHog Code (Claude) in response to a support thread where an agent repeatedly guessed the wrong file-path parameter name when maintaining LLM analytics skills.

Investigated end-to-end before changing anything: confirmed the two skill docs were already internally consistent and matched the API, traced the path/file_path split to the REST design (URL segment vs body field), and found llma-skill-archive was an enabled-but-undocumented tool. Considered three cleanup options — (A) unify the API on path, (B) unify on file_path, (C) document the split — and chose C as the lowest-risk fix that addresses the actual failure mode (agents guessing rather than reading schemas), with A noted as an optional follow-up. Rejected B because it would force the file-object key to file_path everywhere and create more internal inconsistency.


Created with PostHog Code

Document the path vs file_path split across the llma-skill-* tools and add the
missing llma-skill-archive tool to the skills-store and working-with-skills
skills.

The bundled-file path parameter is named file_path on file-get/file-delete
(it's a URL segment) and path on file-create / files[] / file_edits (a body
field), with old_path/new_path on rename. Agents working from memory were
guessing wrong (passing path to file-get yields a /files/undefined/ 404).
Both skills now state the rule explicitly.

llma-skill-archive is an enabled MCP tool that neither skill documented.

Generated-By: PostHog Code
Task-Id: 8ef1a9bc-33e1-48de-92b3-3035b049b876
@github-actions
Copy link
Copy Markdown
Contributor

Hey @andrewm4894! 👋

It looks like your git author email on this PR isn't your @posthog.com address (andrewm4894@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

oxfmt --check normalizes emphasis to underscores; switch *object* to _object_ in the file-path naming note so the markdown formatting check passes.

Generated-By: PostHog Code
Task-Id: 8ef1a9bc-33e1-48de-92b3-3035b049b876
@pauldambra pauldambra marked this pull request as ready for review June 1, 2026 07:38
@pauldambra pauldambra merged commit 5ef32b0 into master Jun 1, 2026
156 checks passed
@assign-reviewers-posthog assign-reviewers-posthog Bot requested a review from a team June 1, 2026 07:39
@pauldambra pauldambra deleted the posthog-code/llma-skills-path-param-clarity branch June 1, 2026 07:39
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Jun 1, 2026

Comments Outside Diff (1)

  1. products/ai_observability/skills/skills-store/SKILL.md, line 235 (link)

    P2 Bridge skill template missing llma-skill-archive in allowed-tools

    The PR adds llma-skill-archive to the tools table and adds a dedicated "Archiving a skill" section, but the local bridge skill template on this line still omits mcp__posthog__llma-skill-archive from its allowed-tools list. An agent who bootstraps the bridge skill from this template and then tries to follow the new archive documentation will get a "tool not permitted" error rather than a /files/undefined/ 404.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: products/ai_observability/skills/skills-store/SKILL.md
    Line: 235
    
    Comment:
    **Bridge skill template missing `llma-skill-archive` in `allowed-tools`**
    
    The PR adds `llma-skill-archive` to the tools table and adds a dedicated "Archiving a skill" section, but the local bridge skill template on this line still omits `mcp__posthog__llma-skill-archive` from its `allowed-tools` list. An agent who bootstraps the bridge skill from this template and then tries to follow the new archive documentation will get a "tool not permitted" error rather than a `/files/undefined/` 404.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
products/ai_observability/skills/skills-store/SKILL.md:235
**Bridge skill template missing `llma-skill-archive` in `allowed-tools`**

The PR adds `llma-skill-archive` to the tools table and adds a dedicated "Archiving a skill" section, but the local bridge skill template on this line still omits `mcp__posthog__llma-skill-archive` from its `allowed-tools` list. An agent who bootstraps the bridge skill from this template and then tries to follow the new archive documentation will get a "tool not permitted" error rather than a `/files/undefined/` 404.

Reviews (1): Last reviewed commit: "chore(llma): fix markdown emphasis style..." | Re-trigger Greptile

@deployment-status-posthog
Copy link
Copy Markdown

deployment-status-posthog Bot commented Jun 1, 2026

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-06-01 08:01 UTC Run
prod-us ✅ Deployed 2026-06-01 08:13 UTC Run
prod-eu ✅ Deployed 2026-06-01 09:02 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants