[RFC] Payload Agent Skill Improvements

[denolfe]

Motivation

Agent Skills let coding agents load capabilities on-demand. We've published the payload skill for agents to load and use when working with Payload projects.

Our eval suite catches regressions on cases we already know about. However, it can't predict what real users will actually type, or how a given model and harness combo will misread it. That's the gap we need help closing.

What we're asking for

Failure scenarios. Cases where your agent had the payload skill loaded and still got it wrong: edited the wrong file, missed a required config property, invented an API, ignored access control, and so on. The more detail you can provide, the better. The goal is to find gaps in the skill's current capabilities and documentation, so we can fix them and make the skill better for everyone.

Each report is a candidate eval case and will guide improvements to the skill.

How to participate

1. Install the skill

The skills CLI is the easiest way to add the skill.

npx skills add https://github.com/payloadcms/skills --skill payload

Alternatively, you can copy the skill files directly into the expected directory for your agent.

2. Use it on real work

Build with Payload like you normally would. The skill loads when your agent sees Payload context (e.g. a payload.config.ts), or you can manually load it if needed.

3. Submit failures

Submit a comment on this Discussion with the details of what went wrong, using the template below. The more specific you can be about the setup, prompt, expected vs actual behavior, and repro steps, the better.

Submission template

Harness/Model: [e.g. Claude Code / Opus 4.7]

Setup:
  Project type: [new app / existing / plugin / etc.]
  Relevant config: [minimal payload.config.ts / collection config / etc.]

Prompt: [verbatim]

Expected: [what the agent should have done]

Actual: [what it did, diff snippet, file edits, or quoted output]

Minimal repro (optional): [snippet or repo link, only if complex]

[nick-vogel]

This is not necessarily strictly on the skill, but I asked Claude Code (Opus 4.7) to implement a slugField. it did it but then it didn't generate the importMap, which should happen on pnpm dev or build, but it doesn't always. it might be worth adding instructions that custom components from LLMs run payload generate:importmap so the custom components are always included before the dev server boots up or build command runs.

[denolfe]

Thanks for bringing this up. The skill could definitely steer the agent in right direction on this 👍

We've actually had discussions internally about integrating import map generation into the build process more reliably.