explicit skill applicability and versioned metadata

[merof-code]

Feature request

Follow up for #2384, #2385. Solves (mostly) #3826. Advances #3862, and, incidentally, the custom-policy groundwork requested in #1766 (closed).

Would you be open to optional, deterministic applicability conditions for skills?

Problem

A skill's description can explain when its guidance is relevant, but that leaves relevance to the model after the content has already enlarged the prompt.

For many rules, applicability is already known. A rule may be relevant to only frontend, backend. And then only to a specific part of it:

• React component changes need our styling and accessibility guidance.
• Laravel controller changes need our request-validation guidance.
• Database migration changes need our schema-compatibility guidance.

Those instructions should not all consume the same prompt budget for every PR.
In a mixed PR, it should also be clear which files each instruction applies to.

Proposal: optional applicability conditions

Illustrative example:

name: laravel-request-validation
description: Review HTTP request validation in Laravel controllers.

pr_agent:
  schema: skills/v1
  id: laravel.request-validation
  when:
    paths:
      - "laravel/app/Http/Controllers/**/*.php"
      - "laravel/app/Http/Requests/**/*.php"
---

Validate incoming request data through a dedicated FormRequest.
Flag controllers that perform request validation inline.

Additionally pr_agent.when.regex[] would allow for keyword activation parameters.

A frontend-only PR would not load this skill. A controller change would select it and identify the matching files before prompt construction.

Proposed behaviour:

• Skills without schema retain existing behaviour - loaded by default.
• Path globs match changed paths in MR; any matching pattern selects the skill.
• Selection happens before allocating the skills token budget.
• The prompt retains the association between the selected skill and its matching files.
• optionally report loaded skills and which files were matched.

In a mixed PR, loading the correct skills may not be enough. each skill can also remain scoped to the files that triggered it. For example, controller-validation guidance should apply to matching controllers, without being interpreted as a requirement for unrelated background jobs.

Path matching along with reporting would solve #3826 and replace our ci script.

Why version the metadata?

This is version pinning, when a new kind of schema will be inevitably introduced, no changes to existing skills or code will be needed.
Or when an existing schema change is not backwards compatable.

Benefits:

• validate conditions
• report unsupported versions clearly,
• evolve the format without changing the meaning of existing files

id field

Currently, pr_agent records resolved issues. That is done by normalizing the response which is not deterministic.
An id would allow issues to be reported in a tuple format,
<laravel.request-validation, filepath, symbol_path>
example:
<'react.rtk-query.invalidation', 'laravel/app/Http/Controllers/SomeController.php', 'SomeController.list'>

In other words: longer term, this could support explicit results per skill/policy and persistent finding identities. Those would be separate extensions; this initial proposal only concerns selecting and identifying guidance.

Compatibility with existing features

This complements rather than replaces existing mechanisms:

• extra_instructions remains for global overrides
• Skills become conditional, reusable guidance.

Future outlook

I see this as complementary to the planned agentic flow. The agent chooses guidance it needs, while explicit rules identifies policies maintainers want checked on every relevant change. Guaranteed to have been checked against.

Deterministic loading would be the first use of that metadata. Longer term, it could also support

• routing policies to different evaluators
• reporting which checks were completed, skipped or uncertain.
• be a place for rule severity and
  all kinds of metadata that does not fit into a toml file. That is the broader direction I’d like to explore in the discussion.

Jev models and other "discriminative model"'s

Some code policies are not easily implemented by AST or linters, while being very much deterministic.
For example, a policy states: "the name of the function must accurately represent what it does". A reviewer can miss it when the MR changes only some part of the function, but the name is no longer representative. A pr_agent is much better at it, but can be lost in tons of context.
A "discriminative model" model like jev would be faster, cheaper and safer to validate policies like that. With a skill yaml format, one addition and perhaps a new schema is all it takes config wise, to label all jev-compatible policies.