chore: remove evals from policy and plugin

[jbaruch]

Author-Model: claude-opus-4-8

Why

The tessl publish pipeline runs the eval suite unconditionally inside tesslio/patch-version-publish, and that trigger is not skippable. When the org is out of credits the eval run fails (Failed to start eval run: ...out of credits) and the publish never advances the registry — observed live on the #190 merge, where the workflow went green but 0.3.94 never reached the registry.

The credit-outage tolerance added in #188 covers the skill-review step but cannot reach inside patch-version-publish. Since evals can't run without credits and block every publish, and we've decided we won't do evals if they gate the registry this way, they're removed entirely.

What was removed

• rules/plugin-evals.md (the entire rule)
• The ## Mandatory Evals section of rules/context-artifacts.md
• The eval-authoring and eval-curation skills
• The whole evals/ scenario suite (43 files)

Cross-references reconciled (no dangling links)

• testing-standards — Fixture Hygiene reference inlined (dates in the filename)
• rule-frontmatter, context-writing-style — dropped evals/ / plugin-evals examples
• skills/migrate-to-plugin — dropped the frozen-eval-scenario-names bullet
• skills/release — dropped the evals/ self-audit example and the criteria.json max_score weight checks
• .github/actions/skill-review/review-skills.sh — dropped the plugin-evals Naming citation
• pyrightconfig.json — dropped the eval-curation Python paths
• .tessl-plugin/plugin.json, .claude/CLAUDE.md, README.md (description, What's New, rules + skills tables, rule count 23→22)

A repo-wide grep confirms no plugin-evals / eval-authoring / eval-curation / Mandatory Evals reference survives outside the CHANGELOG archive.

Verification

• run-diagnostics.sh (shellcheck + pyright) clean
• Full suite {"suites":18,"passed":18,"failed":0}
• tessl plugin lint valid
• Paired cross-family policy review + Copilot

Notes

• The nanoclaw-plugin auto-publish eval carve-outs are now moot but live in those separate repos — not touched here. Happy to open follow-up PRs there on your go-ahead.
• This bundles under the pending 0.3.94 (registry is still 0.3.93; 0.3.94 was bumped on main by feat(ci): native tessl out-of-credits tolerance for the skill-review action (#188) #190's failed publish but never published). Whether removing evals actually unblocks patch-version-publish is the open question this change tests on merge — if patch-version-publish still attempts an eval run with zero scenarios, publishing stays blocked and we'll need a different lever.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Removes Tessl eval scenarios and the associated rule/skills from jbaruch/coding-policy to prevent the unskippable eval run inside tesslio/patch-version-publish from blocking publishes during out-of-credits outages, while reconciling all cross-references and published-plugin surfaces.

Changes:

• Deleted the eval policy surface (rules/plugin-evals.md), eval skills (eval-authoring, eval-curation), and the evals/ suite.
• Removed/updated all documentation and rule references that pointed at evals (rules/skills/README/action comments).
• Updated plugin manifest + ancillary config (e.g., pyrightconfig.json) to stop shipping/typing deleted content.

Reviewed changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
skills/release/SKILL.md	Removes eval-related self-audit/examples from the release workflow guidance.
skills/migrate-to-plugin/SKILL.md	Drops the frozen-eval-scenario-names contract note now that evals are removed.
skills/eval-curation/tests/test_compute_lift.py	Deleted (eval curation test suite removed).
skills/eval-curation/SKILL.md	Deleted (eval curation skill removed).
skills/eval-curation/compute-lift.py	Deleted (eval curation helper script removed).
skills/eval-authoring/SKILL.md	Deleted (eval authoring skill removed).
skills/eval-authoring/REVIEW_CHECKLIST.md	Deleted (eval scenario review guidance removed with eval system).
skills/eval-authoring/LIFT_ANALYSIS.md	Deleted (lift analysis reference removed with eval system).
rules/testing-standards.md	Inlines fixture hygiene guidance previously sourced from plugin-evals.
rules/rule-frontmatter.md	Removes evals/ from scoping examples.
rules/plugin-evals.md	Deleted (eval governance rule removed).
rules/context-writing-style.md	Removes plugin-evals as an example of multi-aspect rules.
rules/context-artifacts.md	Removes evals/** from applyTo, removes Mandatory Evals section, updates “standard directories”.
README.md	Updates repo description + rule counts; removes eval skills and rule from “What’s Included”.
pyrightconfig.json	Removes pyright include paths for deleted eval-curation Python files.
evals/version-bump-reasoning-and-manifest-upda/task.md	Deleted (eval scenario removed).
evals/version-bump-reasoning-and-manifest-upda/criteria.json	Deleted (eval scenario removed).
evals/rule-frontmatter-tile-rule-agreement/task.md	Deleted (eval scenario removed).
evals/rule-frontmatter-tile-rule-agreement/criteria.json	Deleted (eval scenario removed).
evals/rule-frontmatter-mixed-scope-no-narrow/task.md	Deleted (eval scenario removed).
evals/rule-frontmatter-mixed-scope-no-narrow/criteria.json	Deleted (eval scenario removed).
evals/rule-frontmatter-conditional-rule/task.md	Deleted (eval scenario removed).
evals/rule-frontmatter-conditional-rule/criteria.json	Deleted (eval scenario removed).
evals/review-feedback-addressing-conventions/task.md	Deleted (eval scenario removed).
evals/review-feedback-addressing-conventions/criteria.json	Deleted (eval scenario removed).
evals/pr-merge-and-post-merge-cleanup/task.md	Deleted (eval scenario removed).
evals/pr-merge-and-post-merge-cleanup/criteria.json	Deleted (eval scenario removed).
evals/migrate-to-plugin-keeps-contracts/task.md	Deleted (eval scenario removed).
evals/migrate-to-plugin-keeps-contracts/criteria.json	Deleted (eval scenario removed).
evals/install-reviewer-refuses-overwrite/task.md	Deleted (eval scenario removed).
evals/install-reviewer-refuses-overwrite/criteria.json	Deleted (eval scenario removed).
evals/eval-scenario-quality-review-and-repair/task.md	Deleted (eval scenario removed).
evals/eval-scenario-quality-review-and-repair/criteria.json	Deleted (eval scenario removed).
evals/eval-curation-universal-competence-retire/task.md	Deleted (eval scenario removed).
evals/eval-curation-universal-competence-retire/criteria.json	Deleted (eval scenario removed).
evals/eval-curation-task-leak-fix/task.md	Deleted (eval scenario removed).
evals/eval-curation-task-leak-fix/criteria.json	Deleted (eval scenario removed).
evals/eval-curation-criteria-grade-universal-competence/task.md	Deleted (eval scenario removed).
evals/eval-curation-criteria-grade-universal-competence/criteria.json	Deleted (eval scenario removed).
evals/eval-curation-clean-suite-no-action/task.md	Deleted (eval scenario removed).
evals/eval-curation-clean-suite-no-action/criteria.json	Deleted (eval scenario removed).
evals/copilot-review-via-graphql/task.md	Deleted (eval scenario removed).
evals/copilot-review-via-graphql/criteria.json	Deleted (eval scenario removed).
evals/consumer-scaffolds-policy-reviewer/task.md	Deleted (eval scenario removed).
evals/consumer-scaffolds-policy-reviewer/criteria.json	Deleted (eval scenario removed).
evals/ci-and-review-status-polling/task.md	Deleted (eval scenario removed).
evals/ci-and-review-status-polling/criteria.json	Deleted (eval scenario removed).
evals/adopt-fork-pr-same-repo-passthrough/task.md	Deleted (eval scenario removed).
evals/adopt-fork-pr-same-repo-passthrough/criteria.json	Deleted (eval scenario removed).
evals/adopt-fork-pr-adopts-fork-branch/task.md	Deleted (eval scenario removed).
evals/adopt-fork-pr-adopts-fork-branch/criteria.json	Deleted (eval scenario removed).
CHANGELOG.md	Adds a release note about removing evals and reconciled references.
.tessl-plugin/plugin.json	Removes deleted skills and rule from the published plugin manifest.
.github/actions/skill-review/review-skills.sh	Removes a citation to the deleted plugin-evals rule in a comment.
.claude/CLAUDE.md	Removes the deleted plugin-evals rule from the maintainer context imports.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jbaruch]

Fixed in a0cbbc8 — collapsed to a 3-sentence what/why/reference summary; the file-by-file inventory stays in the PR body per CHANGELOG Hygiene.

[github-actions]

Policy loaded: 22 rule files from rules/ (PR head).
All rules pass — no violations found.

Reviewed the eval-removal diff against the PR-head policy, including the changed rule files and the changed release / migrate-to-plugin skill files. Surface sync is consistent: the manifest, README, maintainer context, pyright config, and residual references have been reconciled, and deleted skills are handled by the skill-review action's existing deleted-skill skip path.

Generated by PR Policy Review (OpenAI) for #191 · 80 AIC · ⌖ 4.36 AIC · ⊞ 18.2K

[github-actions]

Policy loaded: 22 rule files from rules/ (PR head).
All rules pass — no violations found.

Reviewed the eval-policy removal, surface sync updates, changed rule/skill prose, manifest, README, changelog entry, pyright include list, and remaining references. No policy violations found.

Generated by PR Policy Review (OpenAI) for #191 · 76.2 AIC · ⌖ 13.2 AIC · ⊞ 16.5K