feat(daily-regen): autonomous spec polish + cross-library similarity audit

[MarkusNeusinger]

Summary

Adds two autonomous pre-flight steps to daily-regen.yml that run before bulk-generate fans out, lifting two quality vectors that today only exist in the local /regen and /update skills into the cloud cadence.

Everything runs through claude-code-action on Claude Max OAuth — no extra API costs — same pinned SHA as impl-generate / impl-review / impl-repair.

What it does

For each spec the pick job selects:

1. Skip-gate (Bash, no LLM) — gh pr list --search "plots/<spec>/ in:files is:open". If any PR is open touching the spec, the polish step is skipped to avoid racing humans or stacking auto-polish PRs. The similarity audit still runs (it's read-only).

2. Spec polish (claude-code-action, --model haiku) — audits the spec across the five update.md §2 dimensions (wording, missing sections, tag completeness, tag quality, tag accuracy). If anything needs work, opens an auto-polish/<spec>/<timestamp> branch + PR with label auto-polish. Never pushes to main directly. Never auto-merges. PR awaits human review. If nothing needs work, prints NOOP and stops. continue-on-error: true so a transient action failure does not block the main pipeline.

3. Cross-library similarity audit (claude-code-action, --model haiku) — reads the 9 review.image_description blobs from plots/<spec>/metadata/python/*.yaml and clusters libraries that converged on the same data scenario / example domain / visual variant beyond what the spec dictated. Optionally drills into impl .py files for ambiguous clusters via the Read tool. Emits /tmp/change-requests.json keyed by library. Project-mandated constants (Okabe-Ito palette positions 1–7, plot size and aspect ratio, theme chrome) are explicitly excluded as cluster signals. continue-on-error: true; if the audit fails, the collect step falls back to empty change_requests.

4. Dispatch bulk-generate with hints — passes change_requests JSON to bulk-generate, which jq-extracts the per-library hint and forwards it as the new change_request input to impl-generate. The hint is staged to /tmp/anyplot-change-request.txt, where the updated impl-generate-claude.md picks it up and treats it as a hard requirement (mirroring regen.md §2c verbatim — hard requirement, no sibling reads, preserve review.strengths, override "no changes for sake of changes").

Why

• Spec drift: specs are currently written once at creation and never revisited. Tag vocab evolves, sections go missing, wording grows vague. Polish-on-cycle keeps them sharp without manual maintenance, and at ~10 cycles/day across 300+ specs, each spec gets touched once a month — drift risk is low.
• Silent convergence: without a similarity check, 9 libs can independently land on the same scenario / domain / variant, producing nine copies of the same chart in different engines — exactly the opposite of the catalog's purpose. The hint-injection breaks the cluster cleanly (one library per cluster, alphabetically later).

Model routing

• The existing daily-regen model input (default haiku, choices haiku/sonnet/opus) is unchanged and still flows to bulk-generate → impl-generate / review / repair.
• The two new pre-flight LLM steps hardcode --model haiku — they're narrow, cheap audits.

dry_run semantics

dry_run=true runs the read-only and decision-only steps so operators can preview what the cycle will do without committing anything:

• Runs: pick, skip-gate, similarity audit (read-only), collect change_requests
• Skipped: spec polish (would open a real PR — side effect), dispatch bulk-generate (would fan out 9 impl-generate jobs)

To preview spec polish in isolation, run a real (non-dry-run) cycle against a single spec: gh workflow run daily-regen.yml -f specification_id=<spec> -f model=haiku. Polish opens a PR; merge or close it manually.

Backwards compatibility

Both new inputs (change_request on impl-generate, change_requests on bulk-generate) default to empty ("" and '{}'). Existing manual triggers without these inputs behave byte-identically to today.

Risks + rollback

• Auto-polish PRs accumulate if humans never review them. Skip-gate prevents duplicates per spec; the spec just doesn't get polished further until reviewed. Acceptable: human stays in control.
• Spec polish prompt has hard rules: no changes to id / issue / created, no semantic changes (data shape, plot type, requirements). Reviewable in commit diffs; revert if anything slips.
• Rollback: revert daily-regen.yml first. Downstream change_request[s] inputs default to empty; remaining changes are no-ops without daily-regen wiring.

Test plan

• CI parses all four workflow YAMLs cleanly
• Manual gh workflow run daily-regen.yml --ref feat/daily-regen-pre-flight -f specification_id=<spec> -f model=haiku -f dry_run=true — confirms pick + preflight-dispatch (skip-gate, similarity, collect) run, polish + bulk-generate dispatch are skipped (dry_run)
• Manual gh workflow run daily-regen.yml --ref feat/daily-regen-pre-flight -f specification_id=<spec> -f model=haiku (no dry_run) — confirms the full chain: polish either NOOPs or opens a PR, similarity emits change_requests, bulk-generate fires, an impl-generate run with non-empty hint shows ::notice::Change request staged: …
• On the auto-polish PR (if produced): id / issue / created unchanged; only wording / sections / tags polished; updated bumped

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adds two LLM-driven pre-flight checks to daily-regen so each scheduled regeneration can optionally polish the spec and inject cross-library divergence hints before fan-out into implementation generation.

Changes:

• Added new Claude workflow prompts for autonomous spec polish and cross-library similarity auditing.
• Threaded per-library change_request / change_requests inputs through daily-regen, bulk-generate, and impl-generate.
• Updated the implementation-generation prompt so a staged divergence hint becomes a binding regeneration requirement.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
prompts/workflow-prompts/spec-polish-claude.md	New prompt defining autonomous spec-polish behavior and PR creation flow.
prompts/workflow-prompts/impl-similarity-claude.md	New prompt defining the read-only cross-library similarity audit and JSON hint output.
prompts/workflow-prompts/impl-generate-claude.md	Adds instructions for consuming a staged divergence hint during regeneration.
.github/workflows/impl-generate.yml	Adds optional change_request input and stages it into /tmp for Claude.
.github/workflows/daily-regen.yml	Replaces direct dispatch with per-spec pre-flight polish/audit/collect/dispatch flow.
.github/workflows/bulk-generate.yml	Adds change_requests input, validates it, and forwards per-library hints to impl-generate.