Refactor apm-review-panel: VERDICT regime -> ADVISORY regime (drop binary APPROVE/REJECT, add doc-writer, restructure comment)

[danielmeppiel]

Problem

The current APM Review Panel skill (apm-review-panel) is structurally adversarial: it computes a binary APPROVE / REJECT verdict and writes a panel-approved / panel-rejected label. The schema forces every finding into one of two buckets — required[] (blocks merge, deterministically forces REJECT) or nits[] (skip-if-you-want). There is no middle ground.

In practice this produces two failure modes:

1. Over-strict verdicts. A panelist reading carefully will lean required for any real-but-not-blocking issue, because nit reads as "ignore me" and the schema offers no third slot. One careful panelist with one borderline required finding flips the entire panel to REJECT.
2. Adversarial comment shape. The comment top-loads Verdict: REJECT + Required before merge (N items) — a gating frame, not a collaborative one. PR authors (especially external contributors) read it as "the bots want me to fix N things" instead of "here is the panel's recommendation, the maintainer decides".

Concrete reference: PR #1084 (manual local panel run)

When PR #1084 (a 101+/1- bug-fix from external contributor @tillig unblocking GHES/GitLab/Bitbucket users) was reviewed manually using the same persona threads but a different output template, the comment (link):

• top-loaded CEO arbitration prose (why this matters strategically)
• followed with a principle alignment block (multi-host, pragmatic-as-npm)
• carried a per-persona summary table (one line per persona, verdict + nit-count + one-liner)
• rendered two mermaid diagrams (component flow + sequence) directly in the body, not buried in extras
• listed 3 prioritized post-merge follow-ups (none blocking)
• ended with ship recommendation prose ("merge as-is")
• collapsed full per-persona findings at the bottom

This is the shape the current panel should produce. It actually helps the contributor and the human reviewer; it does not pretend to be a gate.

Proposal

Refactor apm-review-panel from a VERDICT regime to an ADVISORY regime (mapping onto genesis example 04, not example 05).

What changes

Aspect	Today	Proposed
Verdict	APPROVE / REJECT, deterministic	None. Comment carries CEO ship-recommendation prose; maintainer + author decide.
Severity model	required[] (gates) + nits[] (skip)	findings[] with severity: blocking | recommended | nit. blocking reserved for real correctness/security regressions with explicit rationale.
Labels written	panel-approved XOR panel-rejected, plus panel-review removal	Only panel-review removal (trigger idempotency). No verdict labels.
Comment top	Verdict: REJECT + Required before merge (N)	CEO framing paragraph + principle alignment + per-persona summary table
Mermaid diagrams	Buried in python-architect.extras.diagrams; often missing	Template-required: component diagram + sequence diagram in the body
Roster	py-arch, cli-log, devx, sec, growth, [auth]	+ doc-writer (conditional like auth-expert: docs/, README, CHANGELOG, .apm/skills/, .github/agents/, instructions/, .github/workflows/*.md)
Companion workflow	pr-panel-label-reset.yml strips verdict labels on every push	Becomes obsolete (no verdict labels to strip) — delete or downscope

Why this is better

• Helpful to contributors. The advisory frame matches the mental model of code review on an OSS repo: a panel of reviewers leaving recommendations, not a CI gate. External contributors are not punished by a bot verdict.
• Helpful to maintainers. A summary table + mermaid diagrams + ship-recommendation prose lets the human reviewer scan in 10 seconds. Today's verdict comment is longer and less informative.
• Honest about LLM panels. Binary deterministic verdicts give false confidence — they hide real disagreement under a sum(required) == 0 calculation. Advisory framing makes the panel's actual signal (recommendations, not decisions) explicit.
• Aligned with APM principles. Pragmatic-as-npm: npm does not have a binary "your package quality is REJECTED" gate; it has scanners and recommendations. The panel should mirror that.
• Doc-writer fills a real gap. Documentation drift is a frequent finding raised informally today; making it a first-class persona surfaces it consistently.

Reference

The good comment from PR #1084: #1084 (comment)

The PR refactoring this skill will land alongside this issue and explain WHY/WHAT/HOW with the same diagrams and trade-off analysis.

Out of scope

• Changing the persona definitions themselves (each .agent.md continues to load and reason as today; only the orchestration contract and output shape change).
• Changing the trigger surface (panel-review label + workflow_dispatch continue to work).
• Auto-resolving severity disputes (the CEO arbitration paragraph already handles this; the schema just needs to express the dispute, not gate on it).

[github-actions]

Triage decision

accept

Proposed labels

area/ci-cd
type/refactor
status/accepted
priority/high
agentic-workflows

Milestone

null

Suggested next action

Open the implementing PR with the refactored SKILL.md, updated output template, and the pr-panel-label-reset.yml downscoped or deleted, referencing the advisory comment shape from PR #1084 as the acceptance target.

Suggested issue comment

Thanks for the thorough write-up and the concrete reference from PR #1084 -- that comment shape is a good north star for the advisory regime.

The triage panel accepts this as-is. The proposal is well-scoped: refactor the SKILL.md output contract (severity model, label-set, comment shape), wire in the doc-writer conditional persona already in the roster, and clean up or delete `pr-panel-label-reset.yml`. No CLI-surface or lockfile changes are involved.

A few things worth confirming in the implementing PR:

- Define the `findings[]` item schema explicitly in the SKILL.md output contract section (fields: `severity: blocking | recommended | nit`, `persona`, `summary`, `rationale`) so panelist tasks have an unambiguous return shape.
- Audit downstream automation for any dependency on the `panel-approved` / `panel-rejected` labels before removing them -- the companion workflow is the obvious consumer but worth a grep across workflows.
- The doc-writer persona `.agent.md` exists; the PR just needs to add the conditional-activation rule and the per-lens notes heading to the triage template.

Labelled `priority/high` -- this affects the experience of every contributor who gets a panel review, including first-time external contributors. Happy to review the PR when it lands.

Per-lens notes (collapsed)

DevX UX Expert -- User-Need Reviewer

The user surface here is PR authors -- especially external contributors -- who receive apm-review-panel comments. The current VERDICT regime frames the panel as a CI gate (top-loading "Verdict: REJECT" and "Required before merge (N items)"), which is adversarial to contributors who expect code review to produce recommendations from reviewers, not a binary machine verdict. The proposed advisory shape (CEO framing paragraph, per-persona summary table, mermaid diagrams, ship-recommendation prose) matches how developers experience review on major OSS repos and aligns directly with APM's "pragmatic-as-npm" positioning. The concrete reference from PR #1084 grounds the proposal: the advisory comment is demonstrably more informative and less intimidating. No CLI command or flag changes are implied -- this is entirely a reviewer-output UX change.

Supply Chain Security Expert -- Risk-Surface Reviewer

No risk-surface implication. This issue proposes changes to the output shape and severity model of an internal agent orchestration skill. It does not touch dependency resolution, lockfile integrity, package downloads, signing, auth token handling, MCP trust, or any P/G/S surface. Removing the panel-approved / panel-rejected labels and downscoping the companion pr-panel-label-reset.yml workflow has no security implication. Theme/security and theme/governance are not warranted.

OSS Growth Hacker -- Contributor-Tone Reviewer

Not activated -- author is danielmeppiel with COLLABORATOR association; no new-contributor tone tuning is needed for the suggested reply.

Python Architect -- Architecture Reviewer

The proposal is well-specified at the design level. Key implementation details: (1) the findings[] schema is gestured at (severity: blocking|recommended|nit) but not fully spelled out -- the implementing PR should define the exact fields so panelist task prompts have an unambiguous return contract; (2) removing panel-approved/panel-rejected labels requires confirming no other automation depends on them beyond pr-panel-label-reset.yml; (3) the doc-writer persona already exists as .github/agents/doc-writer.agent.md -- wiring it in is low-risk. Feasibility is high: this is a SKILL.md rewrite and output template change, not a Python module refactor with hard type dependencies. No apm.yml or apm.lock.yaml schema changes are involved. status/accepted is appropriate; no design gate is needed.

Doc Writer -- Documentation Reviewer

Not activated -- this is an internal agent primitive refactor (.apm/skills/ and .github/workflows/); it does not imply new or changed user-facing pages in docs/src/content/docs/, and the suggested comment wording is clear and grounded without requiring doc-site vocabulary alignment.

APM CEO -- Triage Arbiter

Clean accept. This proposal comes from a core collaborator, is thoroughly reasoned, and has a concrete reference implementation. The advisory regime is the right direction: LLM panels give probabilistic signal, not deterministic verdicts, and framing them as advisory is both more honest and more contributor-friendly. The scope is well-bounded -- skill output contract + companion workflow cleanup, no product surface changes. Priority/high is warranted because every PR going through the panel (including first-time external contributor PRs) is affected today. No mega-theme is assigned because this is pure CI/agentic-workflow infra with no product surface implication.

{
  "decision": "accept",
  "decision_detail": "",
  "theme": null,
  "areas": ["area/ci-cd"],
  "type": "type/refactor",
  "status": "status/accepted",
  "priority": "priority/high",
  "preserved_labels": ["agentic-workflows"],
  "milestone": null,
  "next_action": "Open the implementing PR with the refactored SKILL.md, updated output template, and pr-panel-label-reset.yml downscoped or deleted, referencing the advisory comment shape from PR #1084 as the acceptance target.",
  "comment_markdown": "Thanks for the thorough write-up and the concrete reference from PR #1084 -- that comment shape is a good north star for the advisory regime.\n\nThe triage panel accepts this as-is. The proposal is well-scoped: refactor the SKILL.md output contract (severity model, label-set, comment shape), wire in the doc-writer conditional persona already in the roster, and clean up or delete `pr-panel-label-reset.yml`. No CLI-surface or lockfile changes are involved.\n\nA few things worth confirming in the implementing PR:\n\n- Define the `findings[]` item schema explicitly in the SKILL.md output contract section (fields: `severity: blocking | recommended | nit`, `persona`, `summary`, `rationale`) so panelist tasks have an unambiguous return shape.\n- Audit downstream automation for any dependency on the `panel-approved` / `panel-rejected` labels before removing them -- the companion workflow is the obvious consumer but worth a grep across workflows.\n- The doc-writer persona `.agent.md` exists; the PR just needs to add the conditional-activation rule and the per-lens notes heading to the triage template.\n\nLabelled `priority/high` -- this affects the experience of every contributor who gets a panel review, including first-time external contributors. Happy to review the PR when it lands."
}

---

Triage status: agentic proposal pending human ratification.
Silence is approval. Maintainers can:

• Override any label or milestone above by editing it directly --
  human edits are authoritative and will not be reverted on
  subsequent runs.
• Re-trigger triage by applying the status/needs-triage label, or
  by removing status/triaged to enroll the issue in the next
  daily sweep.

Posted by the Triage Panel workflow. See .apm/skills/apm-triage-panel for the panel skill.

Note

🔒 Integrity filter blocked 41 items

The following items were blocked because they don't meet the GitHub integrity level.

• [FEATURE] Extended MCP Server Configuration in apm.yml #20 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Allow install to download and unpack a GitHub release automatically #514 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Skills from .well-known URIs #554 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• Feature: per-target agent frontmatter transformation during deployment #581 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• bug: apm run start picks wrong runtime — system PATH stub takes priority over APM-managed binary, codex v0.116+ incompatible with GitHub Models #605 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Add ability to specify a different configuration file name or path #612 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• bug: apm install plugin@marketplace validation bypasses registry proxy #615 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Renovate supporting apm.yml #639 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] MCP install target for Claude Code (project + user scope) #643 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• Align CONTRIBUTING, Python formatting (Black/isort), and CI #645 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Copilot CLI user-scope instructions: can apm install -g write copilot-instructions.md #650 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• Marketplace add source parity with the Anthropic spec #676 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] PEP 8 Support #690 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] URL-based marketplace work Extension #692 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Create root AGENTS.md documenting all CI-enforced rules for coding agents #695 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [FEATURE] Kiro Support #702 list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• ... and 25 more items

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by Triage Panel · ● 696.8K · ◷