Skip to content

docs(instrumentation): APM Review Panel - 7-agent expert review for non-trivial changes#777

Merged
danielmeppiel merged 4 commits intomainfrom
feat/instrumentation-review-panel
Apr 20, 2026
Merged

docs(instrumentation): APM Review Panel - 7-agent expert review for non-trivial changes#777
danielmeppiel merged 4 commits intomainfrom
feat/instrumentation-review-panel

Conversation

@danielmeppiel
Copy link
Copy Markdown
Collaborator

What

Adds the APM Review Panel -- an orchestration skill plus four new specialist personas wired into the existing python-architect and cli-logging-expert agents -- so any non-trivial change to microsoft/apm automatically gets multi-disciplinary review (architecture, CLI logging, package-manager UX, supply-chain security, strategy, OSS growth) without anyone having to remember to invoke the right agent.

Markdown only. No code, no tests, no runtime behavior change.

Why -- this is instrumentation, per Handbook Ch. 9

This PR maps 1:1 onto the Instrumented Codebase chapter of the Agentic SDLC Handbook:

"Instrumentation is the practice of converting tacit knowledge into structured files that an agent loads as context. ... When an agent makes a mistake, you don't fix the generated code; you fix the context file that failed to prevent the mistake."

The maintainer's review lenses for microsoft/apm -- "is this the right npm-style command shape?", "does this expose a supply-chain surface?", "does this fit the positioning?", "does this help adoption?" -- have lived in one head. This PR externalizes them into versioned .agent.md and SKILL.md primitives that agents (and humans) load on demand.

Mapping to the chapter's primitive types:

Ch. 9 primitive This PR
Agents (.agent.md) -- domain expertise + named patterns + anti-patterns + boundaries 4 new personas, each with all four sections
Skills (SKILL.md) -- reusable decision frameworks that activate on code patterns 4 per-persona auto-activating skills + 1 orchestration skill
Composition -- "primitives form a layered system; the agent's effective context is the composition" The panel is a composition, not a mega-prompt
Feedback loop -- "every failure becomes a one-line addition to the right context file" Each persona is the durable fix-point for its class of review failure

This is the chapter's "ongoing maturity" step: extracting a new orchestration skill once the same coordination pattern has recurred (the auth-expert / logging-expert / doc-writer panel that emerged organically in PR #394 and is documented in the case-study chapter).

Why this helps every OSS contributor

  1. Lower the bar to contribute well. Skills auto-activate at edit time. A first-time contributor who edits cli.py gets the package-manager UX lens; one who touches core/auth.py gets the supply-chain security lens. They get the maintainer's review criteria before opening the PR, not after.
  2. Make review criteria auditable. One file per persona, each ~50-80 lines, ASCII-only, in the repo. No tribal knowledge, no inconsistent bar across reviews.
  3. Compress the maintainer feedback loop. Explicit routing means strategic / breaking-change calls have a single declared arbiter (CEO). Growth implications are surfaced as a side-channel rather than getting lost.
  4. Demonstrate APM dogfooding. APM is the package manager for exactly these primitives. Using them at scale on the APM repo itself is the strongest possible reference example for new users -- and proves the model scales beyond toy examples.
  5. Compounding asset. Per Ch. 9's feedback loop: every future review failure becomes a one-line addition to the right persona file, not a re-litigation of the same call. The instrumentation gets sharper with use.

Routing topology

  python-architect   cli-logging-expert   devx-ux-expert   supply-chain-security-expert
        \________________________|_______________________________/
                                 |
                                 v                       <----  oss-growth-hacker
                              apm-ceo                          (annotates findings;
                        (final call / arbiter)                  updates growth-strategy)
  • Specialists raise findings independently -- no implicit consensus.
  • CEO arbitrates when specialists disagree or when a finding has strategic implications (positioning, breaking change, naming, scope). Uses gh CLI to ground decisions in real repo stats.
  • Growth Hacker is a side-channel to the CEO: never blocks a specialist finding; annotates with conversion implications and maintains WIP/growth-strategy.md.

PROSE compliance

  • Progressive Disclosure -- per-persona SKILL.md files are 30-36 lines; persona detail (47-83 lines each) lives behind markdown links, lazy-loaded only when invoked.
  • Reduced Scope -- each persona has one narrow lens; each skill activates on a focused trigger set.
  • Orchestrated Composition -- 7 atomic personas + 5 thin skills + 1 router; no monolith.
  • Safety Boundaries -- every persona file ends with an explicit ## Boundaries section declaring what it owns and what it defers.
  • Explicit Hierarchy -- global copilot-instructions.md updated with one pointer; per-skill activation triggers narrow scope further; routing topology resolves authority.

Files

Path Purpose
.github/agents/devx-ux-expert.agent.md Package-manager UX persona
.github/agents/supply-chain-security-expert.agent.md Threat-model persona
.github/agents/apm-ceo.agent.md Strategic owner / arbiter
.github/agents/oss-growth-hacker.agent.md Adoption / growth persona
.github/skills/devx-ux/SKILL.md Auto-activates on cli.py / command help
.github/skills/supply-chain-security/SKILL.md Auto-activates on deps/ / auth / lockfile
.github/skills/apm-strategy/SKILL.md Auto-activates on README / MANIFESTO / CHANGELOG / releases
.github/skills/oss-growth/SKILL.md Auto-activates on README / docs / templates
.github/skills/apm-review-panel/SKILL.md Orchestration: roster + routing + workflows + quality gates
.github/copilot-instructions.md One-line pointer surfacing the panel
CHANGELOG.md Unreleased / Added entry

Type of change

  • Documentation
  • Bug fix
  • New feature
  • Maintenance / refactor

Testing

  • Markdown only -- no source code paths affected
  • Strict ASCII audit passed on all 9 new files (per repo encoding rule)
  • All persona files declare explicit ## Boundaries
  • All SKILL.md files use the same frontmatter schema as existing skills (python-architecture, cli-logging-ux, auth)
  • No pytest run required (no Python sources changed)

…ents

Instrumentation per Handbook Ch. 9 (The Instrumented Codebase):
externalize the maintainer's multi-disciplinary review process into
versioned `.agent.md` and `SKILL.md` primitives so any contributor or
agent inherits the same review lenses.

New agents (.github/agents/):
- devx-ux-expert       -- package-manager UX (npm/pip/cargo lens)
- supply-chain-security-expert -- dep confusion, lockfile integrity,
                          token scoping, fail-closed reviews
- apm-ceo              -- strategic owner; uses `gh` CLI for stats;
                          final arbiter for specialist disagreements
- oss-growth-hacker    -- adoption funnel; owns WIP/growth-strategy.md

New per-persona skills (auto-activate on relevant edits):
- devx-ux              -- triggers on cli.py, command help, install/init/run
- supply-chain-security -- triggers on deps/, core/auth.py, lockfile,
                          cleanup chokepoint
- apm-strategy         -- triggers on README/MANIFESTO/PRD/CHANGELOG,
                          release workflows, issue templates
- oss-growth           -- triggers on README/docs/templates, release
                          announcements, growth-strategy.md

New orchestration skill:
- apm-review-panel     -- routes specialists -> CEO; growth-hacker
                          side-channels conversion insights to CEO

Wiring:
- copilot-instructions.md -- one-line pointer surfacing the panel

PROSE-compliant: thin SKILL.md routers (progressive disclosure),
narrow per-agent scope (reduced scope), small composable primitives
(orchestrated composition), explicit `## Boundaries` per persona
(safety boundaries), panel layered on top of global instructions
(explicit hierarchy). All files strict ASCII per repo encoding rule.

No code or test changes; markdown only.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 20, 2026 07:50
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds documentation-only “instrumentation” to the repo: an APM Review Panel orchestration skill plus new specialist personas and per-persona skills, and wires the panel into Copilot guidance so non-trivial changes get multi-lens review by default.

Changes:

  • Add five new .github/skills/*/SKILL.md skills (panel orchestration + DevX UX, supply-chain security, strategy, OSS growth).
  • Add four new .github/agents/*.agent.md personas (DevX UX, supply-chain security, CEO, OSS growth).
  • Update .github/copilot-instructions.md and add a new Unreleased entry in CHANGELOG.md.
Show a summary per file
File Description
CHANGELOG.md Adds an Unreleased entry describing the review panel/personas/skills.
.github/copilot-instructions.md Adds a pointer instructing contributors/agents to invoke the APM Review Panel skill for non-trivial changes.
.github/skills/apm-review-panel/SKILL.md Defines the review-panel roster, routing, workflow blocks, and quality gates.
.github/skills/devx-ux/SKILL.md Adds an auto-activating skill for CLI/package-manager UX review lens.
.github/skills/supply-chain-security/SKILL.md Adds an auto-activating skill for dependency/auth/lockfile/path safety threat-model review.
.github/skills/apm-strategy/SKILL.md Adds an auto-activating skill for positioning/release/breaking-change decisions.
.github/skills/oss-growth/SKILL.md Adds an auto-activating skill for conversion surfaces and growth strategy updates.
.github/agents/devx-ux-expert.agent.md Adds the DevX UX expert persona definition and boundaries.
.github/agents/supply-chain-security-expert.agent.md Adds the supply-chain security expert persona definition and boundaries.
.github/agents/apm-ceo.agent.md Adds the CEO/arbiter persona definition and boundaries.
.github/agents/oss-growth-hacker.agent.md Adds the OSS growth persona definition and boundaries.

Copilot's findings

Comments suppressed due to low confidence (2)

.github/skills/apm-review-panel/SKILL.md:23

  • WIP/growth-strategy.md is referenced as an owned/updated artifact, but there is no WIP/ directory (or that file) in the repo. Either add the file in this PR or update these references to an existing path so links/instructions are not dead.
| [Supply Chain Security Expert](../../agents/supply-chain-security-expert.agent.md) | Threat-Model Reviewer | Dependency identity, lockfile integrity, path safety, token scoping |
| [APM CEO](../../agents/apm-ceo.agent.md) | Strategic Owner / Arbiter | Positioning, breaking-change comms, release decisions, final calls on disagreements |
| [OSS Growth Hacker](../../agents/oss-growth-hacker.agent.md) | Adoption Strategist | Conversion surfaces, story angles, `WIP/growth-strategy.md` |

.github/agents/oss-growth-hacker.agent.md:79

  • This boundary section restricts edits to WIP/growth-strategy.md, but that file/path is not present in the repo. Please add the file or update the reference so the boundary is enforceable in practice.
- You do NOT review code correctness or security.
- You do NOT make final calls -- escalate to CEO with a recommendation.
- You write only to `WIP/growth-strategy.md` and to comments / drafts;
  you do not modify shipped docs without specialist + CEO sign-off.
  • Files reviewed: 11/11 changed files
  • Comments generated: 6

Comment thread CHANGELOG.md
Comment thread .github/skills/apm-review-panel/SKILL.md
Comment thread .github/copilot-instructions.md
Comment thread .github/skills/oss-growth/SKILL.md
Comment thread .github/agents/oss-growth-hacker.agent.md
Comment thread .github/agents/apm-ceo.agent.md Outdated
…tainer-local

Address PR #777 review: WIP/ is intentionally gitignored (private
strategy, may contain competitive thinking). The file references in
the panel docs implied it should exist in the repo, which created a
"missing file" perception for reviewers.

Clarify in all four touch-points (oss-growth-hacker.agent.md,
oss-growth/SKILL.md, apm-review-panel/SKILL.md, apm-ceo.agent.md)
that the file is:

- gitignored (the entire WIP/ directory is excluded)
- maintainer-local; may not exist in every checkout
- created locally on first use if absent
- never staged or committed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel
Copy link
Copy Markdown
Collaborator Author

Thanks for the review.

On the two findings about WIP/growth-strategy.md not existing in the repo: that's intentional. WIP/ is gitignored (.gitignore:67) -- it holds the maintainer's private strategy notes, which can include competitive positioning, draft launch tactics, and unannounced direction. Pushing it would defeat the purpose.

What was actually missing is that the panel docs implied the file should exist in the repo, which made the reference look like a dead link. Pushed 53b8c6c to fix the framing in all four touch-points: the file is now consistently described as gitignored, maintainer-local, created locally on first use if absent, never committed.

So the boundary in oss-growth-hacker.agent.md is still enforceable -- it just enforces locally for whoever is running the panel as a maintainer, rather than against a shared repo file.

danielmeppiel and others added 2 commits April 20, 2026 10:17
The auth resolution + fallback flow is fully documented (with a
mermaid flowchart) in docs/src/content/docs/getting-started/authentication.md
and shipped as a skill resource at packages/apm-guide/.apm/skills/apm-usage/authentication.md.

Surface those as canonical references from auth-expert.agent.md and
auth/SKILL.md so the agent loads them on demand (PROSE progressive
disclosure: link, don't inline) rather than reasoning from a partial
in-agent description.

Also asserts the consistency rule: if implementation diverges from
the mermaid diagram, the diagram is updated in the same PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Following the auth-expert pattern: surface the existing canonical docs
each new persona should treat as source of truth, via lazy-loaded
markdown links (PROSE progressive disclosure -- link, do not inline).

Per persona:
- supply-chain-security-expert: enterprise/security.md (Security
  Model: attack surfaces, "what APM does/does NOT do",
  pre-deployment scanning gate), lockfile-spec.md, governance.md,
  policy-reference.md, and the cleanup.py / path_security.py
  chokepoints.
- devx-ux-expert: reference/cli-commands.md (canonical CLI surface),
  the getting-started funnel (quick-start, installation, first-package),
  introduction/how-it-works.md (system mental-model mermaid), and
  parallel apm-usage skill resources.
- apm-ceo: MANIFESTO.md, PRD.md, README.md, introduction/why-apm.md
  + what-is-apm.md (canonical positioning), enterprise/making-the-case.md
  + adoption-playbook.md (enterprise framing), CHANGELOG.md.
- oss-growth-hacker: README.md (top of funnel), getting-started
  quick-start + first-package (the 5-minute conversion), introduction
  why-apm + what-is-apm (canonical story arc), templates/ (second-use
  retention), CHANGELOG.md (release narrative raw material).

Each section asserts the consistency rule: if the docs and the change
diverge, the docs are updated in the same PR. This makes the docs the
contract rather than the agent's in-memory description.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@danielmeppiel danielmeppiel merged commit a86f07a into main Apr 20, 2026
9 checks passed
@danielmeppiel danielmeppiel deleted the feat/instrumentation-review-panel branch April 20, 2026 08:30
danielmeppiel added a commit that referenced this pull request Apr 21, 2026
- Bump version to 0.9.0 in pyproject.toml and uv.lock
- Move CHANGELOG [Unreleased] entries into [0.9.0] - 2026-04-21
- Add fresh empty [Unreleased] header
- Consolidate the two scattered ### Changed subsections into one
- Backfill missing entries: build-time self-update policy (#675),
  APM Review Panel skill instrumentation (#777)
- Backfill PR refs on a few entries that referenced issue numbers
- Promote the previously-orphaned 'apm install --mcp' / VS Code adapter /
  init Next Steps lines to consistent (#PR) attribution
- Remove stray blank line inside the ### Added block
- Credit external contributors inline (@arika0093 #700, @joostsijm #675,
  @edenfunf #788)

Highlights of 0.9.0:

BREAKING CHANGES
- MCP entry validation hardened (#807)
- Strict-by-default transport selection (#778)
- Whitespace stdio MCP commands now rejected at parse time (#809)

ADDED
- apm install --mcp for declarative MCP server addition (#810)
- --registry flag and MCP_REGISTRY_URL for custom MCP registries (#810)
- HTTP-dependency support via --allow-insecure dual opt-in (#700)
- apm install --ssh / --https flags for transport selection (#778)
- Multi-target apm.yml + --target flag (#628)
- Marketplace UX: view, outdated, validate (#514)
- Build-time self-update policy for package-manager distros (#675)
- APM Review Panel + 4 specialist personas (#777)

This PR is release machinery and is intentionally excluded from the
[0.9.0] section per .github/instructions/changelog.instructions.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants