Skip to content

Add coverage gap audit workflow#258

Merged
Alan-Jowett merged 3 commits into
microsoft:mainfrom
Alan-Jowett:add-coverage-gap-audit
Jun 3, 2026
Merged

Add coverage gap audit workflow#258
Alan-Jowett merged 3 commits into
microsoft:mainfrom
Alan-Jowett:add-coverage-gap-audit

Conversation

@Alan-Jowett
Copy link
Copy Markdown
Member

Summary

Adds a coverage-driven audit path to PromptKit for spec-first maintenance work. The new protocol turns uncovered code into deterministic drift candidates, and the new template packages that method as a reusable audit using existing specification-analysis components.

New Components

Type Name Path Description
protocol coverage-gap-analysis protocols/reasoning/coverage-gap-analysis.md Deterministic coverage-first protocol that normalizes uncovered regions, filters incidental code, and classifies missing validation versus undocumented behavior.
template audit-coverage-gaps templates/audit-coverage-gaps.md Audit template that uses coverage data to trace uncovered regions back to requirements, validation artifacts, and tests.

Design Decisions

  • Reused specification-analyst, investigation-report, and specification-drift instead of adding new persona, format, or taxonomy components because the contribution is a new discovery method, not a new output shape.
  • Named the components around coverage gaps rather than coverage drift to avoid implying a new drift taxonomy category.
  • Added an explicit disambiguation phase so uncovered regions are filtered before classification; this avoids turning inactive paths, generated code, or incidental infrastructure into false D9 findings.
  • Required validation_plan and test_code in the template inputs so the workflow can distinguish D2 missing validation from D11/D12/D13 test-level gaps.
  • Kept the protocol focused on uncovered regions only and explicitly stated that covered code is not cleared by this audit.

Checklist

  • All files have SPDX license headers
  • YAML frontmatter is valid and complete
  • Component names match file names (kebab-case)
  • manifest.yaml updated with all new components
  • No vague instructions in protocols or templates
  • Protocols have numbered, ordered phases
  • Templates have a quality checklist section
  • New components do not conflict with existing ones

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 3, 2026 16:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a coverage-driven audit workflow to PromptKit by introducing a new reasoning protocol for coverage-gap triage and a companion audit template that applies the specification-drift taxonomy to uncovered regions, with manifest.yaml updated to register both components.

Changes:

  • Added coverage-gap-analysis reasoning protocol to turn uncovered regions into reproducible drift candidates and classification steps.
  • Added audit-coverage-gaps template that packages the protocol into an investigation-report-style audit using the existing specification-drift taxonomy.
  • Updated manifest.yaml to include the new protocol and template entries.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
templates/audit-coverage-gaps.md New audit template that consumes coverage artifacts plus spec/validation/test inputs and produces an investigation report.
protocols/reasoning/coverage-gap-analysis.md New deterministic protocol defining phased coverage-gap inventory, disambiguation, tracing, and drift classification.
manifest.yaml Registers the new protocol and template so they can be discovered/assembled by the bootstrap engine.

Comment thread templates/audit-coverage-gaps.md
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 3, 2026 16:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread templates/audit-coverage-gaps.md Outdated
Comment thread protocols/reasoning/coverage-gap-analysis.md
Comment thread protocols/reasoning/coverage-gap-analysis.md
Comment thread templates/audit-coverage-gaps.md
Comment thread protocols/reasoning/coverage-gap-analysis.md Outdated
Comment thread templates/audit-coverage-gaps.md
Address PR feedback by clarifying the allowed drift labels and their scope in the coverage gap audit workflow. Make excluded and inconclusive regions map cleanly to the investigation report structure, add synchronization contracts to behavioral significance, normalize next-action wording, and state the one-region-one-label rule for confirmed findings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Alan-Jowett Alan-Jowett merged commit b2c2d4a into microsoft:main Jun 3, 2026
4 checks passed
@Alan-Jowett Alan-Jowett deleted the add-coverage-gap-audit branch June 3, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants