Add embedded-firmware-engineer persona, spec-invariant-audit protocol, and audit-spec-invariants template#85
Merged
Alan-Jowett merged 1 commit intomicrosoft:mainfrom Mar 28, 2026
Conversation
…, and audit-spec-invariants template New components for adversarial specification analysis against user-supplied invariants: - Persona: embedded-firmware-engineer — domain expertise in boot sequences, flash management, OTA updates, power-fail-safe operations, watchdog timers, and device recovery mechanisms. - Protocol: spec-invariant-audit — 7-phase reasoning protocol that systematically constructs compliant-but-violating interpretations to find spec gaps, ambiguities, contradictions, and missing recovery paths. Domain-agnostic (works with any persona). - Template: audit-spec-invariants — assembles persona + protocol + investigation-report format. Uses configurable persona so it can be applied to any domain (firmware, protocols, safety systems, etc.). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds new PromptKit components to support adversarial auditing of specifications against user-provided invariants, including a reusable reasoning protocol, an embedded-firmware persona, and a new template that emits an investigation report.
Changes:
- Added
spec-invariant-auditreasoning protocol with a 7-phase invariant-focused audit methodology. - Added
audit-spec-invariantstemplate to apply the protocol (with configurable persona) and produce aninvestigation-report. - Added
embedded-firmware-engineerpersona and registered all new components inmanifest.yaml.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
templates/audit-spec-invariants.md |
New template wiring guardrails + spec-invariant-audit protocol into the investigation-report format. |
protocols/reasoning/spec-invariant-audit.md |
New 7-phase reasoning protocol for adversarial invariant audits of specifications. |
personas/embedded-firmware-engineer.md |
New persona focused on embedded firmware failure modes (boot/update/power-loss/recovery). |
manifest.yaml |
Registers the new persona, protocol, and template so the CLI can assemble them. |
|
|
||
| 5. **Format the output** according to the investigation-report format | ||
| with these audit-specific additions: | ||
| - Group findings by invariant violated |
There was a problem hiding this comment.
The instruction to “Group findings by invariant violated” conflicts with the investigation-report format requirement that findings be ordered by severity (Critical first). Please adjust the template guidance so the Findings section remains severity-ordered (e.g., severity buckets first, then group by invariant within each bucket, or provide an invariant-grouped appendix in addition to the severity-ordered findings list).
Suggested change
| - Group findings by invariant violated | |
| - In the primary **Findings** section, maintain severity ordering as required | |
| by the investigation-report format (Critical first, then High, Medium, Low). | |
| Within each severity bucket, you may group or clearly label findings by | |
| invariant violated. | |
| - You may add an appendix that regroups the same findings by invariant | |
| violated for cross-reference. Do not introduce new findings in the appendix; | |
| it must only re-present findings already listed in the severity-ordered | |
| Findings section. |
| interpretation | ||
| - **Disproof attempt**: What you checked to try to disprove this | ||
| finding, and why disproof failed | ||
| - **Confidence**: Confirmed / High-confidence / Needs-domain-check |
There was a problem hiding this comment.
This protocol’s finding “Confidence” scale (Confirmed / High-confidence / Needs-domain-check) doesn’t match the investigation-report format’s required confidence values (High / Medium / Low). To prevent format drift, align the protocol with the format’s scale or explicitly instruct how to map these confidence levels into the format’s allowed values.
Suggested change
| - **Confidence**: Confirmed / High-confidence / Needs-domain-check | |
| - **Confidence**: High / Medium / Low (per investigation-report format) |
Alan-Jowett
pushed a commit
to Alan-Jowett/PromptKit
that referenced
this pull request
Mar 28, 2026
- Template: maintain severity ordering (Critical first) per investigation-report format instead of grouping by invariant. Invariant-grouped appendix is optional. - Protocol: align confidence scale to High/Medium/Low per investigation-report format instead of Confirmed/High-confidence/ Needs-domain-check. Addresses review comments from PR microsoft#85. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Alan-Jowett
added a commit
that referenced
this pull request
Mar 28, 2026
* Fix review feedback on spec-invariant-audit components - Template: maintain severity ordering (Critical first) per investigation-report format instead of grouping by invariant. Invariant-grouped appendix is optional. - Protocol: align confidence scale to High/Medium/Low per investigation-report format instead of Confirmed/High-confidence/ Needs-domain-check. Addresses review comments from PR #85. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update templates/audit-spec-invariants.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Alan Jowett <alan.jowett@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three new components for adversarial specification analysis against user-supplied invariants. The protocol is domain-agnostic; the persona provides embedded firmware expertise; the template assembles them with a configurable persona so any domain can use the audit methodology.
New Components
Design Decisions
Checklist