-
Notifications
You must be signed in to change notification settings - Fork 5
Add protocol engineering templates, protocols, and formats #84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Alan-Jowett
merged 3 commits into
microsoft:main
from
Alan-Jowett:add-protocol-engineering-templates
Mar 27, 2026
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,163 @@ | ||
| <!-- SPDX-License-Identifier: MIT --> | ||
| <!-- Copyright (c) PromptKit Contributors --> | ||
|
|
||
| --- | ||
| name: protocol-delta | ||
| type: format | ||
| description: > | ||
| Output format for protocol specification amendments. Supports three | ||
| presentation styles: amendment document (section-by-section changes), | ||
| redline view (old vs. new text), and standalone revised specification. | ||
| Tracks normative language changes, backward compatibility, and | ||
| cross-reference updates. | ||
| produces: protocol-delta | ||
| consumes: requirements-document | ||
| --- | ||
|
|
||
| # Format: Protocol Delta | ||
|
|
||
| The output MUST be a structured protocol amendment document. The user | ||
| selects the presentation style (amendment, redline, or standalone) via | ||
| the template's `output_style` parameter; the document structure adapts | ||
| accordingly but always includes the metadata sections. | ||
|
|
||
| ## Document Structure | ||
|
|
||
| ```markdown | ||
| # <Protocol Name> — Protocol Delta | ||
|
|
||
| ## 1. Amendment Summary | ||
| <2–4 sentences: what is being changed, why, and the overall impact | ||
| on existing implementations. Classify the highest-severity change | ||
| category (using the protocol-change-categories taxonomy) present | ||
| in this delta.> | ||
|
|
||
| ## 2. Scope and Applicability | ||
| - **Base specification**: <RFC number, spec version, or document title | ||
| being amended> | ||
| - **Presentation style**: Amendment Document | Redline | Standalone | ||
| Revised Specification | ||
| - **Affected protocol roles**: <e.g., client, server, intermediary, all> | ||
| - **Backward compatibility**: Compatible | Requires negotiation | | ||
| Incompatible | ||
| - **Supersedes**: <list of errata, previous amendments, or | ||
| internet-drafts this delta replaces, if any> | ||
|
|
||
| ## 3. Change Register | ||
|
|
||
| <Table summarizing all changes: | ||
|
|
||
| | Change ID | Section | Category | Summary | Severity | | ||
| |-----------|---------|--------------------------------|----------------------------------|----------| | ||
| | CHG-001 | 3.4 | PC5_STATE_MACHINE_MODIFICATION | Add CLOSING state | High | | ||
| | CHG-002 | 4.1 | PC2_CLARIFICATION | Clarify idle timeout measurement | Low |> | ||
|
|
||
| ## 4. Detailed Changes | ||
|
|
||
| ### CHG-<NNN>: <Short Title> | ||
| - **Section**: <section number(s) in the base specification> | ||
| - **Category**: <full label from protocol-change-categories taxonomy, | ||
| e.g., PC5_STATE_MACHINE_MODIFICATION> | ||
| - **Normative keyword changes**: <e.g., SHOULD → MUST, new MUST added, | ||
| none> | ||
| - **Rationale**: <why this change is needed — reference to errata, bug | ||
| report, interoperability issue, new requirement, etc.> | ||
| - **Backward compatibility**: <specific impact on existing | ||
| implementations> | ||
|
|
||
| #### Amendment View (when output_style = amendment) | ||
| **Current text** (from base specification): | ||
| > <quoted text being replaced or modified> | ||
|
|
||
| **Amended text**: | ||
| > <new text, with normative keywords in UPPERCASE> | ||
|
|
||
| **Editor's note**: <optional clarification of intent> | ||
|
|
||
| #### Redline View (when output_style = redline) | ||
| > ~~old text with deletions struck through~~ **new text in bold** | ||
|
|
||
| #### Standalone View (when output_style = standalone) | ||
| <Full revised section text incorporating the change. No diff markers — | ||
| reads as the new normative text.> | ||
|
|
||
| #### Cross-Reference Updates | ||
| - <list of other sections, requirements, or state machine diagrams | ||
| that must be updated to reflect this change> | ||
|
|
||
| #### Affected Requirements | ||
| - <REQ-IDs from an existing requirements document that are modified, | ||
| added, or invalidated by this change. If no requirements document | ||
| exists, state "No existing requirements document provided."> | ||
|
|
||
| ## 5. State Machine Updates | ||
| <If any changes affect the protocol's state machine, provide: | ||
| - Updated state table or diagram | ||
| - New/modified states, transitions, guards, and actions | ||
| - Comparison with the base specification's state machine | ||
|
|
||
| If no state machine changes, state "No state machine changes in this | ||
| delta."> | ||
|
|
||
| ## 6. Message Format Updates | ||
| <If any changes affect wire formats, provide: | ||
| - Updated message format diagrams or field tables | ||
| - New/modified fields with type, size, encoding, and valid values | ||
| - Comparison with the base specification's message formats | ||
|
|
||
| If no message format changes, state "No message format changes in this | ||
| delta."> | ||
|
|
||
| ## 7. IANA Considerations | ||
| <If any changes require IANA registry updates (new code points, new | ||
| registries, modified registration policies), list them here. | ||
| If no IANA considerations, state "No IANA considerations."> | ||
|
|
||
| ## 8. Security Considerations | ||
| <If any changes affect the protocol's security properties, describe: | ||
| - How the security model changes | ||
| - New threats introduced or mitigated | ||
| - Transition risks (e.g., downgrade attacks during deployment) | ||
|
|
||
| If no security impact, state "No security considerations beyond those | ||
| in the base specification."> | ||
|
|
||
| ## 9. Migration and Deployment | ||
| <Guidance for implementers: | ||
| - What must change in existing implementations | ||
| - Recommended deployment sequence (if coordinated rollout is needed) | ||
| - Version negotiation or feature detection mechanisms | ||
| - Transition period considerations> | ||
|
|
||
| ## 10. Consistency Verification | ||
| <Self-check results: | ||
| - All cross-references updated? (list any that changed) | ||
| - State machine still complete? (no unreachable or deadlock states) | ||
| - No conflicting normative statements introduced? | ||
| - IANA registrations consistent with protocol text? | ||
| - Security considerations updated for all PC7 changes?> | ||
|
|
||
| ## 11. Open Questions | ||
| <Unresolved design decisions, ambiguities, or items needing working | ||
| group / community input. For each: what is unresolved, the options, | ||
| and the consequences of each option.> | ||
|
|
||
| ## 12. Revision History | ||
| <Table: | Version | Date | Author | Changes |> | ||
| ``` | ||
|
|
||
| ## Formatting Rules | ||
|
|
||
| - Changes MUST be ordered by section number in the base specification, | ||
| not by severity. | ||
| - Every change MUST have a category from the protocol-change-categories | ||
| taxonomy. | ||
| - Every change MUST state its backward compatibility impact explicitly. | ||
| - Normative keyword changes (MUST, SHOULD, MAY) MUST be highlighted — | ||
| these are the highest-signal elements for reviewers. | ||
| - Quoted text from the base specification MUST be exact — do not | ||
| paraphrase the original. | ||
| - The consistency verification section MUST NOT be omitted — it is the | ||
| self-check that catches cascading errors. | ||
| - If a section has no applicable content, state "None" or "Not | ||
| applicable" — do NOT omit the section. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,220 @@ | ||
| <!-- SPDX-License-Identifier: MIT --> | ||
| <!-- Copyright (c) PromptKit Contributors --> | ||
|
|
||
| --- | ||
| name: protocol-validation-spec | ||
| type: format | ||
| description: > | ||
| Output format for protocol validation specifications. Structures | ||
| conformance tests around state machine coverage, message format | ||
| verification, error handling, and interoperability scenarios. | ||
| Designed to be consumed by an LLM or engineer building a validation | ||
| tool or test suite. | ||
| produces: protocol-validation-spec | ||
| consumes: requirements-document | ||
| --- | ||
|
|
||
| # Format: Protocol Validation Specification | ||
|
|
||
| The output MUST be a structured protocol validation specification with | ||
| the following sections in this exact order. The document serves as a | ||
| blueprint for building a conformance test suite or validation tool. | ||
|
|
||
| ## Document Structure | ||
|
|
||
| ```markdown | ||
| # <Protocol Name> — Protocol Validation Specification | ||
|
|
||
| ## 1. Overview | ||
| <1–2 paragraphs: what protocol is being validated, the validation | ||
| strategy, and the relationship to the protocol specification and any | ||
| requirements documents. State the validation objective — conformance | ||
| testing, interoperability testing, or both.> | ||
|
|
||
| ## 2. Validation Scope | ||
|
|
||
| ### 2.1 Protocol Under Test | ||
| - **Specification**: <RFC number, spec version, or document title> | ||
| - **Protocol roles validated**: <client, server, intermediary, all> | ||
| - **Protocol layers covered**: <which layers of the protocol stack> | ||
|
|
||
| ### 2.2 In Scope | ||
| <What aspects of the protocol will be validated.> | ||
|
|
||
| ### 2.3 Out of Scope | ||
| <What will NOT be validated, and why. E.g., performance testing, | ||
| stress testing, specific transport layers.> | ||
|
|
||
| ### 2.4 Assumptions and Prerequisites | ||
| <Environment requirements, network topology, tooling, peer | ||
| implementations needed for interoperability tests.> | ||
|
|
||
| ## 3. Requirements Traceability | ||
|
|
||
| <Table mapping protocol requirements to validation test groups: | ||
|
|
||
| | Requirement ID | Requirement Summary | Keyword | Test Group(s) | Coverage | | ||
| |----------------|---------------------|---------|---------------|----------| | ||
| | REQ-XXX-001 | ... | MUST | SM-001, MF-001| Full | | ||
| | REQ-XXX-002 | ... | SHOULD | EH-003 | Partial |> | ||
|
|
||
| Coverage values: Full, Partial (with explanation), None (with | ||
| justification — e.g., requires manual testing, requires specific | ||
| hardware). | ||
|
|
||
| ## 4. State Machine Validation | ||
|
|
||
| ### 4.1 State Inventory | ||
| <Table of all protocol states: | ||
|
|
||
| | State ID | State Name | Entry Conditions | Steady-State Invariants |> | ||
|
|
||
| ### 4.2 Transition Tests | ||
|
|
||
| #### SM-<NNN>: <Transition Name> | ||
| - **From state**: <state name> | ||
| - **Trigger**: <event or input that causes the transition> | ||
| - **Expected action**: <what the implementation must do> | ||
| - **Expected next state**: <target state> | ||
| - **Requirement(s)**: <REQ-IDs> | ||
| - **Validation method**: <how to verify — observe output, inspect | ||
| state, check side effects> | ||
| - **Negative case**: <what happens if the trigger occurs in the | ||
| wrong state — the implementation MUST reject/ignore/error> | ||
|
|
||
| ### 4.3 State Coverage Matrix | ||
| <Matrix: states × events → expected behavior (transition, reject, | ||
| ignore). Every cell must be specified. Empty cells are | ||
| underspecification findings.> | ||
|
|
||
| ## 5. Message Format Validation | ||
|
|
||
| ### 5.1 Message Inventory | ||
| <Table of all message types: | ||
|
|
||
| | Message Type | Direction | Required Fields | Optional Fields |> | ||
|
|
||
| ### 5.2 Format Tests | ||
|
|
||
| #### MF-<NNN>: <Message/Field Name> | ||
| - **Message type**: <message type name> | ||
| - **Field**: <field name> | ||
| - **Valid values**: <enumeration, range, or format specification> | ||
| - **Invalid values to test**: <boundary values, out-of-range, | ||
| malformed encoding, truncated messages> | ||
| - **Requirement(s)**: <REQ-IDs> | ||
| - **Validation method**: <send crafted message, verify acceptance | ||
| or rejection behavior> | ||
|
|
||
| ### 5.3 Encoding and Serialization Tests | ||
| <Tests for byte ordering, alignment, padding, length field accuracy, | ||
| and encoding correctness. Reference specific ABNF rules or encoding | ||
| specifications.> | ||
|
|
||
| ## 6. Error Handling Validation | ||
|
|
||
| ### 6.1 Error Inventory | ||
| <Table of all defined error conditions: | ||
|
|
||
| | Error Code/Type | Trigger Condition | Expected Behavior | Recovery |> | ||
|
|
||
| ### 6.2 Error Tests | ||
|
|
||
| #### EH-<NNN>: <Error Scenario Name> | ||
| - **Error condition**: <what triggers the error> | ||
| - **Injection method**: <how to provoke the error — malformed input, | ||
| timeout, out-of-order message, etc.> | ||
| - **Expected response**: <error code, message, state transition> | ||
| - **Recovery verification**: <how to verify the implementation | ||
| recovers correctly after the error> | ||
| - **Requirement(s)**: <REQ-IDs> | ||
|
|
||
| ## 7. Negotiation and Capability Tests | ||
|
|
||
| <If the protocol includes version negotiation, capability exchange, | ||
| or parameter agreement: | ||
|
|
||
| #### NC-<NNN>: <Negotiation Scenario Name> | ||
| - **Scenario**: <e.g., client offers v2, server only supports v1> | ||
| - **Expected outcome**: <agreed version/capability, or failure mode> | ||
| - **Fallback behavior**: <what happens when negotiation fails> | ||
| - **Requirement(s)**: <REQ-IDs> | ||
|
|
||
| If no negotiation mechanisms exist, state "The protocol does not | ||
| define negotiation mechanisms."> | ||
|
|
||
| ## 8. Interoperability Scenarios | ||
|
|
||
| <End-to-end scenarios that verify two implementations can communicate: | ||
|
|
||
| #### IO-<NNN>: <Scenario Name> | ||
| - **Topology**: <e.g., client A ↔ server B, proxy chain> | ||
| - **Scenario steps**: <numbered sequence of actions> | ||
| - **Success criteria**: <what constitutes successful interoperation> | ||
| - **Variations**: <different implementation choices for SHOULD/MAY | ||
| behaviors that should all interoperate> | ||
| - **Requirement(s)**: <REQ-IDs> | ||
|
|
||
| If interoperability testing is out of scope, state why.> | ||
|
|
||
| ## 9. Boundary and Stress Conditions | ||
|
|
||
| <Tests for boundary values, resource limits, and edge cases: | ||
|
|
||
| #### BC-<NNN>: <Boundary Condition Name> | ||
| - **Condition**: <e.g., maximum message size, maximum concurrent | ||
| connections, timer expiration at exact boundary> | ||
| - **Expected behavior**: <what the spec says should happen> | ||
| - **Requirement(s)**: <REQ-IDs> | ||
|
|
||
| Note: This section covers specification-defined boundaries, not | ||
| performance testing.> | ||
|
|
||
| ## 10. Validation Tool Requirements | ||
|
|
||
| <If the validation spec is intended to inform tool construction: | ||
| - **Input format**: <how protocol traffic is captured or injected — | ||
| pcap, API calls, socket-level> | ||
| - **Oracle definition**: <how the tool determines pass/fail — compare | ||
| against state machine model, verify field values, check error codes> | ||
| - **Reporting requirements**: <what the tool should report — per-test | ||
| pass/fail, coverage summary, trace logs> | ||
| - **Automation constraints**: <timing sensitivity, ordering | ||
| requirements, environmental dependencies that affect automation>> | ||
|
|
||
| ## 11. Coverage Summary | ||
| <Aggregate coverage statistics: | ||
|
|
||
| | Category | Total Tests | MUST Coverage | SHOULD Coverage | MAY Coverage | | ||
| |----------|-------------|---------------|-----------------|--------------| | ||
| | State Machine | NN | NN/NN (100%) | NN/NN (NN%) | NN/NN (NN%) | | ||
| | Message Format | NN | ... | ... | ... | | ||
| | Error Handling | NN | ... | ... | ... | | ||
| | Negotiation | NN | ... | ... | ... | | ||
| | Interoperability | NN | ... | ... | ... | | ||
| | Boundary | NN | ... | ... | ... | | ||
|
|
||
| - **Gaps**: <list any requirements with no test coverage and why>> | ||
|
|
||
| ## 12. Revision History | ||
| <Table: | Version | Date | Author | Changes |> | ||
| ``` | ||
|
|
||
| ## Formatting Rules | ||
|
|
||
| - When a requirements document with REQ-IDs is provided, every test MUST | ||
| reference at least one requirement (REQ-ID). When no such document is | ||
| provided, every test MUST instead cite the relevant protocol specification | ||
| section(s) and the normative statement being validated. | ||
| - When requirements with REQ-IDs exist, every MUST requirement MUST have at | ||
| least one test. SHOULD and MAY requirements SHOULD have tests but may be | ||
| deferred with justification. | ||
| - Tests MUST include both positive (valid input → correct behavior) and | ||
| negative (invalid input → correct rejection/error) cases. | ||
| - The state coverage matrix MUST have no empty cells — every | ||
| state × event combination must specify the expected behavior. | ||
| - Test IDs use prefixed sequences: SM- (state machine), MF- (message | ||
| format), EH- (error handling), NC- (negotiation/capability), | ||
| IO- (interoperability), BC- (boundary condition). | ||
| - If a section has no applicable content, state "Not applicable" with | ||
| a brief explanation — do NOT omit the section. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.