Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions formats/protocol-delta.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
<!-- SPDX-License-Identifier: MIT -->
<!-- Copyright (c) PromptKit Contributors -->

---
name: protocol-delta
type: format
description: >
Output format for protocol specification amendments. Supports three
presentation styles: amendment document (section-by-section changes),
redline view (old vs. new text), and standalone revised specification.
Tracks normative language changes, backward compatibility, and
cross-reference updates.
produces: protocol-delta
consumes: requirements-document
---

# Format: Protocol Delta

The output MUST be a structured protocol amendment document. The user
selects the presentation style (amendment, redline, or standalone) via
the template's `output_style` parameter; the document structure adapts
accordingly but always includes the metadata sections.

## Document Structure

```markdown
# <Protocol Name> — Protocol Delta

## 1. Amendment Summary
<2–4 sentences: what is being changed, why, and the overall impact
on existing implementations. Classify the highest-severity change
category (using the protocol-change-categories taxonomy) present
in this delta.>

## 2. Scope and Applicability
- **Base specification**: <RFC number, spec version, or document title
being amended>
- **Presentation style**: Amendment Document | Redline | Standalone
Revised Specification
- **Affected protocol roles**: <e.g., client, server, intermediary, all>
- **Backward compatibility**: Compatible | Requires negotiation |
Incompatible
- **Supersedes**: <list of errata, previous amendments, or
internet-drafts this delta replaces, if any>

## 3. Change Register

<Table summarizing all changes:

| Change ID | Section | Category | Summary | Severity |
|-----------|---------|--------------------------------|----------------------------------|----------|
| CHG-001 | 3.4 | PC5_STATE_MACHINE_MODIFICATION | Add CLOSING state | High |
| CHG-002 | 4.1 | PC2_CLARIFICATION | Clarify idle timeout measurement | Low |>

## 4. Detailed Changes

### CHG-<NNN>: <Short Title>
- **Section**: <section number(s) in the base specification>
- **Category**: <full label from protocol-change-categories taxonomy,
e.g., PC5_STATE_MACHINE_MODIFICATION>
- **Normative keyword changes**: <e.g., SHOULD → MUST, new MUST added,
none>
- **Rationale**: <why this change is needed — reference to errata, bug
report, interoperability issue, new requirement, etc.>
- **Backward compatibility**: <specific impact on existing
implementations>

#### Amendment View (when output_style = amendment)
**Current text** (from base specification):
> <quoted text being replaced or modified>

**Amended text**:
> <new text, with normative keywords in UPPERCASE>

**Editor's note**: <optional clarification of intent>

#### Redline View (when output_style = redline)
> ~~old text with deletions struck through~~ **new text in bold**

#### Standalone View (when output_style = standalone)
<Full revised section text incorporating the change. No diff markers —
reads as the new normative text.>

#### Cross-Reference Updates
- <list of other sections, requirements, or state machine diagrams
that must be updated to reflect this change>

#### Affected Requirements
- <REQ-IDs from an existing requirements document that are modified,
added, or invalidated by this change. If no requirements document
exists, state "No existing requirements document provided.">

## 5. State Machine Updates
<If any changes affect the protocol's state machine, provide:
- Updated state table or diagram
- New/modified states, transitions, guards, and actions
- Comparison with the base specification's state machine

If no state machine changes, state "No state machine changes in this
delta.">

## 6. Message Format Updates
<If any changes affect wire formats, provide:
- Updated message format diagrams or field tables
- New/modified fields with type, size, encoding, and valid values
- Comparison with the base specification's message formats

If no message format changes, state "No message format changes in this
delta.">

## 7. IANA Considerations
<If any changes require IANA registry updates (new code points, new
registries, modified registration policies), list them here.
If no IANA considerations, state "No IANA considerations.">

## 8. Security Considerations
<If any changes affect the protocol's security properties, describe:
- How the security model changes
- New threats introduced or mitigated
- Transition risks (e.g., downgrade attacks during deployment)

If no security impact, state "No security considerations beyond those
in the base specification.">

## 9. Migration and Deployment
<Guidance for implementers:
- What must change in existing implementations
- Recommended deployment sequence (if coordinated rollout is needed)
- Version negotiation or feature detection mechanisms
- Transition period considerations>

## 10. Consistency Verification
<Self-check results:
- All cross-references updated? (list any that changed)
- State machine still complete? (no unreachable or deadlock states)
- No conflicting normative statements introduced?
- IANA registrations consistent with protocol text?
- Security considerations updated for all PC7 changes?>

## 11. Open Questions
<Unresolved design decisions, ambiguities, or items needing working
group / community input. For each: what is unresolved, the options,
and the consequences of each option.>

## 12. Revision History
<Table: | Version | Date | Author | Changes |>
```

## Formatting Rules

- Changes MUST be ordered by section number in the base specification,
not by severity.
- Every change MUST have a category from the protocol-change-categories
taxonomy.
- Every change MUST state its backward compatibility impact explicitly.
- Normative keyword changes (MUST, SHOULD, MAY) MUST be highlighted —
these are the highest-signal elements for reviewers.
- Quoted text from the base specification MUST be exact — do not
paraphrase the original.
- The consistency verification section MUST NOT be omitted — it is the
self-check that catches cascading errors.
- If a section has no applicable content, state "None" or "Not
applicable" — do NOT omit the section.
220 changes: 220 additions & 0 deletions formats/protocol-validation-spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
<!-- SPDX-License-Identifier: MIT -->
<!-- Copyright (c) PromptKit Contributors -->

---
name: protocol-validation-spec
type: format
description: >
Output format for protocol validation specifications. Structures
conformance tests around state machine coverage, message format
verification, error handling, and interoperability scenarios.
Designed to be consumed by an LLM or engineer building a validation
tool or test suite.
produces: protocol-validation-spec
consumes: requirements-document
---

# Format: Protocol Validation Specification

The output MUST be a structured protocol validation specification with
the following sections in this exact order. The document serves as a
blueprint for building a conformance test suite or validation tool.

## Document Structure

```markdown
# <Protocol Name> — Protocol Validation Specification

## 1. Overview
<1–2 paragraphs: what protocol is being validated, the validation
strategy, and the relationship to the protocol specification and any
requirements documents. State the validation objective — conformance
testing, interoperability testing, or both.>

## 2. Validation Scope

### 2.1 Protocol Under Test
- **Specification**: <RFC number, spec version, or document title>
- **Protocol roles validated**: <client, server, intermediary, all>
- **Protocol layers covered**: <which layers of the protocol stack>

### 2.2 In Scope
<What aspects of the protocol will be validated.>

### 2.3 Out of Scope
<What will NOT be validated, and why. E.g., performance testing,
stress testing, specific transport layers.>

### 2.4 Assumptions and Prerequisites
<Environment requirements, network topology, tooling, peer
implementations needed for interoperability tests.>

## 3. Requirements Traceability

<Table mapping protocol requirements to validation test groups:

| Requirement ID | Requirement Summary | Keyword | Test Group(s) | Coverage |
|----------------|---------------------|---------|---------------|----------|
| REQ-XXX-001 | ... | MUST | SM-001, MF-001| Full |
| REQ-XXX-002 | ... | SHOULD | EH-003 | Partial |>

Coverage values: Full, Partial (with explanation), None (with
justification — e.g., requires manual testing, requires specific
hardware).

## 4. State Machine Validation

### 4.1 State Inventory
<Table of all protocol states:

| State ID | State Name | Entry Conditions | Steady-State Invariants |>

### 4.2 Transition Tests

#### SM-<NNN>: <Transition Name>
- **From state**: <state name>
- **Trigger**: <event or input that causes the transition>
- **Expected action**: <what the implementation must do>
- **Expected next state**: <target state>
- **Requirement(s)**: <REQ-IDs>
- **Validation method**: <how to verify — observe output, inspect
state, check side effects>
- **Negative case**: <what happens if the trigger occurs in the
wrong state — the implementation MUST reject/ignore/error>

### 4.3 State Coverage Matrix
<Matrix: states × events → expected behavior (transition, reject,
ignore). Every cell must be specified. Empty cells are
underspecification findings.>

## 5. Message Format Validation

### 5.1 Message Inventory
<Table of all message types:

| Message Type | Direction | Required Fields | Optional Fields |>

### 5.2 Format Tests

#### MF-<NNN>: <Message/Field Name>
- **Message type**: <message type name>
- **Field**: <field name>
- **Valid values**: <enumeration, range, or format specification>
- **Invalid values to test**: <boundary values, out-of-range,
malformed encoding, truncated messages>
- **Requirement(s)**: <REQ-IDs>
- **Validation method**: <send crafted message, verify acceptance
or rejection behavior>

### 5.3 Encoding and Serialization Tests
<Tests for byte ordering, alignment, padding, length field accuracy,
and encoding correctness. Reference specific ABNF rules or encoding
specifications.>

## 6. Error Handling Validation

### 6.1 Error Inventory
<Table of all defined error conditions:

| Error Code/Type | Trigger Condition | Expected Behavior | Recovery |>

### 6.2 Error Tests

#### EH-<NNN>: <Error Scenario Name>
- **Error condition**: <what triggers the error>
- **Injection method**: <how to provoke the error — malformed input,
timeout, out-of-order message, etc.>
- **Expected response**: <error code, message, state transition>
- **Recovery verification**: <how to verify the implementation
recovers correctly after the error>
- **Requirement(s)**: <REQ-IDs>

## 7. Negotiation and Capability Tests

<If the protocol includes version negotiation, capability exchange,
or parameter agreement:

#### NC-<NNN>: <Negotiation Scenario Name>
- **Scenario**: <e.g., client offers v2, server only supports v1>
- **Expected outcome**: <agreed version/capability, or failure mode>
- **Fallback behavior**: <what happens when negotiation fails>
- **Requirement(s)**: <REQ-IDs>

If no negotiation mechanisms exist, state "The protocol does not
define negotiation mechanisms.">

## 8. Interoperability Scenarios

<End-to-end scenarios that verify two implementations can communicate:

#### IO-<NNN>: <Scenario Name>
- **Topology**: <e.g., client A ↔ server B, proxy chain>
- **Scenario steps**: <numbered sequence of actions>
- **Success criteria**: <what constitutes successful interoperation>
- **Variations**: <different implementation choices for SHOULD/MAY
behaviors that should all interoperate>
- **Requirement(s)**: <REQ-IDs>

If interoperability testing is out of scope, state why.>

## 9. Boundary and Stress Conditions

<Tests for boundary values, resource limits, and edge cases:

#### BC-<NNN>: <Boundary Condition Name>
- **Condition**: <e.g., maximum message size, maximum concurrent
connections, timer expiration at exact boundary>
- **Expected behavior**: <what the spec says should happen>
- **Requirement(s)**: <REQ-IDs>

Note: This section covers specification-defined boundaries, not
performance testing.>

## 10. Validation Tool Requirements

<If the validation spec is intended to inform tool construction:
- **Input format**: <how protocol traffic is captured or injected —
pcap, API calls, socket-level>
- **Oracle definition**: <how the tool determines pass/fail — compare
against state machine model, verify field values, check error codes>
- **Reporting requirements**: <what the tool should report — per-test
pass/fail, coverage summary, trace logs>
- **Automation constraints**: <timing sensitivity, ordering
requirements, environmental dependencies that affect automation>>

## 11. Coverage Summary
<Aggregate coverage statistics:

| Category | Total Tests | MUST Coverage | SHOULD Coverage | MAY Coverage |
|----------|-------------|---------------|-----------------|--------------|
| State Machine | NN | NN/NN (100%) | NN/NN (NN%) | NN/NN (NN%) |
| Message Format | NN | ... | ... | ... |
| Error Handling | NN | ... | ... | ... |
| Negotiation | NN | ... | ... | ... |
| Interoperability | NN | ... | ... | ... |
| Boundary | NN | ... | ... | ... |

- **Gaps**: <list any requirements with no test coverage and why>>

## 12. Revision History
<Table: | Version | Date | Author | Changes |>
```

## Formatting Rules

- When a requirements document with REQ-IDs is provided, every test MUST
reference at least one requirement (REQ-ID). When no such document is
provided, every test MUST instead cite the relevant protocol specification
section(s) and the normative statement being validated.
- When requirements with REQ-IDs exist, every MUST requirement MUST have at
least one test. SHOULD and MAY requirements SHOULD have tests but may be
deferred with justification.
- Tests MUST include both positive (valid input → correct behavior) and
negative (invalid input → correct rejection/error) cases.
- The state coverage matrix MUST have no empty cells — every
state × event combination must specify the expected behavior.
- Test IDs use prefixed sequences: SM- (state machine), MF- (message
format), EH- (error handling), NC- (negotiation/capability),
IO- (interoperability), BC- (boundary condition).
- If a section has no applicable content, state "Not applicable" with
a brief explanation — do NOT omit the section.
Loading
Loading