Skip to content

Specification-drift taxonomy: support scoped label groups to reduce prompt overhead #113

@Alan-Jowett

Description

@Alan-Jowett

Problem

The specification-drift taxonomy (taxonomies/specification-drift.md) includes all 16 labels (D1-D16) in a single file. Each audit type uses a disjoint subset:

  • Trifecta audit (audit-traceability): D1-D7
  • Code compliance audit (audit-code-compliance): D8-D10
  • Test compliance audit (audit-test-compliance): D11-D13
  • Integration audit (audit-integration-compliance): D14-D16

When the taxonomy is included in an assembled prompt, the full D1-D16 text is provided regardless of which labels are applicable. For the code compliance audit, D1-D7 and D11-D16 definitions consume ~1,125+ prompt tokens that are never used for classification.

Why simple removal is wrong

Later label groups cross-reference earlier ones:

  • D8-D10 are described as code-level variants of D1-D7 concepts (lines 149-203)
  • D11-D13 reference acceptance criteria concepts from D1-D7 (lines 205-271)
  • The Ranking Criteria section explicitly ranks across ALL groups (e.g., D6/D7/D10/D13/D15 are tier 1)

So the D1-D7 definitions provide semantic context that helps understand D8-D10. Full removal would degrade classification accuracy.

Evidence

A profiling run of three PromptKit audit prompts against the Sonde protocol crate found:

  • Code compliance audit: D1-D7 definitions (~3,200 chars, ~800 tokens) included in prompt. D1-D7 labels never assigned in any finding — only D8, D9, D10 used. (Finding F-005)
  • Test compliance audit: D1-D10 definitions (~5,100 chars, ~1,275 tokens) included in prompt. Only D12 and D13 assigned. (Finding F-008)
  • Combined: ~2,075 tokens of non-primary label definitions across two audits that provide context but are never used for classification

Proposed Change

Restructure the taxonomy to support scoped inclusion while preserving cross-group context:

Option A: Add summary blocks for non-primary groups

Add a condensed summary section at the top of the taxonomy:

## Label Group Summaries (for cross-reference context)
- **D1-D7 (Traceability)**: Document-level drift — untraced requirements,
  orphaned design/test artifacts, assumption conflicts, constraint violations
- **D8-D10 (Code Compliance)**: Code-to-spec drift — unimplemented requirements,
  undocumented behavior, constraint violations in code
- **D11-D13 (Test Compliance)**: Test-to-plan drift — unimplemented test cases,
  untested acceptance criteria, assertion mismatches
- **D14-D16 (Integration)**: Cross-component drift — unspecified flows,
  interface mismatches, untested integration paths

Then the assembly engine or the template frontmatter could specify which label group is primary (full definitions) and which are context-only (summaries).

Option B: Split into per-group files

Split specification-drift.md into:

  • specification-drift-traceability.md (D1-D7 + ranking criteria)
  • specification-drift-code.md (D8-D10 + ranking criteria)
  • specification-drift-test.md (D11-D13 + ranking criteria)
  • specification-drift-integration.md (D14-D16 + ranking criteria)

Each file includes the group summaries for context plus full definitions for its own labels.

Recommendation

Option A is simpler — it requires no manifest or template changes, just adding a summary block to the existing file and letting the assembly process include a label_group parameter to control which group gets full definitions.

Estimated Impact

~800-1,275 prompt token savings per non-primary audit. Applies to code-compliance, test-compliance, and integration audit templates.

Risk

Low — the summary block provides sufficient cross-group context for understanding how labels relate. The Ranking Criteria section (which references all groups) would remain in full.

Source

Session profiling of three specification audits against the Sonde protocol crate (March 2026). Findings F-005, F-008, SI-1, R-002 from the profiling report.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions