specproof

Specification Compiler & Dead Spec Detector

Studies show that 75% of logic errors in production trace back to ambiguous or incomplete specifications. specproof catches these before they become bugs.

What it does

Two modes:

1. Spec Compiler (compile) — Analyzes a natural-language specification (markdown, RFC, Jira-style) and produces:

   • Extracted code entities (functions, classes, API paths, config keys)
   • Ambiguity detection with actionable suggestions
   • Testable acceptance criteria extraction
   • Cross-reference against actual codebase via AST parsing

2. Dead Spec Detector (audit) — Scans documentation directories and computes a "liveness score" — what percentage of code entities referenced in docs actually exist in the codebase.

Install

npm install -g @phoenixaihub/specproof

CLI Usage

# Compile a spec — find ambiguities and extract criteria
specproof compile RFC-042.md --codebase ./src

# Audit all docs against codebase
specproof audit ./docs --codebase ./src

# Pretty-print output
specproof compile spec.md --codebase ./src --pretty

Programmatic API

import { compileSpec, auditDocs } from '@phoenixaihub/specproof';

// Compile a single spec
const result = compileSpec('spec.md', './src');
console.log(result.ambiguities);
console.log(result.acceptance_criteria);
console.log(result.liveness_score);

// Audit all docs
const audit = auditDocs('./docs', './src');
console.log(`Overall liveness: ${audit.overall_liveness}%`);

Output Format

{
  "mode": "compile",
  "spec_file": "RFC-042.md",
  "entities_found": 24,
  "entities_matched": 18,
  "entities_missing": ["processRefund", "RATE_LIMIT_MAX"],
  "ambiguities": [
    {
      "line": 12,
      "text": "the system should handle errors appropriately",
      "suggestion": "Specify: which errors, what handling (retry? log? alert?), what timeout"
    }
  ],
  "acceptance_criteria": [
    { "id": "AC-1", "testable": true, "criterion": "POST /api/payments returns 429 when rate > 100 req/min" },
    { "id": "AC-2", "testable": false, "criterion": "System handles edge cases", "reason": "No edge cases specified" }
  ],
  "liveness_score": 75
}

Algorithms

Entity Extraction

Regex + pattern matching for code identifiers:

• camelCase / PascalCase / snake_case / UPPER_SNAKE identifiers
• API paths (/api/v1/...)
• Backtick-quoted code blocks (highest confidence)
• File paths and config keys

AST Cross-Reference

• Uses tree-sitter to parse TypeScript, JavaScript, and Python codebases
• Builds a symbol table of all declared identifiers
• Falls back to regex extraction when tree-sitter is unavailable
• Matches spec entities against actual codebase symbols

Ambiguity Detection

Flags 20+ vague patterns including:

• "should" → Use "must" or "shall"
• "appropriate" → Define specific criteria
• "the system" → Name the specific component
• "handle errors" → Specify which errors and what handling
• "scalable", "fast", "secure" → Define measurable targets

Constraint Extraction

Identifies testable requirements:

• HTTP status codes (returns 429)
• Timeouts/SLAs (within 500ms)
• Numeric limits (maximum 100 connections)
• Mandatory behaviors (must return, shall not)

Liveness Scoring

(matched entities / total referenced entities) × 100

License

MIT