Skip to content

feat(framework): v1 unified audit prompt template + schema evolution (R11(B))#105

Merged
montfort merged 1 commit intomainfrom
feat/audit-unified-prompt-template
May 5, 2026
Merged

feat(framework): v1 unified audit prompt template + schema evolution (R11(B))#105
montfort merged 1 commit intomainfrom
feat/audit-unified-prompt-template

Conversation

@montfort
Copy link
Copy Markdown
Contributor

@montfort montfort commented May 5, 2026

Summary

Third of 8 PRs implementing the integrated v1 audit-skills iteration described in `Propuesta/devtrail-audit-cli-flow.md` v0.2. Closes the prompt content side of R11(B) of issue #102 — paste-based audits without tool use produce structurally limited findings; the new template enforces the discipline at prompt level.

Strategy: artifact PR, not refactor PR

This PR ships the new template + schema update without yet touching the CLI resolver or removing the v0 templates. PR 4 will do the refactor (switch resolver to read `audit-prompt.md`, delete legacy templates, migrate canonical paths to `.devtrail/audits/`, simplify subcommand). Splitting it this way keeps each PR bisect-safe and reviewable.

What ships

NEW `dist/.devtrail/audit-prompts/audit-prompt.md` (325 lines)

Lifts integrally the seven universal sections from Sentinel's pre-DevTrail `audit/SKILL.md` skill (contributed via issue #102):

  1. ⛔ REGLA ABSOLUTA — SOLO LECTURA — read-only enforcement: "Si encuentras un bug, DOCUMENTALO. NO lo corrijas. Violación de esta regla invalida toda la auditoría."
  2. Tu rol — anti-cheerleader pattern.
  3. Reglas de alcance — out-of-scope handled as separate notes, not as defects.
  4. Paso 2 verificación obligatoria — chain of tool-use evidence: locate → read FULL file → trace handler→service→repo→storage → read ≥2 test cases → cite `path:line`. Includes an explicit "Disciplina de evidencia" sub-block requiring tool-call-backed citations.
  5. Paso 5 calibración severidad contra config REAL — anti-inflation/anti-deflation rules with mandatory checklist (active driver, feature flags, build tags, DB role, deployment scope). Etapa 12 example preserved verbatim as labeled real adopter case.
  6. Lo que NO debes hacer — closing reinforcement.
  7. Formato de salida — required frontmatter shape + body structure.

Sentinel hardcodes parameterized

Sentinel hardcode DevTrail v1 placeholder
`specs/001-sentinel-mvp/tasks.md` `{{charter_path}}` / `{{charter_content}}`
`## Etapa N:` heading `{{charter_id}}` / `{{charter_title}}`
Project description "Sentinel..." `{{project_context}}` (NEW, optional)
`go vet/build/test` commands preserved as block-quote with Rust/TS/Python parallels
`internal/modules/identity/` paths preserved as didactic illustrations (labeled)

UPDATE `audit-output.schema.v0.json`

  • `audit_role` enum extended to `["auditor", "auditor-primary", "auditor-secondary"]` — v1 unified + v0 legacy coexist during transition.
  • NEW optional `evidence_citations: integer (>=0)` — review-skill weighting hint.
  • `calibratorOutput.auditors_reconciled.maxItems` removed: v1 supports N≥2 auditors.

Credit

Explicit attribution to José Villaseñor Montfort (StrangeDaysTech) for the contributed source material at top (HTML comment header) and bottom (closing italics).

Tests

9 fixture tests in `cli/tests/audit_template_test.rs`:

NOT in this PR (deferred to PR 4)

  • DELETE `auditor-primary.md`, `auditor-secondary.md`, `calibrator-reconciler.md`.
  • CLI resolver switch to read `audit-prompt.md`.
  • Canonical path migration (`audit/charters/` → `.devtrail/audits/`).
  • Subcommand simplification (`--prepare` / `--merge-reports`).
  • Deprecation shims.

Test plan

  • `cargo test --test audit_template_test` → 9/9 green
  • `cargo test` (full suite) → all passing, no regressions in existing 14 charter_audit + 8 audit_skill tests
  • No version bump (lands together with PRs 4-8 in the integrated v1 release)

Phase v1 progress

PR Title Status
1 R10 resolver fix merged (#103)
2 `git_range` default → `origin/main..HEAD` with fallback merged (#104)
3 Unified `audit-prompt.md` template (Sentinel skill lift) this PR
4 CLI subcommand simplified + canonical paths + deprecation shims pending
5 New skill `devtrail-audit-execute` (3 platforms) pending
6 Updated `audit-prompt` and `audit-review` skills (review.md consolidated) pending
7 Adopter docs (3 langs) pending
8 Bump `fw-4.9.0` / `cli-3.10.0` + CHANGELOG + tag release pending

🤖 Generated with Claude Code

…(issue #102 R11(B))

Lifts the seven universal sections from Sentinel's pre-DevTrail audit
skill (audit/SKILL.md, contributed via issue #102) into a unified
DevTrail v1 prompt template, parameterizing the project-specific
hardcodes against the Charter doc / git range / originating AILOGs.
This is the artifact PR — it ships the new template and schema update
without yet touching the CLI resolver or removing the v0 templates.
PR 4 will refactor the CLI to use this artifact and clean up legacy.

NEW: dist/.devtrail/audit-prompts/audit-prompt.md (325 lines)

Lifts integrally the seven universal sections that the issue #102
follow-up identified as portable:

  1. ⛔ REGLA ABSOLUTA — SOLO LECTURA (read-only enforcement: "Si
     encuentras un bug, DOCUMENTALO. NO lo corrijas. Violación de
     esta regla invalida toda la auditoría.")
  2. Tu rol — anti-cheerleader pattern.
  3. Reglas de alcance — out-of-scope handled as separate notes, not
     as defects of the unit being audited.
  4. Paso 2 verificación obligatoria — chain of tool-use evidence:
     locate → read FULL file → trace handler→service→repo→storage →
     read ≥2 test cases → cite path:line. Includes an explicit
     "Disciplina de evidencia" sub-block requiring tool-call-backed
     citations (R11(B) of issue #102 — paste-based audits without
     tool use produce structurally limited findings).
  5. Paso 5 calibración severidad contra config REAL — anti-inflation
     and anti-deflation rules with the obligatory checklist (active
     driver, feature flags, build tags, DB role, deployment scope)
     before declaring Critical/High. Etapa 12 example (Pub/Sub stub
     vs gochannel active) preserved verbatim as a labeled real
     adopter case.
  6. Lo que NO debes hacer — closing reinforcement (no modify, no
     inflate, no consult external sources).
  7. Formato de salida — required frontmatter shape + body structure
     (Resumen, Verificación de compilación, Trazabilidad tarea por
     tarea, Hallazgos por severidad, Notas fuera de alcance,
     Evaluación de cierre, Conclusión).

Sentinel hardcodes parameterized:

  - `specs/001-sentinel-mvp/tasks.md` → `{{charter_path}}` /
    `{{charter_content}}`.
  - `## Etapa N:` heading → `{{charter_id}}` / `{{charter_title}}`.
  - Project description "Sentinel es un monolito modular en Go..." →
    `{{project_context}}` (NEW placeholder, free-form, optional).
  - `internal/modules/identity/` and similar Go-specific paths →
    preserved as didactic illustrations (labeled as such), not
    parameterized — adopters in other stacks read them as patterns.
  - `go vet`, `go build`, `go test` commands → preserved in a
    block-quote labeled "Examples by stack" with parallels for Rust,
    TypeScript, and Python so non-Go adopters have a starting point.
  - `originating_ailogs[]` → `{{ailog_paths}}` + `{{ailog_contents}}`
    (DevTrail-specific, not in original Sentinel skill).

Credit: explicit attribution at top (HTML comment header) and bottom
(closing italics) to José Villaseñor Montfort (StrangeDaysTech) for
the contributed source material via issue #102.

UPDATE: dist/.devtrail/schemas/audit-output.schema.v0.json

  - audit_role enum extended to ["auditor", "auditor-primary",
    "auditor-secondary"]. v1 unified value is "auditor"; legacy v0
    values still accepted during the v0→v1 transition. New audits
    should use "auditor".
  - NEW optional field `evidence_citations: integer (>=0)` —
    review-skill weighting hint that records how many path:line
    citations the auditor included.
  - calibratorOutput.auditors_reconciled.maxItems removed: v1
    supports N≥2 auditors per cycle (previously fixed at exactly 2).
  - $comment updated with v1 evolution context.

NEW: cli/tests/audit_template_test.rs (9 fixture tests)

  - unified_template_ships_at_canonical_path
  - unified_template_has_seven_universal_sections
  - unified_template_declares_expected_placeholders (incl. the new
    {{project_context}})
  - unified_template_preserves_etapa_12_didactic_example
  - unified_template_credits_sentinel_contribution
  - unified_template_enforces_evidence_discipline
  - schema_accepts_v1_unified_audit_role (legacy + new coexist)
  - schema_declares_evidence_citations_optional
  - schema_calibrator_supports_n_auditors (no maxItems)

NOT in this PR (deferred to PR 4):

  - DELETE auditor-primary.md, auditor-secondary.md,
    calibrator-reconciler.md (still present, still used by current
    CLI resolver — leaves PR 3 bisect-safe).
  - CLI resolver switch to read audit-prompt.md.
  - Canonical path migration (audit/charters/ → .devtrail/audits/).
  - Subcommand simplification (--prepare / --merge-reports).
  - Deprecation shims for --calibrate / --finalize.

Test plan:
  - cargo test --test audit_template_test → 9/9 green
  - cargo test (full suite) → all passing, no regressions in the
    existing 14 charter_audit tests or the 8 audit_skill tests
  - No version bump (lands together with PRs 4-8 in the integrated
    v1 release per Propuesta/devtrail-audit-cli-flow.md v0.2 §5)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant