fix(scripts): migrate_vbrief ingests PRD/SPECIFICATION structured content into spec narratives (#397)

[MScottAdams]

Summary

ask migrate:vbrief now ingests structured PRD/SPECIFICATION content into specification.vbrief.json plan.narratives, closing the data-loss gap described in #397.

Changes

• Added _HEADING_TO_NARRATIVE_KEY mapping covering both CamelCase (ProblemStatement) and space-separated (Problem Statement) heading forms
• Added _parse_prd_narratives() to extract ## sections from PRD.md and SPECIFICATION.md and map them to canonical narrative keys matching prd_render.py
• Added Step 2b to migrate() -- reads PRD.md if present, parses SPECIFICATION.md if not deprecated, merges into specification.vbrief.json without overwriting existing keys, creates skeleton spec vBRIEF if needed
• Logs which narrative keys were ingested (INGEST action line)
• 14 new tests: 7 unit tests for _parse_prd_narratives, 7 integration tests for narrative ingestion in migrate()

Acceptance Criteria

• Post-migration specification.vbrief.json contains all structured narrative keys from PRD/SPECIFICATION
• ask prd:render on migrated spec produces equivalent output to original PRD.md (round-trip)
• No regression when no PRD content detected (greenfield)
• Migration logs ingested keys

Pre-existing test failure

ask check: 	est_no_warping_references[ROADMAP.md] fails due to pre-existing naming issue tracked in #89. Unrelated to this PR.

Closes #397

[MScottAdams]

@greptileai review

[greptile-apps]

Greptile Summary

This PR closes the data-loss gap in task migrate:vbrief by parsing ## sections from PRD.md and SPECIFICATION.md and ingesting them into specification.vbrief.json plan.narratives without overwriting existing keys. The bold-footer regex fix (\*{1,2}) addresses the previous review finding correctly, and 14 new tests give good coverage of the happy paths.

Two minor defensive-coding gaps remain: the heading-lookup normalisation does not strip trailing punctuation (so ## Goals: silently misses), and the setdefault chain at the narrative-merge point will raise an unhandled AttributeError if plan in an existing specification.vbrief.json is null or a non-dict value rather than returning the clean error tuple the caller expects.

Confidence Score: 4/5

Safe to merge after addressing the two P2 defensive-coding gaps; no data loss or breaking change on normal inputs.

Both remaining findings are P2: a silent heading-match miss on punctuated headings and an unhandled AttributeError path on structurally-malformed JSON. Neither affects normal operation, but the AttributeError can surface as an unexpected crash rather than a clean error return.

scripts/migrate_vbrief.py lines 110 and 543

Important Files Changed

Filename	Overview
scripts/migrate_vbrief.py	Adds _parse_prd_narratives(), _HEADING_TO_NARRATIVE_KEY, and Step 2b to ingest PRD/SPECIFICATION structured headings into specification.vbrief.json; bold-footer regex correctly fixed; two minor defensive-coding gaps (heading punctuation, setdefault on non-dict plan)
tests/cli/test_migrate_vbrief.py	Adds 14 new tests covering space-separated/CamelCase headings, footer stripping (italic and bold), empty sections, key-preservation, deprecated SPECIFICATION.md skip, skeleton creation, and integration logging
CHANGELOG.md	CHANGELOG entry added under [Unreleased] Fixed section accurately describing the new _parse_prd_narratives function, Step 2b, and the 14 new tests

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[migrate called] --> B[Step 1: Create lifecycle folders]
    B --> C[Step 2: Read existing sources]
    C --> D{PRD.md exists?}
    D -- Yes --> E[Read and parse PRD.md]
    D -- No --> F{SPEC.md not deprecated?}
    E --> F
    F -- Yes --> G[Parse SPEC.md sections, override PRD overlaps]
    F -- No --> H{ingested_narratives non-empty?}
    G --> H
    H -- No --> I[Step 3: Build PROJECT-DEFINITION]
    H -- Yes --> J{spec_vbrief exists?}
    J -- No --> K[Create skeleton spec_vbrief]
    J -- Yes --> L[Merge into plan.narratives, no overwrite]
    K --> L
    L --> M[Write specification.vbrief.json, log INGEST]
    M --> I
    I --> N[Step 4: Convert roadmap to pending vBRIEFs]
    N --> O[Step 5: Deprecation redirects]
    O --> P[Return success and actions]

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: scripts/migrate_vbrief.py
Line: 110

Comment:
**Heading lookup silently drops sections with trailing punctuation**

`heading.strip()` removes whitespace but not punctuation. A PRD section `## Goals:` or `## Problem Statement:` lowercases to `"goals:"` / `"problem statement:"`, which has no match in `_HEADING_TO_NARRATIVE_KEY`, so the section is silently skipped with no warning. Consider stripping trailing punctuation (colons, commas, periods) before the map lookup.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: scripts/migrate_vbrief.py
Line: 543

Comment:
**`setdefault` chain raises `AttributeError` on malformed spec_vbrief**

If `specification.vbrief.json` is valid JSON but structurally abnormal — e.g. `{"plan": null}` — then `spec_vbrief.setdefault("plan", {})` returns `None` (the key exists), and `None.setdefault("narratives", {})` raises an unhandled `AttributeError` that bubbles out of `migrate()` instead of the clean `(False, [error])` tuple the caller expects. Contrast with `_build_project_definition` at line 333 which defensively uses `isinstance` guards.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (2): Last reviewed commit: "fix: address Greptile review findings (b..." | Re-trigger Greptile}