diff --git a/.gitignore b/.gitignore index e57d445..e5a7360 100644 --- a/.gitignore +++ b/.gitignore @@ -421,3 +421,4 @@ FodyWeavers.xsd CoderMind/.genreleases/ release_notes.md workspace/ +.cmind \ No newline at end of file diff --git a/CoderMind/README.hi-IN.md b/CoderMind/README.hi-IN.md index 77855d0..0498675 100644 --- a/CoderMind/README.hi-IN.md +++ b/CoderMind/README.hi-IN.md @@ -133,15 +133,9 @@ uvx --from "git+https://github.com/microsoft/RPG-ZeroRepo.git#subdirectory=Coder 4. फॉरवर्ड पाइपलाइन चलाएँ: ```text - /cmind.feature_spec - /cmind.feature_build - /cmind.feature_refactor + /cmind.feature_construct [Optional] /cmind.feature_edit - /cmind.build_skeleton - /cmind.build_data_flow - /cmind.design_base_classes - /cmind.design_interfaces - /cmind.plan_tasks + /cmind.plan /cmind.code_gen [Optional] /cmind.rpg_edit ``` @@ -149,8 +143,8 @@ uvx --from "git+https://github.com/microsoft/RPG-ZeroRepo.git#subdirectory=Coder > [!IMPORTANT] > **हर Coding Agent का इनवोकेशन थोड़ा अलग होता है**: > -> - **Claude Code**: चैट में सीधे `/cmind.feature_spec ...` टाइप करें — slash command पहचाने जाते हैं और संबंधित workflow ट्रिगर हो जाता है। -> - **GitHub Copilot CLI**: slash command समर्थित नहीं हैं (कस्टम agent समर्थित हैं), इसलिए पहले `/agent cmind.feature_spec` से लक्ष्य agent पर स्विच करें, फिर `start` टाइप करके इसका अंतर्निहित workflow चलाएँ। +> - **Claude Code**: चैट में सीधे `/cmind.feature_construct ...` टाइप करें — slash command पहचाने जाते हैं और संबंधित workflow ट्रिगर हो जाता है। +> - **GitHub Copilot CLI**: slash command समर्थित नहीं हैं (कस्टम agent समर्थित हैं), इसलिए पहले `/agent cmind.feature_construct` से लक्ष्य agent पर स्विच करें, फिर `start` टाइप करके इसका अंतर्निहित workflow चलाएँ। CoderMind क्रमिक रूप से `~/.cmind/workspaces//data/rpg.json` बनाता है और इसका उपयोग आवश्यकताओं, प्लानिंग आउटपुट, जनरेटेड कोड और dependency जानकारी को संरेखित रखने के लिए करता है। आपके वर्कस्पेस की स्रोत फ़़ाइलें दूषित नहीं होंगी। @@ -192,7 +186,7 @@ CoderMind क्रमिक रूप से `~/.cmind/workspaces/ - /cmind.feature_build - /cmind.feature_refactor + /cmind.feature_construct [Optional] /cmind.feature_edit - /cmind.build_skeleton - /cmind.build_data_flow - /cmind.design_base_classes - /cmind.design_interfaces - /cmind.plan_tasks + /cmind.plan /cmind.code_gen [Optional] /cmind.rpg_edit ``` @@ -149,8 +143,8 @@ uvx --from "git+https://github.com/microsoft/RPG-ZeroRepo.git#subdirectory=Coder > [!IMPORTANT] > **コーディングエージェントごとに呼び出し方が異なります**： > -> - **Claude Code**：チャットにそのまま `/cmind.feature_spec ...` と入力します。slash command が認識され、対応する workflow がトリガーされます。 -> - **GitHub Copilot CLI**：slash command はサポートされません（カスタム agent はサポート）。まず `/agent cmind.feature_spec` で目的の agent に切り替え、その後 `start` と入力して内蔵の workflow を実行します。 +> - **Claude Code**：チャットにそのまま `/cmind.feature_construct ...` と入力します。slash command が認識され、対応する workflow がトリガーされます。 +> - **GitHub Copilot CLI**：slash command はサポートされません（カスタム agent はサポート）。まず `/agent cmind.feature_construct` で目的の agent に切り替え、その後 `start` と入力して内蔵の workflow を実行します。 CoderMind は `~/.cmind/workspaces//data/rpg.json` を段階的に作成し、それを使って要件・計画成果物・生成コード・依存情報を整合した状態に保ちます。ワークスペースのソースファイルは汚染されません。 @@ -192,7 +186,7 @@ CoderMind は `~/.cmind/workspaces//data/rpg.json` を段階的に ```text my-project/ -├── docs/ # /cmind.feature_spec 用の任意の要件ドキュメント +├── docs/ # /cmind.feature_construct 用の任意の要件ドキュメント ├── .github/ or .claude/ # Coding Agent のコマンド定義と設定 ├── .vscode/ # 該当する場合の Copilot/VS Code MCP 設定 ├── .cmind/ # 生成されたレポートと設定ファイル @@ -241,7 +235,7 @@ cmind update ## 今後の機能 -- **よりシンプルな生成コマンド:** 現在の多段階の生成フローを、`/cmind.generate_repo`、`/cmind.generate_feature`、`/cmind.plan` などのより少ないコマンドにまとめます。 +- **よりシンプルな生成コマンド:** 現在の多段階の生成フローを、`/cmind.generate_repo` や `/cmind.generate_feature` などのより少ないコマンドにまとめます。`/cmind.plan` は 0.1.4 でリリース済みです。 - **多言語サポート:** Go、C++、Rust、JavaScript/TypeScript などのサポートを追加します。 - **より多くのプラットフォーム連携:** さまざまなシステム上の異なる AI コーディングエージェントについて、CLI と VS Code 拡張ワークフローを横断して CoderMind をサポートします。 diff --git a/CoderMind/README.ko-KR.md b/CoderMind/README.ko-KR.md index 66be341..da3212a 100644 --- a/CoderMind/README.ko-KR.md +++ b/CoderMind/README.ko-KR.md @@ -133,15 +133,9 @@ uvx --from "git+https://github.com/microsoft/RPG-ZeroRepo.git#subdirectory=Coder 4. 포워드 파이프라인을 실행합니다: ```text - /cmind.feature_spec - /cmind.feature_build - /cmind.feature_refactor + /cmind.feature_construct [Optional] /cmind.feature_edit - /cmind.build_skeleton - /cmind.build_data_flow - /cmind.design_base_classes - /cmind.design_interfaces - /cmind.plan_tasks + /cmind.plan /cmind.code_gen [Optional] /cmind.rpg_edit ``` @@ -149,8 +143,8 @@ uvx --from "git+https://github.com/microsoft/RPG-ZeroRepo.git#subdirectory=Coder > [!IMPORTANT] > **Coding Agent마다 호출 방식이 조금씩 다릅니다**: > -> - **Claude Code**: 채팅에 직접 `/cmind.feature_spec ...` 을 입력하면 slash command가 인식되어 해당 workflow가 트리거됩니다. -> - **GitHub Copilot CLI**: slash command는 지원하지 않으나(커스텀 agent는 지원), 먼저 `/agent cmind.feature_spec` 으로 대상 agent로 전환한 다음 `start` 를 입력해 내장된 workflow를 실행합니다. +> - **Claude Code**: 채팅에 직접 `/cmind.feature_construct ...` 을 입력하면 slash command가 인식되어 해당 workflow가 트리거됩니다. +> - **GitHub Copilot CLI**: slash command는 지원하지 않으나(커스텀 agent는 지원), 먼저 `/agent cmind.feature_construct` 으로 대상 agent로 전환한 다음 `start` 를 입력해 내장된 workflow를 실행합니다. CoderMind은 `~/.cmind/workspaces//data/rpg.json` 을 점진적으로 생성하고, 이를 사용해 요구사항, 계획 산출물, 생성된 코드, 의존성 정보를 정합 상태로 유지합니다. 워크스페이스의 소스 파일은 오염되지 않습니다. @@ -192,7 +186,7 @@ CoderMind은 `~/.cmind/workspaces//data/rpg.json` 을 점진적으 ```text my-project/ -├── docs/ # /cmind.feature_spec 용 선택적 요구사항 문서 +├── docs/ # /cmind.feature_construct 용 선택적 요구사항 문서 ├── .github/ or .claude/ # Coding Agent 커맨드 정의 및 설정 ├── .vscode/ # 해당하는 경우 Copilot/VS Code MCP 구성 ├── .cmind/ # 생성된 리포트와 설정 파일 @@ -241,7 +235,7 @@ cmind update ## 예정된 기능 -- **더 간단한 생성 커맨드:** 현재의 다단계 생성 흐름을 `/cmind.generate_repo`, `/cmind.generate_feature`, `/cmind.plan` 등 더 적은 커맨드로 통합합니다. +- **더 간단한 생성 커맨드:** 현재의 다단계 생성 흐름을 `/cmind.generate_repo`, `/cmind.generate_feature` 등 더 적은 커맨드로 통합합니다. `/cmind.plan` 은 0.1.4 에서 출시되었습니다. - **다국어 지원:** Go, C++, Rust, JavaScript/TypeScript 등을 추가로 지원합니다. - **더 많은 플랫폼 통합:** 다양한 시스템에서 서로 다른 AI 코딩 에이전트의 CLI 및 VS Code 확장 워크플로에 걸쳐 CoderMind을 지원합니다. diff --git a/CoderMind/README.md b/CoderMind/README.md index 917ad86..390dbf4 100644 --- a/CoderMind/README.md +++ b/CoderMind/README.md @@ -136,15 +136,9 @@ Use this path when you want CoderMind to turn requirements into a new codebase. 4. Run the forward pipeline: ```text - /cmind.feature_spec - /cmind.feature_build - /cmind.feature_refactor + /cmind.feature_construct [Optional] /cmind.feature_edit - /cmind.build_skeleton - /cmind.build_data_flow - /cmind.design_base_classes - /cmind.design_interfaces - /cmind.plan_tasks + /cmind.plan /cmind.code_gen [Optional] /cmind.rpg_edit ``` @@ -152,8 +146,8 @@ Use this path when you want CoderMind to turn requirements into a new codebase. > [!IMPORTANT] > **Coding Agents are invoked slightly differently**: > -> - **Claude Code**: type `/cmind.feature_spec ...` directly in the chat — slash commands are recognised and dispatch the matching workflow. -> - **GitHub Copilot CLI**: slash commands are not supported (custom agents are), so first run `/agent cmind.feature_spec` to switch to the target agent, then type `start` to run its built-in workflow. +> - **Claude Code**: type `/cmind.feature_construct ...` directly in the chat — slash commands are recognised and dispatch the matching workflow. +> - **GitHub Copilot CLI**: slash commands are not supported (custom agents are), so first run `/agent cmind.feature_construct` to switch to the target agent, then type `start` to run its built-in workflow. CoderMind progressively builds `rpg.json` in the home-side runtime directory (`~/.cmind/workspaces//data/rpg.json`) and uses it to keep requirements, planning artifacts, generated code, and dependency information aligned. Your workspace source files are not polluted. @@ -195,7 +189,7 @@ Use this path when you already have a repository and want an AI agent to underst ```text my-project/ -├── docs/ # Optional requirement docs for /cmind.feature_spec +├── docs/ # Optional requirement docs for /cmind.feature_construct ├── .github/ or .claude/ # Coding Agent command definitions and settings ├── .vscode/ # Copilot/VS Code MCP configuration when applicable ├── .cmind/ # Generated reports and configuration files @@ -244,7 +238,7 @@ cmind update ## Upcoming Features -- **Simpler generation commands:** merge the current multi-step generation flow into fewer commands, such as `/cmind.generate_repo`, `/cmind.generate_feature`, and `/cmind.plan`. +- **Simpler generation commands:** merge the current multi-step generation flow into fewer commands, such as `/cmind.generate_repo` and `/cmind.generate_feature`. `/cmind.plan` has shipped in 0.1.4. - **Multi-language support:** add support for Go, C++, Rust, JavaScript/TypeScript, and more. - **More platform integrations:** support CoderMind across CLI and VS Code extension workflows for different AI coding agents on different systems. diff --git a/CoderMind/README.zh-CN.md b/CoderMind/README.zh-CN.md index 0b436dc..bbbad6a 100644 --- a/CoderMind/README.zh-CN.md +++ b/CoderMind/README.zh-CN.md @@ -133,15 +133,9 @@ uvx --from "git+https://github.com/microsoft/RPG-ZeroRepo.git#subdirectory=Coder 4. 运行正向流水线： ```text - /cmind.feature_spec - /cmind.feature_build - /cmind.feature_refactor + /cmind.feature_construct [Optional] /cmind.feature_edit - /cmind.build_skeleton - /cmind.build_data_flow - /cmind.design_base_classes - /cmind.design_interfaces - /cmind.plan_tasks + /cmind.plan /cmind.code_gen [Optional] /cmind.rpg_edit ``` @@ -149,8 +143,8 @@ uvx --from "git+https://github.com/microsoft/RPG-ZeroRepo.git#subdirectory=Coder > [!IMPORTANT] > **不同 Coding Agent 的调用方式略有不同**： > -> - **Claude Code**：直接在对话中输入 `/cmind.feature_spec ...`，slash command 会被识别并触发对应 workflow。 -> - **GitHub Copilot CLI**：不支持 slash command（但支持自定义 agent），需要先 `/agent cmind.feature_spec` 切换到目标 agent，然后输入 `start` 让它执行内置的 workflow。 +> - **Claude Code**：直接在对话中输入 `/cmind.feature_construct ...`，slash command 会被识别并触发对应 workflow。 +> - **GitHub Copilot CLI**：不支持 slash command（但支持自定义 agent），需要先 `/agent cmind.feature_construct` 切换到目标 agent，然后输入 `start` 让它执行内置的 workflow。 CoderMind 会渐进式地在 home-side 运行时目录（`~/.cmind/workspaces//data/rpg.json`）里创建 `rpg.json`，并用它把需求、规划产物、生成的代码和依赖信息保持对齐。你的工作区源文件不会被污染。 @@ -192,7 +186,7 @@ CoderMind 会渐进式地在 home-side 运行时目录（`~/.cmind/workspaces/` | Run Phase 1 feature specification, build, and refactor in one step — recommended | | `/cmind.feature_spec ` | Create structured feature specifications from user input or `docs/` files | | `/cmind.feature_build` | Generate and expand the feature tree from specifications | | `/cmind.feature_refactor` | Refactor feature tree into modular component architecture | | `/cmind.feature_edit ` | Edit feature tree nodes before skeleton planning — optional | +> `/cmind.feature_construct` is the simplest way to drive Phase 1 end-to-end. Use +> the individual Phase 1 commands only when you want to debug or re-run a +> specific stage. + ### Phase 2: RPG Construction and Planning | Command | Description | | ------- | ----------- | +| `/cmind.plan` | Run all five Phase-2 stages in one step with automatic resume — recommended | | `/cmind.build_skeleton` | Build repository file skeleton from component architecture; creates `.cmind/data/rpg.json` | | `/cmind.build_data_flow` | Build inter-component data flow DAG and update the RPG | | `/cmind.design_base_classes` | Design shared base classes and data structures | | `/cmind.design_interfaces` | Design function/class interfaces with type hints and docstrings | | `/cmind.plan_tasks` | Plan dependency-ordered implementation task batches | +> `/cmind.plan` is the simplest way to drive Phase 2 end-to-end. Use +> the individual commands above only when you want to debug or +> re-run a specific stage. + ### Phase 3: Code Generation and Surgical Edits | Command | Description | @@ -49,31 +59,69 @@ Both directions produce the same RPG structure at `.cmind/data/rpg.json`, enabli ## Phase 1: Feature Specification -### `/cmind.feature_spec` +### `/cmind.feature_construct` -Create structured feature specifications from user input or documentation files. +Run the full Phase 1 pipeline (`feature_spec` → `feature_build` → `feature_refactor`) in one step. This is the recommended entry point for creating the feature tree that feeds `/cmind.plan`. **Input modes:** -- **Direct input:** provide a description after the command -- **Auto-detect:** omit input to auto-detect `docs/*.md` files +- **Direct input:** provide requirements after the command. +- **Auto-detect:** omit input to use existing `docs/*.md` files automatically. +- **Inline prompt:** if neither direct input nor usable docs exist, the command asks for requirements and then continues in the same flow. -**Output:** +**Output:** the three Phase 1 JSON artefacts — `feature_spec.json`, `feature_build.json`, and `feature_tree.json` — in the CoderMind data store. + +**Process:** + +1. **Probe progress** — runs `cmind script feature_construct.py --check-only --json` to see which Phase 1 stages already have valid artifacts. +2. **Generate requirements artifacts when needed** — follows the `/cmind.feature_spec` workflow for direct text or `docs/*.md` sources. +3. **Run/resume** — executes `cmind script feature_construct.py`, skipping completed stages and cascading downstream rebuilds when an upstream stage reruns. +4. **Optional expansion** — after completion, the user can expand features through the existing `feature_build --mode suggest-directions` and `--mode step2 --direction ` flow; refactor is rerun afterward so `feature_tree.json` stays aligned. + +**CLI flags forwarded after `$ARGUMENTS`:** + +- `--check-only` — show Phase 1 progress without modifying artifacts or running stages. +- `--json` — with `--check-only`, emit the progress report as JSON. +- `--force` — rebuild all Phase 1 stages. +- `--dry-run` — print the commands that would run without modifying artifacts. +- `--verbose` — forward native verbose logging flags. +- `--no-trajectory` — disable trajectory recording where supported. +- `--max-iter-refactor N` — forward to `feature_refactor.py` as `--max-iterations N`. +- `--review-threshold N` — forward to `feature_build.py --mode step1`. +- `--review-max-iterations N` — forward to `feature_build.py --mode step1`. + +Use `--` to separate options from requirement text: ```text -.cmind/data/feature_spec/ -├── evidence/ # Source evidence files -│ ├── user_input.md # From direct user input, or -│ ├── 01_project_charter.md -│ └── ... -├── feature_spec.md # Meta + Background + NFR -└── features/ # Feature tree documents - ├── FT-001.md - ├── FT-002.md - └── ... +/cmind.feature_construct --check-only +/cmind.feature_construct --check-only --json +/cmind.feature_construct --review-threshold 99 -- Build a CLI tool for managing Docker containers +/cmind.feature_construct Build a CLI tool for managing Docker containers +/cmind.feature_construct # Auto-detect docs/ files ``` -Also generates `.cmind/data/feature_spec.json`. +**Next step:** `/cmind.plan` is the default handoff after Phase 1. If the final tree needs small adjustments, run `/cmind.feature_edit `. The granular Phase 1 commands remain available for debug and surgical reruns; `/cmind.build_skeleton` is a Phase 2 granular fallback, not the default next step. + +--- + +### `/cmind.feature_spec` + +Generate a structured feature specification from user input or +documentation files. + +> **Recommended workflow:** Use `/cmind.feature_construct` for the +> end-to-end Phase 1 flow. `/cmind.feature_spec` is the granular +> single-stage command, kept available for debugging and surgical reruns. + +**Input modes:** + +- **Direct input:** provide a description after the command +- **Auto-detect:** omit input to auto-detect `docs/*.md` files + +**Output:** `.cmind/data/feature_spec.json` — a Pydantic-validated JSON +document containing `meta`, `repository_name`, `repository_purpose`, +`background_and_overview`, `non_functional_requirements`, and a +recursive `functional_requirements` tree. **Examples:** @@ -159,6 +207,69 @@ Edit feature tree nodes before repository planning begins. ## Phase 2: RPG Construction and Planning +### `/cmind.plan` + +Run the full Phase-2 pipeline (`build_skeleton` → `build_data_flow` → +`design_base_classes` → `design_interfaces` → `plan_tasks`) in one +step. This is the recommended entry point for Phase 2. + +**Input:** `~/.cmind/workspaces//data/feature_tree.json` (produced by +`/cmind.feature_construct` or `/cmind.feature_refactor`) + +**Output:** every artifact produced by the five individual commands — +`skeleton.json`, `data_flow.json`, `base_classes.json`, +`interfaces.json`, `tasks.json`, plus `rpg.json` and the +`data_flow_viz.html` visualization. + +**Process:** + +1. **Probe progress** — runs `cmind script plan.py --check-only --json` + to see which stages already have valid artifacts. +2. **Decide** — based on the probe result, the command prompts you + **once** with one of three options: + - All five stages already done → `Overwrite` or `Exit`. + - Partial progress (some stages done) → `Continue`, `Restart`, or `Exit`. + - Fresh workspace → no prompt; runs the full pipeline. +3. **Run** — executes the chosen mode through `cmind script plan.py`. + Each stage's stdout is streamed live and also written to a per-stage + log under `~/.cmind/workspaces//logs/`. +4. **Verify** — after every stage's build script, the corresponding + `check_*.py` script re-runs to validate the produced artifact. If + verification fails the pipeline stops and prints recovery hints. + +**Resume semantics:** the command treats `type == "update"` from each +`check_*.py` as the source of truth for "this stage is done". If you +press Ctrl-C halfway through, running `/cmind.plan` again automatically +resumes from the first not-done stage. When any earlier stage is +re-run, every downstream stage is rebuilt too so artifacts never drift +apart. + +**CLI flags forwarded after `$ARGUMENTS`:** + +- `--force` — discard existing artifacts and rebuild every stage. +- `--max-iter-skeleton N`, `--max-iter-data-flow N`, + `--max-iter-base-classes N`, `--max-iter-interfaces N` — + override iteration counts for the corresponding stage. +- `--verbose` — forward `--verbose` to every sub-script. +- `--no-trajectory` — forward `--no-trajectory` where supported. + +**Examples:** + +```text +/cmind.plan +/cmind.plan --verbose +/cmind.plan --force # rebuild everything +/cmind.plan --max-iter-skeleton 15 +``` + +To inspect progress without running anything: + +```bash +cmind script plan.py --check-only +``` + +--- + ### `/cmind.build_skeleton` Build the repository file skeleton from the component architecture. This is where the forward pipeline first creates the RPG. @@ -462,10 +573,9 @@ All intermediate data is stored in `.cmind/data/`: | File | Produced by | Description | | ---- | ----------- | ----------- | -| `feature_spec/` | `feature_spec` | Evidence and feature specification documents | -| `feature_spec.json` | `feature_spec` | Structured feature specification | -| `feature_build.json` | `feature_build` | Expanded feature tree | -| `feature_tree.json` | `feature_refactor` / `feature_edit` | Component architecture | +| `feature_spec.json` | `feature_construct` / `feature_spec` | Structured feature specification | +| `feature_build.json` | `feature_construct` / `feature_build` | Expanded feature tree | +| `feature_tree.json` | `feature_construct` / `feature_refactor` / `feature_edit` | Component architecture | | `skeleton.json` | `build_skeleton` | File skeleton | | `skeleton_summary.txt` | `build_skeleton` | Human-readable skeleton summary | | `rpg.json` | `build_skeleton` / `encode`, then updated by later commands | Repository Planning Graph | diff --git a/CoderMind/scripts/check_base_classes.py b/CoderMind/scripts/check_base_classes.py index b9a0909..91308f8 100644 --- a/CoderMind/scripts/check_base_classes.py +++ b/CoderMind/scripts/check_base_classes.py @@ -108,14 +108,14 @@ def inspect_state(base_classes_path: Path) -> Dict[str, Any]: """Inspect current state and determine action needed. Returns dict with: - - state: "error" | "init" | "update" + - type: "error" | "init" | "update" - message: description - details: additional info """ # Check if base_classes.json exists if not base_classes_path.exists(): return { - "state": "init", + "type": "init", "message": "base_classes.json not found - need to run design_base_classes", "details": {} } @@ -126,7 +126,7 @@ def inspect_state(base_classes_path: Path) -> Dict[str, Any]: data = json.load(f) except json.JSONDecodeError as e: return { - "state": "error", + "type": "error", "message": f"Invalid JSON in base_classes.json: {e}", "details": {} } @@ -134,7 +134,7 @@ def inspect_state(base_classes_path: Path) -> Dict[str, Any]: # Check for error field if "error" in data: return { - "state": "error", + "type": "error", "message": f"Base classes has error: {data['error']}", "details": {} } @@ -143,7 +143,7 @@ def inspect_state(base_classes_path: Path) -> Dict[str, Any]: is_valid, errors = validate_base_classes_structure(data) if not is_valid: return { - "state": "error", + "type": "error", "message": "Base classes structure or syntax is invalid", "details": {"errors": errors} } @@ -161,7 +161,7 @@ def inspect_state(base_classes_path: Path) -> Dict[str, Any]: ds_file_paths = [ds.get("file_path", "") for ds in data_structures if ds.get("file_path")] return { - "state": "update", + "type": "update", "message": "Base classes are valid", "details": { "file_count": len(base_classes), @@ -178,7 +178,7 @@ def inspect_state(base_classes_path: Path) -> Dict[str, Any]: def print_state(result: Dict[str, Any]) -> None: """Print state information.""" - state = result["state"] + state = result["type"] message = result["message"] details = result.get("details", {}) @@ -259,7 +259,7 @@ def main(): result = inspect_state(args.input) # In verbose mode, include raw base_classes data - if args.verbose and result.get("state") == "update": + if args.verbose and result.get("type") == "update": base_classes_data = load_json(args.input) if base_classes_data: result["base_classes"] = base_classes_data.get("base_classes", []) @@ -273,7 +273,7 @@ def main(): print_state(result) # Return exit code based on state - if result["state"] == "error": + if result["type"] == "error": return 1 return 0 diff --git a/CoderMind/scripts/check_data_flow.py b/CoderMind/scripts/check_data_flow.py index 12b5004..1482031 100644 --- a/CoderMind/scripts/check_data_flow.py +++ b/CoderMind/scripts/check_data_flow.py @@ -168,14 +168,14 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: """Inspect current state and determine action needed. Returns dict with: - - state: "error" | "init" | "warning" | "update" + - type: "error" | "init" | "warning" | "update" - message: description - details: additional info """ # Check if data_flow.json exists if not data_flow_path.exists(): return { - "state": "init", + "type": "init", "message": "data_flow.json not found - need to run build_data_flow", "details": {} } @@ -186,7 +186,7 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: data_flow = json.load(f) except json.JSONDecodeError as e: return { - "state": "error", + "type": "error", "message": f"Invalid JSON in data_flow.json: {e}", "details": {} } @@ -194,7 +194,7 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: # Check for error field if "error" in data_flow: return { - "state": "error", + "type": "error", "message": f"Data flow has error: {data_flow['error']}", "details": {} } @@ -203,7 +203,7 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: is_valid, errors = validate_data_flow_structure(data_flow) if not is_valid: return { - "state": "error", + "type": "error", "message": "Data flow structure is invalid", "details": {"errors": errors} } @@ -223,14 +223,14 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: if not is_consistent: return { - "state": "warning", + "type": "warning", "message": "Component mismatch between skeleton and data flow", "details": xval_details } # All good return { - "state": "update", + "type": "update", "message": "Data flow is valid and consistent", "details": { "edge_count": len(data_flow.get("data_flow", [])), @@ -242,7 +242,7 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: except Exception as e: # Skeleton load failed, just validate data flow return { - "state": "update", + "type": "update", "message": f"Data flow is valid (skeleton check skipped: {e})", "details": { "edge_count": len(data_flow.get("data_flow", [])), @@ -252,7 +252,7 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: # No skeleton to compare return { - "state": "update", + "type": "update", "message": "Data flow is valid (no skeleton to cross-validate)", "details": { "edge_count": len(data_flow.get("data_flow", [])), @@ -263,7 +263,7 @@ def inspect_state(data_flow_path: Path, skeleton_path: Path) -> Dict[str, Any]: def print_state(result: Dict[str, Any]) -> None: """Print state information.""" - state = result["state"] + state = result["type"] message = result["message"] details = result.get("details", {}) @@ -338,7 +338,7 @@ def main(): result = inspect_state(args.data_flow, args.skeleton) # In verbose mode, include all edges and component details - if args.verbose and result.get("state") == "update": + if args.verbose and result.get("type") == "update": data_flow_data = load_json(args.data_flow) if data_flow_data: result["edges"] = data_flow_data.get("data_flow", []) @@ -353,7 +353,7 @@ def main(): print_state(result) # Print verbose details - if args.verbose and result.get("state") == "update": + if args.verbose and result.get("type") == "update": edges = result.get("edges", []) if edges: print("\nData Flow Edges:") @@ -365,7 +365,7 @@ def main(): print(f"\nSubtree Order: {' → '.join(subtree_order)}") # Return exit code based on state - if result["state"] == "error": + if result["type"] == "error": return 1 return 0 diff --git a/CoderMind/scripts/check_skeleton.py b/CoderMind/scripts/check_skeleton.py index 3f32304..bb2699d 100644 --- a/CoderMind/scripts/check_skeleton.py +++ b/CoderMind/scripts/check_skeleton.py @@ -385,7 +385,15 @@ def main() -> None: action="store_true", help="Include detailed file list and all feature mismatches" ) - + parser.add_argument( + "--json", + action="store_true", + help=( + "Accepted for compatibility with the unified check_*.py contract used by " + "plan.py; this script already prints JSON unconditionally." + ), + ) + args = parser.parse_args() result = inspect_state() diff --git a/CoderMind/scripts/check_tasks.py b/CoderMind/scripts/check_tasks.py index 99ecb15..4d3e6fc 100644 --- a/CoderMind/scripts/check_tasks.py +++ b/CoderMind/scripts/check_tasks.py @@ -75,13 +75,25 @@ def get_all_units_from_tasks(tasks_path: Path) -> Set[str]: """ with open(tasks_path, 'r', encoding='utf-8') as f: data = json.load(f) - + units = set() - + + # ``plan_tasks.py`` emits auxiliary scaffolding tasks (README, + # requirements.txt, integration tests, cross-module wiring, UI + # polish, main entry, comprehensive tests) using synthetic file-path + # placeholders wrapped in angle-brackets such as ```` or + # ``_Foo``. These are by design not present in + # ``interfaces.json``; skipping them prevents false-positive + # ``missing_in_interfaces`` warnings from the cross-validation step. + def _is_synthetic(file_path: str) -> bool: + return file_path.startswith("<") and ">" in file_path + # Support planned_tasks_dict format if "planned_tasks_dict" in data: for component_name, files_dict in data["planned_tasks_dict"].items(): for file_path, task_list in files_dict.items(): + if _is_synthetic(file_path): + continue for task in task_list: # units_key contains the unit names for unit_name in task.get("units_key", []): @@ -92,9 +104,9 @@ def get_all_units_from_tasks(tasks_path: Path) -> Set[str]: for unit in batch.get("units", []): file_path = unit.get("file_path", "") unit_name = unit.get("unit_name", "") - if file_path and unit_name: + if file_path and unit_name and not _is_synthetic(file_path): units.add(f"{file_path}::{unit_name}") - + return units diff --git a/CoderMind/scripts/feature/prompts/__init__.py b/CoderMind/scripts/feature/prompts/__init__.py new file mode 100644 index 0000000..1e027fb --- /dev/null +++ b/CoderMind/scripts/feature/prompts/__init__.py @@ -0,0 +1,20 @@ +"""LLM prompt templates for feature-related stages. + +This package collects prompt templates grouped by stage. The historical +single-file module ``feature/prompts.py`` is preserved as +``feature.prompts.legacy`` and re-exported here so existing imports +(``from feature.prompts import PROMPT_TEMPLATE_BUILD_FEATURE`` etc.) +continue to work unchanged. + +New stages add their prompts in dedicated submodules and re-export from +here as needed. +""" + +# Back-compat: re-export every public symbol from the legacy module. +from .legacy import * # noqa: F401,F403 + +# New, dedicated prompt modules. +from .spec import ( # noqa: F401 + PROMPT_TEMPLATE_FEATURE_SPEC_SYSTEM, + PROMPT_TEMPLATE_FEATURE_SPEC_USER, +) diff --git a/CoderMind/scripts/feature/prompts.py b/CoderMind/scripts/feature/prompts/legacy.py similarity index 100% rename from CoderMind/scripts/feature/prompts.py rename to CoderMind/scripts/feature/prompts/legacy.py diff --git a/CoderMind/scripts/feature/prompts/spec.py b/CoderMind/scripts/feature/prompts/spec.py new file mode 100644 index 0000000..4b65a6c --- /dev/null +++ b/CoderMind/scripts/feature/prompts/spec.py @@ -0,0 +1,225 @@ +"""LLM prompt templates for the ``feature_spec`` stage. + +The Phase-1 ``feature_spec`` stage converts raw requirements (either a free- +form user description or a set of ``docs/*.md`` files) into a single, +strictly-validated ``feature_spec.json``. + +These prompts replace the historical 1008-line +``templates/commands/feature_spec.md`` slash-command document. The +intermediate Markdown artefacts (``evidence/*.md``, ``feature_spec.md``, +``features/FT-*.md``) are *no longer generated*; the LLM emits the final +JSON directly, validated against ``feature.schemas.spec.FeatureSpecOutput``. + +Schema knowledge — field meanings, ID conventions, MIU principle, etc. — +lives both here (in the prompt body) and in the Pydantic ``Field`` +descriptions; the two are intentionally aligned. +""" + +from __future__ import annotations + + +# =========================================================================== +# System prompt — stable, contains the schema contract & quality rules. +# =========================================================================== + +PROMPT_TEMPLATE_FEATURE_SPEC_SYSTEM = r""" +## Role + +You are a senior software architect. Your task is to read raw project +requirements and emit a single, well-structured ``feature_spec.json`` +document that downstream code-generation stages can consume directly. + +## Output Contract (MANDATORY) + +Emit **exactly one** ``...`` block at the end of +your response. The block must contain a JSON object validating against +the following Pydantic schema (snake_case attribute names are the JSON keys): + +```python +class Evidence(BaseModel): + id: str # "--NNN" + source: str # filename, or "user_input" + line_start: int # 1-based inclusive + line_end: int + +class Meta(BaseModel): + project_types: list[str] # subset of {WEB, API, SERVICE, PIPELINE, + # CLI, GUI, GAME, LIBRARY} + project_notes: str # ≤500 chars + generated_at: str # "YYYY-MM-DD" + source_documents: list[str] # ["doc1.md", ...] or ["user_input"] + +class BackgroundItem / NfrItem(BaseModel): + id: str # "BG-NNN" / "NFR-NNN" (1-based, zero-padded) + title: str # Title Case, a few words + description: str # 1-3 sentences, English + evidence: list[Evidence] = [] # OPTIONAL + +class FeatureNode(BaseModel): + id: str # "FT-NNN[-NNN]*", segments = depth + name: str # Title Case, 2-4 English words preferred + description: str # 1 sentence describing WHAT (not HOW) + evidence: list[Evidence] = [] # OPTIONAL + children: list[FeatureNode] = [] + +class FeatureSpecOutput(BaseModel): + meta: Meta + background_and_overview: list[BackgroundItem] + non_functional_requirements: list[NfrItem] + functional_requirements: list[FeatureNode] + repository_name: str # 1-3 words, kebab-case + repository_purpose: str # 1-2 sentences +``` + +You may include reasoning, planning notes or commentary outside the +```` block. Use a single ``...`` block for +that if you wish; downstream tooling will ignore it. + +## Quality Rules + +### Project types + +Choose from this whitelist (multi-select allowed; at least one required): + +| Token | Use it for | +| --- | --- | +| WEB | HTTP endpoints rendering HTML for browsers | +| API | JSON / GraphQL endpoints, no HTML rendering | +| SERVICE | long-running daemon, worker, bot, scheduler | +| PIPELINE | batch data processing (ETL, DAG, ML training) | +| CLI | command-line entry point with subcommands or arguments | +| GUI | desktop window with widgets | +| GAME | interactive real-time application with a rendering loop | +| LIBRARY | importable package, no end-user interface | + +### Repository identity + +- ``repository_name``: concise, kebab-case, 1-3 words (e.g. ``todo-list-app``). +- ``repository_purpose``: 1-2 sentences capturing the core objective. + +### Background & NFR + +- Cluster source mentions by topic; each entry is one topic. +- ``description`` is a short paragraph synthesising the topic — not a + verbatim quote. +- Sequence IDs from 001, in source-order. + +### Functional requirement tree + +- **Top-level Domains**: 3-8 domains is typical; group by responsibility. +- **Hierarchy depth**: as deep as needed to make leaves Minimum + Implementable Units, but **at most 6 levels**. Simple projects often + stop at depth 3. +- **Leaf nodes (no children)** must satisfy the MIU principle: + - Single verb + single object (no "and"/"or"). + - Independently testable; observable input→output or state change. + - Atomic — one function/method scope; assignable as one dev task. + - Describable in one sentence. +- **Intermediate / organisation nodes** group children; they need not be + MIU and typically have empty ``evidence``. +- **IDs**: ``FT-001``, ``FT-001-002``, ``FT-001-002-003`` etc. Segments = + depth. Use zero-padded 3-digit numbers. Numbering is sequential within + each parent. + +### Evidence + +- When source documents are available, include ``evidence`` entries + pointing back to the exact line range that justifies an item. +- ``id`` format: ``--NNN`` + - source_prefix derivation: + 1. Strip leading digits + extension (``02_charter.md`` → ``charter``). + 2. ≤20 chars → use full name (e.g. ``charter``). + 3. >20 chars → uppercase initials of underscore-separated words + (e.g. ``requirements_specification`` → ``RS``). + 4. Duplicate-prefix conflict → append ``_2``, ``_3``, … + - The literal prefix ``user_input`` is reserved for inline requirement text. +- For ``user_input`` mode you may emit one consolidated evidence per + item with ``line_start=1, line_end=`` based on the line count + of the user's text — or leave ``evidence: []`` entirely. + +### Content extraction policy + +- Extract **what the system does / is**, not project management + artefacts (project risks, external risks, operational processes, + inter-document references, open questions). +- For tables: extract each row feature independently. +- Faithfully reflect the granularity of the source — do not invent + features that aren't supported by the source. + +### Style + +- All output text is **English** (regardless of source-document language + for proper nouns; titles and descriptions are in English). +- Markdown special characters in descriptions must be properly JSON-escaped. + +## Self-Verification (MANDATORY before emitting ) + +Coverage completeness is a **hard requirement** — schema validity alone +is not enough. Inside a single ```` block, do the following +checklist before producing the JSON: + +1. **Enumerate sources.** List every section header, table row, bullet + point, and inline statement in the source material that describes a + capability, behaviour, data shape, or constraint. + +2. **Map each item.** For each enumerated item, write one of: + + - ``→ `` — covered by the listed spec id (e.g. ``→ FT-002-001``, + ``→ NFR-002``, ``→ BG-001``). + - ``→ excluded: `` — explicitly justify the omission using a + reason from the Content extraction policy (e.g. "project risk", + "open question", "operational process"). + +3. **Resolve gaps.** If any item lacks both a mapping and a + justification, **patch the spec** (add or refine an entry) before + moving on. Do not emit JSON with unmapped items. + +4. **Sanity checks.** + + - Every leaf ``FeatureNode`` (no children) is a Minimum Implementable + Unit (single verb + single object, testable, one-sentence-describable). + Refactor any non-MIU leaves into intermediate nodes with MIU children. + - Every ``BackgroundItem`` / ``NfrItem`` / ``FeatureNode`` id follows + the format rules and is unique within its scope. + - ``meta.project_types`` and ``meta.source_documents`` are + non-empty. + +Only after the four-step checklist passes, emit the ```` +block. +""" + + +# =========================================================================== +# User prompt template — supplies the actual input documents / text. +# =========================================================================== + +PROMPT_TEMPLATE_FEATURE_SPEC_USER = r""" +## Input + +The following is the raw requirement material. Convert it into a complete +``feature_spec.json`` per the schema and quality rules in the system +prompt. + +**Generation date** (use this for ``meta.generated_at``): {generated_at} + +**Source kind**: {source_kind} + +**Source documents** (use these names in ``meta.source_documents`` and +in evidence ``source`` fields): {source_documents} + +--- + +{input_blob} + +--- + +Remember: emit one ``{{...}}`` block containing +the full ``feature_spec.json`` object. Do not split into multiple JSON +blocks. Do not wrap in Markdown code fences inside the block. +""" + + +__all__ = [ + "PROMPT_TEMPLATE_FEATURE_SPEC_SYSTEM", + "PROMPT_TEMPLATE_FEATURE_SPEC_USER", +] diff --git a/CoderMind/scripts/feature/schemas/__init__.py b/CoderMind/scripts/feature/schemas/__init__.py new file mode 100644 index 0000000..97f5555 --- /dev/null +++ b/CoderMind/scripts/feature/schemas/__init__.py @@ -0,0 +1,21 @@ +"""Pydantic schemas for feature stage outputs.""" + +from .spec import ( + BackgroundItem, + Evidence, + FeatureNode, + FeatureSpecOutput, + Meta, + NfrItem, + ProjectType, +) + +__all__ = [ + "BackgroundItem", + "Evidence", + "FeatureNode", + "FeatureSpecOutput", + "Meta", + "NfrItem", + "ProjectType", +] diff --git a/CoderMind/scripts/feature/schemas/spec.py b/CoderMind/scripts/feature/schemas/spec.py new file mode 100644 index 0000000..b5773b5 --- /dev/null +++ b/CoderMind/scripts/feature/schemas/spec.py @@ -0,0 +1,252 @@ +"""Pydantic schemas for ``feature_spec.json``. + +The schema mirrors the historical ``feature_spec.json`` shape produced by +``feature_spec_to_json.py`` so that downstream stages (``feature_build``, +``build_skeleton``, …) can consume it without modification. + +Reference sample:: + + CoderMind/workspace/codermind-bench/project_templates/ + decoder/cmind.build_skeleton/data/feature_spec.json + +Top-level fields (order matches the sample):: + + meta: dict + background_and_overview: list[BackgroundItem] + non_functional_requirements:list[NfrItem] + functional_requirements: list[FeatureNode] # recursive tree + repository_name: str + repository_purpose: str + +Evidence +-------- +``evidence`` is optional on every item. When the LLM can identify a clear +source line range (typically when running from ``docs/*.md``) it should +fill it; for ``user_input``-derived items the field stays empty. + +Downstream consumers MUST tolerate missing/empty evidence +(use ``item.get("evidence", [])`` or rely on the Pydantic default). +""" + +from __future__ import annotations + +from typing import List, Literal + +from pydantic import BaseModel, Field + + +ProjectType = Literal[ + "WEB", + "API", + "SERVICE", + "PIPELINE", + "CLI", + "GUI", + "GAME", + "LIBRARY", +] + + +class Evidence(BaseModel): + """A traceable source-line citation for a spec item.""" + + id: str = Field( + ..., + description=( + "Evidence identifier, formed as ``--``. " + "``source_prefix`` derives from the source filename " + "(strip leading digits + extension; ≤20 chars use full name, " + "else uppercase initial of each underscore-separated word). " + "User-input mode always uses prefix ``user_input``." + ), + ) + source: str = Field( + ..., + description=( + "Source filename (e.g. ``02_requirements_specification.md``) " + "or the literal string ``user_input`` for inline requirement text." + ), + ) + line_start: int = Field( + ..., + ge=1, + description="1-based inclusive start line in the source document.", + ) + line_end: int = Field( + ..., + ge=1, + description="1-based inclusive end line in the source document.", + ) + + +class Meta(BaseModel): + """Repository-level metadata.""" + + project_types: List[ProjectType] = Field( + ..., + min_length=1, + description=( + "UPPERCASE tokens drawn from the 8-item whitelist. " + "Examples: ``['WEB']``, ``['WEB','CLI']``, ``['API','SERVICE']``." + ), + ) + project_notes: str = Field( + ..., + max_length=500, + description=( + "Free-form short paragraph (≤500 chars) capturing details the " + "whitelist cannot express — framework choices, special domain " + "requirements, anything unique." + ), + ) + generated_at: str = Field( + ..., + description="Generation date in ``YYYY-MM-DD`` format.", + ) + source_documents: List[str] = Field( + ..., + description=( + "Names of source artefacts that fed this spec, e.g. " + "``['01_charter.md','02_spec.md']`` or ``['user_input']``." + ), + ) + + +class BackgroundItem(BaseModel): + """One background / overview entry.""" + + id: str = Field( + ..., + description="Stable identifier, formatted as ``BG-NNN`` (1-based).", + ) + title: str = Field( + ..., + description="Short title (a few words) summarising the entry.", + ) + description: str = Field( + ..., + description="One-to-three-sentence description in English.", + ) + evidence: List[Evidence] = Field( + default_factory=list, + description="Optional source-line citations supporting this entry.", + ) + + +class NfrItem(BaseModel): + """One non-functional requirement entry.""" + + id: str = Field( + ..., + description="Stable identifier, formatted as ``NFR-NNN`` (1-based).", + ) + title: str = Field( + ..., + description=( + "Short title in Title Case, e.g. ``Data Persistence``, ``Logging``." + ), + ) + description: str = Field( + ..., + description="One-to-three-sentence description in English.", + ) + evidence: List[Evidence] = Field( + default_factory=list, + description="Optional source-line citations supporting this entry.", + ) + + +class FeatureNode(BaseModel): + """One node in the functional-requirement tree. + + Each node is either a *Domain* (top level), *Sub-domain* (intermediate) + or *Leaf feature*. The same shape is used at all levels — depth is + inferred from the ``id`` segment count and from ``children``. + + Hierarchy rules + --------------- + * ``id`` segments equal the depth, e.g. ``FT-001`` is a domain, + ``FT-001-002`` is a sub-domain, ``FT-001-002-003`` is a leaf, etc. + * Depth is project-driven — simple projects may stop at level 2 or 3; + complex projects may reach 5 or 6. Hard ceiling: 6 levels. + * Leaf nodes (``children == []``) must be a Minimum Implementable Unit + (MIU): single verb + single object, no "and"/"or", independently + testable, scope ≤ one function/method, describable in one sentence. + * Intermediate / organisation nodes group children and need not satisfy + MIU; their ``evidence`` is typically empty. + """ + + id: str = Field( + ..., + description=( + "Stable identifier ``FT-NNN[-NNN]*`` matching depth. " + "Use zero-padded 3-digit segments." + ), + ) + name: str = Field( + ..., + description=( + "Title-Case feature name (2-4 English words preferred). " + "Avoid abbreviations; reflect the core responsibility." + ), + ) + description: str = Field( + ..., + description=( + "Describes *what* the feature does, not *how*. " + "One concise sentence is best." + ), + ) + evidence: List[Evidence] = Field( + default_factory=list, + description=( + "Optional source-line citations. Leaf nodes derived from docs " + "should include them; organisation nodes typically have none." + ), + ) + children: List["FeatureNode"] = Field( + default_factory=list, + description="Child feature nodes; empty list means this is a leaf.", + ) + + +# Resolve the forward reference so recursive validation works. +FeatureNode.model_rebuild() + + +class FeatureSpecOutput(BaseModel): + """Top-level model representing the full ``feature_spec.json``. + + Field order intentionally mirrors the historical sample to maximise + diff-friendliness when comparing old vs new outputs. + """ + + meta: Meta + background_and_overview: List[BackgroundItem] + non_functional_requirements: List[NfrItem] + functional_requirements: List[FeatureNode] + repository_name: str = Field( + ..., + description=( + "Concise repository name (1-3 words, kebab-case). " + "Used by downstream stages for code generation and reporting." + ), + ) + repository_purpose: str = Field( + ..., + description=( + "One-to-two-sentence description of the core objective of the " + "repository." + ), + ) + + +__all__ = [ + "ProjectType", + "Evidence", + "Meta", + "BackgroundItem", + "NfrItem", + "FeatureNode", + "FeatureSpecOutput", +] diff --git a/CoderMind/scripts/feature/spec.py b/CoderMind/scripts/feature/spec.py new file mode 100644 index 0000000..4ff8db1 --- /dev/null +++ b/CoderMind/scripts/feature/spec.py @@ -0,0 +1,366 @@ +"""Phase-1 ``feature_spec`` stage — direct JSON generation. + +Reads raw requirements (inline text or ``docs/*.md`` files), drives an LLM +via :class:`LLMClient`, and writes a validated ``feature_spec.json`` ready +for downstream stages (``feature_build`` etc.) to consume. + +This module replaces the historical Markdown-intermediary pipeline +(``feature_spec.md`` slash command + ``feature_spec_to_json.py`` parser). +The LLM emits the final JSON directly, validated against +:class:`feature.schemas.spec.FeatureSpecOutput`. + +Public surface +-------------- +:func:`generate_feature_spec` + Programmatic entry point — used by ``scripts/feature_spec.py`` (CLI) + and the ``feature_construct`` orchestrator. + +:class:`InputSource` + Resolved input description (docs/ vs inline text). + +:func:`resolve_input_source` + Decide which input to use given CLI flags / defaults. + +:func:`probe` + Check whether ``FEATURE_SPEC_FILE`` already contains a valid spec. +""" + +from __future__ import annotations + +import json +import logging +from dataclasses import dataclass, field +from datetime import date +from pathlib import Path +from typing import List, Optional + +from common.llm_client import LLMClient +from common.paths import FEATURE_SPEC_FILE, WORKSPACE_ROOT +from common.trajectory import load_or_create_trajectory + +from .prompts.spec import ( + PROMPT_TEMPLATE_FEATURE_SPEC_SYSTEM, + PROMPT_TEMPLATE_FEATURE_SPEC_USER, +) +from .schemas.spec import FeatureSpecOutput + + +logger = logging.getLogger(__name__) + + +# Default ``docs/`` directory lives at the workspace root. +DEFAULT_DOCS_DIR: Path = WORKSPACE_ROOT / "docs" + + +# =========================================================================== +# Input source resolution +# =========================================================================== + + +@dataclass(frozen=True) +class InputSource: + """Resolved input source for ``feature_spec`` generation. + + Exactly one of ``text`` / ``docs`` is populated, matching ``kind``. + """ + + kind: str # "user_input" | "docs" + text: Optional[str] = None + docs: List[Path] = field(default_factory=list) + + @property + def source_documents(self) -> List[str]: + """Names suitable for ``meta.source_documents``.""" + if self.kind == "user_input": + return ["user_input"] + return [p.name for p in self.docs] + + +class NoInputAvailable(RuntimeError): + """Raised when neither inline text nor ``docs/*.md`` exist.""" + + +def resolve_input_source( + *, + input_text: Optional[str] = None, + docs_dir: Optional[Path] = None, +) -> InputSource: + """Pick the input source for spec generation. + + Resolution order: + + 1. ``input_text`` (non-empty after stripping) → user_input mode. + 2. Markdown files under ``docs_dir`` (default + :data:`DEFAULT_DOCS_DIR`) → docs mode. + 3. Otherwise raise :class:`NoInputAvailable`. + + The function never prompts the user — interactive fallback is the + slash-command layer's responsibility. + """ + if input_text and input_text.strip(): + return InputSource(kind="user_input", text=input_text.strip()) + + docs_dir = docs_dir or DEFAULT_DOCS_DIR + if docs_dir.is_dir(): + md_files = sorted(p for p in docs_dir.glob("*.md") if p.is_file()) + if md_files: + return InputSource(kind="docs", docs=md_files) + + raise NoInputAvailable( + f"No requirement text provided and no Markdown files found in " + f"{docs_dir}. Provide --input-text or populate the docs directory." + ) + + +# =========================================================================== +# Probe / check-only helpers +# =========================================================================== + + +def probe(spec_file: Path = FEATURE_SPEC_FILE) -> dict: + """Inspect the on-disk ``feature_spec.json`` and report status. + + Status values: ``missing`` | ``valid`` | ``invalid``. + The returned dict is JSON-serialisable and intended for both + human-readable display and consumption by the orchestrator / + slash-command layer. + """ + payload = { + "stage": "feature_spec", + "path": str(spec_file), + "status": "missing", + "message": "", + } + + if not spec_file.exists(): + payload["message"] = f"{_display_path(spec_file)} does not exist" + return payload + + try: + data = json.loads(spec_file.read_text(encoding="utf-8")) + except (json.JSONDecodeError, OSError) as exc: + payload["status"] = "invalid" + payload["message"] = f"cannot parse {_display_path(spec_file)}: {exc}" + return payload + + try: + FeatureSpecOutput.model_validate(data) + except Exception as exc: # pydantic.ValidationError, etc. + payload["status"] = "invalid" + payload["message"] = ( + f"{_display_path(spec_file)} does not match FeatureSpecOutput " + f"schema: {str(exc)[:200]}" + ) + return payload + + payload["status"] = "valid" + payload["message"] = f"{_display_path(spec_file)} is valid" + return payload + + +def _display_path(p: Path) -> str: + """Render *p* as ``~/...`` when under ``$HOME``, else absolute.""" + try: + rel = p.relative_to(Path.home()) + return f"~/{rel}" + except ValueError: + return str(p) + + +# =========================================================================== +# LLM generation +# =========================================================================== + + +def _build_input_blob(source: InputSource) -> str: + """Format the input material as the LLM should see it.""" + if source.kind == "user_input": + return f"### user_input\n\n{source.text or ''}" + + chunks: List[str] = [] + for doc in source.docs: + try: + text = doc.read_text(encoding="utf-8") + except OSError as exc: + raise RuntimeError(f"cannot read {doc}: {exc}") from exc + chunks.append(f"### {doc.name}\n\n{text}") + return "\n\n---\n\n".join(chunks) + + +def _build_user_prompt(source: InputSource) -> str: + return PROMPT_TEMPLATE_FEATURE_SPEC_USER.format( + generated_at=date.today().isoformat(), + source_kind=source.kind, + source_documents=", ".join(source.source_documents), + input_blob=_build_input_blob(source), + ) + + +def _call_llm( + source: InputSource, + *, + llm: LLMClient, + max_retries: int = 3, +) -> FeatureSpecOutput: + """Invoke the LLM and return a validated :class:`FeatureSpecOutput`.""" + user_prompt = _build_user_prompt(source) + _, result, _ = llm.call_structured( + system_prompt=PROMPT_TEMPLATE_FEATURE_SPEC_SYSTEM, + user_prompt=user_prompt, + response_model=FeatureSpecOutput, + max_retries=max_retries, + purpose="feature_spec", + ) + if result is None: + raise RuntimeError( + "LLM failed to produce a valid feature_spec.json after " + f"{max_retries} attempts (see LLM trace logs for details)." + ) + return result + + +# =========================================================================== +# Main entry — used by the CLI wrapper and by orchestrator +# =========================================================================== + + +@dataclass +class GenerationResult: + """Outcome of :func:`generate_feature_spec`.""" + + skipped: bool + spec: Optional[FeatureSpecOutput] + output_path: Path + reason: str = "" + + +def generate_feature_spec( + *, + input_text: Optional[str] = None, + docs_dir: Optional[Path] = None, + output_path: Path = FEATURE_SPEC_FILE, + force: bool = False, + enable_trajectory: bool = True, + llm: Optional[LLMClient] = None, +) -> GenerationResult: + """Generate ``feature_spec.json`` from the resolved input. + + Behaviour summary: + + * If ``output_path`` already contains a valid spec and ``force`` is + ``False``, skip (no LLM call, no file write). + * Otherwise call the LLM, validate the output, and atomically write + the file. + + Atomic write: contents are first written to ``.tmp``, + then renamed. On any failure no partial file is left behind. + + Raises: + NoInputAvailable: if no requirement source could be resolved. + RuntimeError: on LLM failure / schema validation failure. + """ + # Skip if already valid and not forced. + if not force and output_path.exists(): + existing = probe(output_path) + if existing["status"] == "valid": + return GenerationResult( + skipped=True, + spec=None, + output_path=output_path, + reason="already valid (use --force to regenerate)", + ) + + source = resolve_input_source(input_text=input_text, docs_dir=docs_dir) + logger.info( + "feature_spec source: %s (%d items)", + source.kind, + 1 if source.kind == "user_input" else len(source.docs), + ) + + # Trajectory + LLM client setup mirroring feature_build / feature_refactor. + trajectory = None + step_id = None + if enable_trajectory: + trajectory = load_or_create_trajectory("feature_spec") + trajectory.start(metadata={ + "source_kind": source.kind, + "source_documents": source.source_documents, + "output_path": str(output_path), + }) + step = trajectory.add_step( + "feature_spec", + "Generate feature_spec.json directly from requirements", + ) + trajectory.start_step(step.step_id) + step_id = step.step_id + + if llm is None: + llm = LLMClient(trajectory=trajectory, step_id=step_id) + elif trajectory is not None: + llm.set_trajectory(trajectory, step_id=step_id) + + try: + spec = _call_llm(source, llm=llm) + _atomic_write_json(output_path, spec) + except Exception as exc: + if trajectory is not None and step_id is not None: + trajectory.fail_step(step_id, str(exc)) + trajectory.fail(str(exc)) + raise + + if trajectory is not None and step_id is not None: + trajectory.complete_step(step_id, { + "top_level_features": len(spec.functional_requirements), + "background_items": len(spec.background_and_overview), + "nfr_items": len(spec.non_functional_requirements), + }) + trajectory.complete(metadata={ + "repository_name": spec.repository_name, + }) + logger.info("[OK] Trajectory saved to: %s", trajectory.trajectory_file) + + return GenerationResult( + skipped=False, + spec=spec, + output_path=output_path, + reason="generated", + ) + + +# =========================================================================== +# Helpers +# =========================================================================== + + +def _atomic_write_json(path: Path, model: FeatureSpecOutput) -> None: + """Write *model* to *path* atomically. + + The model is dumped via Pydantic to preserve field order and the + declared schema, then written to ``.tmp`` and renamed. On any + failure the temp file is cleaned up so no partial artefact remains. + """ + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(path.suffix + ".tmp") + try: + # ``model_dump_json`` keeps field ordering and renders unicode. + payload = model.model_dump_json(indent=2) + tmp.write_text(payload + "\n", encoding="utf-8") + tmp.replace(path) # atomic on POSIX + except Exception: + # Best-effort cleanup of any partial temp file. + try: + tmp.unlink(missing_ok=True) + except OSError: + pass + raise + + +__all__ = [ + "DEFAULT_DOCS_DIR", + "GenerationResult", + "InputSource", + "NoInputAvailable", + "generate_feature_spec", + "probe", + "resolve_input_source", +] diff --git a/CoderMind/scripts/feature_construct.py b/CoderMind/scripts/feature_construct.py new file mode 100644 index 0000000..87b966a --- /dev/null +++ b/CoderMind/scripts/feature_construct.py @@ -0,0 +1,466 @@ +#!/usr/bin/env python3 +"""Phase 1 feature construction facade orchestrator.""" + +from __future__ import annotations + +import argparse +import json +import shutil +import signal +import subprocess +import sys +import time +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Optional + +from common.paths import FEATURE_BUILD_FILE as _FEATURE_BUILD_FILE +from common.paths import FEATURE_SPEC_FILE as _FEATURE_SPEC_FILE +from common.paths import FEATURE_TREE_FILE as _FEATURE_TREE_FILE + +_SCRIPTS_DIR = Path(__file__).resolve().parent + +FEATURE_SPEC_FILE = _FEATURE_SPEC_FILE +FEATURE_BUILD_FILE = _FEATURE_BUILD_FILE +FEATURE_TREE_FILE = _FEATURE_TREE_FILE + +_LOGICAL_PATHS = { + "feature_spec": ".cmind/data/feature_spec.json", + "feature_build": ".cmind/data/feature_build.json", + "feature_refactor": ".cmind/data/feature_tree.json", +} + +_RESET_REASONS = {"forced", "upstream rebuilt"} + +_REQUIRED_FEATURE_SPEC_FIELDS = ( + "meta", + "repository_name", + "repository_purpose", + "background_and_overview", + "functional_requirements", + "non_functional_requirements", +) + + +@dataclass(frozen=True) +class Stage: + name: str + build_script: str + + +STAGES: tuple[Stage, ...] = ( + Stage(name="feature_spec", build_script="feature_spec.py"), + Stage(name="feature_build", build_script="feature_build.py"), + Stage(name="feature_refactor", build_script="feature_refactor.py"), +) + + +@dataclass +class StageState: + stage: Stage + type: str = "init" + message: str = "" + done: bool = False + will_run: bool = False + reason: str = "" + raw: dict[str, Any] = field(default_factory=dict) + + +def _resolve_invoker() -> list[str]: + cmind = shutil.which("cmind") + if cmind: + return [cmind, "script"] + return [sys.executable] + + +def _script_argv(invoker: list[str], script_name: str) -> list[str]: + # Use ``.stem`` so the check matches both ``cmind`` (POSIX) and + # ``cmind.exe`` (Windows packaging) uniformly. + if Path(invoker[0]).stem == "cmind": + return [*invoker, script_name] + return [*invoker, str(_SCRIPTS_DIR / script_name)] + + +def _run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + argv = [*_script_argv(invoker, script_name), *extra] + proc = subprocess.run(argv, check=False) + return proc.returncode + + +def _load_json_object(path: Path) -> tuple[Optional[dict[str, Any]], Optional[str]]: + try: + with path.open("r", encoding="utf-8") as handle: + data = json.load(handle) + except FileNotFoundError: + return None, "missing" + except json.JSONDecodeError as exc: + return None, f"invalid JSON: {exc.msg}" + except OSError as exc: + return None, f"cannot read file: {exc}" + + if not isinstance(data, dict): + return None, "JSON root must be an object" + return data, None + + +def _has_content(value: Any) -> bool: + if value is None: + return False + if isinstance(value, str): + return bool(value.strip()) + if isinstance(value, (list, dict)): + return bool(value) + return True + + +def _state(stage: Stage, type_: str, message: str, raw: Optional[dict[str, Any]] = None) -> StageState: + return StageState( + stage=stage, + type=type_, + message=message, + done=(type_ == "update"), + raw=raw or {}, + ) + + +def _check_feature_spec(stage: Stage) -> StageState: + data, error = _load_json_object(FEATURE_SPEC_FILE) + logical = _LOGICAL_PATHS[stage.name] + if error == "missing": + return _state(stage, "init", f"{logical} is missing") + if error: + return _state(stage, "warning", f"{logical} is not complete: {error}") + + missing = [field for field in _REQUIRED_FEATURE_SPEC_FIELDS if not _has_content(data.get(field))] + if missing: + return _state( + stage, + "warning", + f"{logical} is missing required fields: {', '.join(missing)}", + {"missing_fields": missing}, + ) + return _state(stage, "update", f"{logical} is valid") + + +def _check_feature_build(stage: Stage) -> StageState: + data, error = _load_json_object(FEATURE_BUILD_FILE) + logical = _LOGICAL_PATHS[stage.name] + if error == "missing": + return _state(stage, "init", f"{logical} is missing") + if error: + return _state(stage, "warning", f"{logical} is not complete: {error}") + return _state(stage, "update", f"{logical} is valid JSON", {"keys": sorted(data.keys())}) + + +def _check_feature_refactor(stage: Stage) -> StageState: + data, error = _load_json_object(FEATURE_TREE_FILE) + logical = _LOGICAL_PATHS[stage.name] + if error == "missing": + return _state(stage, "init", f"{logical} is missing") + if error: + return _state(stage, "warning", f"{logical} is not complete: {error}") + + components = data.get("components") + if isinstance(components, (list, dict)) and components: + return _state(stage, "update", f"{logical} has components") + return _state(stage, "warning", f"{logical} has no non-empty components collection") + + +def _check_stage(stage: Stage) -> StageState: + if stage.name == "feature_spec": + return _check_feature_spec(stage) + if stage.name == "feature_build": + return _check_feature_build(stage) + if stage.name == "feature_refactor": + return _check_feature_refactor(stage) + return _state(stage, "error", f"unknown stage: {stage.name}") + + +def probe() -> list[StageState]: + return [_check_stage(stage) for stage in STAGES] + + +def decide(states: list[StageState], force: bool) -> None: + cascade = False + for state in states: + if force: + state.will_run = True + state.reason = "forced" + continue + if cascade: + state.will_run = True + state.reason = "upstream rebuilt" + continue + if state.type == "update": + state.will_run = False + state.reason = "up-to-date" + else: + state.will_run = True + state.reason = f"type={state.type}" + cascade = True + + +_GLYPH = {"update": "✓", "init": "·", "warning": "!", "error": "✗"} + + +def _format_table(states: list[StageState]) -> str: + rows = ["Stage Type Done Action"] + rows.append("-" * 52) + for state in states: + glyph = _GLYPH.get(state.type, "?") + action = "run" if state.will_run else "skip" + rows.append( + f"{state.stage.name:<16} {glyph} {state.type:<8} " + f"{'yes' if state.done else 'no ':<3} {action}" + ) + return "\n".join(rows) + + +def _print_probe_summary(states: list[StageState]) -> None: + done = sum(1 for state in states if state.done) + total = len(states) + first_pending = next((state.stage.name for state in states if not state.done), None) + print(f"Feature construction progress: {done}/{total} stages complete.") + if first_pending: + print(f"Next pending stage: {first_pending}") + else: + print("All Phase 1 stages are up-to-date.") + print() + print(_format_table(states)) + + +def _check_only_payload(states: list[StageState]) -> dict[str, Any]: + done = sum(1 for state in states if state.done) + total = len(states) + next_pending = next((state.stage.name for state in states if not state.done), None) + return { + "total": total, + "done": done, + "completed": done, + "next": next_pending, + "stages": [ + { + "name": state.stage.name, + "type": state.type, + "message": state.message, + "done": state.done, + "will_run": state.will_run, + "reason": state.reason, + } + for state in states + ], + } + + +def _emit_check_only_json(states: list[StageState]) -> None: + print(json.dumps(_check_only_payload(states), indent=2)) + + +def _format_number(value: float) -> str: + if value.is_integer(): + return str(int(value)) + return str(value) + + +def _build_args_for(stage: Stage, args: argparse.Namespace) -> list[str]: + extra: list[str] = [] + if stage.name == "feature_spec": + # ``feature_spec.py`` accepts the standard facade flags only; + # inline requirement text reaches it via the slash-command layer + # (which calls ``feature_spec.py --input-text "..."`` directly + # before resuming the orchestrator). + if args.force: + extra.append("--force") + if args.verbose: + extra.append("--verbose") + if args.no_trajectory: + extra.append("--no-trajectory") + elif stage.name == "feature_build": + extra.extend(["--mode", "step1"]) + if args.review_threshold is not None: + extra.extend(["--review-threshold", _format_number(args.review_threshold)]) + if args.review_max_iterations is not None: + extra.extend(["--review-max-iterations", str(args.review_max_iterations)]) + if args.verbose: + extra.append("--verbose") + if args.no_trajectory: + extra.append("--no-trajectory") + elif stage.name == "feature_refactor": + if args.max_iter_refactor is not None: + extra.extend(["--max-iterations", str(args.max_iter_refactor)]) + if args.verbose: + extra.extend(["--log-level", "DEBUG"]) + if args.no_trajectory: + extra.append("--no-trajectory") + return extra + + +def _debug_args_for(stage: Stage) -> list[str]: + if stage.name == "feature_build": + return ["--verbose"] + if stage.name == "feature_refactor": + return ["--log-level", "DEBUG"] + return [] + + +def _target_output_for(stage: Stage) -> Optional[Path]: + return { + "feature_spec": FEATURE_SPEC_FILE, + "feature_build": FEATURE_BUILD_FILE, + "feature_refactor": FEATURE_TREE_FILE, + }.get(stage.name) + + +def _should_reset_output(state: StageState) -> bool: + if not state.will_run or _target_output_for(state.stage) is None: + return False + return state.reason in _RESET_REASONS or state.type not in {"init", "update"} + + +def _reset_output_if_needed(state: StageState) -> None: + target = _target_output_for(state.stage) + if target is not None and _should_reset_output(state): + target.unlink(missing_ok=True) + + +def _parse_args(argv: Optional[list[str]] = None) -> argparse.Namespace: + parser = argparse.ArgumentParser( + prog="feature_construct.py", + description="Run the Phase 1 feature construction pipeline with automatic resume.", + ) + parser.add_argument("--check-only", action="store_true", help="Probe all stages and exit.") + parser.add_argument("--json", action="store_true", help="With --check-only, emit JSON.") + parser.add_argument("--force", action="store_true", help="Rebuild all Phase 1 stages.") + parser.add_argument("--dry-run", action="store_true", help="Print commands without executing them.") + parser.add_argument("--verbose", action="store_true", help="Forward native verbose logging flags.") + parser.add_argument("--no-trajectory", action="store_true", help="Disable trajectory recording where supported.") + parser.add_argument( + "--max-iter-refactor", + type=int, + default=None, + metavar="N", + help="Override feature_refactor.py --max-iterations.", + ) + parser.add_argument( + "--review-threshold", + type=float, + default=None, + metavar="N", + help="Forward to feature_build.py --review-threshold.", + ) + parser.add_argument( + "--review-max-iterations", + type=int, + default=None, + metavar="N", + help="Forward to feature_build.py --review-max-iterations.", + ) + return parser.parse_args(argv) + + +def _install_sigint_handler() -> None: + def _handle(signum: int, frame: Any) -> None: + print("\n[feature_construct] interrupted — rerun `cmind script feature_construct.py` to resume.") + sys.exit(130) + + signal.signal(signal.SIGINT, _handle) + + +def _print_failure_hint( + invoker: list[str], + stage: Stage, + rc: int, + *, + phase: str, +) -> None: + """Print recovery hints to stderr after a stage fails. + + Commands are rendered using the current *invoker* so the hint stays + correct whether ``cmind`` is on ``$PATH`` (``cmind script ``) + or the Python fallback is in use (``python ``). + """ + debug_args = _debug_args_for(stage) + + def _fmt(name: str, *extra: str) -> str: + # Render the command using the basename of the invoker so + # users see e.g. ``cmind script ...`` or ``python ...`` rather + # than the resolved absolute path that subprocess actually uses. + argv = _script_argv(invoker, name) + display = [Path(argv[0]).name, *argv[1:], *extra] + return " ".join(display) + + print(file=sys.stderr) + print(f"X {stage.name} {phase} failed (exit {rc})", file=sys.stderr) + print(f" Resume : {_fmt('feature_construct.py')}", file=sys.stderr) + print(f" Debug : {_fmt(stage.build_script, *debug_args)}", file=sys.stderr) + print(f" Status : {_fmt('feature_construct.py', '--check-only')}", file=sys.stderr) + + +def main(argv: Optional[list[str]] = None) -> int: + args = _parse_args(argv) + _install_sigint_handler() + invoker = _resolve_invoker() + + states = probe() + decide(states, force=args.force) + + if args.check_only: + if args.json: + _emit_check_only_json(states) + else: + _print_probe_summary(states) + return 0 + + runnable = [state for state in states if state.will_run] + if not runnable: + print("All Phase 1 stages are already complete — nothing to do.") + print("Use `--force` to rebuild from scratch.") + print("Next: `/cmind.plan` to build the Repository Planning Graph (RPG).") + return 0 + + print(f"Feature construction pipeline: {len(runnable)} of {len(states)} stages to run.") + print(_format_table(states)) + print() + + if args.dry_run: + for state in runnable: + cmd = _script_argv(invoker, state.stage.build_script) + cmd += _build_args_for(state.stage, args) + print("DRY-RUN >", " ".join(cmd)) + return 0 + + started = time.monotonic() + for state in states: + if not state.will_run: + print(f"skip {state.stage.name:<16} ({state.reason})") + continue + + stage_started = time.monotonic() + print(f"run {state.stage.name:<16} {state.stage.build_script} ...") + _reset_output_if_needed(state) + rc = _run_stage(invoker, state.stage.build_script, _build_args_for(state.stage, args)) + if rc != 0: + _print_failure_hint(invoker, state.stage, rc, phase="build") + return rc + + verify = _check_stage(state.stage) + if verify.type != "update": + print( + f" verification failed: {verify.type} — {verify.message}", + file=sys.stderr, + ) + _print_failure_hint(invoker, state.stage, 1, phase="check") + return 1 + + elapsed = time.monotonic() - stage_started + print(f"done {state.stage.name:<16} in {elapsed:.1f}s") + + total_elapsed = time.monotonic() - started + print() + print(f"Feature construct complete in {total_elapsed:.1f}s.") + print("Next: `/cmind.plan` to build the Repository Planning Graph (RPG).") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/CoderMind/scripts/feature_spec.py b/CoderMind/scripts/feature_spec.py new file mode 100644 index 0000000..82cc026 --- /dev/null +++ b/CoderMind/scripts/feature_spec.py @@ -0,0 +1,166 @@ +#!/usr/bin/env python3 +"""Phase-1 ``feature_spec`` stage — CLI wrapper. + +This is the entry point invoked by ``cmind script feature_spec.py`` and +by the ``feature_construct`` orchestrator. All real work happens in +:mod:`feature.spec`; this module only translates CLI arguments and +formats human-readable output. + +Examples +-------- + +Generate from ``docs/*.md`` (auto-detected):: + + cmind script feature_spec.py + +Generate from inline requirement text:: + + cmind script feature_spec.py --input-text "Build a CLI for managing Docker containers" + +Probe current state without modifying anything:: + + cmind script feature_spec.py --check-only --json +""" + +from __future__ import annotations + +import argparse +import json +import logging +import sys +from pathlib import Path +from typing import Optional + +from common.paths import FEATURE_SPEC_FILE + +from feature.spec import ( + DEFAULT_DOCS_DIR, + NoInputAvailable, + generate_feature_spec, + probe, +) + + +def _configure_logging(verbose: bool) -> None: + level = logging.DEBUG if verbose else logging.INFO + logging.basicConfig( + level=level, + format="%(asctime)s - %(levelname)s - %(message)s", + ) + + +def _parse_args(argv: Optional[list[str]] = None) -> argparse.Namespace: + parser = argparse.ArgumentParser( + prog="feature_spec.py", + description=( + "Generate feature_spec.json directly from requirements " + "(inline text or docs/*.md), via a strict Pydantic-validated " + "LLM call." + ), + ) + parser.add_argument( + "--docs", + type=Path, + default=None, + metavar="DIR", + help=( + "Directory containing requirement Markdown files " + f"(default: {DEFAULT_DOCS_DIR})." + ), + ) + parser.add_argument( + "--input-text", + default=None, + metavar="TEXT", + help=( + "Inline requirement description. Overrides --docs." + ), + ) + parser.add_argument( + "--output", + type=Path, + default=FEATURE_SPEC_FILE, + metavar="PATH", + help=f"Output path (default: {FEATURE_SPEC_FILE}).", + ) + parser.add_argument( + "--force", + action="store_true", + help="Overwrite even if a valid feature_spec.json already exists.", + ) + parser.add_argument( + "--check-only", + action="store_true", + help="Probe the output state and exit without invoking the LLM.", + ) + parser.add_argument( + "--json", + action="store_true", + help="With --check-only, emit JSON instead of human-readable text.", + ) + parser.add_argument( + "--verbose", + action="store_true", + help="Enable DEBUG logging.", + ) + parser.add_argument( + "--no-trajectory", + action="store_true", + help="Disable trajectory recording.", + ) + return parser.parse_args(argv) + + +def _emit_probe(payload: dict, as_json: bool) -> None: + if as_json: + print(json.dumps(payload, indent=2)) + return + status = payload["status"] + marker = { + "valid": "[OK]", + "missing": "[--]", + "invalid": "[!!]", + }.get(status, "[??]") + print(f"{marker} feature_spec: {payload['message']}") + + +def main(argv: Optional[list[str]] = None) -> int: + args = _parse_args(argv) + _configure_logging(args.verbose) + + if args.check_only: + _emit_probe(probe(args.output), as_json=args.json) + return 0 + + try: + result = generate_feature_spec( + input_text=args.input_text, + docs_dir=args.docs, + output_path=args.output, + force=args.force, + enable_trajectory=not args.no_trajectory, + ) + except NoInputAvailable as exc: + print(f"[FAIL] {exc}", file=sys.stderr) + return 2 + except RuntimeError as exc: + print(f"[FAIL] {exc}", file=sys.stderr) + return 1 + + if result.skipped: + print(f"[SKIP] feature_spec: {result.reason}") + print(f" Output: {result.output_path}") + return 0 + + assert result.spec is not None # generated path always sets spec + spec = result.spec + print(f"[OK] feature_spec written to {result.output_path}") + print(f" Repository: {spec.repository_name}") + print(f" Top-level features: {len(spec.functional_requirements)}") + print(f" Background items: {len(spec.background_and_overview)}") + print(f" NFR items: {len(spec.non_functional_requirements)}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/CoderMind/scripts/feature_spec_to_json.py b/CoderMind/scripts/feature_spec_to_json.py deleted file mode 100644 index d6ec4e5..0000000 --- a/CoderMind/scripts/feature_spec_to_json.py +++ /dev/null @@ -1,444 +0,0 @@ -#!/usr/bin/env python3 -"""Build feature specification JSON from Markdown documentation files. - -This script parses: - - feature_spec.md: Contains meta, background, NFR sections and feature tree links - - features/*.md: Contains detailed feature hierarchies - -Output: A structured JSON file with all parsed content. - -Usage: - cmind script feature_spec_to_json.py [--input-dir DIR] [--output FILE] [--no-evidence] - -Arguments: - --input-dir Directory containing feature_spec.md and features/ folder - Default: .cmind/data/feature_spec - --output Output JSON file path - Default: feature_spec.json in input directory - --no-evidence Exclude evidence fields from output for compact JSON -""" - -import argparse -import json -import re -import sys -from pathlib import Path -from typing import Optional - -# Use the canonical paths from common.paths so the output location -# matches what downstream stages (feature_build, feature_build_validation, -# ...) expect. That resolves to -# ``~/.cmind/workspaces//data/feature_spec.json`` rather than the -# workspace-local ``.cmind/data/feature_spec.json`` this script used -# to compute on its own — a mismatch that previously broke the -# feature_spec → feature_build handoff. -from common.paths import FEATURE_SPEC_FILE - - -def parse_evidence_line(line: str) -> Optional[dict]: - """Parse an evidence reference line. - - Format: " - evidence_id | document.md Lstart-Lend". - """ - line = line.strip() - if not line.startswith("- "): - return None - - content = line[2:].strip() - - # Pattern: "evidence_id | document.md Lstart-Lend" or "evidence_id | document.md Lstart" - match = re.match(r'^([^\|]+)\s*\|\s*(\S+)\s+L(\d+)(?:-L?(\d+))?$', content) - if match: - evidence_id = match.group(1).strip() - document = match.group(2).strip() - line_start = int(match.group(3)) - line_end = int(match.group(4)) if match.group(4) else line_start - return { - "evidence_id": evidence_id, - "document_id": document, - "line_start": line_start, - "line_end": line_end - } - - return None - - -def parse_meta_section(lines: list, start_idx: int) -> tuple: - """Parse the Meta section.""" - meta = {} - i = start_idx - - while i < len(lines): - line = lines[i].strip() - - # Stop at next section - if line.startswith("## ") and not line.startswith("## Meta"): - break - - if line.startswith("- **Repository Name**:"): - meta["repository_name"] = line.split(":", 1)[1].strip() - elif line.startswith("- **Repository Purpose**:"): - meta["repository_purpose"] = line.split(":", 1)[1].strip() - elif line.startswith("- **Project Types**:"): - # Comma- or bracket-list of UPPERCASE tokens, e.g. "WEB, CLI" - # or "[WEB, CLI]" or '["WEB", "CLI"]' (JSON-style). Strip - # wrappers, split on comma, then strip stray quotes/whitespace - # from each token. validate_project_types() further filters - # against the whitelist. - raw = line.split(":", 1)[1].strip() - raw = raw.strip("[]").strip() - tokens = [] - for t in raw.split(","): - t = t.strip().strip('"').strip("'").strip() - if t: - tokens.append(t) - meta["project_types"] = tokens - elif line.startswith("- **Project Notes**:"): - meta["project_notes"] = line.split(":", 1)[1].strip() - elif line.startswith("- **Generated At**:"): - meta["generated_at"] = line.split(":", 1)[1].strip() - elif line.startswith("- **Source Documents**:"): - docs = line.split(":", 1)[1].strip() - meta["source_documents"] = [d.strip() for d in docs.split(",")] - - i += 1 - - return meta, i - - -def parse_bg_or_nfr_item(lines: list, start_idx: int, include_evidence: bool = True) -> tuple: - """Parse a single BG or NFR item.""" - i = start_idx - line = lines[i].strip() - - # Parse header: "### BG-001: Title" or "### NFR-001: Title" - match = re.match(r'^###\s+(BG|NFR)-(\d+):\s+(.+)$', line) - if not match: - return None, i + 1 - - item_type = match.group(1) - item_num = match.group(2) - title = match.group(3).strip() - item_id = f"{item_type}-{item_num}" - - item = { - "id": item_id, - "title": title, - } - if include_evidence: - item["evidence"] = [] - - i += 1 - in_evidence = False - - while i < len(lines): - line = lines[i].strip() - - # Stop at next item or section - if line.startswith("### ") or line.startswith("## "): - break - - if line.startswith("- **Description**:"): - item["description"] = line.split(":", 1)[1].strip() - elif line.startswith("- **Evidence**:"): - in_evidence = True - elif in_evidence and line.startswith("- ") and include_evidence: - evidence = parse_evidence_line(line) - if evidence: - item["evidence"].append(evidence) - elif not line.startswith("-") and line: - in_evidence = False - - i += 1 - - return item, i - - -def parse_background_section(lines: list, start_idx: int, include_evidence: bool = True) -> tuple: - """Parse the Background section.""" - backgrounds = [] - i = start_idx - - while i < len(lines): - line = lines[i].strip() - - # Stop at next major section - if line.startswith("## ") and not line.startswith("## Background"): - break - - if line.startswith("### BG-"): - item, i = parse_bg_or_nfr_item(lines, i, include_evidence) - if item: - backgrounds.append(item) - else: - i += 1 - - return backgrounds, i - - -def parse_nfr_section(lines: list, start_idx: int, include_evidence: bool = True) -> tuple: - """Parse the NFR section.""" - nfrs = [] - i = start_idx - - while i < len(lines): - line = lines[i].strip() - - # Stop at next major section (or end) - if line.startswith("## ") and not line.startswith("## NFR"): - break - - if line.startswith("### NFR-"): - item, i = parse_bg_or_nfr_item(lines, i, include_evidence) - if item: - nfrs.append(item) - else: - i += 1 - - return nfrs, i - - -def parse_feature_tree_links(lines: list, start_idx: int) -> tuple: - """Parse Feature Tree links to get feature file references.""" - links = [] - i = start_idx - - while i < len(lines): - line = lines[i].strip() - - # Stop at next section - if line.startswith("## ") and not line.startswith("## Feature Tree"): - break - - # Pattern: "- [FT-001: Title](features/FT-001.md)" - match = re.match(r'^-\s+\[([^\]]+)\]\(([^\)]+)\)$', line) - if match: - title = match.group(1) - path = match.group(2) - links.append({"title": title, "path": path}) - - i += 1 - - return links, i - - -def parse_feature_file(file_path: Path, include_evidence: bool = True) -> Optional[dict]: - """Parse a single feature file (e.g., FT-001.md).""" - if not file_path.exists(): - return None - - content = file_path.read_text(encoding="utf-8") - lines = content.split("\n") - - feature = None - stack = [] # Stack to track parent features at each level - - i = 0 - while i < len(lines): - line = lines[i] - stripped = line.strip() - - # Match feature headers at any level - # # FT-001: Title (level 1) - # ## FT-001-001: Title (level 2) - # ### FT-001-001-001: Title (level 3) - header_match = re.match(r'^(#+)\s+(FT-[\d-]+):\s+(.+)$', stripped) - - if header_match: - level = len(header_match.group(1)) - feature_id = header_match.group(2) - name = header_match.group(3).strip() - - new_feature = { - "id": feature_id, - "name": name, - "description": "", - "children": [] - } - if include_evidence: - new_feature["evidence"] = [] - - # Parse description and evidence - i += 1 - in_evidence = False - - while i < len(lines): - current = lines[i].strip() - - # Stop if we hit another header - if re.match(r'^#+\s+(FT-[\d-]+):', current): - break - - if current.startswith("- **Description**:"): - new_feature["description"] = current.split(":", 1)[1].strip() - elif current.startswith("- **Evidence**:"): - in_evidence = True - elif in_evidence and current.startswith("- ") and include_evidence: - evidence = parse_evidence_line(current) - if evidence: - new_feature["evidence"].append(evidence) - elif not current.startswith("-") and current: - in_evidence = False - - i += 1 - - # Determine where to place this feature - if level == 1: - feature = new_feature - stack = [(1, feature)] - else: - # Find parent at level - 1 - while stack and stack[-1][0] >= level: - stack.pop() - - if stack: - parent = stack[-1][1] - parent["children"].append(new_feature) - - stack.append((level, new_feature)) - else: - i += 1 - - return feature - - -def parse_feature_spec(input_dir: Path, include_evidence: bool = True) -> dict: - """Parse the complete feature specification from Markdown files.""" - spec_file = input_dir / "feature_spec.md" - - if not spec_file.exists(): - raise FileNotFoundError(f"feature_spec.md not found in {input_dir}") - - content = spec_file.read_text(encoding="utf-8") - lines = content.split("\n") - - result = { - "meta": {}, - "background_and_overview": [], - "non_functional_requirements": [], - "functional_requirements": [] - } - - i = 0 - while i < len(lines): - line = lines[i].strip() - - if line == "## Meta": - meta, i = parse_meta_section(lines, i + 1) - result["meta"] = meta - elif line == "## Background": - backgrounds, i = parse_background_section(lines, i + 1, include_evidence) - result["background_and_overview"] = backgrounds - elif line == "## NFR": - nfrs, i = parse_nfr_section(lines, i + 1, include_evidence) - result["non_functional_requirements"] = nfrs - else: - i += 1 - - # Scan features/ directory for feature files - features_dir = input_dir / "features" - if features_dir.exists(): - for feature_file in sorted(features_dir.glob("FT-*.md")): - feature = parse_feature_file(feature_file, include_evidence) - if feature: - result["functional_requirements"].append(feature) - - # Extract repository info from meta - if "repository_name" in result["meta"]: - result["repository_name"] = result["meta"].pop("repository_name") - if "repository_purpose" in result["meta"]: - result["repository_purpose"] = result["meta"].pop("repository_purpose") - - return result - - -def main(): - parser = argparse.ArgumentParser( - description="Convert Markdown feature specification to JSON format" - ) - parser.add_argument( - "--input-dir", - type=Path, - default=None, - help="Directory containing feature_spec.md and features/ folder" - ) - parser.add_argument( - "--output", - type=Path, - default=None, - help="Output JSON file path" - ) - parser.add_argument( - "--no-evidence", - action="store_true", - default=True, - help="Exclude evidence fields from output" - ) - - args = parser.parse_args() - - # Determine input directory - if args.input_dir: - input_dir = args.input_dir - else: - # Try to find .cmind/data/feature_spec relative to current directory - cwd = Path.cwd() - default_path = cwd / ".cmind" / "data" / "feature_spec" - if default_path.exists(): - input_dir = default_path - else: - # Try relative to script location - script_dir = Path(__file__).parent - input_dir = script_dir.parent / "data" / "feature_spec" - - if not input_dir.exists(): - print(f"Error: Input directory not found: {input_dir}", file=sys.stderr) - sys.exit(1) - - # Determine output file - if args.output: - output_file = args.output - else: - # Default to the canonical location from common.paths so - # downstream stages (feature_build) can find it. The output - # lives in the home-side data dir. - output_file = FEATURE_SPEC_FILE - - include_evidence = not args.no_evidence - - print(f"Parsing feature specification from: {input_dir.name}") - print(f"Include evidence: {include_evidence}") - - try: - spec = parse_feature_spec(input_dir, include_evidence) - - # Write output - output_file.parent.mkdir(parents=True, exist_ok=True) - with open(output_file, "w", encoding="utf-8") as f: - json.dump(spec, f, indent=2, ensure_ascii=False) - - # Print summary — use only the file name so stdout stays - # workspace-independent; the agent cannot access home-side paths. - print(f"\nOutput written to: {output_file.name}") - print(f" - Repository: {spec.get('repository_name', 'N/A')}") - print(f" - Background items: {len(spec.get('background_and_overview', []))}") - print(f" - NFR items: {len(spec.get('non_functional_requirements', []))}") - print(f" - Top-level features: {len(spec.get('functional_requirements', []))}") - - # Count total feature nodes - def count_features(features: list) -> int: - count = len(features) - for f in features: - count += count_features(f.get("children", [])) - return count - - total_features = count_features(spec.get("functional_requirements", [])) - print(f" - Total feature nodes: {total_features}") - - except Exception as e: - print(f"Error: {e}", file=sys.stderr) - sys.exit(1) - - -if __name__ == "__main__": - main() diff --git a/CoderMind/scripts/plan.py b/CoderMind/scripts/plan.py new file mode 100644 index 0000000..a0121e4 --- /dev/null +++ b/CoderMind/scripts/plan.py @@ -0,0 +1,575 @@ +#!/usr/bin/env python3 +"""Plan Orchestrator Script. + +Run the full RPG planning pipeline in one shot, replacing the +five sequential slash-commands ``/cmind.build_skeleton`` → +``/cmind.build_data_flow`` → ``/cmind.design_base_classes`` → +``/cmind.design_interfaces`` → ``/cmind.plan_tasks``. + +Design contract +--------------- + +This script is intentionally non-interactive. All user-facing +"continue / restart / exit" decisions belong to the slash-command +template (``templates/commands/plan.md``); this script only +implements the three execution modes the template chooses from: + +* ``--check-only [--json]`` — probe every stage's ``check_*.py`` + script and print a progress report, then exit 0. This is how + the template inspects the workspace before prompting the user. + +* (default) — *resume mode*: skip stages whose check returns + ``type == "update"``; run every other stage in dependency order. + Once any stage gets (re)built, all downstream stages are forced + to rebuild too, so up- and down-stream artifacts never drift + apart. + +* ``--force`` — discard the current progress and rebuild all five + stages from scratch. + +Sub-scripts are invoked via ``cmind script `` when the +``cmind`` CLI is on ``$PATH`` (so each stage gets its own +``logs/.log`` and inner-git snapshot, courtesy of the +dispatcher). When ``cmind`` is missing, the script falls back +to a direct ``python `` invocation. + +Exit codes +---------- + +* 0 — pipeline finished successfully (or nothing to do) +* 2 — argument error +* 130 — interrupted with Ctrl-C +* N — exit code of the first failing sub-stage (passed through) +""" + +from __future__ import annotations + +import argparse +import json +import shutil +import signal +import subprocess +import sys +import time +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Optional + +# Sub-scripts live in the same directory as this file (bundled under +# cmind_cli/core_pack/scripts/ in the installed wheel). +_SCRIPTS_DIR = Path(__file__).resolve().parent + + +# --------------------------------------------------------------------------- +# Stage table — single source of truth for the pipeline. +# --------------------------------------------------------------------------- + +@dataclass(frozen=True) +class Stage: + """One step of the planning pipeline.""" + + name: str # short id used by the user and the template + build_script: str # the .py runner under scripts/ + check_script: str # the .py probe under scripts/ + max_iter_flag: Optional[str] # CLI flag used by the build script, if any + + +STAGES: tuple[Stage, ...] = ( + Stage( + name="skeleton", + build_script="build_skeleton.py", + check_script="check_skeleton.py", + max_iter_flag="--max-iterations", + ), + Stage( + name="data_flow", + build_script="build_data_flow.py", + check_script="check_data_flow.py", + max_iter_flag="--max-iterations", + ), + Stage( + name="base_classes", + build_script="design_base_classes.py", + check_script="check_base_classes.py", + max_iter_flag="--max-iterations", + ), + Stage( + name="interfaces", + build_script="design_interfaces.py", + check_script="check_interfaces.py", + # design_interfaces uses a different flag name than the others. + max_iter_flag="--max-file-iterations", + ), + Stage( + name="tasks", + build_script="plan_tasks.py", + check_script="check_tasks.py", + max_iter_flag=None, # plan_tasks.py takes no iteration count. + ), +) + +# Post-pipeline helper scripts. Always run on a successful pipeline +# so the user gets an up-to-date summary + visualization. +POST_STEPS: tuple[str, ...] = ( + "summary_skeleton.py", + "generate_viz.py", +) + + +# --------------------------------------------------------------------------- +# Subprocess helpers. +# --------------------------------------------------------------------------- + +def _resolve_invoker() -> list[str]: + """Return the argv prefix used to invoke a sub-script. + + Prefer the ``cmind`` CLI so the dispatcher tees each stage's + output to ``~/.cmind/workspaces//logs/.log`` and + snapshots the inner git repo automatically. Fall back to a + direct python invocation when ``cmind`` is not on ``$PATH``. + """ + cmind = shutil.which("cmind") + if cmind: + return [cmind, "script"] + return [sys.executable] # script path appended by caller + + +def _script_argv(invoker: list[str], script_name: str) -> list[str]: + """Build the argv needed to invoke ``script_name`` via ``invoker``.""" + if Path(invoker[0]).stem == "cmind": + return [*invoker, script_name] + return [*invoker, str(_SCRIPTS_DIR / script_name)] + + +def _run_check(invoker: list[str], script_name: str) -> dict[str, Any]: + """Run a check_*.py script and parse its JSON stdout. + + The check scripts print exactly one JSON object on stdout when + invoked with ``--json``. We capture it without printing to the + parent terminal so the user is not flooded by 5 raw JSON blobs + during probing. ``--json`` is the unified contract across all + ``check_*.py`` scripts; ``check_skeleton.py`` accepts it as a + no-op for compatibility. + """ + argv = [*_script_argv(invoker, script_name), "--json"] + try: + proc = subprocess.run( + argv, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + check=False, + ) + except FileNotFoundError as exc: + return {"type": "error", "message": f"cannot invoke {argv[0]}: {exc}"} + + text = (proc.stdout or b"").decode("utf-8", errors="replace").strip() + if not text: + return { + "type": "error", + "message": f"{script_name} produced no output (exit {proc.returncode})", + } + try: + return json.loads(text) + except json.JSONDecodeError: + # Some checks may emit human-readable lines before the JSON + # object; take the last brace-balanced block. + last_obj = _extract_last_json_object(text) + if last_obj is not None: + return last_obj + return { + "type": "error", + "message": f"{script_name} returned non-JSON output", + } + + +def _extract_last_json_object(text: str) -> Optional[dict[str, Any]]: + """Best-effort: pull the last ``{...}`` block out of ``text``. + + Robust to unmatched ``}`` characters that may appear in surrounding + log lines (e.g. error messages quoting JSON fragments). When depth + would go negative, reset the parser state so a later, well-formed + object can still be captured. + """ + depth = 0 + start = -1 + last: Optional[str] = None + for i, ch in enumerate(text): + if ch == "{": + if depth == 0: + start = i + depth += 1 + elif ch == "}": + if depth == 0: + # Stray ``}`` outside any object — ignore and stay reset. + continue + depth -= 1 + if depth == 0 and start >= 0: + last = text[start : i + 1] + if last is None: + return None + try: + obj = json.loads(last) + return obj if isinstance(obj, dict) else None + except json.JSONDecodeError: + return None + + +def _run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + """Run a build_*/design_* script and stream its output live.""" + argv = [*_script_argv(invoker, script_name), *extra] + proc = subprocess.run(argv, check=False) + return proc.returncode + + +# --------------------------------------------------------------------------- +# Progress probing and decision logic. +# --------------------------------------------------------------------------- + +@dataclass +class StageState: + stage: Stage + type: str = "error" # init | update | warning | error + message: str = "" + done: bool = False + will_run: bool = False + reason: str = "" + raw: dict[str, Any] = field(default_factory=dict) + + +def probe(invoker: list[str]) -> list[StageState]: + """Run every check_*.py and return a parallel list of states.""" + states: list[StageState] = [] + for stage in STAGES: + result = _run_check(invoker, stage.check_script) + type_ = str(result.get("type", "error")) + states.append( + StageState( + stage=stage, + type=type_, + message=str(result.get("message", "")), + done=(type_ == "update"), + raw=result, + ) + ) + return states + + +def decide(states: list[StageState], force: bool) -> None: + """Mark each state's ``will_run`` / ``reason`` in place. + + Rule: any stage with ``type != "update"`` runs. Once any + stage runs, *all* downstream stages run too (cascade), so + derived artifacts never get out of sync with regenerated + upstream ones. ``--force`` flips every stage to ``will_run``. + """ + cascade = False + for state in states: + if force: + state.will_run = True + state.reason = "forced" + continue + if cascade: + state.will_run = True + state.reason = "upstream rebuilt" + continue + if state.type == "update": + state.will_run = False + state.reason = "up-to-date" + else: + state.will_run = True + state.reason = f"type={state.type}" + cascade = True + + +# --------------------------------------------------------------------------- +# Pretty-printing. +# --------------------------------------------------------------------------- + +_GLYPH = {"update": "✓", "init": "·", "warning": "!", "error": "✗"} + + +def _format_table(states: list[StageState]) -> str: + rows = ["Stage Type Done Action"] + rows.append("-" * 50) + for s in states: + glyph = _GLYPH.get(s.type, "?") + action = "run" if s.will_run else "skip" + rows.append( + f"{s.stage.name:<14} {glyph} {s.type:<7} " + f"{'yes' if s.done else 'no ':<3} {action}" + ) + return "\n".join(rows) + + +def _print_probe_summary(states: list[StageState]) -> None: + done = sum(1 for s in states if s.done) + total = len(states) + first_pending = next((s.stage.name for s in states if not s.done), None) + print(f"Planning progress: {done}/{total} stages complete.") + if first_pending: + print(f"Next pending stage: {first_pending}") + else: + print("All stages are up-to-date.") + print() + print(_format_table(states)) + + +def _emit_check_only_json(states: list[StageState]) -> None: + done = sum(1 for s in states if s.done) + total = len(states) + next_pending = next((s.stage.name for s in states if not s.done), None) + payload = { + "total": total, + "done": done, + "next": next_pending, + "stages": [ + { + "name": s.stage.name, + "type": s.type, + "message": s.message, + "done": s.done, + } + for s in states + ], + } + print(json.dumps(payload, indent=2)) + + +# --------------------------------------------------------------------------- +# Build-args assembly. +# --------------------------------------------------------------------------- + +def _build_args_for(stage: Stage, args: argparse.Namespace) -> list[str]: + """Collect CLI args to forward to ``stage.build_script``.""" + extra: list[str] = [] + if stage.max_iter_flag is not None: + value = getattr(args, f"max_iter_{stage.name}", None) + if value is not None: + extra.extend([stage.max_iter_flag, str(value)]) + if args.verbose: + extra.append("--verbose") + if args.no_trajectory: + extra.append("--no-trajectory") + return extra + + +# --------------------------------------------------------------------------- +# Entry point. +# --------------------------------------------------------------------------- + +def _parse_args(argv: Optional[list[str]] = None) -> argparse.Namespace: + p = argparse.ArgumentParser( + prog="plan.py", + description=( + "Run the full RPG planning pipeline (skeleton → data_flow → " + "base_classes → interfaces → tasks) with automatic resume." + ), + ) + p.add_argument( + "--check-only", + action="store_true", + help="Probe every stage and print progress, then exit. No build runs.", + ) + p.add_argument( + "--json", + action="store_true", + help="With --check-only, emit a machine-readable JSON progress report.", + ) + p.add_argument( + "--force", + action="store_true", + help="Ignore current progress and rebuild every stage from scratch.", + ) + p.add_argument( + "--dry-run", + action="store_true", + help="Print the commands that would run without executing them.", + ) + p.add_argument( + "--verbose", + action="store_true", + help="Forward --verbose to every sub-script.", + ) + p.add_argument( + "--no-trajectory", + action="store_true", + help="Forward --no-trajectory to every sub-script that supports it.", + ) + # Per-stage iteration overrides (only the four stages that take one). + for stage in STAGES: + if stage.max_iter_flag is None: + continue + p.add_argument( + f"--max-iter-{stage.name.replace('_', '-')}", + dest=f"max_iter_{stage.name}", + type=int, + default=None, + metavar="N", + help=f"Override iteration count for the '{stage.name}' stage.", + ) + return p.parse_args(argv) + + +def _install_sigint_handler() -> None: + def _handle(signum: int, frame: Any) -> None: # noqa: ARG001 + print("\n[plan] interrupted — rerun `cmind script plan.py` to resume.") + sys.exit(130) + + signal.signal(signal.SIGINT, _handle) + + +def main(argv: Optional[list[str]] = None) -> int: + args = _parse_args(argv) + _install_sigint_handler() + invoker = _resolve_invoker() + + # --- Phase 1: probe ---------------------------------------------------- + states = probe(invoker) + decide(states, force=args.force) + + if args.check_only: + if args.json: + _emit_check_only_json(states) + else: + _print_probe_summary(states) + return 0 + + # --- Phase 1b: prerequisite check -------------------------------------- + # If the very first stage cannot even start (its input is missing or + # invalid), abort cleanly so the user gets a helpful pointer instead + # of a confusing failure from the build script itself. ``--dry-run`` + # bypasses this so users can preview commands without an initialised + # workspace. + head = states[0] + if head.type == "error" and not args.dry_run: + print( + f"Cannot start the planning pipeline: {head.message}", + file=sys.stderr, + ) + print( + "Run `/cmind.feature_construct` first to produce " + "`feature_tree.json`, then re-run `/cmind.plan`.", + file=sys.stderr, + ) + return 2 + + # --- Phase 2: short-circuit when nothing to do ------------------------- + runnable = [s for s in states if s.will_run] + if not runnable: + print("All 5 planning stages are already complete — nothing to do.") + print("Use `cmind script plan.py --force` to rebuild from scratch.") + return 0 + + # --- Phase 3: announce plan ------------------------------------------- + print(f"Planning pipeline: {len(runnable)} of {len(states)} stages to run.") + print(_format_table(states)) + print() + + if args.dry_run: + for s in runnable: + cmd = _script_argv(invoker, s.stage.build_script) + cmd += _build_args_for(s.stage, args) + print("DRY-RUN ▸", " ".join(cmd)) + for post in POST_STEPS: + print("DRY-RUN ▸", " ".join(_script_argv(invoker, post))) + return 0 + + # --- Phase 4: execute -------------------------------------------------- + started = time.monotonic() + for s in states: + if not s.will_run: + print(f"⏭ {s.stage.name:<14} skip ({s.reason})") + continue + + stage_started = time.monotonic() + print(f"▶ {s.stage.name:<14} running {s.stage.build_script} ...") + build_extra = _build_args_for(s.stage, args) + rc = _run_stage(invoker, s.stage.build_script, build_extra) + if rc != 0: + _print_failure_hint(invoker, s.stage, rc, phase="build") + return rc + + # Re-run the check to confirm the artifact came out valid. Parse + # its JSON quietly; surface details only when the verification + # fails, otherwise the user would see a JSON dump after every + # stage. + # + # ``update`` -> stage is fully valid; continue. + # ``warning`` -> artefact is usable but a soft inconsistency was + # detected (e.g. tasks.json having auxiliary tasks + # without a 1:1 interface mapping). Print the + # message and continue; do not fail the pipeline. + # ``init`` / ``error`` -> artefact is missing or unusable; fail. + verify = _run_check(invoker, s.stage.check_script) + verify_type = verify.get("type", "error") + if verify_type == "warning": + print( + f" warning: {verify.get('message', 'no message')}" + f" (continuing)", + file=sys.stderr, + ) + elif verify_type != "update": + print( + f" verification failed: {verify_type} — " + f"{verify.get('message', 'no message')}", + file=sys.stderr, + ) + for err in verify.get("validation_errors", [])[:5]: + print(f" - {err}", file=sys.stderr) + _print_failure_hint(invoker, s.stage, 1, phase="check") + return 1 + + elapsed = time.monotonic() - stage_started + print(f"✓ {s.stage.name:<14} done in {elapsed:.1f}s") + + # --- Phase 5: post-pipeline helpers ----------------------------------- + print() + print("Running post-pipeline helpers ...") + for post in POST_STEPS: + print(f"▶ {post}") + rc = _run_stage(invoker, post, []) + if rc != 0: + print(f" warning: {post} exited with {rc} (continuing)") + + total_elapsed = time.monotonic() - started + print() + print(f"Plan complete in {total_elapsed:.1f}s.") + print("Next: `/cmind.code_gen` to generate source code.") + print("Graph: see the 'Writing visualization to:' line above for the generated HTML path.") + return 0 + + +def _print_failure_hint( + invoker: list[str], + stage: Stage, + rc: int, + *, + phase: str, +) -> None: + """Print recovery hints to stderr after a stage fails. + + ``phase`` is ``"build"`` or ``"check"``; the debug command points at + the script that actually failed so the user can reproduce. Commands + are rendered using the current *invoker* so the hint stays correct + whether ``cmind`` is on ``$PATH`` (``cmind script ``) or the + Python fallback is in use (``python ``). + """ + debug_script = stage.build_script if phase == "build" else stage.check_script + + def _fmt(name: str, *extra: str) -> str: + # Render the command using the basename of the invoker so + # users see e.g. ``cmind script ...`` or ``python ...`` rather + # than the resolved absolute path that subprocess actually uses. + argv = _script_argv(invoker, name) + display = [Path(argv[0]).name, *argv[1:], *extra] + return " ".join(display) + + print(file=sys.stderr) + print(f"✗ {stage.name} {phase} failed (exit {rc})", file=sys.stderr) + print(f" Resume : {_fmt('plan.py')}", file=sys.stderr) + print(f" Debug : {_fmt(debug_script, '--verbose')}", file=sys.stderr) + print(f" Status : {_fmt('plan.py', '--check-only')}", file=sys.stderr) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/CoderMind/templates/commands/build_data_flow.md b/CoderMind/templates/commands/build_data_flow.md index 7f2b34b..0c7b55c 100644 --- a/CoderMind/templates/commands/build_data_flow.md +++ b/CoderMind/templates/commands/build_data_flow.md @@ -24,7 +24,7 @@ Unless it is explicitly empty, you may assume it is always available as `$ARGUME Run the script `cmind script check_data_flow.py` to verify the current state. -1. Inspect the `state` field in the output: +1. Inspect the `type` field in the output: * `error` → Display the error message and stop. Instruct user to fix the error or regenerate. Terminate this command. * `init` → Proceed to Step 2. diff --git a/CoderMind/templates/commands/design_base_classes.md b/CoderMind/templates/commands/design_base_classes.md index 1f775dc..db15d2d 100644 --- a/CoderMind/templates/commands/design_base_classes.md +++ b/CoderMind/templates/commands/design_base_classes.md @@ -22,7 +22,7 @@ Unless it is explicitly empty, you may assume it is always available as `$ARGUME Run the script `cmind script check_base_classes.py` to verify the current state. -1. Inspect the `state` field in the output: +1. Inspect the `type` field in the output: * `error` → Display the error message and stop. Instruct user to fix the error or regenerate. Terminate this command. * `init` → Proceed to Step 2. diff --git a/CoderMind/templates/commands/feature_construct.md b/CoderMind/templates/commands/feature_construct.md new file mode 100644 index 0000000..a131f33 --- /dev/null +++ b/CoderMind/templates/commands/feature_construct.md @@ -0,0 +1,270 @@ +--- +description: Build the complete Phase 1 feature tree from requirements in one step, with automatic resume on failure +name: cmind.feature_construct +--- + +## User Input + +```text +$ARGUMENTS +``` + +`/cmind.feature_construct` is the **default entry point** for Phase 1. +It orchestrates three stages and produces three artefacts in the +CoderMind data store. + +| Stage | Artefact | +| --- | --- | +| `feature_spec` | `feature_spec.json` | +| `feature_build` | `feature_build.json` | +| `feature_refactor` | `feature_tree.json` | + +Each stage is a standalone Python helper that drives an LLM through +`LLMClient.call_structured(...)` with a Pydantic-validated schema — no +intermediate Markdown artefacts are produced. + +## Argument Parsing + +Supported facade options (forward as-is to the orchestrator): + +- `--check-only` — probe stage status and exit. +- `--json` — with `--check-only`, emit JSON. +- `--force` — rebuild every stage from scratch. +- `--dry-run` — list commands without executing. +- `--verbose` — DEBUG logging. +- `--no-trajectory` — skip trajectory recording. +- `--max-iter-refactor N` — override `feature_refactor.py --max-iterations`. +- `--review-threshold N` — forward to `feature_build.py`. +- `--review-max-iterations N` — forward to `feature_build.py`. + +If options and requirement text are both present, split them with `--`: + +```text +/cmind.feature_construct --review-threshold 99 -- Build a CLI for managing containers +``` + +If no options are present, treat the whole argument string as requirement +text. Requirement text is **not** forwarded to the orchestrator; instead, +when the spec stage runs the slash command passes it via +`--input-text ""` to `feature_spec.py`. + +### Check-only short-circuits + +If the user invoked `/cmind.feature_construct --check-only` (optionally +with `--json`), run exactly: + +```bash +cmind script feature_construct.py --check-only [--json] +``` + +Show the orchestrator output verbatim and stop. Do not inspect `docs/`, +do not ask for requirements, do not run any stage. + +## Workflow + +> [!WARNING] +> A full Phase 1 run typically takes a few minutes (todo-list scale) +> up to ~30 minutes for larger projects. Do not interrupt it; if you +> must, re-run this command and it will resume from the first +> incomplete stage. + +### Step 1: Probe progress + +```bash +cmind script feature_construct.py --check-only --json +``` + +Parse: + +- `total` — always 3. +- `done` — count of stages whose `type` is `update` (artefact present and valid). +- `next` — first incomplete stage name, or `null` if all done. +- `stages[*]` — per-stage `status`, `message`, `will_run`, `reason`. + +### Step 2: Determine requirement source + +Only relevant when `feature_spec` is incomplete or the user chose to +overwrite. Priority: + +1. **Requirement text after the command** — capture into a variable to + pass via `--input-text` when the spec stage runs. +2. **`docs/*.md` files** — auto-detected by `feature_spec.py`; no flag + needed. +3. **Inline prompt** — if neither exists, ask the user once in this + chat for requirements, capture the text, then continue. Do not ask + them to rerun the slash command. + +If the user supplied new requirement text while any Phase 1 artefact +already exists and `--force` was **not** supplied, ask one real decision +before changing anything: + +```text +Phase 1 artefacts already exist, and new requirements were provided. + +What would you like to do? + [R] Restart — regenerate Phase 1 from the new requirements (--force) + [E] Exit — keep existing artefacts unchanged +``` + +`R` → continue with `--force`. `E` → terminate after Step 7. + +### Step 3: Choose execution mode + +**Case A — all three stages already complete (`done == total`)** + +```text +All Phase 1 stages are already complete: + ✓ feature_spec + ✓ feature_build + ✓ feature_refactor + +What would you like to do? + [X] Expand features — suggest expansion directions, expand selected ones, then rerun refactor + [O] Overwrite — regenerate Phase 1 from scratch (--force) + [E] Exit — keep existing artefacts and proceed to /cmind.plan +``` + +- `X` → go to **Step 6** (expansion flow). +- `O` → ensure a requirement source exists per Step 2, then rerun the + orchestrator with `--force`. +- `E` → show the completion guidance in Step 7 and stop. + +**Case B — fresh or incomplete workspace (`done < total`)** + +If `done == 0`, inform the user and start immediately: + +```text +Starting Phase 1 feature construction (3 stages). This may take a while. +``` + +If `0 < done < total`, resume from `next` automatically. Do not ask for +stage parameters unless the user explicitly requested an overwrite in +Step 2. + +### Step 4: Run the orchestrator + +The orchestrator (`feature_construct.py`) runs each stage in order +(`feature_spec` → `feature_build` → `feature_refactor`), skipping +already-complete stages, and validates each artefact after the stage +finishes. + +If requirement text was supplied (Step 2 case 1 or 3), the slash command +must invoke `feature_spec.py` directly for the spec stage so that +`--input-text` can be passed — then resume the rest of the pipeline: + +```bash +# Spec stage with inline text +cmind script feature_spec.py --input-text "" [--force] [other facade flags] + +# Remaining stages (orchestrator skips feature_spec since it now exists) +cmind script feature_construct.py [facade flags] +``` + +Otherwise (auto-detected `docs/*.md` or pure resume) just run: + +```bash +cmind script feature_construct.py [facade flags] +``` + +If restart was chosen in Step 2 or 3 (or `--force` was supplied), append +`--force` to the orchestrator command. + +If `--dry-run` was supplied, stream the dry-run command list and stop +without modifying artefacts. + +### Step 5: Stream output and handle failures + +Stream stdout/stderr verbatim. The orchestrator prints one progress +line per stage and validates each generated artefact. + +If a stage exits non-zero, surface the orchestrator's recovery hints. +Typical recovery commands: + +```bash +cmind script feature_construct.py --check-only # see which stage failed +cmind script feature_construct.py # resume from failed stage +cmind script feature_spec.py --verbose # debug spec stage +cmind script feature_build.py --verbose # debug build stage +cmind script feature_refactor.py --log-level DEBUG # debug refactor stage +``` + +For spec-stage failures, the most common cause is an LLM call that +failed to produce a schema-valid JSON after retries. Re-run with +`--verbose` to surface the trajectory file location in the script's +own log output; do not attempt to locate it manually. + +### Step 6: Optional feature expansion + +Offered both after normal completion and from Case A in Step 3: + +```text +Would you like to expand the feature tree beyond the current specification? + [Y] Yes — suggest expansion directions + [N] No — finish here +``` + +If `Y`, perform one or more *expansion rounds*. Each round is a closed +cycle: + +1. **Suggest directions:** + + ```bash + cmind script feature_build.py --mode suggest-directions + ``` + + Add `--verbose` / `--no-trajectory` only if the user supplied those + facade options. + +2. Parse the JSON output and show directions as a numbered Markdown + table. + +3. Ask for comma-separated direction numbers (or `N` to finish this + round). Normalise numeric input before passing it on. + +4. **Run directed expansion:** + + ```bash + cmind script feature_build.py --mode step2 --direction "" + ``` + + Forward `--review-max-iterations`, `--verbose`, `--no-trajectory` if + supplied. Do not forward `--review-threshold` to `step2`. + +5. **Run refactor immediately** so `feature_tree.json` reflects the + expanded `feature_build.json` and the user can inspect the actual + tree before deciding on further rounds: + + ```bash + cmind script feature_refactor.py + ``` + + Forward `--max-iterations ` when `--max-iter-refactor N` was + supplied, `--log-level DEBUG` when `--verbose` was supplied, and + `--no-trajectory` when supplied. + +6. **Ask whether to start another round:** + + ```text + Expansion round complete. The feature tree has been refactored. + + Would you like another expansion round? + [Y] Yes — suggest more directions + [N] No — finish here + ``` + + If `Y`, repeat from step 1. If `N`, continue to Step 7. + +### Step 7: Completion message + +On success or when the all-complete case exits without changes: + +```text +Feature construct complete. + +Next step: + /cmind.plan — build the Repository Planning Graph (RPG) + +Optional refinements: + Rerun this command and choose [X] — expand the feature tree further + /cmind.feature_edit — adjust the final tree if it is unsatisfactory +``` diff --git a/CoderMind/templates/commands/feature_edit.md b/CoderMind/templates/commands/feature_edit.md index 7ac3971..87ee702 100644 --- a/CoderMind/templates/commands/feature_edit.md +++ b/CoderMind/templates/commands/feature_edit.md @@ -51,17 +51,17 @@ Inspect the `type` field in the output: - `file_not_found`: ```markdown - > **Error**: The file `.cmind/data/feature_tree.json` does not exist. + > **Error**: No feature tree found. > - > Please run `/cmind.feature_refactor` first to generate the feature tree. + > Please run `/cmind.feature_construct` first to build the feature tree. ``` - `field_empty` or `field_missing`: ```markdown - > **Error**: The file exists but the `components` field is missing or empty. + > **Error**: The feature tree exists but is incomplete. > - > Please run `/cmind.feature_refactor` to generate a valid feature tree structure. + > Please run `/cmind.feature_construct` to rebuild a valid feature tree. ``` 2. **If `type` is `"ready"`**: Proceed to Step 2. @@ -133,5 +133,5 @@ After the script completes: If you are satisfied with the feature tree and ready to proceed to the **next step**, run: ```text - /cmind.build_skeleton + /cmind.plan ``` diff --git a/CoderMind/templates/commands/feature_spec.md b/CoderMind/templates/commands/feature_spec.md index 853016c..f68d63d 100644 --- a/CoderMind/templates/commands/feature_spec.md +++ b/CoderMind/templates/commands/feature_spec.md @@ -1,5 +1,5 @@ --- -description: Create structured feature specifications from user input or documentation files +description: Extract a structured feature specification from requirement docs or inline text name: cmind.feature_spec --- @@ -9,1000 +9,103 @@ name: cmind.feature_spec $ARGUMENTS ``` -Text provided after `/cmind.feature_spec` will be used as the feature description. If empty, the agent will automatically detect and use files in the `docs/` directory. +This command is a thin wrapper around `cmind script feature_spec.py`. +For the recommended end-to-end Phase 1 flow (spec → build → refactor), +use `/cmind.feature_construct` instead. The granular command remains +available for debugging and single-stage reruns. -## Capabilities +## Argument Parsing -- **Dual Input Modes**: Accepts either directly provided user descriptions or auto-detects `docs/*.md` files -- **Evidence-Based Extraction**: All feature specifications are traceable to source text with line numbers -- **Sub-Document Architecture**: Generates modular Markdown files for improved maintainability -- **Hierarchical Feature Tree**: Supports unlimited nesting levels using numeric outline format -- **Unified Processing Flow**: Both input modes converge to the same downstream workflow +Supported facade options (forward as-is to the helper): -## Output Directory Structure +- `--check-only` — probe output state without invoking the LLM. +- `--json` — with `--check-only`, emit JSON. +- `--force` — overwrite an existing valid `feature_spec.json`. +- `--verbose` — DEBUG logging. +- `--no-trajectory` — skip trajectory recording. -```text -.cmind/data/feature_spec/ -├── evidence/ # Step 2 output -│ ├── user_input.md # (from user input) or -│ ├── 01_project_charter.md # (from docs/) -│ ├── 02_requirements_specification.md -│ └── ... -├── feature_spec.md # Step 3 output (Meta + Background + NFR) -└── features/ # Step 4 output (Feature Tree) - ├── FT-001.md - ├── FT-002.md - └── ... -``` - -## Workflow - -**Working Directory**: All relative paths are based on the project root. - -### Step 1: Determine Input Source - -#### 1.1 Check User Input - -If `$ARGUMENTS` is **not empty**: - -```markdown -## ✓ User Input Detected - -- **Mode**: User-provided description -- **Input Length**: characters -- **Output Directory**: .cmind/data/feature_spec/ - -Processing user-provided feature description. -``` - -→ Proceed to **Step 2A: Process User Input** - -#### 1.2 Check docs/ Directory - -If `$ARGUMENTS` is **empty**, check for documentation files: - -1. List all `.md` files in the `./docs` directory. - -2. If files are found: - - ```markdown - ## Documentation Detected - - No feature description provided. Found the following Markdown files in `docs/` directory: - - - 01_project_charter.md - - 02_requirements_specification.md - - ... - - **Use these documents to create feature specification? (Y/N)** - ``` - - **Wait for user response:** - - - **If "Y"**: → Proceed to **Step 2B: Process Documentation Files** - - **If "N"**: → Display message and terminate - -#### 1.3 No Available Input - -If neither user input nor documentation files are available: - -```markdown -## No Available Input Source - -The feature specification process requires one of the following: - -1. **Provide a feature description as input:** - - `/cmind.feature_spec ` - -2. **Place documentation files in the `docs/` directory:** - - Add requirement or design documents to the `docs/` directory, then run: - - `/cmind.feature_spec` -``` - -→ **Terminate agent execution** - ---- - -### Step 2A: Process User Input → Evidence - -Convert user input to Evidence file format. - -#### 2A.1: Parse User Input - -1. **Analyze user input** to identify: - - Background information (context, goals, scope, design philosophy, etc.) - - Functional requirements (features, behaviors, operations, interface contracts, etc.) - - Non-functional requirements (constraints, performance, security, assumptions, risks, etc.) - -2. **Assign sequential IDs** in format `user_input-{category}-{sequence}`: - - Background: `user_input-BG-001`, `user_input-BG-002`, ... - - FR: `user_input-FR-001`, `user_input-FR-002`, ... - - NFR: `user_input-NFR-001`, `user_input-NFR-002`, ... - -3. **Virtual line numbers**: User input is not a file, use virtual line numbers (split user input text by lines, counting from line 1) - -#### 2A.2: Generate Evidence File - -Create `.cmind/data/feature_spec/evidence/user_input.md`: - -```markdown -# Evidence: user_input.md - -## Background / Overview - -### [user_input-BG-001] L1-L{end} -**Parent Title**: - - -> {Background content extracted from user input} - -## FR (Functional Requirement) - -### [user_input-FR-001] L{start}-L{end} -**Parent Title**: - - -> {Functional requirement extracted from user input} - -### [user_input-FR-002] L{start}-L{end} -**Parent Title**: - - -> {Another functional requirement} - -## NFR (Non-Functional Requirement) - -### [user_input-NFR-001] L{start}-L{end} -**Parent Title**: - - -> {Non-functional requirement extracted from user input} -``` - -#### 2A.3: Display Progress - -```markdown -## ✓ User Input Processing Complete - -- **Evidence File**: .cmind/data/feature_spec/evidence/user_input.md -- **Background Entries**: -- **FR Entries**: -- **NFR Entries**: -- **Total Evidence**: -``` - -→ Proceed to **Step 3: Generate Main File** - ---- - -### Step 2B: Process Documentation Files → Evidence - -Extract Evidence from each documentation file. - -> **Important: Process One Document at a Time** -> -> Process documents **one by one**, completing each before moving to the next. -> -> - Avoid processing all documents at once to prevent context overflow and quality degradation -> - Display progress after completing each document, then continue to the next - -#### 2B.1: Process Each Document - -For each `.md` file in `docs/`, create a corresponding evidence file. - -**ID Format:** +If options and requirement text are both present, split them with `--`: ```text -{file_prefix}-{category}-{sequence} - │ │ │ - │ │ └── 001, 002, 003... (numbered independently per category) - │ └────── BG / FR / NFR - └──────────── File name abbreviation or full name +/cmind.feature_spec --force -- Build a CLI for managing Docker containers ``` -**File Prefix Generation Rules:** - -1. Remove numeric prefix and extension (e.g., `01_project_charter.md` → `project_charter`) -2. Choose scheme based on length: - - **≤ 20 characters**: Use full name as prefix - - **> 20 characters**: Use abbreviation (capitalize first letter of each word) -3. **Handle duplicate prefixes**: If generated prefix matches an existing prefix, append sequence number `_2`, `_3`... - -**Examples:** - -| Filename | File Prefix | BG ID | FR ID | NFR ID | -| -------- | ----------- | ----- | ----- | ------ | -| `01_project_charter.md` | `project_charter` | `project_charter-BG-001` | `project_charter-FR-001` | `project_charter-NFR-001` | -| `02_requirements_specification.md` | `RS` | `RS-BG-001` | `RS-FR-001` | `RS-NFR-001` | -| `05_interface_data_contract.md` | `IDC` | `IDC-BG-001` | `IDC-FR-001` | `IDC-NFR-001` | - -**Abbreviation Generation Rules:** - -1. Split by `_` into words -2. Take first letter of each word -3. Convert to uppercase - -**Duplicate Prefix Handling Examples:** - -| Filename | Generated Prefix | Conflict Handling | Final Prefix | -| -------- | ---------------- | ----------------- | ------------ | -| `docs/api_reference.md` | `api_reference` | No conflict | `api_reference` | -| `docs/sub/api_reference.md` | `api_reference` | Already exists | `api_reference_2` | -| `docs/other/api_reference.md` | `api_reference` | Already exists | `api_reference_3` | - -**Special Cases:** - -- User input always uses prefix `user_input` - -#### 2B.2: Generate File ID Mapping Table +If no options are present, treat the whole argument string as requirement +text. -Before processing documents, scan all documents and generate the ID prefix mapping table: - -```markdown -## File ID Mapping Table - -| # | File Path | File Prefix | -|---|-----------|-------------| -| 1 | docs/01_project_charter.md | `project_charter` | -| 2 | docs/02_requirements_specification.md | `RS` | -| 3 | docs/sub/api_reference.md | `api_reference` | -| 4 | docs/other/api_reference.md | `api_reference_2` | -| ... | ... | ... | - -**Total Documents**: -``` - -This mapping table will be used to trace Evidence sources later. - -#### 2B.3: Extract Evidence from Each Document - -For each document, read the complete content and extract evidence: - -1. **Identify section headers** (e.g., "3.1 DAG Authoring and Definition") - as Parent Title -2. **Record line numbers** for traceability -3. **Preserve original text** verbatim copy -4. **Categorize**: Background / Overview, FR, NFR - -**Field Source Description:** - -- **Parent Title**: Section header from source (for tracing source location) - -**Evidence Category Quick Reference:** - -| Category | Definition | Typical Content | -| -------- | ---------- | --------------- | -| **BG** | System context, goals, design philosophy | Project background, system boundaries, design principles, terminology definitions | -| **FR** | What the system **does** | Module responsibilities, APIs, user interactions, data flows, interface contracts | -| **NFR** | System's **quality attributes and constraints** | Performance, security, scalability, assumptions, constraints, technical risks | - -**FR Extraction Granularity Requirements:** - -- Extract each independent feature point as one Evidence -- Describe "what" not "how" -- Extract each table row feature independently - -**Content NOT to Extract:** -Project risks, external risks, operational processes, inter-document references, open questions (undecided items) - -#### 2B.4: **Generate Evidence Files** - -For each document, create `.cmind/data/feature_spec/evidence/{document_name}.md`: - -```markdown -# Evidence: {document_name}.md - -## Background / Overview - -### [{prefix}-BG-001] L{start}-L{end} -**Parent Title**: - - -> {Original excerpt} - -### [{prefix}-BG-002] L{start}-L{end} -**Parent Title**: {Section header} - -> {Original excerpt} - -## FR (Functional Requirement) - -### [{prefix}-FR-001] L{start}-L{end} -**Parent Title**: {Section header, e.g., 3.1 DAG Authoring & Definition} - -> {Original excerpt} - -### [{prefix}-FR-002] L{start}-L{end} -**Parent Title**: {Section header} - -> {Original excerpt} - -## NFR (Non-Functional Requirement) - -### [{prefix}-NFR-001] L{start}-L{end} -**Parent Title**: {Section header, e.g., 4.1 Performance} - -> {Original excerpt} -``` - -#### 2B.5: Display Progress - -After processing each document: - -```markdown -## Document Processing Progress - -### ✓ .md -- **Evidence File**: .cmind/data/feature_spec/evidence/.md -- **Background**: | **FR**: | **NFR**: - -### ✓ .md -- **Evidence File**: .cmind/data/feature_spec/evidence/.md -- **Background**: | **FR**: | **NFR**: - -... - ---- - -**Total Documents**: -**Total Evidence Entries**: -``` - -→ Proceed to **Step 3: Generate Main File** - ---- - -### Step 3: Generate Main File - -Generate the main feature specification file containing Meta, Background, and NFR. - -#### 3.1: Read All Evidence Files - -Load all `.md` files from `.cmind/data/feature_spec/evidence/`. - -#### 3.2: Determine Repository Information - -Derive from evidence: - -- **Repository Name**: Concise name (1-3 words, kebab-case) -- **Repository Purpose**: 1-2 sentences describing the core objective -- **Project Types**: Identify which user-facing surfaces the project exposes. - Output a list of UPPERCASE tokens drawn from this 8-item whitelist. - At least one token is required; multiple are allowed when the project - exposes more than one surface. - - | Token | Use it for | - | --- | --- | - | `WEB` | HTTP endpoints that render HTML pages for browsers | - | `API` | JSON / GraphQL endpoints, no HTML rendering | - | `SERVICE` | long-running daemon, worker, bot, scheduler | - | `PIPELINE` | batch data processing (ETL, Airflow DAG, Spark job, ML training) with a clear start and end | - | `CLI` | command-line entry point with subcommands or arguments | - | `GUI` | desktop window with widgets | - | `GAME` | interactive real-time application with a rendering loop | - | `LIBRARY` | importable package, no end-user interface | - - Examples: `[WEB]`, `[WEB, CLI]`, `[API, SERVICE]`, `[PIPELINE, CLI]`, - `[GAME, LIBRARY]`. - -- **Project Notes**: A short paragraph (≤ 500 chars) capturing details the - whitelist cannot express — framework choices, special domain - requirements, anything unique. Examples: - - - `REST API only, no HTML pages — clients are mobile apps` - - `Discord bot using discord.py, runs as long-lived daemon` - - `Textual TUI with arrow-key navigation` - - `Airflow DAG, scheduled daily, reads from S3` - -#### 3.3: Merge Background Entries - -1. Group Background evidence from all files semantically -2. Generate Description for each group -3. Preserve all original evidence references - -#### 3.4: Merge NFR Entries - -1. Group NFR evidence from all files by category (Performance, Security, Scalability, etc.) -2. Generate Description for each group -3. Preserve all original evidence references - -#### 3.5: Generate feature_spec.md - -Create `.cmind/data/feature_spec/feature_spec.md`: - -```markdown -# Feature Specification - -## Meta - -- **Repository Name**: {repository_name} -- **Repository Purpose**: {repository_purpose} -- **Project Types**: {comma-separated UPPERCASE tokens, e.g. WEB, CLI} -- **Project Notes**: {short paragraph ≤ 500 chars} -- **Generated At**: {YYYY-MM-DD} -- **Source Documents**: {comma-separated list} - -## Background - -### BG-001: {Name} -- **Description**: {Description} -- **Evidence**: - - {ID} | {source} L{line_number} - -### BG-002: {Name} -- **Description**: {Description} -- **Evidence**: - - {ID} | {source} L{line_number} - -## NFR - -### NFR-001: {Name} -- **Description**: {Description} -- **Evidence**: - - {ID} | {source} L{line_number} - -### NFR-002: {Name} -- **Description**: {Description} -- **Evidence**: - - {ID} | {source} L{line_number} -``` - -#### 3.6: Display Progress - -```markdown -## ✓ Main File Generation Complete - -- **Output File**: .cmind/data/feature_spec/feature_spec.md -- **Repository Name**: {name} -- **Background Entries**: -- **NFR Entries**: -``` - -→ Proceed to **Step 4: Generate Feature Domain Files** - ---- - -### Step 4: Generate Feature Domain Files - -Identify feature domains based on FR Evidence and generate detailed feature files. - -#### 4.1: Identify Feature Domains - -Read original excerpts from all FR evidence and cluster by feature semantics: - -**Inference Steps:** - -1. **Read original excerpts** - Understand the `> {original excerpt}` content and semantics of each FR -2. **Semantic clustering** - Group functionally related FRs with similar responsibilities into the same feature domain -3. **Name feature domains** - Choose a descriptive name for each cluster - -**Clustering Principles:** - -- **Functional relevance** - Features within the same domain should be logically closely related -- **Responsibility cohesion** - Features within the same domain should serve the same or similar goals -- **Clear boundaries** - Different domains should have clear responsibility boundaries - -**Feature Domain Naming Convention:** - -- Use 2-4 English words -- Title Case (capitalize first letter of each word) -- Avoid abbreviations, maintain readability -- Reflect the core responsibility of the domain - -#### 4.2: Build Feature Tree Hierarchy - -**Build hierarchical structure based on feature semantics from original excerpts:** - -1. **Semantic analysis**: Read original excerpts of each FR, understand its functional responsibility -2. **Hierarchy inference**: Determine hierarchy level based on abstraction level and containment relationships - - High level: General feature descriptions → as parent nodes - - Low level: Specific feature points → as leaf nodes -3. **Establish relationships**: Group specific features under related abstract features -4. **Reference information**: Chapter structure from Parent Title can serve as reference for hierarchy division, but final decisions are based on feature semantics - -#### 4.3: Generate Feature Files - -For each domain, create `.cmind/data/feature_spec/features/FT-{NNN}.md`: - -```markdown -# FT-001: {Domain Name} -- **Description**: {Domain description} - -## FT-001-001: {Sub-domain Name} -- **Description**: {Sub-domain description} - -### FT-001-001-001: {Feature Name} -- **Description**: {Feature description} -- **Evidence**: - - {ID} | {source} L{line_number} - -### FT-001-001-002: {Feature Name} -- **Description**: {Feature description} -- **Evidence**: - - {ID} | {source} L{line_number} - -## FT-001-002: {Sub-domain Name} -- **Description**: {Sub-domain description} - -### FT-001-002-001: {Feature Name} -- **Description**: {Feature description} -- **Evidence**: - - {ID} | {source} L{line_number} - -#### FT-001-002-001-001: {Sub-feature Name} -- **Description**: {Sub-feature description} -- **Evidence**: - - {ID} | {source} L{line_number} -``` - -**Hierarchical Structure (determined by heading level):** - -| Heading | Level | Description | -| ------- | ----- | ----------- | -| `#` | Domain | FT-001 | -| `##` | Sub-domain | FT-001-001 | -| `###` | Feature | FT-001-001-001 | -| `####` | Sub-feature | FT-001-001-001-001 | -| `#####` | Deeper level | FT-001-001-001-001-001 | -| `######` | Deepest level | FT-001-001-001-001-001-001 | - -**Rules:** - -- Heading level determines hierarchy depth -- ID follows immediately after heading marker, format is `FT-XXX-XXX-...` -- Number of `-` separated segments in ID = heading level -- Supports up to 6 levels (Markdown heading limit) -- **Hierarchy depth is not fixed** - Determined by actual project needs, not required to use all levels -- **Leaf nodes can be at any level** - For simple projects, leaf nodes may be at level 2 or 3; complex projects may reach level 5 or 6 - -**Node Types and Evidence:** - -| Node Type | Evidence | Description | -| --- | --- | --- | -| **Document nodes** | **Required** | Feature descriptions from documents, regardless of hierarchy level | -| **Organization nodes** | None | Abstract layers created for organizational structure, used to group child nodes | - -**Judgment Principles:** - -- **Be faithful to documents** - The feature tree should reflect the granularity described in documents -- **Prefer document leaf nodes** - If document descriptions are already precise to specific features (meeting leaf node conditions), adopt them directly -- **Document nodes are leaf nodes** - Feature descriptions from documents serve as leaf nodes (unless the document also describes sub-features) -- **Organization nodes have no Evidence** - Abstract layers inferred for grouping have no direct document source - -**Leaf Node Conditions:** - -1. **Precise to specific feature** - Describes an independent, identifiable feature point -2. **Not implementation details** - Describes "what" not "how" -3. **Has Evidence support** - Traceable to documents - -**Note**: If document-described feature granularity is coarse (not precise to specific features), it still serves as a leaf node at the current stage; subsequent `feature_build` workflow can further refine. - -**Evidence Relationships:** - -- One node can reference multiple evidence items (same feature described in multiple documents) -- One evidence item corresponds to one node (one feature description corresponds to one feature point) - -#### 4.4: Display Progress - -```markdown -## ✓ Feature Domain Files Generation Complete - -| Domain | File | Feature Count | -|--------|------|---------------| -| FT-001: Core Orchestration | features/FT-001.md | | -| FT-002: User Interface | features/FT-002.md | | -| FT-003: Extensibility | features/FT-003.md | | - -**Total Feature Files**: -**Total Feature Count**: -``` - -→ Proceed to **Step 5: Convert to JSON** - ---- - -### Step 5: Convert to JSON - -Convert generated Markdown feature specification files to JSON format. +## Workflow -#### 5.1: Run Conversion Script +### Step 1: Check-only short-circuits -Execute the following command: +If `--check-only` was supplied, run: ```bash -cmind script feature_spec_to_json.py -``` - -#### 5.2: Verify Output - -Confirm conversion results based on script log output. The script will output logs in a format similar to: - -```text -Parsing feature specification from: .cmind/data/feature_spec -Include evidence: True - -Output written to: .cmind/data/feature_spec.json - - Repository: {name} - - Background items: - - NFR items: - - Top-level features: - - Total feature nodes: -``` - -Display results based on log information: - -```markdown -## ✓ JSON Conversion Complete - -- **Output File**: .cmind/data/feature_spec.json -- **Repository**: {from log} -- **Background Entries**: {from log} -- **NFR Entries**: {from log} -- **Top-level Features**: {from log} -- **Total Feature Nodes**: {from log} -``` - -→ Proceed to **Step 6: Results Report** - ---- - -### Step 6: Results Report - -#### 6.1: Summary - -```markdown -## Feature Specification Complete - -### Output Files - -| File | Description | -|------|-------------| -| .cmind/data/feature_spec/evidence/*.md | Evidence files | -| .cmind/data/feature_spec/feature_spec.md | Main specification file | -| .cmind/data/feature_spec/features/FT-*.md | Feature domain files | -| .cmind/data/feature_spec.json | JSON format specification file | - -### Statistics - -| Metric | Count | -|--------|-------| -| Source Documents | | -| Evidence Entries | | -| Background Entries | | -| Feature Domains | | -| Total Features | | -| NFR Entries | | - -### Next Steps - -To expand and build the feature tree, run: - -`/cmind.feature_build` -``` - ---- - -## Appendix A: Evidence File Format - -### Template - -```markdown -# Evidence: {document_filename} - -## Background / Overview - -### [{prefix}-BG-{NNN}] L{start}-L{end} -**Parent Title**: - - -> {Original excerpt} - -## FR (Functional Requirement) - -### [{prefix}-FR-{NNN}] L{start}-L{end} -**Parent Title**: {Section header} - -> {Original excerpt} - -## NFR (Non-Functional Requirement) - -### [{prefix}-NFR-{NNN}] L{start}-L{end} -**Parent Title**: {Section header} - -> {Original excerpt} -``` - -### Example - -```markdown -# Evidence: 02_requirements_specification.md - -## Background / Overview - -### [RS-BG-001] L9-L12 -**Parent Title**: - - -> This document captures the functional and non-functional requirements for Apache Airflow, a platform to programmatically author, schedule, and monitor workflows. - -### [RS-BG-002] L17-L29 -**Parent Title**: 2. Target Users and Usage Scenarios - -> #### Data Engineer - "Pipeline Builder" -> **Background:** Software engineer specializing in data infrastructure -> ... - -## FR (Functional Requirement) - -### [RS-FR-001] L83-L83 -**Parent Title**: 3.1 DAG Authoring & Definition - -> | FR-DA-001 | Define workflows as Python code using a clear API | Must Have | Core value proposition | - -### [RS-FR-002] L84-L84 -**Parent Title**: 3.1 DAG Authoring & Definition - -> | FR-DA-002 | Express task dependencies explicitly | Must Have | Foundation of scheduling | - -### [RS-FR-003] L95-L95 -**Parent Title**: 3.2 Task Scheduling - -> | FR-TS-001 | Schedule DAG runs based on time intervals (cron-like) | Must Have | Primary scheduling mode | - -## NFR (Non-Functional Requirement) - -### [RS-NFR-001] L184-L184 -**Parent Title**: 4.1 Performance - -> | NFR-P-001 | Scheduler latency | < 1 minute from schedule time to task queuing | - -### [RS-NFR-002] L194-L194 -**Parent Title**: 4.2 Scalability - -> | NFR-S-001 | Support horizontal scaling of workers | Distributed execution backends | -``` - -### Field Descriptions - -| Field | Source | Description | -| --- | --- | --- | -| **Parent Title** | Extracted from source | Section header (for tracing source location); `-` for user input mode | - -### Line Number Format Specification - -All line numbers use unified `L{start}-L{end}` format: - -- Single line: `L83-L83` -- Multiple lines: `L21-L24` - ---- - -## Appendix B: Main File Format (feature_spec.md) - -### Template - -```markdown -# Feature Specification - -## Meta - -- **Repository Name**: {repository_name} -- **Repository Purpose**: {repository_purpose} -- **Generated At**: {date} -- **Source Documents**: {comma-separated list} - -## Background - -### BG-{NNN}: {Name} -- **Description**: {Description} -- **Evidence**: - - {ID} | {source} L{line_number} - -## NFR - -### NFR-{NNN}: {Name} -- **Description**: {Description} -- **Evidence**: - - {ID} | {source} L{line_number} -``` - -### Example - -```markdown -# Feature Specification - -## Meta - -- **Repository Name**: apache-airflow -- **Repository Purpose**: A platform to programmatically author, schedule, and monitor batch-oriented workflows as code with explicit dependencies and reliable execution. -- **Generated At**: 2026-02-05 -- **Source Documents**: 01_project_charter.md, 02_requirements_specification.md, 03_domain_analysis.md, 04_system_design_overview.md, 05_interface_data_contract.md, 06_assumptions_constraints_risks.md - -## Background - -### BG-001: Fragmented Tooling Ecosystem -- **Description**: Organizations currently mix cron jobs, custom scripts, and proprietary scheduling systems without unified workflow definition, execution, and monitoring capabilities. -- **Evidence**: - - project_charter-BG-001 | 01_project_charter.md L21-L24 - - RS-BG-001 | 02_requirements_specification.md L9-L12 - -### BG-002: Batch-Oriented Workflow Focus -- **Description**: Apache Airflow focuses on batch-oriented workflow orchestration where work is divided into discrete tasks with clear dependencies. -- **Evidence**: - - project_charter-BG-002 | 01_project_charter.md L43-L49 - -## NFR - -### NFR-001: Performance -- **Description**: Scheduler latency less than 1 minute from schedule time to task queuing, support 100+ DAGs, UI response under 3 seconds. -- **Evidence**: - - RS-NFR-001 | 02_requirements_specification.md L184-L184 - -### NFR-002: Scalability -- **Description**: Support horizontal scaling of workers, distributed execution backends, and multi-scheduler deployment. -- **Evidence**: - - RS-NFR-002 | 02_requirements_specification.md L194-L194 - -### NFR-003: Security -- **Description**: Encrypt sensitive data at rest, support authentication/authorization mechanisms, and provide audit logging. -- **Evidence**: - - RS-NFR-003 | 02_requirements_specification.md L210-L210 +cmind script feature_spec.py --check-only [--json] ``` ---- - -## Appendix C: Feature Domain File Format (features/FT-XXX.md) - -### Template +Show the output verbatim and stop without inspecting `docs/` or +generating anything. -```markdown -# FT-{NNN}: {Domain Name} -- **Description**: {Domain description} +### Step 2: Determine requirement source -## FT-{NNN}-001: {Sub-domain Name} -- **Description**: {Sub-domain description} +Priority: -### FT-{NNN}-001-001: {Feature Name} -- **Description**: {Feature description} -- **Evidence**: - - {ID} | {source} L{line_number} +1. **Requirement text** after the command (and after `--` if options are + present). Pass via `--input-text ""`. +2. **`docs/*.md` files**. If text is empty and `docs/` contains usable + Markdown files, no flag is needed — `feature_spec.py` auto-detects. +3. **Inline prompt**. If neither exists, ask the user for requirements in + this conversation, then pass the captured text via `--input-text`. -### FT-{NNN}-001-002: {Feature Name} -- **Description**: {Feature description} -- **Evidence**: - - {ID} | {source} L{line_number} +Do **not** display the legacy "Use these documents? (Y/N)" confirmation — +auto-detection already covers it. -## FT-{NNN}-002: {Sub-domain Name} -- **Description**: {Sub-domain description} +### Step 3: Overwrite decision -### FT-{NNN}-002-001: {Feature Name} -- **Description**: {Feature description} -- **Evidence**: - - {ID} | {source} L{line_number} -``` - -### Example - -```markdown -# FT-001: Core Orchestration -- **Description**: Core workflow orchestration capabilities including DAG authoring, scheduling, and execution. - -## FT-001-001: DAG Authoring and Definition -- **Description**: Enable users to define workflows as Python code with explicit task dependencies. - -### FT-001-001-001: Python DAG Definition API -- **Description**: Define workflows using DAG context manager pattern with dag_id, schedule, start_date, default_args, catchup, and tags parameters. -- **Evidence**: - - project_charter-FR-001 | 01_project_charter.md L116-L118 - - RS-FR-001 | 02_requirements_specification.md L83-L83 - -### FT-001-001-002: Task Dependency Declaration -- **Description**: Express task dependencies explicitly using >> operator, << operator, chain(), and set_upstream/set_downstream methods. -- **Evidence**: - - RS-FR-002 | 02_requirements_specification.md L84-L84 - -### FT-001-001-003: DAG Parameterization -- **Description**: Support parameterized DAGs with Variables, Params, and Jinja templating. -- **Evidence**: - - RS-FR-003 | 02_requirements_specification.md L85-L85 - -## FT-001-002: Task Scheduling -- **Description**: Time-based and dependency-based task scheduling capabilities. - -### FT-001-002-001: Cron-based Scheduling -- **Description**: Schedule DAG runs based on cron expressions and preset intervals (@daily, @hourly, @weekly, etc.). -- **Evidence**: - - project_charter-FR-002 | 01_project_charter.md L119-L120 - - RS-FR-004 | 02_requirements_specification.md L95-L95 - -### FT-001-002-002: Data-aware Scheduling -- **Description**: Trigger DAG runs based on dataset updates and data availability. -- **Evidence**: - - RS-FR-005 | 02_requirements_specification.md L96-L96 - -## FT-001-003: Task Execution -- **Description**: Task execution management with multiple backends and reliability features. - -### FT-001-003-001: Executor Backends -- **Description**: Support multiple execution backends for different deployment scenarios. - -#### FT-001-003-001-001: Local Executor -- **Description**: Execute tasks locally using multiprocessing for development and small deployments. -- **Evidence**: - - RS-FR-010 | 02_requirements_specification.md L107-L107 - -#### FT-001-003-001-002: Celery Executor -- **Description**: Distribute task execution across Celery workers for horizontal scaling. -- **Evidence**: - - SDO-FR-005 | 04_system_design_overview.md L89-L89 - -#### FT-001-003-001-003: Kubernetes Executor -- **Description**: Execute each task in a separate Kubernetes pod for isolation and scalability. -- **Evidence**: - - SDO-FR-006 | 04_system_design_overview.md L91-L91 - -### FT-001-003-002: Retry and Failure Handling -- **Description**: Automatic task retry with configurable attempts, delays, and exponential backoff. -- **Evidence**: - - RS-FR-011 | 02_requirements_specification.md L108-L108 -``` - ---- - -## Evidence ID Format Description - -### ID Structure +If `feature_spec.json` already exists and `--force` was not supplied, +the helper exits with `[SKIP]`. To regenerate, ask the user: ```text -{file_prefix}-{category}-{sequence} - │ │ │ - │ │ └── 001, 002, 003... (numbered independently per category) - │ └────── BG / FR / NFR - └──────────── File name abbreviation or full name +feature_spec.json already exists. Regenerate? + [F] Force regenerate [E] Keep existing ``` -### Category Meanings - -| Category | Full Name | Description | -| --- | --- | --- | -| BG | Background | Background information, context, goals, terminology definitions | -| FR | Functional Requirement | Functional requirements, features, behaviors, operations | -| NFR | Non-Functional Requirement | Performance, security, scalability, assumptions, constraints, technical risks | +`F` → rerun the command with `--force`; `E` → stop. -### Examples +### Step 4: Run -| ID | Interpretation | -| --- | --- | -| `RS-FR-001` | requirements_specification - FR - 1st entry | -| `RS-NFR-003` | requirements_specification - NFR - 3rd entry | -| `project_charter-BG-001` | project_charter - Background - 1st entry | -| `IDC-FR-004` | interface_data_contract - FR - 4th entry | -| `user_input-FR-001` | user_input - FR - 1st entry | +```bash +cmind script feature_spec.py [--input-text ""] [--force] [--no-trajectory] [--verbose] +``` ---- +Stream stdout / stderr from the helper. The helper handles: -## Quality Standards +- Reading docs / inline text. +- Calling the LLM with a strict Pydantic schema (`FeatureSpecOutput`). +- Validating the output against the schema (no markdown intermediaries). +- Writing `feature_spec.json` atomically (on failure, no partial file). -### Evidence Quality +### Step 5: Recovery hints on failure -- [ ] Original text preserved verbatim -- [ ] Line numbers accurately traceable -- [ ] Each evidence entry has unique ID (format: `{prefix}-{BG|FR|NFR}-{sequence}`) -- [ ] Correctly categorized (Background / FR / NFR) -- [ ] Summary is feature description/summary (extracted from source or generated based on content) -- [ ] Parent Title is section header from source (`-` for user input) +If the helper exits non-zero: -### Feature Tree Quality +- Exit code `2` (`NoInputAvailable`) — neither `--input-text` nor + `docs/*.md` was found. Re-invoke with either inline text or after + populating `docs/`. +- Exit code `1` (LLM failure / schema validation failure) — re-run with + `--verbose` to surface the trajectory file location in the script's + own log output; do not attempt to locate it manually. -- [ ] Hierarchical structure is logically consistent -- [ ] Numbering follows decimal outline format -- [ ] ID matches numbering structure -- [ ] Document nodes (feature descriptions from documents) have Evidence -- [ ] Organization nodes (abstract layers for grouping) have no Evidence -- [ ] Each document node has clear feature boundaries (describable in one sentence) -- [ ] Node descriptions say "what" not "how" -- [ ] No orphaned or duplicate features + ```bash + cmind script feature_spec.py --verbose + ``` -### File Quality +### Step 6: Completion -- [ ] All files saved to correct locations -- [ ] File links in feature_spec.md are valid -- [ ] Markdown formatting is correct -- [ ] No broken references between files +On success, the helper prints summary stats (repository, top-level +feature count, BG / NFR counts) and writes the spec to the home-side +data store. -```text +For the standard end-to-end Phase 1 flow, use +`/cmind.feature_construct` — it runs this stage plus the remaining +Phase 1 steps in one command. diff --git a/CoderMind/templates/commands/plan.md b/CoderMind/templates/commands/plan.md new file mode 100644 index 0000000..0f1f8c8 --- /dev/null +++ b/CoderMind/templates/commands/plan.md @@ -0,0 +1,138 @@ +--- +description: Build the complete Phase 2 Repository Planning Graph (RPG) from the feature tree in one step, with automatic resume on failure +name: cmind.plan +--- + +## User Input + +```text +$ARGUMENTS +``` + +`$ARGUMENTS` is forwarded verbatim to `cmind script plan.py` (for +example, `--verbose`, `--max-iter-skeleton 15`, or `--force`). +If empty, proceed with default behavior. + +## **Outline** + +Given the feature tree produced by `/cmind.feature_construct`, this +command builds the complete Repository Planning Graph (RPG) in a single +non-interactive run with automatic resume on failure. + +> [!WARNING] +> A full pipeline run can take from a few minutes to over an hour +> depending on project size. Set your terminal timeout to at least +> **240 minutes** before running. Do **not** interrupt it; if you +> must, re-run this command and it will resume from where it stopped. + +### Step 1: Probe progress + +Run the orchestrator in probe mode and capture the JSON report: + +```bash +cmind script plan.py --check-only --json +``` + +Parse the JSON. The fields you need: + +* `total` — total number of stages (always 5) +* `done` — count of stages whose `type` is `update` +* `next` — name of the first not-done stage (or `null` if all done) +* `stages[*].name`, `stages[*].done` + +### Step 2: One decision (the only prompt of this command) + +Choose **exactly one** case based on `done` vs `total`: + +**Case A — Everything already done (`done == total`):** + +Display this prompt and wait for the user's choice: + +```text +All 5 planning stages are already complete: + ✓ skeleton + ✓ data_flow + ✓ base_classes + ✓ interfaces + ✓ tasks + +What would you like to do? + [O] Overwrite — regenerate everything from scratch + [E] Exit — keep existing artifacts and proceed to /cmind.code_gen +``` + +* `O` → execute: `cmind script plan.py --force $ARGUMENTS` +* `E` → terminate this command; remind the user that `/cmind.code_gen` + is the next step. + +**Case B — Fresh workspace (`done == 0`):** + +Do **not** prompt. Briefly inform the user and proceed: + +```text +Starting the full planning pipeline (5 stages). This may take a while. +``` + +Then execute: `cmind script plan.py $ARGUMENTS` + +**Case C — Partial progress (`0 < done < total`):** + +Display this prompt, with each stage marked using its real status +from `stages[*].done` (`✓` for done, `▸` for the first not-done one, +`·` for the rest): + +```text +Planning is partially complete: / stages done. + skeleton + data_flow + base_classes + interfaces + tasks + +Last completed stage: +Stopped at: + +What would you like to do? + [C] Continue — resume from `` and finish the pipeline + [R] Restart — discard progress and regenerate everything + [E] Exit — do nothing +``` + +* `C` → execute: `cmind script plan.py $ARGUMENTS` +* `R` → execute: `cmind script plan.py --force $ARGUMENTS` +* `E` → terminate this command. + +### Step 3: Stream the orchestrator's output + +When you execute the orchestrator (cases A → O, B, C → C/R above), +stream its stdout/stderr to the user as-is. The orchestrator already +prints one progress line per stage and a final summary; do not add +your own commentary on top of every line. + +### Step 4: On failure + +If the orchestrator exits non-zero, it has already printed a +`✗ ... failed` line plus three recovery hints. Surface those +hints verbatim. The most common follow-ups are: + +```bash +# Re-check progress (no side effects). +cmind script plan.py --check-only + +# Resume from where it failed (default behavior). +cmind script plan.py + +# Debug a single stage interactively. +cmind script .py --verbose +``` + +### Step 5: On success + +Tell the user: + +```text +Planning pipeline complete. + +Next: + /cmind.code_gen — generate source code from the plan +``` diff --git a/CoderMind/tests/test_feature_construct_orchestrator.py b/CoderMind/tests/test_feature_construct_orchestrator.py new file mode 100644 index 0000000..6e6c965 --- /dev/null +++ b/CoderMind/tests/test_feature_construct_orchestrator.py @@ -0,0 +1,381 @@ +"""Unit tests for the Phase 1 feature construction orchestrator.""" + +from __future__ import annotations + +import importlib.util +import json +import sys +from pathlib import Path + +import pytest + +_REPO = Path(__file__).resolve().parents[1] +_SCRIPTS = _REPO / "scripts" + +if str(_SCRIPTS) not in sys.path: + sys.path.insert(0, str(_SCRIPTS)) + +_SPEC = importlib.util.spec_from_file_location( + "feature_construct_orchestrator", + _SCRIPTS / "feature_construct.py", +) +assert _SPEC is not None and _SPEC.loader is not None +feature_construct = importlib.util.module_from_spec(_SPEC) +sys.modules["feature_construct_orchestrator"] = feature_construct +_SPEC.loader.exec_module(feature_construct) + + +@pytest.fixture +def artifact_paths(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> dict[str, Path]: + paths = { + "feature_spec": tmp_path / "feature_spec.json", + "feature_build": tmp_path / "feature_build.json", + "feature_refactor": tmp_path / "feature_tree.json", + } + monkeypatch.setattr(feature_construct, "FEATURE_SPEC_FILE", paths["feature_spec"]) + monkeypatch.setattr(feature_construct, "FEATURE_BUILD_FILE", paths["feature_build"]) + monkeypatch.setattr(feature_construct, "FEATURE_TREE_FILE", paths["feature_refactor"]) + return paths + + +def _write_json(path: Path, data: object) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(data), encoding="utf-8") + + +def _write_text(path: Path, text: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + +def _valid_feature_spec() -> dict[str, object]: + return { + "meta": {"generated_at": "2026-05-25", "project_types": ["CLI"]}, + "repository_name": "sample-cli", + "repository_purpose": "Build a sample CLI.", + "background_and_overview": [{"id": "BG-001", "description": "Users need a CLI."}], + "functional_requirements": [{"id": "FT-001", "name": "CLI", "children": []}], + "non_functional_requirements": [{"id": "NFR-001", "description": "Fast startup."}], + } + + +def _states(types: list[str]) -> list["feature_construct.StageState"]: + assert len(types) == len(feature_construct.STAGES) + return [ + feature_construct.StageState(stage=stage, type=t, done=(t == "update")) + for stage, t in zip(feature_construct.STAGES, types) + ] + + +class TestStageRegistry: + def test_three_stages_in_canonical_order(self) -> None: + assert [stage.name for stage in feature_construct.STAGES] == [ + "feature_spec", + "feature_build", + "feature_refactor", + ] + + @pytest.mark.parametrize("stage", feature_construct.STAGES) + def test_every_stage_has_a_build_script(self, stage: "feature_construct.Stage") -> None: + assert (_SCRIPTS / stage.build_script).is_file(), stage.build_script + + +class TestCompletionDetection: + def test_missing_artifacts_are_incomplete(self, artifact_paths: dict[str, Path]) -> None: + states = feature_construct.probe() + assert [state.type for state in states] == ["init", "init", "init"] + assert [state.done for state in states] == [False, False, False] + + def test_valid_artifacts_are_complete(self, artifact_paths: dict[str, Path]) -> None: + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + _write_json(artifact_paths["feature_build"], {"feature_tree": {}}) + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "core"}]}) + + states = feature_construct.probe() + assert [state.type for state in states] == ["update", "update", "update"] + assert [state.done for state in states] == [True, True, True] + + def test_feature_spec_requires_downstream_fields(self, artifact_paths: dict[str, Path]) -> None: + spec = _valid_feature_spec() + spec.pop("functional_requirements") + _write_json(artifact_paths["feature_spec"], spec) + + state = feature_construct.probe()[0] + assert state.type == "warning" + assert state.done is False + assert "functional_requirements" in state.message + + def test_feature_refactor_requires_non_empty_components(self, artifact_paths: dict[str, Path]) -> None: + _write_json(artifact_paths["feature_refactor"], {"components": []}) + + state = feature_construct.probe()[2] + assert state.type == "warning" + assert state.done is False + assert "components" in state.message + + +class TestCheckOnlyJson: + def test_json_payload_reports_progress(self, artifact_paths: dict[str, Path], capsys: pytest.CaptureFixture[str]) -> None: + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + _write_json(artifact_paths["feature_build"], {"feature_tree": {}}) + + rc = feature_construct.main(["--check-only", "--json"]) + captured = capsys.readouterr() + payload = json.loads(captured.out) + + assert rc == 0 + assert payload["total"] == 3 + assert payload["done"] == 2 + assert payload["completed"] == 2 + assert payload["next"] == "feature_refactor" + assert [stage["name"] for stage in payload["stages"]] == [ + "feature_spec", + "feature_build", + "feature_refactor", + ] + assert [stage["done"] for stage in payload["stages"]] == [True, True, False] + + +class TestExecutionReset: + def test_force_removes_stale_output_sensitive_artifacts_before_stage_invocation( + self, + artifact_paths: dict[str, Path], + monkeypatch: pytest.MonkeyPatch, + ) -> None: + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + _write_json(artifact_paths["feature_build"], {"stale": "build"}) + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "stale"}]}) + calls: list[str] = [] + + def fake_run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + calls.append(script_name) + if script_name == "feature_spec.py": + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + elif script_name == "feature_build.py": + assert not artifact_paths["feature_build"].exists() + _write_json(artifact_paths["feature_build"], {"feature_tree": {"fresh": True}}) + elif script_name == "feature_refactor.py": + assert not artifact_paths["feature_refactor"].exists() + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "fresh"}]}) + return 0 + + monkeypatch.setattr(feature_construct, "_run_stage", fake_run_stage) + + rc = feature_construct.main(["--force"]) + + assert rc == 0 + assert calls == ["feature_spec.py", "feature_build.py", "feature_refactor.py"] + + def test_cascade_removes_stale_downstream_artifacts_before_stage_invocation( + self, + artifact_paths: dict[str, Path], + monkeypatch: pytest.MonkeyPatch, + ) -> None: + spec = _valid_feature_spec() + spec.pop("repository_purpose") + _write_json(artifact_paths["feature_spec"], spec) + _write_json(artifact_paths["feature_build"], {"stale": "build"}) + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "stale"}]}) + calls: list[str] = [] + + def fake_run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + calls.append(script_name) + if script_name == "feature_spec.py": + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + elif script_name == "feature_build.py": + assert not artifact_paths["feature_build"].exists() + _write_json(artifact_paths["feature_build"], {"feature_tree": {"fresh": True}}) + elif script_name == "feature_refactor.py": + assert not artifact_paths["feature_refactor"].exists() + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "fresh"}]}) + return 0 + + monkeypatch.setattr(feature_construct, "_run_stage", fake_run_stage) + + rc = feature_construct.main([]) + + assert rc == 0 + assert calls == ["feature_spec.py", "feature_build.py", "feature_refactor.py"] + + def test_invalid_output_sensitive_artifact_is_removed_before_stage_invocation( + self, + artifact_paths: dict[str, Path], + monkeypatch: pytest.MonkeyPatch, + ) -> None: + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + _write_text(artifact_paths["feature_build"], "{") + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "stale"}]}) + calls: list[str] = [] + + def fake_run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + calls.append(script_name) + if script_name == "feature_build.py": + assert not artifact_paths["feature_build"].exists() + _write_json(artifact_paths["feature_build"], {"feature_tree": {"fresh": True}}) + elif script_name == "feature_refactor.py": + assert not artifact_paths["feature_refactor"].exists() + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "fresh"}]}) + return 0 + + monkeypatch.setattr(feature_construct, "_run_stage", fake_run_stage) + + rc = feature_construct.main([]) + + assert rc == 0 + assert calls == ["feature_build.py", "feature_refactor.py"] + + def test_all_up_to_date_skip_path_does_not_remove_artifacts( + self, + artifact_paths: dict[str, Path], + monkeypatch: pytest.MonkeyPatch, + ) -> None: + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + _write_json(artifact_paths["feature_build"], {"feature_tree": {}}) + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "core"}]}) + + def fail_run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + pytest.fail(f"unexpected stage run: {script_name}") + + monkeypatch.setattr(feature_construct, "_run_stage", fail_run_stage) + + rc = feature_construct.main([]) + + assert rc == 0 + assert artifact_paths["feature_build"].exists() + assert artifact_paths["feature_refactor"].exists() + + @pytest.mark.parametrize("argv", [["--check-only"], ["--check-only", "--json"]]) + def test_check_only_does_not_remove_artifacts_or_run_stages( + self, + argv: list[str], + artifact_paths: dict[str, Path], + monkeypatch: pytest.MonkeyPatch, + ) -> None: + _write_text(artifact_paths["feature_spec"], "{") + _write_json(artifact_paths["feature_build"], {"stale": "build"}) + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "stale"}]}) + + def fail_run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + pytest.fail(f"unexpected stage run: {script_name}") + + monkeypatch.setattr(feature_construct, "_run_stage", fail_run_stage) + + rc = feature_construct.main(argv) + + assert rc == 0 + assert artifact_paths["feature_spec"].exists() + assert artifact_paths["feature_build"].exists() + assert artifact_paths["feature_refactor"].exists() + + def test_dry_run_does_not_remove_artifacts_or_run_stages( + self, + artifact_paths: dict[str, Path], + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], + ) -> None: + _write_json(artifact_paths["feature_spec"], _valid_feature_spec()) + _write_text(artifact_paths["feature_build"], "{") + _write_json(artifact_paths["feature_refactor"], {"components": [{"name": "stale"}]}) + + def fail_run_stage(invoker: list[str], script_name: str, extra: list[str]) -> int: + pytest.fail(f"unexpected stage run: {script_name}") + + monkeypatch.setattr(feature_construct, "_run_stage", fail_run_stage) + + rc = feature_construct.main(["--dry-run"]) + captured = capsys.readouterr() + + assert rc == 0 + assert "DRY-RUN >" in captured.out + assert artifact_paths["feature_build"].exists() + assert artifact_paths["feature_refactor"].exists() + + +class TestDecideCascade: + def test_all_update_means_nothing_runs(self) -> None: + states = _states(["update", "update", "update"]) + feature_construct.decide(states, force=False) + assert [state.will_run for state in states] == [False, False, False] + + def test_fresh_workspace_runs_everything(self) -> None: + states = _states(["init", "init", "init"]) + feature_construct.decide(states, force=False) + assert [state.will_run for state in states] == [True, True, True] + + def test_partial_resume_runs_from_first_incomplete_stage(self) -> None: + states = _states(["update", "init", "update"]) + feature_construct.decide(states, force=False) + assert [state.will_run for state in states] == [False, True, True] + assert "upstream" in states[2].reason + + def test_upstream_warning_cascades_to_downstream_update(self) -> None: + states = _states(["warning", "update", "update"]) + feature_construct.decide(states, force=False) + assert [state.will_run for state in states] == [True, True, True] + assert "upstream" in states[1].reason + + def test_force_runs_everything(self) -> None: + states = _states(["update", "update", "update"]) + feature_construct.decide(states, force=True) + assert [state.will_run for state in states] == [True, True, True] + assert [state.reason for state in states] == ["forced", "forced", "forced"] + + +class TestBuildArgs: + def test_feature_build_forwards_review_options(self) -> None: + ns = feature_construct._parse_args([ + "--review-threshold", + "99", + "--review-max-iterations", + "4", + ]) + stage = next(stage for stage in feature_construct.STAGES if stage.name == "feature_build") + args = feature_construct._build_args_for(stage, ns) + + assert args[:2] == ["--mode", "step1"] + assert args[args.index("--review-threshold") + 1] == "99" + assert args[args.index("--review-max-iterations") + 1] == "4" + + def test_feature_refactor_maps_facade_iteration_flag(self) -> None: + ns = feature_construct._parse_args(["--max-iter-refactor", "7"]) + stage = next(stage for stage in feature_construct.STAGES if stage.name == "feature_refactor") + args = feature_construct._build_args_for(stage, ns) + + assert "--max-iterations" in args + assert args[args.index("--max-iterations") + 1] == "7" + assert "--max-iter-refactor" not in args + + def test_verbose_and_no_trajectory_use_native_stage_names(self) -> None: + ns = feature_construct._parse_args(["--verbose", "--no-trajectory"]) + build_stage = next(stage for stage in feature_construct.STAGES if stage.name == "feature_build") + refactor_stage = next(stage for stage in feature_construct.STAGES if stage.name == "feature_refactor") + + build_args = feature_construct._build_args_for(build_stage, ns) + refactor_args = feature_construct._build_args_for(refactor_stage, ns) + + assert "--verbose" in build_args + assert "--no-trajectory" in build_args + assert "--log-level" in refactor_args + assert refactor_args[refactor_args.index("--log-level") + 1] == "DEBUG" + assert "--no-trajectory" in refactor_args + + def test_feature_spec_receives_supported_facade_options(self) -> None: + # ``feature_spec.py`` (the new Python+LLMClient pipeline) accepts + # the standard facade flags --force / --verbose / --no-trajectory. + ns = feature_construct._parse_args(["--force", "--verbose", "--no-trajectory"]) + stage = next(stage for stage in feature_construct.STAGES if stage.name == "feature_spec") + args = feature_construct._build_args_for(stage, ns) + assert "--force" in args + assert "--verbose" in args + assert "--no-trajectory" in args + # Build/refactor-only options must NOT leak into feature_spec. + ns2 = feature_construct._parse_args([ + "--review-threshold", "99", + "--review-max-iterations", "3", + "--max-iter-refactor", "5", + ]) + args2 = feature_construct._build_args_for(stage, ns2) + assert "--review-threshold" not in args2 + assert "--review-max-iterations" not in args2 + assert "--max-iter-refactor" not in args2 + assert "--max-iterations" not in args2 diff --git a/CoderMind/tests/test_plan_orchestrator.py b/CoderMind/tests/test_plan_orchestrator.py new file mode 100644 index 0000000..b60cce4 --- /dev/null +++ b/CoderMind/tests/test_plan_orchestrator.py @@ -0,0 +1,209 @@ +"""Unit tests for the planning orchestrator's pure logic. + +Covers the decision rules of ``scripts/plan.py``: + +* ``decide()`` cascade behaviour +* probe-result parsing (``_extract_last_json_object``) +* CLI flag wiring for max-iteration overrides +* checker JSON field contracts used by the orchestrator + +The build sub-scripts themselves are *not* exercised here because they +would require real LLM calls; this test focuses on deterministic logic. +""" + +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +import pytest + +_REPO = Path(__file__).resolve().parents[1] +_SCRIPTS = _REPO / "scripts" + +if str(_SCRIPTS) not in sys.path: + sys.path.insert(0, str(_SCRIPTS)) + +# ``plan.py`` is shipped under ``scripts/`` and not installed as a +# package, so load it via importlib. +_SPEC = importlib.util.spec_from_file_location("plan_orchestrator", _SCRIPTS / "plan.py") +assert _SPEC is not None and _SPEC.loader is not None +plan = importlib.util.module_from_spec(_SPEC) +sys.modules["plan_orchestrator"] = plan +_SPEC.loader.exec_module(plan) + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +def _states(types: list[str]) -> list["plan.StageState"]: + """Build a list of StageState objects from the given type sequence.""" + assert len(types) == len(plan.STAGES) + return [ + plan.StageState(stage=stage, type=t, done=(t == "update")) + for stage, t in zip(plan.STAGES, types) + ] + + +def _load_script(name: str): + spec = importlib.util.spec_from_file_location(name, _SCRIPTS / f"{name}.py") + assert spec is not None and spec.loader is not None + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +# --------------------------------------------------------------------------- +# decide() cascade rule +# --------------------------------------------------------------------------- + +class TestDecideCascade: + def test_all_update_means_nothing_runs(self) -> None: + states = _states(["update"] * 5) + plan.decide(states, force=False) + assert [s.will_run for s in states] == [False] * 5 + + def test_fresh_workspace_runs_everything(self) -> None: + states = _states(["init"] * 5) + plan.decide(states, force=False) + assert [s.will_run for s in states] == [True] * 5 + + def test_partial_resume_runs_from_first_non_update(self) -> None: + # skeleton + data_flow done, base_classes init, rest init + states = _states(["update", "update", "init", "init", "init"]) + plan.decide(states, force=False) + assert [s.will_run for s in states] == [False, False, True, True, True] + + def test_cascade_forces_downstream_even_if_update(self) -> None: + # Inconsistent state: skeleton needs rebuild but base_classes + # is "update" (e.g. user manually deleted skeleton.json). + # Cascade rule must force base_classes to rebuild anyway. + states = _states(["init", "update", "update", "update", "update"]) + plan.decide(states, force=False) + assert [s.will_run for s in states] == [True, True, True, True, True] + # downstream reasons should mention cascade + assert "upstream" in states[1].reason + + def test_warning_triggers_run(self) -> None: + states = _states(["update", "warning", "update", "update", "update"]) + plan.decide(states, force=False) + # data_flow runs because warning; downstream cascades. + assert [s.will_run for s in states] == [False, True, True, True, True] + + def test_force_runs_everything(self) -> None: + states = _states(["update"] * 5) + plan.decide(states, force=True) + assert all(s.will_run for s in states) + assert all(s.reason == "forced" for s in states) + + +# --------------------------------------------------------------------------- +# _extract_last_json_object — tolerant JSON parsing. +# --------------------------------------------------------------------------- + +class TestExtractLastJsonObject: + def test_pure_json(self) -> None: + obj = plan._extract_last_json_object('{"type": "init"}') + assert obj == {"type": "init"} + + def test_json_with_trailing_text(self) -> None: + text = '{"type": "update", "ok": true}\n📸 snapshot abc123' + obj = plan._extract_last_json_object(text) + assert obj == {"type": "update", "ok": True} + + def test_json_with_leading_text(self) -> None: + text = 'Running ...\n{"type": "init"}' + obj = plan._extract_last_json_object(text) + assert obj == {"type": "init"} + + def test_takes_last_object_when_multiple(self) -> None: + text = '{"first": 1}{"type": "update"}' + obj = plan._extract_last_json_object(text) + assert obj == {"type": "update"} + + def test_returns_none_on_garbage(self) -> None: + assert plan._extract_last_json_object("no braces here") is None + assert plan._extract_last_json_object("{not json}") is None + + +# --------------------------------------------------------------------------- +# Per-stage max-iter flag wiring. +# --------------------------------------------------------------------------- + +class TestBuildArgs: + def test_max_iter_skeleton_uses_max_iterations_flag(self) -> None: + ns = plan._parse_args(["--max-iter-skeleton", "7"]) + skeleton = plan.STAGES[0] + assert skeleton.name == "skeleton" + args = plan._build_args_for(skeleton, ns) + assert "--max-iterations" in args + assert args[args.index("--max-iterations") + 1] == "7" + + def test_max_iter_interfaces_uses_max_file_iterations_flag(self) -> None: + # design_interfaces.py has a different flag name than the others. + ns = plan._parse_args(["--max-iter-interfaces", "4"]) + interfaces = next(s for s in plan.STAGES if s.name == "interfaces") + args = plan._build_args_for(interfaces, ns) + assert "--max-file-iterations" in args + assert "--max-iterations" not in args + assert args[args.index("--max-file-iterations") + 1] == "4" + + def test_tasks_stage_has_no_max_iter_flag(self) -> None: + # plan_tasks.py takes no iteration count; --max-iter-* must not be + # forwarded even if some other stage's flag is set. + ns = plan._parse_args(["--max-iter-skeleton", "9"]) + tasks = next(s for s in plan.STAGES if s.name == "tasks") + args = plan._build_args_for(tasks, ns) + assert all(not a.startswith("--max") for a in args) + + def test_verbose_forwarded(self) -> None: + ns = plan._parse_args(["--verbose"]) + args = plan._build_args_for(plan.STAGES[0], ns) + assert "--verbose" in args + + def test_no_trajectory_forwarded(self) -> None: + ns = plan._parse_args(["--no-trajectory"]) + args = plan._build_args_for(plan.STAGES[0], ns) + assert "--no-trajectory" in args + + +# --------------------------------------------------------------------------- +# Stage table sanity — guard against silent registry drift. +# --------------------------------------------------------------------------- + +class TestCheckerContracts: + @pytest.mark.parametrize( + ("script_name", "args"), + [ + ("check_data_flow", (Path("missing-data-flow.json"), Path("missing-skeleton.json"))), + ("check_base_classes", (Path("missing-base-classes.json"),)), + ], + ) + def test_plan_checkers_emit_type_not_state(self, script_name: str, args: tuple[Path, ...]) -> None: + checker = _load_script(script_name) + result = checker.inspect_state(*args) + assert result["type"] == "init" + assert "state" not in result + + +class TestStageRegistry: + def test_five_stages_in_canonical_order(self) -> None: + assert [s.name for s in plan.STAGES] == [ + "skeleton", + "data_flow", + "base_classes", + "interfaces", + "tasks", + ] + + @pytest.mark.parametrize("stage", plan.STAGES) + def test_every_stage_has_a_build_and_check_script(self, stage: plan.Stage) -> None: + assert (_SCRIPTS / stage.build_script).is_file(), stage.build_script + assert (_SCRIPTS / stage.check_script).is_file(), stage.check_script + + @pytest.mark.parametrize("post_script", plan.POST_STEPS) + def test_post_step_scripts_exist(self, post_script: str) -> None: + assert (_SCRIPTS / post_script).is_file(), post_script