redesign dev.kit: context.yaml pipeline, AGENTS.md with rules + manifests, promptfoo eval

[fqjony]

Summary

• Migrate from .dev-kit/manifest.json to .rabbit/context.yaml as the canonical repo-to-agent handoff
• Redesign AGENTS.md with anti-drift Rules, traceable Config manifests, and full 15-step workflow
• Add promptfoo eval proving dev.kit context improves agent accuracy (100% with vs 75% without)
• Add spinners, status indicators, and [next] navigation to all commands

Command surface

Command	What it does
dev.kit	Validate env, detect repo, show next steps
dev.kit repo	Analyze 7 factors, detect config manifests, write .rabbit/context.yaml
dev.kit agent	Generate AGENTS.md (auto-creates context.yaml if missing)
dev.kit learn	Scan Claude + Codex sessions, write incremental lessons artifact

All commands support --json. All show spinners during slow phases.

Workflow

dev.kit → dev.kit repo → dev.kit agent → dev.kit learn
env        context.yaml   AGENTS.md       lessons artifact

dev.kit agent auto-generates context — no manual steps required.

AGENTS.md redesign

Now a comprehensive agent guide with:

• Rules — no scanning, verify before commit, follow the workflow
• Config manifests — every YAML listed with one-line description of what it controls
• Full workflow — 15 steps with operational notes (not truncated)
• Lessons — referenced from prior agent sessions

Scaffold (config-driven)

dev.kit repo --scaffold now reads from repo-scaffold.yaml:

• Baseline dirs + files (docs, .gitignore, PR template)
• Archetype-specific structure (shell-cli gets bin, lib/commands, etc.)
• Factor-driven files (missing config → .env.example, missing runtime → Dockerfile)

promptfoo eval

make eval runs 8 test cases comparing agent accuracy with vs without AGENTS.md:

• With context: 100% pass (knows verify command, reads priority refs, avoids scanning, identifies gaps)
• Without context: 75% pass (resorts to find . and tree, can't identify structural gaps)

UX

• Braille spinners during all slow phases (analyzing repo, writing context, scanning sessions)
• Status indicators: ✓ present / ◦ partial / ✗ missing / · n/a
• [next] navigation section on every command output

Test plan

• 44/44 test suite passes (make test)
• promptfoo eval: 14/16 pass (2 expected baseline failures)
• Spinner verified in interactive terminal
• dev.kit repo --scaffold creates baseline + archetype + factor files
• Manual: verify AGENTS.md quality on an external repo

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR refactors dev.kit into a phase-based workflow (env → repo → agent), enriches repo context output (archetypes, factors, gap messages), and improves UX/performance via progressive output and caching.

Changes:

• Replaces legacy audit/bridge/status command surfaces with new home output plus repo, agent, and learn flows backed by JSON templates and config catalogs.
• Adds config-driven detection/archetype catalogs and repo scaffolding/manifest generation (including WordPress fixture coverage).
• Introduces performance improvements (per-process cache, env context cache) and updates docs/tests accordingly.

Reviewed changes

Copilot reviewed 61 out of 69 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/suite.sh	Reworks integration suite into grouped runs; adds repo/agent expectations.
tests/run.sh	Removes worker-based test runner script.
tests/local-udx.sh	Adds local UDX repo scanning contract checks (JSON).
tests/fixtures/wordpress-repo/wp-content/mu-plugins/bootstrap.php	Adds WordPress fixture signal.
tests/fixtures/wordpress-repo/wp-config.php	Adds WordPress fixture root config.
tests/fixtures/wordpress-repo/package.json	Adds WordPress fixture Node manifest.
tests/fixtures/wordpress-repo/README.md	Adds WordPress fixture docs signal.
tests/fixtures/wordpress-repo/.rabbit/infra_configs/development/develop-alex/aws-cloudfront-distribution.yaml	Adds infra config fixture signal.
tests/fixtures/wordpress-repo/.rabbit/infra_configs/development/aws-cloudfront-distribution.yaml	Adds infra config fixture signal.
tests/fixtures/wordpress-repo/.github/workflows/deploy.yml	Adds workflow fixture signal.
tests/fixtures/codex-home/sessions/2026/04/02/rollout-2026-04-02T20-54-19-019d4f54-eddc-7350-a757-3bb578d24f99.jsonl	Adds learn-session fixture input.
src/templates/repo.json	Adds JSON contract template for dev.kit repo.
src/templates/learn.json	Adds JSON contract template for dev.kit learn.
src/templates/bridge.json.tmpl	Removes legacy bridge JSON template.
src/templates/audit.json.tmpl	Removes legacy audit JSON template.
src/templates/agent.json	Adds JSON contract template for dev.kit agent.
src/configs/repo-scaffold.yaml	Introduces repo scaffold catalog (baseline/archetype/factor).
src/configs/learning-workflows.yaml	Introduces learn workflow catalog and session pattern rules.
src/configs/knowledge-base.yaml	Introduces knowledge base hierarchy/source config.
src/configs/development-workflows.yaml	Introduces development workflow contract catalog.
src/configs/development-practices.yaml	Introduces practice messages catalog.
src/configs/detection-signals.yaml	Restructures detection signals into config keys; adds prune dirs.
src/configs/context-config.yaml	Adds priority refs, repo markers, and note paths config.
src/configs/archetype-signals.yaml	Adds archetype-specific signal lists (WP/K8s/Terraform).
src/configs/archetype-rules.yaml	Adds archetype rules + human descriptions.
lib/modules/utils.sh	Adds per-process cache + YAML helpers + safer JSON array escaping.
lib/modules/rule_catalog.sh	Removes separate rule catalog reader module.
lib/modules/repo_workflows.sh	Adds workflow contract + entrypoints JSON.
lib/modules/repo_signals.sh	Refactors detection catalog reads + repo root/marker logic + find pruning.
lib/modules/repo_scaffold.sh	Adds manifest writer + gaps JSON + scaffold apply.
lib/modules/repo_reports.sh	Enriches factor JSON with rule messages; adds agent contract output.
lib/modules/repo_navigation.sh	Adds workflow/dependency navigation helpers (not wired into bootstrap list).
lib/modules/repo_inspector.sh	Removes compatibility shim.
lib/modules/repo_factors.sh	Adds factor status caching; adjusts dependencies applicability and doc partial logic.
lib/modules/repo_archetypes.sh	Adds facet-based archetype detection driven by catalogs and caching.
lib/modules/output.sh	Adds output formatting helpers for progressive UX.
lib/modules/local_env.sh	Adds env tool inventory caching + capability derivation.
lib/modules/learning_sources.sh	Adds session discovery + rule/flow scoring for learn command.
lib/modules/detection_catalog.sh	Removes legacy detection catalog reader module.
lib/modules/config_catalog.sh	Adds unified config catalog reader (archetypes/context/rules/workflows).
lib/modules/bootstrap.sh	Defines module list and public commands; updates command file resolution.
lib/commands/status.sh	Removes legacy status command.
lib/commands/repo.sh	Adds repo learn/check/scaffold modes; manifest + AGENTS generation.
lib/commands/learn.sh	Adds learn command output (text + JSON).
lib/commands/bridge.sh	Removes legacy bridge command.
lib/commands/audit.sh	Removes legacy audit command.
lib/commands/agent.sh	Adds agent command reading manifest and generating AGENTS.md.
docs/workflow.md	Updates docs to phase-based workflow model.
docs/todo.md	Removes legacy TODO doc.
docs/pull-requests.md	Removes legacy PR guidance doc.
docs/overview.md	Adds new high-level product overview.
docs/engineering-guide.md	Removes legacy engineering guide doc.
docs/development.md	Removes legacy development doc.
docs/detection-facets.md	Adds facet detection documentation.
docs/commands.md	Adds new command reference.
docs/architecture.md	Adds architecture/config/module map documentation.
context7.json	Removes Context7 config file.
bin/scripts/uninstall.sh	Uses output helpers for formatted uninstall output.
bin/scripts/install.sh	Uses output helpers for formatted install output.
bin/dev-kit	Adds home command behavior + progressive output + module path loading + proc cache cleanup.
bin/completions/dev.kit.bash	Adds completion handling for action --json.
assets/dev-kit-bridge.svg	Removes legacy diagram asset.
assets/compliance-improve.svg	Removes legacy diagram asset.
README.md	Rewrites README around env/repo/agent model and updated docs links.
Makefile	Adds local/worker test targets.
.gitignore	Ignores repo manifest and agent artifacts.
.githooks/pre-push	Disables pre-push hook.

Comments suppressed due to low confidence (6)

lib/modules/utils.sh:1

• dev_kit_cache_set appends new values but dev_kit_cache_get returns the first matching key (due to exit), so repeated sets for the same key will return stale data. Consider either (a) making cache_get return the last match, or (b) making cache_set rewrite the cache file while removing older entries for that key (atomic temp-file + mv) to keep correctness and avoid unbounded growth.
  lib/modules/repo_signals.sh:1
• The repo-root cache compares DEV_KIT_REPO_ROOT_CACHE_INPUT against the normalized repo_dir (absolute path), but stores "$input_dir" when caching. If the caller passes a relative path or a different-but-equivalent path, the cache will never hit. Store the same normalized path you compare against (e.g., the resolved repo_dir used for the search) to make caching effective and predictable.
  lib/modules/repo_workflows.sh:1
• The JSON emitted for refs_json does not escape values; any ref containing quotes, backslashes, or other JSON-sensitive characters will produce invalid JSON. Prefer building the JSON array via existing helpers that JSON-escape (e.g., produce newline-delimited refs then pipe through dev_kit_lines_to_json_array), or apply escaping per element before printing.
  lib/modules/repo_scaffold.sh:1
• The scaffold planner does not use repo-scaffold.yaml or the archetype argument, so --scaffold currently only plans docs/ (and any explicitly passed extras) rather than the baseline/archetype/factor-driven structure described in config and the PR description. To make --scaffold functional, build the plan from src/configs/repo-scaffold.yaml (baseline + archetype + factor-driven items) and include both directory and file creation actions.
  lib/modules/repo_scaffold.sh:1
• Factor IDs are hardcoded here (and similarly in lib/commands/repo.sh). This risks drift if factors are renamed/added elsewhere. Prefer iterating a single canonical source (e.g., dev_kit_repo_factor_ids) to keep scaffold gaps and CLI output aligned without duplicating lists.
  lib/modules/repo_navigation.sh:1
• repo_navigation.sh defines dev_kit_cache_get / dev_kit_cache_put with different semantics/signatures than the per-process cache helpers added in lib/modules/utils.sh. If repo_navigation.sh is ever sourced alongside utils, these names will collide and break callers. Renaming these to something module-scoped (e.g., dev_kit_mem_cache_get/put or dev_kit_repo_nav_cache_*) would avoid hard-to-debug runtime overrides.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 79 out of 87 changed files in this pull request and generated 12 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 79 out of 87 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 76 out of 84 changed files in this pull request and generated 7 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

learn-last-run looks like ephemeral state (an epoch timestamp) that will change on every dev.kit learn run and will cause constant working-tree churn. Consider treating this as a purely local artifact (gitignore it and/or store it under $DEV_KIT_HOME) rather than committing it in-repo.

Suggested change

1776175365

[Copilot]

prune_dirs contains duplicates (e.g., dist and .next appear twice). This doesn’t change behavior but adds noise and longer find expressions. Consider deduplicating the list to keep the config easier to maintain.

[Copilot]

This doc references dev_kit_catalog_value(), but that function does not exist in lib/modules/config_catalog.sh (parsing is done via functions like dev_kit_yaml_config_list, dev_kit_yaml_mapping_scalar, etc.). Please update the documentation to reference the actual parsing entrypoints so readers can trace the code correctly.

Suggested change

	All catalogs use the same schema: `kind`, `version`, `config`. Parsed by `lib/modules/config_catalog.sh` via `dev_kit_catalog_value()`.
	All catalogs use the same schema: `kind`, `version`, `config`. Parsed by `lib/modules/config_catalog.sh` through the YAML helper entrypoints such as `dev_kit_yaml_config_list` and `dev_kit_yaml_mapping_scalar`.

[Copilot]

This fixture JSONL embeds a developer-local absolute path in payload.cwd (/Users/...). That leaks local filesystem structure into the repo and makes fixtures less portable. Please replace it with a neutral placeholder path (or omit the field) while keeping the fixture semantics intact.

[Copilot]

This README example output includes make build / make run. Because dev.kit discovers build/run commands by regex-scanning markdown, these example strings can get mis-detected as the repo’s real entrypoints (and they currently show up in generated context for this repo). Consider changing the example to avoid matching real command patterns (or adjust detection to require an explicit commands contract) so docs don’t create false positives.

Suggested change

	verify: make test
	build: make build
	run: make run
	verify: <project verify command>
	build: <project build command>
	run: <project run command>

[Copilot]

AGENTS.md advertises make build and make run, but this repository’s Makefile does not define build or run targets (only test, test-docker, eval, etc.). This appears to be a false positive coming from markdown example parsing, and it will mislead agents/humans following the contract. Either add real build/run targets or adjust command detection/docs so the generated contract only lists executable commands.

Suggested change

	build: make build
	run: make run

[Copilot]

The generated .rabbit/context.yaml lists build: make build and run: make run, but this repo’s Makefile does not have build or run targets. Since this file is the canonical handoff artifact, the command contract needs to be accurate; otherwise downstream dev.kit agent output and automation will drift. Fix by adding those targets or tightening detection so example docs don’t get promoted into commands.

Suggested change

	build: make build
	run: make run