Ship context compiler and automatic project upgrades

[JimmyMcBride]

Summary

• make brain context compile the primary summary-first context surface and ship the supporting compiler, boundary graph, verification recipe, telemetry, explain, stats, and promotion flows
• bundle the Brain skill into the running binary, keep installs copy-only, and refresh existing Brain skills during update/install flows
• add automatic project soft migrations with a repo-local ledger, update-time migration for the current repo, lazy repair on later project entry, and visible migration status/remediation in brain update and brain doctor
• tighten bootstrap and adoption so fresh Brain repos start as migration-current instead of appearing pending until a later preflight runs
• update docs, maintainer workflow notes, and skills/brain/SKILL.md to teach the compiler-era workflow and automatic project-upgrade lifecycle

Testing

• go test ./internal/projectcontext ./cmd
• brain session run --project . -- go test ./...
• brain session run --project . -- go build ./...
• go run . skills install --scope local --agent codex --agent openclaw --project .
• go run . context migrate --project .

[Copilot]

Pull request overview

This PR completes the “compiler-era” workflow shift by making brain context compile the primary context surface, adding compiler-supporting structure/telemetry/promotion flows, and wiring automatic project soft-migrations into update + project-entry preflight.

Changes:

• Add a normalized boundary graph (BoundaryGraph) and extend live context collection with boundary responsibilities/adjacency plus derived verification recipes.
• Add promotion review/suggestion plumbing (session closeout suggestions + distill now renders promotion review) and human renderers for compiler telemetry (context explain/stats).
• Add automatic project soft-migrations (run during brain update, and lazily during project-entry preflight) and update docs/skills/guidance to match the new model.

Reviewed changes

Copilot reviewed 119 out of 119 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
skills/brain/SKILL.md	Updates Brain skill guidance to center context compile, explain/stats, promotions, and upgrade workflow.
internal/structure/manager_test.go	Adds coverage ensuring boundary graph exposes compiler-facing relations.
internal/structure/boundary_graph.go	Introduces normalized boundary graph model + file/test/adjacency/responsibility derivation.
internal/session/telemetry_render.go	Adds human renderers for packet explain + context stats.
internal/session/promotion.go	Adds promotion review building + suggestion shaping from session signals.
internal/promotion/promotion_test.go	Adds tests for promotion categories and promotability assessment behavior.
internal/projectcontext/testdata/agents.golden	Updates generated agents contract to reference context compile and promotion-aware closeout.
internal/projectcontext/testdata/agent-integration-updated-codex.golden	Updates agent integration guidance for compile-first workflow and blocked-finish remediation.
internal/projectcontext/testdata/agent-integration-created-claude.golden	Same as above for Claude integration fixture.
internal/projectcontext/testdata/agent-integration-appended-codex.golden	Same as above for appended Codex integration fixture.
internal/projectcontext/packet.go	Adds stable compiled packet schema + hashing.
internal/projectcontext/compiler_test.go	Adds tests for base contract and source-summary item generation.
internal/livecontext/verification_recipes.go	Derives verification recipes from policy/Makefile/package scripts/CI/session history.
internal/livecontext/manager_test.go	Updates tests to use BoundaryGraph and covers recipe derivation.
internal/livecontext/manager.go	Switches structural input to BoundaryGraph, adds recipe output, enriches touched boundaries, adjusts nearby test derivation.
internal/distill/manager_test.go	Updates distill test expectations to promotion-review + promotable targets.
internal/distill/manager.go	Refactors distill-from-session to be promotion-review driven and only propose promotable targets.
docs/why.md	Updates positioning language to match the narrower “memory/context” framing.
docs/usage.md	Reframes usage around compile-first context workflow, explain/stats, upgrade/migration lifecycle.
docs/skills.md	Documents new update+migration behavior and maintainer validation steps.
docs/project-workflows.md	Updates repo workflow doc to compile-first + promotion-aware closeout guidance.
docs/project-overview.md	Updates overview wording to reference compiled context.
docs/project-architecture.md	Adds new internal packages (promotion, taskcontext) to architecture map.
docs/architecture.md	Updates high-level architecture narrative for compiler + promotion + optional workflows.
cmd/update.go	Runs skill refresh and then project migrations (via binary runner) and reports migration status.
cmd/skill_refresh.go	Adds project migration runner + summarization; adds lazy migration preflight helper.
cmd/session.go	Surfaces promotion suggestions in session output.
cmd/root.go	Changes preflight gating to include project migrations, not just skill repair.
cmd/read.go	Records packet expansion when reading notes.
cmd/doctor.go	Adds project_migrations doctor check from migration planner state.
AGENTS.md	Updates repo contract to compile-first workflow + promotion-aware closeout messaging.
.brain/resources/references/maintainer-global-refresh.md	Updates maintainer steps to validate migrations + bundled skill from branch binary.
.brain/planning/stories/update-docs-and-the-brain-skill-to-teach-the-compiler-model.md	Adds/updates planning story describing doc/skill compiler-model work.
.brain/planning/stories/tune-closeout-suggestion-ux-and-suppression-rules.md	Adds/updates planning story for closeout suggestion UX + suppression.
.brain/planning/stories/teach-docs-and-the-brain-skill-about-explain-and-stats.md	Adds/updates planning story for explain/stats guidance.
.brain/planning/stories/teach-docs-and-the-brain-skill-about-context-compile.md	Adds/updates planning story for compile guidance.
.brain/planning/stories/teach-docs-and-the-brain-skill-about-automatic-project-upgrades.md	Adds/updates planning story for automatic project upgrades guidance.
.brain/planning/stories/surface-utility-aware-reasons-and-threshold-diagnostics.md	Adds/updates planning story for utility-aware diagnostics work.
.brain/planning/stories/surface-project-migration-status-and-remediation.md	Adds/updates planning story for migration health surfaces and remediation.
.brain/planning/stories/run-project-migrations-during-update-and-project-entry.md	Adds/updates planning story for update + lazy preflight migration flow.
.brain/planning/stories/render-packet-output-with-provenance-ambiguities-and-verification-hints.md	Adds/updates planning story for packet output requirements.
.brain/planning/stories/reframe-cli-help-and-compatibility-guidance-around-context-compile.md	Adds/updates planning story for UX migration to compile-first.
.brain/planning/stories/record-packet-expansion-verification-and-closeout-linkage.md	Adds/updates planning story for telemetry linkage.
.brain/planning/stories/record-packet-composition-during-context-compile.md	Adds/updates planning story for compile recording.
.brain/planning/stories/normalize-structural-output-into-compiler-boundary-records.md	Adds/updates planning story for boundary graph normalization.
.brain/planning/stories/link-durable-notes-and-generated-context-to-boundaries.md	Adds/updates planning story for boundary-aware note selection.
.brain/planning/stories/integrate-verification-and-test-surfaces-into-compiled-packets.md	Adds/updates planning story for verification/test integration.
.brain/planning/stories/integrate-promotion-gate-with-durable-update-workflows.md	Adds/updates planning story for promotion gate integration.
.brain/planning/stories/implement-first-wave-project-soft-migrations.md	Adds/updates planning story for migrations implementation.
.brain/planning/stories/implement-first-wave-candidate-gathering-and-deterministic-selection.md	Adds/updates planning story for deterministic selection.
.brain/planning/stories/generate-summary-and-anchor-forms-for-major-context-sources.md	Adds/updates planning story for summary+anchor generation.
.brain/planning/stories/extract-verification-recipes-from-repo-and-successful-command-sources.md	Adds/updates planning story for recipe extraction.
.brain/planning/stories/extend-session-state-for-packet-records.md	Adds/updates planning story for session packet record extensions.
.brain/planning/stories/expose-boundary-adjacency-and-responsibilities-to-compiler-consumers.md	Adds/updates planning story for adjacency/responsibilities exposure.
.brain/planning/stories/derive-nearby-tests-from-changed-files-and-boundaries.md	Adds/updates planning story for nearby test derivation.
.brain/planning/stories/define-promotable-and-non-promotable-categories-for-compiler-era-brain.md	Adds/updates planning story for promotion categories.
.brain/planning/stories/connect-closeout-suggestions-to-packet-history-verification-and-durable-state.md	Adds/updates planning story for closeout suggestion grounding.
.brain/planning/stories/build-tiny-base-contract-extraction.md	Adds/updates planning story for base contract extraction.
.brain/planning/stories/build-file-to-boundary-and-boundary-to-test-relations.md	Adds/updates planning story for boundary/test relations.
.brain/planning/stories/blend-utility-with-deterministic-task-boundary-and-workflow-signals.md	Adds/updates planning story for blending utility signals.
.brain/planning/stories/balance-multi-boundary-packets-and-surface-boundary-diagnostics.md	Adds/updates planning story for multi-boundary balancing.
.brain/planning/stories/audit-generated-guidance-and-reduce-static-level-language.md	Adds/updates planning story for removing level-centric wording.
.brain/planning/stories/add-promotion-gate-classification-and-diagnostics.md	Adds/updates planning story for promotion gate diagnostics.
.brain/planning/stories/add-project-migration-ledger-and-planner.md	Adds/updates planning story for migration ledger/planner.
.brain/planning/stories/add-local-packet-telemetry-event-model-and-storage.md	Adds/updates planning story for telemetry storage.
.brain/planning/stories/add-deterministic-packet-recording-tests-and-no-session-fallback-behavior.md	Adds/updates planning story for packet-recording test coverage.
.brain/planning/stories/add-context-stats-for-signal-noise-and-expansion-patterns.md	Adds/updates planning story for stats.
.brain/planning/stories/add-context-explain-surfaces-for-packet-rationale.md	Adds/updates planning story for explain.
.brain/planning/stories/add-context-compile-command-and-stable-packet-schema.md	Adds/updates planning story for compile command + schema.
.brain/planning/stories/add-conservative-utility-wording-and-diagnostic-fields.md	Adds/updates planning story for conservative utility wording.
.brain/planning/stories/add-conservative-utility-and-noise-scoring-inputs.md	Adds/updates planning story for utility/noise inputs.
.brain/planning/stories/add-compiler-context-item-and-anchor-types.md	Adds/updates planning story for context item + anchor types.
.brain/planning/stories/add-closeout-time-promotion-suggestion-logic.md	Adds/updates planning story for closeout promotion suggestions.
.brain/planning/stories/add-bounded-telemetry-retention-and-linkage-tests.md	Adds/updates planning story for bounded telemetry tests.
.brain/planning/stories/add-boundary-aware-candidate-generation-inputs.md	Adds/updates planning story for boundary-aware candidates.
.brain/planning/specs/v4-session-closeout-promotion-suggestions.md	Adds promotion-suggestion spec.
.brain/planning/specs/v4-promotion-gate-and-durable-promotion-types.md	Adds promotion gate + category spec.
.brain/planning/specs/v4-context-compiler-ux-migration.md	Adds UX migration spec.
.brain/planning/specs/v3-utility-aware-context-ranking.md	Adds utility-aware ranking spec.
.brain/planning/specs/v3-context-utility-analysis-surfaces.md	Adds explain/stats spec.
.brain/planning/specs/v3-context-packet-telemetry.md	Adds telemetry spec.
.brain/planning/specs/v2-verification-and-test-surface-derivation.md	Adds verification/test derivation spec.
.brain/planning/specs/v2-repo-boundary-graph.md	Adds boundary graph spec.
.brain/planning/specs/v2-boundary-aware-context-selection.md	Adds boundary-aware selection spec.
.brain/planning/specs/v1-packet-provenance-and-session-recording.md	Adds packet provenance spec.
.brain/planning/specs/v1-context-packet-compiler.md	Adds packet compiler spec.
.brain/planning/specs/v1-base-contract-and-summary-anchors.md	Adds base contract + summary/anchor spec.
.brain/planning/specs/release-install-and-update-flow.md	Updates lifecycle spec to include project migrations.
.brain/planning/epics/v4-session-closeout-promotion-suggestions.md	Adds epic tracking for closeout suggestions.
.brain/planning/epics/v4-promotion-gate-and-durable-promotion-types.md	Adds epic tracking for promotion gate.
.brain/planning/epics/v4-context-compiler-ux-migration.md	Adds epic tracking for compile-first UX migration.
.brain/planning/epics/v3-utility-aware-context-ranking.md	Adds epic tracking for utility-aware ranking.
.brain/planning/epics/v3-context-utility-analysis-surfaces.md	Adds epic tracking for explain/stats.
.brain/planning/epics/v3-context-packet-telemetry.md	Adds epic tracking for telemetry.
.brain/planning/epics/v2-verification-and-test-surface-derivation.md	Adds epic tracking for verification/test surfaces.
.brain/planning/epics/v2-repo-boundary-graph.md	Adds epic tracking for boundary graph.
.brain/planning/epics/v2-boundary-aware-context-selection.md	Adds epic tracking for boundary-aware selection.
.brain/planning/epics/v1-packet-provenance-and-session-recording.md	Adds epic tracking for packet provenance.
.brain/planning/epics/v1-context-packet-compiler.md	Adds epic tracking for packet compiler.
.brain/planning/epics/v1-base-contract-and-summary-anchors.md	Adds epic tracking for base contract.
.brain/planning/epics/release-install-and-update-flow.md	Updates lifecycle epic to include project migrations.
.brain/context/workflows.md	Updates repo workflow guidance for compile-first + promotion-aware closeout.
.brain/context/memory-policy.md	Adds non-promotable guidance (speculative/scratch by default).
.brain/context/current-state.md	Updates current state snapshot + local notes (includes future-dated entries).
.brain/context/architecture.md	Updates repo architecture map for new packages.
.brain/brainstorms/context-substrate-direction.md	Adds brainstorm describing the narrower “context substrate” direction.
.brain/brainstorms/context-compiler-rollout.md	Adds brainstorm describing phased rollout plan for compiler/promotion/telemetry.

Comments suppressed due to low confidence (1)

cmd/root.go:211

• Project-repair preflight currently runs for all brain context ... subcommands (top-level command name is context). If a project migration is broken, this preflight will error before brain context refresh / brain context migrate can run, but those commands are part of the documented remediation path. Consider excluding the relevant context remediation subcommands from preflight (or otherwise ensuring preflight cannot block them), so users can still run refresh/migrate to recover a broken migration state.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.