diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 4bbd2dd..bbc8bf1 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -70,9 +70,9 @@ The four layers, brand-named and emitted cross-tool: drift from. Prose rules in CLAUDE.md get acknowledged and then forgotten after compaction; a guard does not. Every enforceable invariant belongs here. - **mcp** — the protocol layer. Forge ships one stdio server (`src/cortex_mcp.js`) - exposing 19 MCP tools: the substrate checks (`substrate_check` / `predict_impact` / - `assumption_gate` / …), memory reads AND writes (`forge_remember`, ledger - ratify/retract), and ops/health — the full table is in docs/GUIDE.md. + exposing 20 MCP tools: the substrate checks (`substrate_check` / `predict_impact` / + `assumption_gate` / `rank_code` / …), memory reads AND writes (`forge_remember`, + ledger ratify/retract), and ops/health — the full table is in docs/GUIDE.md. Cross-cutting concerns thread through all four: **atlas** (the code graph), **lean** (minimalism — shipped as _both_ a tool and a Stop-guard, so it applies whether or not diff --git a/CHANGELOG.md b/CHANGELOG.md index 081a43d..f628dd4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,47 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +### Added + +- **`forge rank` — load-bearing code, measured.** Weighted PageRank centrality over the + atlas graph (same edge priors as the blast-radius search), Tarjan SCC circular-import + clusters over the directed import graph, and Hopcroft–Tarjan articulation points + (chokepoint files whose removal splits the repo) — joined with each file's + past-incident history from the evidence ledger: `hazard = centrality × (1 + history)`, + where history is the val()-weighted sum of lesson and session claims naming the file. + Structurally central code that has already bitten the team outranks equally central + code that hasn't. Deterministic end to end (sorted-order power iteration, no + `Math.random`), fail-open without a ledger, and exposed to every MCP-capable agent as + the `rank_code` tool (20 MCP tools total). + +- **Time-travel for team memory.** The ledger is append-only and every record carries + its day, so past beliefs are recomputable — now they are queryable: `forge ledger at +` rebuilds any past day's state with `val` scored by that day's evidence and + clock, and `forge ledger diff []` classifies what changed between two + days (appeared / retired / strengthened / weakened, with an epsilon floor). Pure + functions in the ledger core (`stateAt` is a lattice morphism — it commutes with the + CRDT merge, property-tested), no new storage, no clock reads. +- **Merkle state root.** `stateRoot()` hashes the whole verified ledger state into one + permutation-invariant root (leaf per claim over its logs in canonical order, shard + hashes over the store's 2-hex-char prefixes — so divergence is localized, not just + detected). Surfaced as `forge ledger root` and used by `ledger sync --dir` as an + O(state-read) already-in-sync fast path — the ref transport's tree-SHA equality + already was this check; now the dir transport has one too. + +### Changed + +- **`impact()` dequeues in O(1).** The label-correcting blast-radius search in + `src/atlas.js` drained its frontier with `queue.shift()` — O(n) per dequeue on V8 + arrays, quadratic on large frontiers — and rescanned the start set with a linear + `includes` inside the inner loop. The queue now drains through an index pointer and + the start set is a `Set`; processing order, and therefore every reported confidence, + is unchanged. A new test pins the max-product diamond semantics any future rewrite + must preserve. +- **Lesson glob compilation is memoized.** `matchScore` runs per (lesson × file) on + every PreToolUse hook and recompiled the same trigger-glob RegExp each time; compiled + globs are now cached in a module-level map bounded by the distinct globs in the + lesson set. + ## [0.27.4] - 2026-08-04 ### Fixed diff --git a/README.md b/README.md index 5bfdcda..1f75552 100644 --- a/README.md +++ b/README.md @@ -188,7 +188,7 @@ git pull && forge ledger merge On Claude Code the substrate then runs on **every prompt automatically** via a `UserPromptSubmit` hook — advisory only, silent on clean tasks. Every other tool gets a -native config rule plus **19 MCP tools** it can call itself — pre-action checks +native config rule plus **20 MCP tools** it can call itself — pre-action checks (`substrate_check`, `predict_impact`, `assumption_gate`, `route_task`, `scope_files`), memory reads and writes, and ops/health — the full list with schemas is in [`docs/GUIDE.md`](docs/GUIDE.md#mcp-tools). @@ -223,7 +223,7 @@ that never clobbers your existing settings (skip it with `install.sh --no-settin | | `forge harden` | wire the pre-commit gate (gitleaks + commit gate) + sandbox settings | | | `forge catalog` | Start-Here index of every tool / crew / guard | | | `forge brand` | print the brand token map | -| **Memory & team** | `forge ledger` | proof-carrying memory — stats / verify / show / blame / query / ratify / retract / merge / sync / import | +| **Memory & team** | `forge ledger` | proof-carrying memory — stats / verify / show / blame / query / at / diff / root / ratify / retract / merge / sync / import | | | `forge recall` | cross-session personal memory — list / add / consolidate | | | `forge remember` | durable, repo-committable fact | | | `forge brain` | portable project-memory index | @@ -237,6 +237,7 @@ that never clobbers your existing settings (skip it with `install.sh --no-settin | | `forge preflight` | assumption / info-gap check | | | `forge route` | cheapest capable model tier (`route gateway` emits LiteLLM config) | | | `forge impact` | predict blast radius for a symbol or file | +| | `forge rank` | load-bearing code — PageRank centrality × past-incident history, circular imports, chokepoint files | | | `forge scope` | cluster + surface coupled files | | | `forge imagine` | consequence sim + minimal dry-run suite (`--run` executes it sandboxed) | | | `forge context` | budgeted context assembly + completeness gate | diff --git a/ROADMAP.md b/ROADMAP.md index 20fe96c..641e034 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -38,7 +38,7 @@ confidence only from independent oracles, and merges across teammates conflict-f exposing the complexity tiers as model aliases; point `ANTHROPIC_BASE_URL` at the proxy and every model call routes through it. - **MCP server** — the cortex MCP server (`src/cortex_mcp.js`) exposes read-path - tools for ledger, brain, atlas, recall, cost, substrate, and dashboard (19 MCP tools + tools for ledger, brain, atlas, recall, cost, substrate, and dashboard (20 MCP tools as of 0.8.x, including the write tools added in 0.8.0). - **Cost dashboard** — `forge dash` serves a local HTML dashboard showing model spend, event timeline, and ledger health from `.forge/` data. diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 83f98b3..b7671fb 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -25,15 +25,15 @@ recipes, and how to extend each piece. If you just want to get going, the Every command is real and wired. Grouped by what it does: -| Group | Commands | -| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Config / cross-tool sync** | `forge init` · `forge sync` · `forge tools` · `forge doctor` · `forge update` · `forge docs` · `forge config` · `forge harden` · `forge catalog` · `forge brand` | -| **Memory & ledger (PCM)** | `forge ledger` · `forge recall` · `forge remember` · `forge brain` · `forge cortex` · `forge reuse` · `forge handoff` · `forge decide` · `forge know` | -| **Code graph & retrieval** | `forge atlas` · `forge stack` · `forge context` | -| **Substrate / pre-action** | `forge substrate` · `forge preflight` · `forge route` · `forge impact` · `forge scope` · `forge imagine` · `forge anchor` · `forge diagnose` · `forge lean` · `forge cost` | -| **Verification & safety** | `forge verify` · `forge precommit` · `forge radar` · `forge scan` · `forge spec` | -| **UI / design** | `forge taste` · `forge uicheck` | -| **Dashboard** | `forge dash` | +| Group | Commands | +| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Config / cross-tool sync** | `forge init` · `forge sync` · `forge tools` · `forge doctor` · `forge update` · `forge docs` · `forge config` · `forge harden` · `forge catalog` · `forge brand` | +| **Memory & ledger (PCM)** | `forge ledger` · `forge recall` · `forge remember` · `forge brain` · `forge cortex` · `forge reuse` · `forge handoff` · `forge decide` · `forge know` | +| **Code graph & retrieval** | `forge atlas` · `forge stack` · `forge context` | +| **Substrate / pre-action** | `forge substrate` · `forge preflight` · `forge route` · `forge impact` · `forge rank` · `forge scope` · `forge imagine` · `forge anchor` · `forge diagnose` · `forge lean` · `forge cost` | +| **Verification & safety** | `forge verify` · `forge precommit` · `forge radar` · `forge scan` · `forge spec` | +| **UI / design** | `forge taste` · `forge uicheck` | +| **Dashboard** | `forge dash` | Storage in one line: the code graph is `.forge/atlas.json` (plain JSON, not SQLite); the ledger is content-addressed claims under `.forge/ledger/` (git-committable, union-merge). @@ -282,6 +282,38 @@ Forge impact — blast radius - src/session.js ``` +### `forge rank` — what here is dangerous to touch? + +The standing companion to `forge impact`: impact answers "what breaks if I change X", +rank answers "which X-es should I worry about at all". Weighted PageRank over the atlas +graph scores structural centrality (using the same edge weights the blast-radius search +trusts), Tarjan SCC finds circular-import clusters, articulation points find chokepoint +files whose removal would split the import graph — and the ledger join is the part +nobody else has: each file's past-incident history (val()-weighted lesson and session +claims that name it) multiplies into `hazard = centrality × (1 + history)`, so central +code that has already bitten the team outranks equally central code that hasn't. Run +`forge atlas build` first. Also exposed to every MCP-capable agent as `rank_code`. + +```console +$ forge rank --top 3 + graph: 9285 nodes, 27083 edges + + files (hazard = centrality × 1+history): + ████████ 1.000 src/cli.js + ███████░ 0.840 src/ledger.js + ██████░░ 0.718 src/ledger_store.js + + symbols (centrality): + 0.004878 epochDay src/util.js + 0.004005 sync src/sync.js + 0.003897 loadClaims src/ledger_store.js + + circular imports: none + + chokepoints (removal splits the import graph): + src/sync.js splits off 10 subtree(s) +``` + ### `forge scope ` — can this be split into sessions? Groups the files you name into independent clusters and surfaces coupled files you @@ -720,6 +752,23 @@ Forge ledger blame — lesson 3f2a91c04d7e 0.93 juber ``` +The store is append-only and every record carries its day, so **time-travel is +recomputed, never guessed**: `forge ledger at ` rebuilds the beliefs of any past +day — which claims existed, with `val` scored by _that_ day's evidence and clock — and +`forge ledger diff []` reports what changed between two days: claims +that appeared, were retired, strengthened, or weakened. `forge ledger root` prints one +Merkle root over the whole verified state (per-shard hashes localize any divergence); +two replicas share a root exactly when they share a state, which is also the fast path +`ledger sync --dir` uses to skip merges that have nothing to do. + +```console +$ forge ledger diff 2026-07-01 + appeared 2 · retired 1 · strengthened 1 · weakened 0 + new fact b19b2961 · → 0.55 demo shared + gone lesson 3f2a91c0 0.82 → · stale port rule + up fact 88ac02d1 0.50 → 0.63 build needs node 20 +``` + The rest of the surface, briefly: `forge ledger merge ` folds in any other ledger tree (a teammate's checkout, a worktree, a backup) — `merged: 3 new claim(s), 5 new record(s) — conflict-free`, in any order; `query ""` ranks live claims by the @@ -1152,7 +1201,7 @@ one extra turn, exactly when that turn was owed. > `forge substrate "" --json` (or the MCP tool `substrate_check`). If > `okToProceed` is false, ask the questions first; read `impact.impactedFiles` before editing. -…and exposes the substrate as **19 MCP tools** any MCP-capable agent can call directly +…and exposes the substrate as **20 MCP tools** any MCP-capable agent can call directly (the stdio server is launched with `forge cortex-mcp`, wired automatically via the emitted `.mcp.json`): @@ -1173,6 +1222,7 @@ emitted `.mcp.json`): | `forge_remember` | **write**: add a durable project fact | | `forge_ledger_ratify` | **write**: human-ratify a claim into a decision | | `forge_ledger_retract` | **write**: tombstone a claim | +| `rank_code` | load-bearing files/symbols, cycles, chokepoints | | `forge_diagnose` | doom-loop failure check | | `forge_doctor` | health check | | `forge_provider_status` | provider detection + gateway reachability | diff --git a/mintlify/cli/memory.mdx b/mintlify/cli/memory.mdx index 2ecdba7..e715767 100644 --- a/mintlify/cli/memory.mdx +++ b/mintlify/cli/memory.mdx @@ -53,6 +53,9 @@ forge ledger verify # re-check claims are in normal form forge ledger show # a claim and its evidence forge ledger blame # who minted it, every oracle outcome, per-author trust forge ledger query "" # retrieve by relevance +forge ledger at # beliefs as of any past day — recomputed, never guessed +forge ledger diff [] # appeared / retired / strengthened / weakened +forge ledger root # Merkle root over the verified state (sync fast path) forge ledger ratify # human accept forge ledger retract # tombstone a claim forge ledger merge # fold a teammate's ledger in, conflict-free diff --git a/mintlify/cli/substrate.mdx b/mintlify/cli/substrate.mdx index bd49a24..9d81825 100644 --- a/mintlify/cli/substrate.mdx +++ b/mintlify/cli/substrate.mdx @@ -43,6 +43,17 @@ Predict the blast radius for a symbol or file from the atlas graph. forge impact ``` +## `forge rank` + +Load-bearing code: weighted PageRank centrality over the atlas graph joined with +past-incident history from the evidence ledger (`hazard = centrality × 1+history`), +plus circular-import clusters (Tarjan SCC) and chokepoint files (articulation points). +Also exposed as the `rank_code` MCP tool. + +```bash +forge rank [--top ] [--json] +``` + ## `forge scope` Decompose files into independent clusters — plus coupled files you didn't name. diff --git a/mintlify/concepts/config-compiler.mdx b/mintlify/concepts/config-compiler.mdx index bbb1d18..311b9ba 100644 --- a/mintlify/concepts/config-compiler.mdx +++ b/mintlify/concepts/config-compiler.mdx @@ -53,7 +53,7 @@ Each layer is brand-named and emitted cross-tool. does not. Every enforceable invariant belongs here. - Forge ships one stdio server (`src/cortex_mcp.js`) exposing 19 MCP tools: the + Forge ships one stdio server (`src/cortex_mcp.js`) exposing 20 MCP tools: the substrate checks (`substrate_check` / `predict_impact` / `assumption_gate` / …), memory reads _and_ writes (`forge_remember`, ledger ratify/retract), and ops/health. diff --git a/mintlify/quickstart.mdx b/mintlify/quickstart.mdx index fb456b5..9869d29 100644 --- a/mintlify/quickstart.mdx +++ b/mintlify/quickstart.mdx @@ -70,7 +70,7 @@ forge substrate "Change verifyToken in src/auth.js to require length > 20; updat On Claude Code the substrate runs on **every prompt automatically** via a `UserPromptSubmit` hook — advisory only, silent on clean tasks. Every other tool gets - a native config rule plus 19 MCP tools it can call itself. + a native config rule plus 20 MCP tools it can call itself. If `forge substrate` says `ASK FIRST`, ask the returned questions before editing. Read diff --git a/src/atlas.js b/src/atlas.js index e2feb4e..0cdb09d 100644 --- a/src/atlas.js +++ b/src/atlas.js @@ -588,7 +588,10 @@ function targetIds(atlas, target) { return matches.map((n) => n.id); } -const EDGE_WEIGHT = { +// Exported for rank.js — PageRank centrality weights edges with the same priors the +// blast-radius search uses, so "load-bearing" and "impacted" can never disagree on +// what an edge kind is worth. +export const EDGE_WEIGHT = { calls: 0.95, imports: 0.85, inherits: 0.92, @@ -664,6 +667,7 @@ export function impact( { threshold = 0.1, maxHops = 6, decay = 0.85, llm, run, verify } = {}, ) { const starts = targetIds(atlas, target); + const startSet = new Set(starts); const { nodeById, incoming } = adjacency(atlas); const visited = new Map(); const queue = starts.map((id) => ({ @@ -673,11 +677,18 @@ export function impact( path: [id], edgeKinds: [], })); - while (queue.length) { - const current = queue.shift(); + // Label-correcting search: a node re-enters the queue whenever a better path is + // found, so the loop converges to the max-product confidence. The queue is drained + // with an index pointer (queue.shift() is O(n) on V8 arrays — quadratic on large + // frontiers). A heap-based best-first variant is the upgrade seam if graphs ever + // outgrow this; it would change processing order, so it must re-prove the diamond + // max-product test before landing. + let head = 0; + while (head < queue.length) { + const current = queue[head++]; if (!current || current.hop >= maxHops) continue; for (const edge of incoming.get(current.id) || []) { - if (starts.includes(edge.source)) continue; + if (startSet.has(edge.source)) continue; const nextConfidence = current.confidence * (EDGE_WEIGHT[edge.kind] || 0.5) * (edge.confidence ?? 1) * decay; if (nextConfidence < threshold) continue; diff --git a/src/cli.js b/src/cli.js index 5ea44f2..f4a0d27 100755 --- a/src/cli.js +++ b/src/cli.js @@ -857,6 +857,79 @@ HANDLERS.ledger = async (argv) => { ); return; } + // `at` / `diff` / `root` — the temporal surface. The store is append-only and every + // record carries its day, so a past day's beliefs are recomputed, never guessed. + const parseDay = (s) => { + if (/^\d{1,6}$/.test(s ?? "")) return Number(s); // bare epoch-day + const t = Date.parse(`${s}T00:00:00Z`); + return Number.isNaN(t) ? null : Math.floor(t / 86_400_000); + }; + if (sub === "at") { + const day = parseDay(args[2]); + if (day === null) { + console.error(`usage: ${BRAND.cli} ledger at [--json]`); + process.exitCode = 1; + return; + } + const lg = await import("./ledger.js"); + const live = lg.liveClaims(lg.stateAt(ls.loadState(dir), day)); + const rows = live + .map((c) => ({ + id: c.id, + kind: c.kind, + val: Number(lg.val(c, day).toFixed(4)), + tombstoned: Boolean(c.tombstone), + text: lg.claimText(c).slice(0, 90), + })) + .sort((a, b) => b.val - a.val || (a.id < b.id ? -1 : 1)); + if (json) return console.log(JSON.stringify({ day, claims: rows.length, rows }, null, 2)); + heading(`${BRAND.brand} ledger — beliefs as of day ${day}\n`); + const byKind = {}; + for (const r of rows) byKind[r.kind] = (byKind[r.kind] ?? 0) + 1; + console.log( + ` claims: ${rows.length} (${rows.filter((r) => r.tombstoned).length} tombstoned)`, + ); + for (const [kind, n] of Object.entries(byKind)) console.log(` ${kind}: ${n}`); + for (const r of rows.slice(0, 10)) + console.log( + ` ${bar(r.val, 8)} ${r.val.toFixed(3)} ${paint(r.kind.padEnd(9), "accent")} ${paint(r.id.slice(0, 8), "dim")} ${r.text}`, + ); + return; + } + if (sub === "diff") { + const a = parseDay(args[2]); + const b = args[3] ? parseDay(args[3]) : nowDay; + if (a === null || b === null) { + console.error( + `usage: ${BRAND.cli} ledger diff [] [--json]`, + ); + process.exitCode = 1; + return; + } + const lg = await import("./ledger.js"); + const d = lg.beliefDiff(ls.loadState(dir), a, b); + if (json) return console.log(JSON.stringify({ since: a, until: b, ...d }, null, 2)); + heading(`${BRAND.brand} ledger — what changed, day ${a} → ${b}\n`); + console.log( + ` appeared ${d.appeared.length} · retired ${d.retired.length} · ${paint(`strengthened ${d.strengthened.length}`, "ok")} · ${paint(`weakened ${d.weakened.length}`, "warn")}`, + ); + const row = (label, r) => + console.log( + ` ${label} ${paint(r.kind.padEnd(9), "accent")} ${paint(r.id.slice(0, 8), "dim")} ${r.from === null ? "· " : r.from.toFixed(2)} → ${r.to === null ? "·" : r.to.toFixed(2)} ${r.text.slice(0, 70)}`, + ); + for (const r of d.appeared.slice(0, 5)) row(paint("new ", "ok"), r); + for (const r of d.retired.slice(0, 5)) row(paint("gone", "err"), r); + for (const r of d.strengthened.slice(0, 5)) row(paint("up ", "ok"), r); + for (const r of d.weakened.slice(0, 5)) row(paint("down", "warn"), r); + return; + } + if (sub === "root") { + const lg = await import("./ledger.js"); + const r = lg.stateRoot(ls.loadState(dir)); + if (json) return console.log(JSON.stringify(r, null, 2)); + console.log(r.root); // bare hex on stdout — scriptable ("are we in sync?" is one diff) + return; + } if (sub === "sync") { const { ledgerSync, defaultRef } = await import("./ledger_sync.js"); const di = args.indexOf("--dir"); @@ -882,7 +955,12 @@ HANDLERS.ledger = async (argv) => { table([ [paint("target", "dim"), r.dir], [paint("pulled", "dim"), `${r.pulled.claims} claim(s), ${r.pulled.records} record(s)`], - [paint("pushed", "dim"), `${r.pushed.claims} claim(s), ${r.pushed.records} record(s)`], + [ + paint("pushed", "dim"), + r.upToDate + ? paint("up to date — state roots match, nothing to merge", "dim") + : `${r.pushed.claims} claim(s), ${r.pushed.records} record(s)`, + ], ]), ); } else { @@ -929,7 +1007,7 @@ HANDLERS.ledger = async (argv) => { return; } console.error( - `ledger: unknown subcommand "${sub}" (stats | verify | show | blame | query | ratify | retract --reason "" | merge | sync [--dir |--remote |--ref ] | import) [--personal] [--json]`, + `ledger: unknown subcommand "${sub}" (stats | verify | show | blame | query | at | diff [] | root | ratify | retract --reason "" | merge | sync [--dir |--remote |--ref ] | import) [--personal] [--json]`, ); process.exitCode = 1; return; @@ -1115,6 +1193,44 @@ HANDLERS.atlas = async (argv) => { } return; }; +HANDLERS.rank = async (argv) => { + const { rankReport } = await import("./rank.js"); + const json = argv.includes("--json"); + const ti = argv.indexOf("--top"); + const top = ti >= 0 ? Math.max(1, Number(argv[ti + 1]) || 15) : 15; + const r = rankReport(process.cwd(), { top }); + if (!r.built) { + console.error(` no index — run \`${BRAND.cli} atlas build\` first`); + process.exitCode = 1; + return; + } + if (json) return console.log(JSON.stringify(r, null, 2)); + heading(`${BRAND.brand} rank — load-bearing code\n`); + console.log(paint(` graph: ${r.nodes} nodes, ${r.edges} edges`, "dim")); + console.log(paint("\n files (hazard = centrality × 1+history):", "accent")); + const maxHazard = r.topFiles[0]?.hazard || 1; + for (const f of r.topFiles) + console.log( + ` ${bar(f.hazard / maxHazard, 8)} ${f.hazard.toFixed(3)} ${f.file}${ + f.incidents ? paint(` (${f.incidents} past incident(s))`, "warn") : "" + }`, + ); + console.log(paint("\n symbols (centrality):", "accent")); + for (const s of r.topSymbols) + console.log(` ${s.score.toFixed(6)} ${s.name} ${paint(s.file, "dim")}`); + if (r.cycles.length) { + console.log(paint(`\n circular imports: ${r.cycles.length} cluster(s)`, "warn")); + for (const c of r.cycles.slice(0, 5)) console.log(` [${c.length}] ${c.join(" ⇄ ")}`); + } else { + console.log(paint("\n circular imports: none", "dim")); + } + if (r.chokepoints.length) { + console.log(paint("\n chokepoints (removal splits the import graph):", "accent")); + for (const c of r.chokepoints.slice(0, 10)) + console.log(` ${c.file} ${paint(`splits off ${c.splits} subtree(s)`, "dim")}`); + } + return; +}; HANDLERS.scan = async (argv) => { const { scan } = await import("./skillgate.js"); const target = argv[1]; diff --git a/src/commands.js b/src/commands.js index f482128..fb6db8b 100644 --- a/src/commands.js +++ b/src/commands.js @@ -87,13 +87,26 @@ export const COMMANDS = { cortex: "self-correcting project memory — status / why ", deja: "anti-repetition — have you done this task before? ranks prior solved/verified sessions", ledger: - "evidence-referenced memory — stats / verify / show / blame / query / ratify / retract / merge / sync / import", + "evidence-referenced memory — stats / verify / show / blame / query / at / diff / root / ratify / retract / merge / sync / import", reuse: "proof-carrying code cache — query / mint --file / stats", context: "budgeted context assembly + completeness gate — what an edit NEEDS known", preflight: "assumption check — what a task names that the repo doesn't define", config: "provider setup — show / switch / add providers, set default model", route: "recommend the cheapest capable model for a task (+ gateway config)", impact: "predict blast radius for a symbol or file from the atlas graph", + rank: { + summary: + "load-bearing code — PageRank centrality × past-incident history, circular-dependency clusters, chokepoint files", + usage: "forge rank [--top ] [--json]", + flags: [ + { + flag: "--top ", + desc: "how many files/symbols to list (default 15)", + }, + { flag: "--json", desc: "machine-readable full report" }, + ], + examples: ["forge rank", "forge rank --top 5 --json"], + }, substrate: "one pre-action gate: assumptions, route, impact, scope, memory, verify", scope: "decompose files into independent clusters (+ coupled files you didn't name)", anchor: "goal-drift check — are your actual (git) changes still on the stated goal?", @@ -172,6 +185,7 @@ export const GROUPS = { "report", "deja", "reuse", + "rank", ], }; diff --git a/src/cortex_mcp.js b/src/cortex_mcp.js index 0d1a5e2..2166d4e 100644 --- a/src/cortex_mcp.js +++ b/src/cortex_mcp.js @@ -59,7 +59,9 @@ async function callTool(name, args = {}) { return JSON.stringify(assessTask(String(args.task ?? "")), null, 2); if (name === "predict_impact") return JSON.stringify( - predictImpact(root, String(args.target ?? ""), { threshold: Number(args.threshold ?? 0.1) }), + predictImpact(root, String(args.target ?? ""), { + threshold: Number(args.threshold ?? 0.1), + }), null, 2, ); @@ -153,6 +155,10 @@ async function callTool(name, args = {}) { remember(store, String(args.name ?? ""), String(args.body ?? "")); return `Remembered "${args.name}" in ${store}.`; } + if (name === "rank_code") { + const { rankReport } = await import("./rank.js"); + return JSON.stringify(rankReport(root, { top: Number(args.top ?? 15) || 15 }), null, 2); + } if (name === "forge_ledger_ratify") { const { ratify, repoLedger, getClaimByPrefix } = await import("./ledger_store.js"); const { gitAuthor } = await import("./util.js"); diff --git a/src/ledger.js b/src/ledger.js index 2e06924..aa5a0df 100644 --- a/src/ledger.js +++ b/src/ledger.js @@ -617,3 +617,146 @@ export function liveClaims(state) { })) .sort((a, b) => (a.id < b.id ? -1 : 1)); } + +// --------------------------------------------------------------------------- +// Temporal views + Merkle state root. The store is append-only and every record +// carries its day, so any past day's beliefs are RECOMPUTABLE, never guessed — +// and a whole state can be summarized in one permutation-invariant hash. +// --------------------------------------------------------------------------- + +/** + * The state as it stood at end of `day`: claims whose mint day ≤ day — the earliest + * provenance-log record, falling back to the claim's inline mint record, then 0 + * (mint-day-unknown must not hide a claim) — with every log filtered to records with + * t ≤ day. Pure; inputs are not mutated. A lattice morphism: + * stateAt(merge(a,b), d) ≡ merge(stateAt(a,d), stateAt(b,d)) — property-tested next + * to the semilattice suite. + * @param {{claims:any, evidence:any, provenance:any, tombstones:any}} state + * @param {number} day epoch day (inclusive cutoff) + */ +export function stateAt(state, day) { + const cut = (recs = []) => sortRecords(recs).filter((r) => (r.t ?? 0) <= day); + const out = emptyState(); + for (const [id, c] of Object.entries(state.claims ?? {})) { + const provAll = sortRecords(state.provenance?.[id] ?? []); + const mintDay = provAll.length ? (provAll[0].t ?? 0) : (c.provenance?.t ?? 0); + if (mintDay > day) continue; // minted after `day` — didn't exist yet + out.claims[id] = c; + out.provenance[id] = provAll.filter((r) => (r.t ?? 0) <= day); + out.evidence[id] = cut(state.evidence?.[id] ?? []); + out.tombstones[id] = cut(state.tombstones?.[id] ?? []); + } + return out; +} + +/** + * What changed between two days — with val() evaluated AT each day's own clock, so + * this answers "what did we believe then", not "what does today think of then". + * appeared: minted in (dayA, dayB] + * retired: first tombstone lands in (dayA, dayB] + * strengthened: |Δval| ≥ epsilon upward weakened: downward + * @param {{claims:any, evidence:any, provenance:any, tombstones:any}} state + * @param {number} dayA earlier day + * @param {number} dayB later day + * @param {{epsilon?:number, halfLife?:number}} [opts] + * @returns {{appeared:any[], retired:any[], strengthened:any[], weakened:any[]}} + * rows are {id, kind, text, from, to} sorted by |Δval| desc then id asc + */ +export function beliefDiff( + state, + dayA, + dayB, + { epsilon = 0.05, halfLife = DEFAULT_HALF_LIFE_DAYS } = {}, +) { + const before = new Map(liveClaims(stateAt(state, dayA)).map((c) => [c.id, c])); + const after = liveClaims(stateAt(state, dayB)); + const appeared = []; + const retired = []; + const moved = []; + for (const c of after) { + const prev = before.get(c.id); + const text = claimText(c).slice(0, 120); + if (!prev) { + appeared.push({ + id: c.id, + kind: c.kind, + text, + from: null, + to: round4(val(c, dayB, { halfLife })), + }); + continue; + } + if (!prev.tombstone && c.tombstone) { + retired.push({ + id: c.id, + kind: c.kind, + text, + from: round4(val(prev, dayA, { halfLife })), + to: null, + }); + continue; + } + const from = val(prev, dayA, { halfLife }); + const to = val(c, dayB, { halfLife }); + if (Math.abs(to - from) >= epsilon) + moved.push({ + id: c.id, + kind: c.kind, + text, + from: round4(from), + to: round4(to), + }); + } + const byDelta = (a, b) => + Math.abs((b.to ?? 0) - (b.from ?? 0)) - Math.abs((a.to ?? 0) - (a.from ?? 0)) || + (a.id < b.id ? -1 : 1); + appeared.sort(byDelta); + retired.sort((a, b) => (b.from ?? 0) - (a.from ?? 0) || (a.id < b.id ? -1 : 1)); + return { + appeared, + retired, + strengthened: moved.filter((m) => m.to > m.from).sort(byDelta), + weakened: moved.filter((m) => m.to < m.from).sort(byDelta), + }; +} + +const round4 = (x) => Number(x.toFixed(4)); + +/** + * Merkle root over a ledger state. Leaf per claim id = hash of the claim's canonical + * content plus its three logs in sortRecords order; shard hash per 2-hex-char id + * prefix (the store's on-disk sharding) over its sorted "id:leaf" lines; root over + * the sorted "prefix:shardHash" lines. Permutation- and merge-order-invariant by + * construction: two replicas share a root ⇔ their verified states are identical, and + * when they differ the differing shard hashes localize where. + * @param {{claims:any, evidence:any, provenance:any, tombstones:any}} state + * @returns {{root:string, shards:Record, claims:number}} + */ +export function stateRoot(state) { + const leaves = new Map(); // prefix → "id:leafHash" lines + const ids = Object.keys(state.claims ?? {}).sort(); + for (const id of ids) { + const leaf = contentHash( + canonicalize({ + claim: state.claims[id], + evidence: sortRecords(state.evidence?.[id] ?? []), + provenance: sortRecords(state.provenance?.[id] ?? []), + tombstones: sortRecords(state.tombstones?.[id] ?? []), + }), + ); + const prefix = id.slice(0, 2); + if (!leaves.has(prefix)) leaves.set(prefix, []); + leaves.get(prefix).push(`${id}:${leaf}`); + } + /** @type {Record} */ + const shards = {}; + for (const prefix of [...leaves.keys()].sort()) + shards[prefix] = contentHash(leaves.get(prefix).join("\n")); + const root = contentHash( + Object.keys(shards) + .sort() + .map((p) => `${p}:${shards[p]}`) + .join("\n"), + ); + return { root, shards, claims: ids.length }; +} diff --git a/src/ledger_sync.js b/src/ledger_sync.js index b8d8caf..55ca4cf 100644 --- a/src/ledger_sync.js +++ b/src/ledger_sync.js @@ -19,7 +19,7 @@ import { execFileSync } from "node:child_process"; import { existsSync } from "node:fs"; import { BRAND } from "./brand.js"; -import { canonicalize } from "./ledger.js"; +import { canonicalize, stateRoot } from "./ledger.js"; import { importState, loadState, mergeDirs } from "./ledger_store.js"; import { gitAuthor } from "./util.js"; @@ -152,6 +152,21 @@ export function syncDir(localDir, otherDir) { dir: otherDir, reason: `no sync directory at ${otherDir}`, }; + // Merkle fast path: two replicas sharing a state root are already the same + // VERIFIED state, so the (much costlier) per-record union merge has nothing to do. + // Honest limitation: garbage lines that readLog rejects are invisible to the root, + // so equal-modulo-garbage dirs skip the merge without quarantining that garbage — + // `ledger merge` still does. (The ref transport's tree-SHA equality already IS this + // check; only the dir transport lacked one.) + if (stateRoot(loadState(localDir)).root === stateRoot(loadState(otherDir)).root) + return { + ok: true, + mode: "dir", + dir: otherDir, + upToDate: true, + pulled: { claims: 0, records: 0, quarantined: 0 }, + pushed: { claims: 0, records: 0, quarantined: 0 }, + }; const pulled = mergeDirs(localDir, otherDir); const pushed = mergeDirs(otherDir, localDir); return { ok: true, mode: "dir", dir: otherDir, pulled, pushed }; diff --git a/src/lessons.js b/src/lessons.js index 7e9c191..5ef1357 100644 --- a/src/lessons.js +++ b/src/lessons.js @@ -129,12 +129,20 @@ export function contradict(lesson, nowDay) { return next; } -/** Compile a `*` / `**` glob to an anchored RegExp (no sentinel chars — single pass). */ -const globToRe = (glob) => { - const body = glob - .replace(/[.+^${}()|[\]\\]/g, "\\$&") - .replace(/\*\*|\*/g, (m) => (m === "**" ? ".*" : "[^/]*")); - return new RegExp(`^${body}$`); +/** Compile a `*` / `**` glob to an anchored RegExp (no sentinel chars — single pass). + * Memoized: matchScore runs per (lesson × file) on every PreToolUse hook, and the + * cache is bounded in practice by the distinct trigger globs in the lesson set. */ +const GLOB_RE_CACHE = new Map(); +export const globToRe = (glob) => { + let re = GLOB_RE_CACHE.get(glob); + if (!re) { + const body = glob + .replace(/[.+^${}()|[\]\\]/g, "\\$&") + .replace(/\*\*|\*/g, (m) => (m === "**" ? ".*" : "[^/]*")); + re = new RegExp(`^${body}$`); + GLOB_RE_CACHE.set(glob, re); + } + return re; }; // Tokens every path shares — matching on them would make a lesson keyed to @@ -194,7 +202,10 @@ export function selectForInjection(lessons, context, { budget = 12, nowDay = 0 } const val = validity(x.lesson); const scopeW = SCOPE_WEIGHT[x.lesson.scope] ?? 0.5; const recencyBoost = 1 + (nowDay - x.lesson.lastConfirmedDay <= 14 ? 0.2 : 0); - return { lesson: x.lesson, score: rel * rec * val * scopeW * recencyBoost }; + return { + lesson: x.lesson, + score: rel * rec * val * scopeW * recencyBoost, + }; }) .sort((a, b) => b.score - a.score); diff --git a/src/mcp_tools.js b/src/mcp_tools.js index b5b9bcf..4f10491 100644 --- a/src/mcp_tools.js +++ b/src/mcp_tools.js @@ -68,8 +68,14 @@ export const TOOLS = [ inputSchema: { type: "object", properties: { - target: { type: "string", description: "symbol name, qualified name, or file" }, - threshold: { type: "number", description: "confidence threshold, default 0.1" }, + target: { + type: "string", + description: "symbol name, qualified name, or file", + }, + threshold: { + type: "number", + description: "confidence threshold, default 0.1", + }, }, required: ["target"], }, @@ -124,7 +130,10 @@ export const TOOLS = [ inputSchema: { type: "object", properties: { - query: { type: "string", description: "what you are about to do or looking for" }, + query: { + type: "string", + description: "what you are about to do or looking for", + }, }, required: ["query"], }, @@ -162,7 +171,10 @@ export const TOOLS = [ inputSchema: { type: "object", properties: { - name: { type: "string", description: "short slug for the fact (used as filename)" }, + name: { + type: "string", + description: "short slug for the fact (used as filename)", + }, body: { type: "string", description: "the fact content (markdown)" }, }, required: ["name", "body"], @@ -180,6 +192,20 @@ export const TOOLS = [ required: ["id"], }, }, + { + name: "rank_code", + description: + "Which code is load-bearing and dangerous to touch — PageRank centrality over the Forge atlas graph joined with past-incident history from the evidence ledger, plus circular-dependency clusters and chokepoint files whose removal disconnects the import graph.", + inputSchema: { + type: "object", + properties: { + top: { + type: "number", + description: "how many files/symbols to return (default 15)", + }, + }, + }, + }, { name: "forge_ledger_retract", description: @@ -188,7 +214,10 @@ export const TOOLS = [ type: "object", properties: { id: { type: "string", description: "claim ID or unique prefix" }, - reason: { type: "string", description: "why the claim is being retracted" }, + reason: { + type: "string", + description: "why the claim is being retracted", + }, }, required: ["id", "reason"], }, diff --git a/src/rank.js b/src/rank.js new file mode 100644 index 0000000..a7a584c --- /dev/null +++ b/src/rank.js @@ -0,0 +1,319 @@ +// forge rank — load-bearing code, measured. Three classical graph readings of the atlas +// plus one join nobody else has: WHERE the structure says a change propagates widely +// (weighted PageRank centrality), WHERE the dependency graph is knotted (Tarjan SCC → +// circular-dependency clusters), WHERE the import graph would split if a file vanished +// (Hopcroft–Tarjan articulation points), and — the original part — how often each file +// has ALREADY bitten the team, from the evidence ledger (val()-weighted lesson and +// session-summary claims that name it). hazard = centralityNorm × (1 + history): +// structurally central code that has hurt before outranks equally central code that +// hasn't. DATA may be a table; DECISIONS are these formulas. +// +// Determinism: no Math.random anywhere — PageRank is a fixed-order power iteration over +// sorted node ids, ties break by (score desc, id asc), so two machines always print the +// same ranking for the same atlas. Substrate/route integration is a deliberate SEAM: +// rank ships standalone (CLI + MCP) first; feeding hazard into route complexity or the +// substrate advisory is a later, separately-measured step. +import { EDGE_WEIGHT, load } from "./atlas.js"; +import { val } from "./ledger.js"; +import { loadClaims, repoLedger } from "./ledger_store.js"; +import { globToRe } from "./lessons.js"; +import { directedImportGraph } from "./scope.js"; +import { epochDay, toPosix } from "./util.js"; + +/** Node kinds that are containers/artifacts, not symbols — excluded from the symbol view. */ +const NON_SYMBOL_KINDS = new Set(["module", "doc", "config", "unknown"]); + +/** + * Weighted PageRank over the atlas graph. An edge source→target means "source depends + * on target", so the random surfer walks WITH dependency direction and rank accrues to + * what is depended upon. Edge weight = EDGE_WEIGHT[kind] × edge.confidence; out-weights + * are normalized per source; dangling mass is redistributed uniformly. + * @param {{nodes?:any[], edges?:any[]}} atlas + * @param {{damping?:number, maxIter?:number, tol?:number}} [opts] + * @returns {Map} node id → score (scores sum to ~1) + */ +export function pagerank(atlas, { damping = 0.85, maxIter = 60, tol = 1e-9 } = {}) { + const ids = [...new Set((atlas.nodes ?? []).map((n) => n.id))].sort(); + const n = ids.length; + const scores = new Map(); + if (!n) return scores; + const index = new Map(ids.map((id, i) => [id, i])); + // out[i] = [[j, weight]...] in deterministic (source-sorted, insertion) order + const out = ids.map(() => []); + const outWeight = new Float64Array(n); + const sortedEdges = [...(atlas.edges ?? [])].sort( + (a, b) => a.source.localeCompare(b.source) || a.target.localeCompare(b.target), + ); + for (const e of sortedEdges) { + if (e.unresolved) continue; + const s = index.get(e.source); + const t = index.get(e.target); + if (s === undefined || t === undefined || s === t) continue; + const w = (EDGE_WEIGHT[e.kind] ?? 0.5) * (e.confidence ?? 1); + if (w <= 0) continue; + out[s].push([t, w]); + outWeight[s] += w; + } + let r = new Float64Array(n).fill(1 / n); + for (let iter = 0; iter < maxIter; iter++) { + const next = new Float64Array(n).fill((1 - damping) / n); + let dangling = 0; + for (let i = 0; i < n; i++) { + if (!out[i].length) { + dangling += r[i]; + continue; + } + const share = (damping * r[i]) / outWeight[i]; + for (const [t, w] of out[i]) next[t] += share * w; + } + const danglingShare = (damping * dangling) / n; + let l1 = 0; + for (let i = 0; i < n; i++) { + next[i] += danglingShare; + l1 += Math.abs(next[i] - r[i]); + } + r = next; + if (l1 < tol) break; + } + for (let i = 0; i < n; i++) scores.set(ids[i], r[i]); + return scores; +} + +/** + * File- and symbol-level centrality views. A file's score is the sum of its nodes' + * PageRank; the symbol view keeps only definition nodes (functions/classes/types). + * @param {{nodes?:any[], edges?:any[]}} atlas + * @param {{damping?:number, maxIter?:number, tol?:number}} [opts] + * @returns {{files:{file:string,score:number}[], symbols:{id:string,name:string,file:string,score:number}[]}} + */ +export function centrality(atlas, opts) { + const scores = pagerank(atlas, opts); + const byFile = new Map(); + const symbols = []; + for (const node of [...(atlas.nodes ?? [])].sort((a, b) => a.id.localeCompare(b.id))) { + const s = scores.get(node.id) ?? 0; + if (node.file) byFile.set(node.file, (byFile.get(node.file) ?? 0) + s); + if (!NON_SYMBOL_KINDS.has(node.kind) && node.name && node.file) + symbols.push({ id: node.id, name: node.name, file: node.file, score: s }); + } + const files = [...byFile.entries()].map(([file, score]) => ({ file, score })); + files.sort((a, b) => b.score - a.score || a.file.localeCompare(b.file)); + symbols.sort((a, b) => b.score - a.score || a.id.localeCompare(b.id)); + return { files, symbols }; +} + +/** + * Circular-import clusters: Tarjan's strongly connected components (iterative — no + * recursion, deep chains can't blow the stack) over the DIRECTED import graph from + * scope.directedImportGraph. Deliberately NOT the atlas call edges: unique-name call + * resolution is too noisy across files (one coincidental name match glues unrelated + * files into a mega-component — measured on this very repo). Import statements are + * the ground truth for "circular dependency". Components of ≥2 files are cycles. + * @param {{nodes:string[], edges:Map>}} graph directedImportGraph output + * @returns {string[][]} each component sorted; list sorted by size desc, then first file + */ +export function cycles(graph) { + const adj = graph.edges; + const files = [...graph.nodes].sort(); + const disc = new Map(); + const low = new Map(); + const onStack = new Set(); + const stack = []; + const comps = []; + let clock = 0; + for (const start of files) { + if (disc.has(start)) continue; + /** @type {{v:string, i:number}[]} iterative Tarjan frames: node + neighbor cursor */ + const frames = [{ v: start, i: 0 }]; + const neighbors = new Map([[start, [...(adj.get(start) ?? [])].sort()]]); + disc.set(start, clock); + low.set(start, clock); + clock++; + stack.push(start); + onStack.add(start); + while (frames.length) { + const frame = frames[frames.length - 1]; + const { v } = frame; + const ns = neighbors.get(v); + if (frame.i < ns.length) { + const w = ns[frame.i++]; + if (!disc.has(w)) { + disc.set(w, clock); + low.set(w, clock); + clock++; + stack.push(w); + onStack.add(w); + neighbors.set(w, [...(adj.get(w) ?? [])].sort()); + frames.push({ v: w, i: 0 }); + } else if (onStack.has(w)) { + low.set(v, Math.min(low.get(v), disc.get(w))); + } + } else { + frames.pop(); + if (frames.length) { + const parent = frames[frames.length - 1].v; + low.set(parent, Math.min(low.get(parent), low.get(v))); + } + if (low.get(v) === disc.get(v)) { + const comp = []; + let w; + do { + w = stack.pop(); + onStack.delete(w); + comp.push(w); + } while (w !== v); + if (comp.length >= 2) comps.push(comp.sort()); + } + } + } + } + comps.sort((a, b) => b.length - a.length || a[0].localeCompare(b[0])); + return comps; +} + +/** + * Chokepoint files: articulation points (Hopcroft–Tarjan, iterative) of the undirected + * import graph — files whose removal disconnects part of the repo. `splits` counts the + * subtrees that would break off (the bigger, the more load-bearing the file). + * @param {{nodes:string[], edges:Map>}} graph scope.importGraph output + * @returns {{file:string, splits:number}[]} sorted by splits desc, then file asc + */ +export function chokepoints(graph) { + const disc = new Map(); + const low = new Map(); + const splits = new Map(); + let clock = 0; + for (const root of [...graph.nodes].sort()) { + if (disc.has(root)) continue; + let rootChildren = 0; + /** @type {{v:string, parent:string|null, i:number}[]} DFS frames (iterative — no recursion) */ + const frames = [{ v: root, parent: null, i: 0 }]; + const neighbors = new Map([[root, [...(graph.edges.get(root) ?? [])].sort()]]); + disc.set(root, clock); + low.set(root, clock); + clock++; + while (frames.length) { + const frame = frames[frames.length - 1]; + const { v, parent } = frame; + const ns = neighbors.get(v); + if (frame.i < ns.length) { + const w = ns[frame.i++]; + if (w === parent) continue; + if (disc.has(w)) { + low.set(v, Math.min(low.get(v), disc.get(w))); + continue; + } + disc.set(w, clock); + low.set(w, clock); + clock++; + neighbors.set(w, [...(graph.edges.get(w) ?? [])].sort()); + frames.push({ v: w, parent: v, i: 0 }); + } else { + frames.pop(); + if (!frames.length) continue; + const p = frames[frames.length - 1].v; + low.set(p, Math.min(low.get(p), low.get(v))); + if (p === root) rootChildren++; + else if (low.get(v) >= disc.get(p)) splits.set(p, (splits.get(p) ?? 0) + 1); + } + } + if (rootChildren > 1) splits.set(root, rootChildren - 1); + } + const out = [...splits.entries()].map(([file, s]) => ({ file, splits: s })); + out.sort((a, b) => b.splits - a.splits || a.file.localeCompare(b.file)); + return out; +} + +/** + * The team-history overlay — how much verified memory already points at each file. + * Per file: Σ val(claim) over lesson claims whose trigger.files glob-match it and + * summary claims (deja session records) that list it. val() is the ledger's + * time-decayed Beta posterior, so stale incidents fade on the same clock everything + * else in the substrate uses. Pure; fail-open — no claims → all zeros. + * @param {any[]} claims live ledger claims (loadClaims output) + * @param {string[]} files repo-relative file paths + * @param {number} nowDay epoch day for val() + * @returns {Map} file → history + */ +export function history(claims, files, nowDay) { + const out = new Map(files.map((f) => [f, { weight: 0, hits: 0 }])); + for (const claim of claims ?? []) { + let touched = []; + if (claim.kind === "lesson") { + const globs = claim.body?.trigger?.files ?? []; + if (globs.length) + touched = files.filter((f) => globs.some((g) => globToRe(String(g)).test(f))); + } else if (claim.kind === "summary") { + const set = new Set((claim.body?.files ?? []).map((f) => toPosix(String(f)))); + touched = files.filter((f) => set.has(f)); + } + if (!touched.length) continue; + const w = val(claim, nowDay); + for (const f of touched) { + const h = out.get(f); + h.weight += w; + h.hits += 1; + } + } + return out; +} + +const round6 = (x) => Number(x.toFixed(6)); + +/** + * The impure assembler the CLI and MCP tool call: load the atlas (missing → build hint), + * the import graph, and — best-effort — the ledger, then compose the report. + * @param {string} root + * @param {{top?:number}} [opts] + * @returns {{built:boolean, nodes?:number, edges?:number, topFiles?:any[], topSymbols?:any[], + * cycles?:string[][], chokepoints?:{file:string,splits:number}[]}} + */ +export function rankReport(root, { top = 15 } = {}) { + const atlas = load(root); + if (!atlas) return { built: false }; + // One walk serves both graph readings: cycles need the directed edges, articulation + // points the undirected view derived from them. + const directed = directedImportGraph(root); + const undirected = new Map(directed.nodes.map((f) => [f, new Set()])); + for (const [f, targets] of directed.edges) { + for (const t of targets) { + undirected.get(f).add(t); + undirected.get(t)?.add(f); + } + } + const { files, symbols } = centrality(atlas); + let claims = []; + try { + claims = loadClaims(repoLedger(root)); + } catch { + claims = []; // no ledger (or unreadable) → structural ranking only + } + const hist = history( + claims, + files.map((f) => f.file), + epochDay(), + ); + const maxScore = files[0]?.score || 1; + const ranked = files.map(({ file, score }) => { + const h = hist.get(file) ?? { weight: 0, hits: 0 }; + return { + file, + score: round6(score), + history: round6(h.weight), + incidents: h.hits, + hazard: round6((score / maxScore) * (1 + h.weight)), + }; + }); + ranked.sort((a, b) => b.hazard - a.hazard || a.file.localeCompare(b.file)); + return { + built: true, + nodes: (atlas.nodes ?? []).length, + edges: (atlas.edges ?? []).length, + topFiles: ranked.slice(0, top), + topSymbols: symbols + .slice(0, top) + .map((s) => ({ name: s.name, file: s.file, score: round6(s.score) })), + cycles: cycles(directed), + chokepoints: chokepoints({ nodes: directed.nodes, edges: undirected }), + }; +} diff --git a/src/scope.js b/src/scope.js index c160281..50e1ee5 100644 --- a/src/scope.js +++ b/src/scope.js @@ -42,6 +42,20 @@ function resolveSpec(fromRel, spec, root, fileSet) { /** Build an UNDIRECTED file→file import graph (coupling is symmetric for decomposition). */ export function importGraph(root) { + const { nodes, edges } = directedImportGraph(root); + const undirected = new Map(nodes.map((f) => [f, new Set()])); + for (const [f, targets] of edges) { + for (const t of targets) { + undirected.get(f).add(t); + undirected.get(t)?.add(f); + } + } + return { nodes, edges: undirected }; +} + +/** The DIRECTED form (importer → imported) — what cycle detection needs; the + * undirected view above is derived from it. Same walk, same resolver. */ +export function directedImportGraph(root) { const files = []; walk(root, root, files); const fileSet = new Set(files); @@ -56,10 +70,7 @@ export function importGraph(root) { for (const re of IMPORT_RES) { for (const m of text.matchAll(re)) { const target = resolveSpec(f, m[1], root, fileSet); - if (target && target !== f) { - edges.get(f).add(target); - edges.get(target)?.add(f); - } + if (target && target !== f) edges.get(f).add(target); } } } diff --git a/test/atlas.test.js b/test/atlas.test.js index 5abc8f3..808aa09 100644 --- a/test/atlas.test.js +++ b/test/atlas.test.js @@ -243,3 +243,28 @@ test("C function DEFINITIONS index but prototype declarations do not (same-line assert.ok(names.includes("add"), "definition indexed"); assert.ok(!names.includes("dup"), "a bare prototype (ends in ;) is not a definition"); }); + +test("impact takes the max-product path through a diamond, not the first-found one", () => { + // Two routes from dependent D back to target S: a direct `contains` edge + // (0.45 × 0.85 decay = 0.3825) and a two-hop `calls` chain through M + // ((0.95 × 0.85)² = 0.6521). Label correction must keep the stronger long + // path — this pins the semantics any future queue/heap rewrite must preserve. + const atlas = { + nodes: [ + { id: "s.js::S", name: "S", kind: "function", file: "s.js" }, + { id: "m.js::M", name: "M", kind: "function", file: "m.js" }, + { id: "d.js::D", name: "D", kind: "function", file: "d.js" }, + ], + edges: [ + { source: "d.js::D", target: "s.js::S", kind: "contains" }, + { source: "m.js::M", target: "s.js::S", kind: "calls" }, + { source: "d.js::D", target: "m.js::M", kind: "calls" }, + ], + symbols: [], + }; + const r = impact(atlas, "S"); + const d = r.impacted.find((x) => x.id === "d.js::D"); + assert.ok(d, "D is in the blast radius"); + assert.equal(d.confidence, 0.6521, "max-product confidence wins over first-found"); + assert.equal(d.hopDistance, 2, "the winning path is the two-hop calls chain"); +}); diff --git a/test/cortex_mcp.test.js b/test/cortex_mcp.test.js index bbf5974..4a3d880 100644 --- a/test/cortex_mcp.test.js +++ b/test/cortex_mcp.test.js @@ -13,7 +13,12 @@ import { handle } from "../src/cortex_mcp.js"; process.env.FORGE_LEDGER_ONLY = "0"; test("handle: initialize advertises the forge-cortex server", async () => { - const r = await handle({ jsonrpc: "2.0", id: 1, method: "initialize", params: {} }); + const r = await handle({ + jsonrpc: "2.0", + id: 1, + method: "initialize", + params: {}, + }); assert.equal(r.result.serverInfo.name, "forge-cortex"); }); @@ -36,11 +41,37 @@ test("handle: tools/list exposes the cortex + preflight tools", async () => { "forge_remember", "forge_ledger_ratify", "forge_ledger_retract", + "rank_code", ]) { assert.ok(names.includes(t), `exposes ${t}`); } }); +test("rank_code without an atlas answers built:false over stdio, never a throw", () => { + const root = mkdtempSync(join(tmpdir(), "forge-mcp-rank-")); + const requests = [ + JSON.stringify({ jsonrpc: "2.0", id: 1, method: "initialize", params: {} }), + JSON.stringify({ + jsonrpc: "2.0", + id: 2, + method: "tools/call", + params: { name: "rank_code", arguments: {} }, + }), + ].join("\n"); + const r = spawnSync("node", [SERVER], { + input: `${requests}\n`, + encoding: "utf8", + env: { ...process.env, FORGE_ROOT: root }, + timeout: 10000, + }); + const responses = r.stdout + .trim() + .split("\n") + .map((l) => JSON.parse(l)); + const call = responses.find((x) => x.id === 2); + assert.deepEqual(JSON.parse(call.result.content[0].text), { built: false }); +}); + test("handle: notifications get no response; unknown methods error", async () => { assert.equal(await handle({ method: "notifications/initialized" }), null); assert.equal((await handle({ id: 9, method: "bogus" })).error.code, -32601); @@ -124,7 +155,10 @@ test("forge_ledger_retract returns error for missing claim via stdio", () => { jsonrpc: "2.0", id: 2, method: "tools/call", - params: { name: "forge_ledger_retract", arguments: { id: "nonexistent", reason: "test" } }, + params: { + name: "forge_ledger_retract", + arguments: { id: "nonexistent", reason: "test" }, + }, }), ].join("\n"); const r = spawnSync("node", [SERVER], { diff --git a/test/ledger.test.js b/test/ledger.test.js index 2844ef0..93db56e 100644 --- a/test/ledger.test.js +++ b/test/ledger.test.js @@ -2,6 +2,7 @@ import assert from "node:assert/strict"; import { test } from "node:test"; import { authorTrust, + beliefDiff, canonicalize, claimId, claimText, @@ -19,6 +20,9 @@ import { sealRecord, shingles, sketch, + sortRecords, + stateAt, + stateRoot, UNRESOLVED_VAL_CAP, val, } from "../src/ledger.js"; @@ -551,3 +555,133 @@ test("val with trust: a distrusted author's evidence moves confidence less", () assert.ok(weighted < flat, "trust scales the evidence weight down"); assert.ok(weighted > 0.5, "but a confirmation still counts for something"); }); + +// --- temporal views + Merkle state root ---------------------------------------------- + +test("stateAt hides a claim minted later and evidence appended later; val returns to the prior", () => { + const early = mintClaim({ kind: "fact", body: { name: "e", text: "early" }, t: 5 }).claim; + const late = mintClaim({ kind: "fact", body: { name: "l", text: "late" }, t: 50 }).claim; + const s = state( + [early, late], + { [early.id]: [ev("confirm", 6), ev("confirm", 40)] }, + {}, + { [early.id]: [prov("alice", 5)], [late.id]: [prov("bob", 50)] }, + ); + const then = liveClaims(stateAt(s, 10)); + assert.deepEqual( + then.map((c) => c.id), + [early.id], + "the day-50 claim did not exist on day 10", + ); + assert.equal(then[0].evidence.length, 1, "day-40 evidence is not visible on day 10"); + const now = liveClaims(stateAt(s, 60)); + assert.equal(now.length, 2, "both claims exist by day 60"); + assert.ok( + val(then[0], 10) !== + val( + now.find((c) => c.id === early.id), + 60, + ), + "belief strength is recomputed with that day's evidence and clock", + ); +}); + +test("stateAt commutes with mergeStates — a lattice morphism, so replicas agree on history", () => { + const c1 = mintClaim({ kind: "fact", body: { name: "m1", text: "one" }, t: 1 }).claim; + const c2 = mintClaim({ kind: "fact", body: { name: "m2", text: "two" }, t: 2 }).claim; + const sA = state([c1], { [c1.id]: [ev("confirm", 3)] }, {}, { [c1.id]: [prov("alice", 1)] }); + const sB = state( + [c1, c2], + { [c1.id]: [ev("contradict", 30)] }, + { [c2.id]: [tomb("dup", 40, "bob")] }, + { [c1.id]: [prov("bob", 1)], [c2.id]: [prov("bob", 2)] }, + ); + const canon = (s) => canonicalize(liveClaims(s)); + for (const day of [0, 1, 5, 30, 40, 99]) + assert.equal( + canon(stateAt(mergeStates(sA, sB), day)), + canon(mergeStates(stateAt(sA, day), stateAt(sB, day))), + `morphism holds at day ${day}`, + ); + assert.equal( + canon(stateAt(sA, 7)), + canon(stateAt(stateAt(sA, 7), 7)), + "stateAt is idempotent at the same day", + ); +}); + +test("beliefDiff classifies appeared / retired / strengthened / weakened and respects epsilon", () => { + const grew = mintClaim({ kind: "fact", body: { name: "g", text: "grew" }, t: 1 }).claim; + const sank = mintClaim({ kind: "fact", body: { name: "s", text: "sank" }, t: 1 }).claim; + const born = mintClaim({ kind: "fact", body: { name: "b", text: "born" }, t: 20 }).claim; + const gone = mintClaim({ kind: "fact", body: { name: "x", text: "gone" }, t: 1 }).claim; + const still = mintClaim({ kind: "fact", body: { name: "q", text: "still" }, t: 1 }).claim; + const s = state( + [grew, sank, born, gone, still], + { + [grew.id]: [ev("confirm", 12), ev("confirm", 13), ev("confirm", 14)], + [sank.id]: [ev("confirm", 2), ev("contradict", 12), ev("contradict", 13)], + }, + { [gone.id]: [tomb("obsolete", 15, "alice")] }, + { + [grew.id]: [prov("a", 1)], + [sank.id]: [prov("a", 1)], + [born.id]: [prov("a", 20)], + [gone.id]: [prov("a", 1)], + [still.id]: [prov("a", 1)], + }, + ); + const d = beliefDiff(s, 10, 30); + assert.deepEqual( + d.appeared.map((r) => r.id), + [born.id], + "minted inside the window", + ); + assert.deepEqual( + d.retired.map((r) => r.id), + [gone.id], + "tombstoned inside the window", + ); + assert.deepEqual( + d.strengthened.map((r) => r.id), + [grew.id], + "confirms raised val", + ); + assert.deepEqual( + d.weakened.map((r) => r.id), + [sank.id], + "contradictions sank val", + ); + assert.ok(!d.strengthened.some((r) => r.id === still.id), "no-news claim stays out"); + const strict = beliefDiff(s, 10, 30, { epsilon: 0.99 }); + assert.equal( + strict.strengthened.length + strict.weakened.length, + 0, + "a large epsilon silences movement rows", + ); +}); + +test("stateRoot: replicas merged in any order share one root; one new record moves exactly one shard", () => { + const c1 = mintClaim({ kind: "fact", body: { name: "r1", text: "one" }, t: 1 }).claim; + const c2 = mintClaim({ kind: "fact", body: { name: "r2", text: "two" }, t: 2 }).claim; + const sA = state([c1], { [c1.id]: [ev("confirm", 3)] }, {}, { [c1.id]: [prov("alice", 1)] }); + const sB = state([c1, c2], {}, {}, { [c2.id]: [prov("bob", 2)] }); + const ab = stateRoot(mergeStates(sA, sB)); + const ba = stateRoot(mergeStates(sB, sA)); + assert.equal(ab.root, ba.root, "merge order cannot leak into the root"); + assert.equal(ab.claims, 2); + const grown = mergeStates(sA, sB); + grown.evidence[c2.id] = sortRecords([ev("confirm", 9)]); + const after = stateRoot(grown); + assert.notEqual(after.root, ab.root, "new evidence changes the root"); + const changed = Object.keys(after.shards).filter((p) => after.shards[p] !== ab.shards[p]); + assert.deepEqual(changed, [c2.id.slice(0, 2)], "divergence is localized to the touched shard"); +}); + +test("stateRoot distinguishes states that liveClaims-level summaries could conflate", () => { + const c = mintClaim({ kind: "fact", body: { name: "t", text: "tomb" }, t: 1 }).claim; + const plain = state([c], {}, {}, { [c.id]: [prov("a", 1)] }); + const tombed = state([c], {}, { [c.id]: [tomb("done", 2, "a")] }, { [c.id]: [prov("a", 1)] }); + assert.notEqual(stateRoot(plain).root, stateRoot(tombed).root); + assert.notEqual(stateRoot(state([])).root, stateRoot(plain).root, "empty ≠ one-claim"); +}); diff --git a/test/ledger_sync.test.js b/test/ledger_sync.test.js index 73d417e..3b036cf 100644 --- a/test/ledger_sync.test.js +++ b/test/ledger_sync.test.js @@ -4,7 +4,7 @@ import { mkdtempSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { test } from "node:test"; -import { canonicalize, mintClaim } from "../src/ledger.js"; +import { canonicalize, mintClaim, stateRoot } from "../src/ledger.js"; import { loadClaims, loadState, putClaim } from "../src/ledger_store.js"; import { defaultRun, ledgerSync, stateBytes, syncDir, syncTarget } from "../src/ledger_sync.js"; @@ -108,6 +108,25 @@ test("syncDir: bidirectional union → both dirs byte-identical", () => { assert.equal(canonicalize(loadState(a)), canonicalize(loadState(b)), "byte-identical state"); }); +test("syncDir: equal state roots short-circuit; a new record re-arms the real merge", () => { + const a = tmp(); + const b = tmp(); + mint(a, "x", "shared fact"); + mint(b, "x", "shared fact"); // same content → same id → same state root + const first = syncDir(a, b); + assert.equal(first.upToDate, true, "identical replicas skip the merge entirely"); + assert.deepEqual(first.pulled, { claims: 0, records: 0, quarantined: 0 }); + mint(b, "y", "known only to B"); + const second = syncDir(a, b); + assert.ok(!second.upToDate, "a diverged replica takes the real merge path"); + assert.equal(second.pulled.claims, 1, "the new claim crosses over"); + assert.equal( + stateRoot(loadState(a)).root, + stateRoot(loadState(b)).root, + "after the merge both replicas share one state root again", + ); +}); + test("syncDir: missing target degrades honestly (no throw)", () => { const a = tmp(); const r = syncDir(a, join(tmp(), "does-not-exist")); diff --git a/test/rank.test.js b/test/rank.test.js new file mode 100644 index 0000000..6be93e5 --- /dev/null +++ b/test/rank.test.js @@ -0,0 +1,180 @@ +import assert from "node:assert/strict"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { build } from "../src/atlas.js"; +import { mintClaim } from "../src/ledger.js"; +import { appendEvidence, putClaim, repoLedger } from "../src/ledger_store.js"; +import { centrality, chokepoints, cycles, history, pagerank, rankReport } from "../src/rank.js"; +import { directedImportGraph, importGraph } from "../src/scope.js"; + +// Synthetic atlas: three modules all call into util — util must out-rank everything. +const hubAtlas = () => ({ + nodes: [ + { id: "a.js::fa", name: "fa", kind: "function", file: "a.js" }, + { id: "b.js::fb", name: "fb", kind: "function", file: "b.js" }, + { id: "c.js::fc", name: "fc", kind: "function", file: "c.js" }, + { id: "util.js::help", name: "help", kind: "function", file: "util.js" }, + ], + edges: [ + { source: "a.js::fa", target: "util.js::help", kind: "calls" }, + { source: "b.js::fb", target: "util.js::help", kind: "calls" }, + { source: "c.js::fc", target: "util.js::help", kind: "calls" }, + ], + symbols: [], +}); + +test("pagerank ranks the symbol every module depends on above the leaves, scores sum to 1", () => { + const scores = pagerank(hubAtlas()); + const util = scores.get("util.js::help"); + for (const leaf of ["a.js::fa", "b.js::fb", "c.js::fc"]) + assert.ok(util > scores.get(leaf), `depended-upon hub out-ranks ${leaf}`); + const sum = [...scores.values()].reduce((a, b) => a + b, 0); + assert.ok(Math.abs(sum - 1) < 1e-6, `scores form a distribution (sum=${sum})`); +}); + +test("pagerank is deterministic — two runs are identical and symmetric nodes tie exactly", () => { + const a = pagerank(hubAtlas()); + const b = pagerank(hubAtlas()); + assert.deepEqual([...a.entries()], [...b.entries()], "identical input → identical output"); + assert.equal(a.get("a.js::fa"), a.get("b.js::fb"), "structurally symmetric nodes score equal"); +}); + +test("centrality aggregates node scores per file and keeps only definition symbols", () => { + const atlas = hubAtlas(); + atlas.nodes.push({ + id: "module:util.js", + name: "util.js", + kind: "module", + file: "util.js", + }); + const { files, symbols } = centrality(atlas); + assert.equal(files[0].file, "util.js", "hub file ranks first"); + assert.ok( + symbols.every((s) => s.name !== "util.js"), + "module nodes stay out of the symbol view", + ); + assert.equal(symbols[0].name, "help", "hub symbol ranks first"); +}); + +test("cycles finds the a⇄b import cycle and leaves the acyclic file out", () => { + const root = mkdtempSync(join(tmpdir(), "forge-rank-")); + writeFileSync(join(root, "a.js"), 'import "./b.js";\nexport const a = 1;\n'); + writeFileSync(join(root, "b.js"), 'import "./a.js";\nexport const b = 1;\n'); + writeFileSync(join(root, "c.js"), 'import "./a.js";\nexport const c = 1;\n'); + const comps = cycles(directedImportGraph(root)); + assert.deepEqual(comps, [["a.js", "b.js"]], "exactly the mutual-import pair, sorted"); +}); + +test("chokepoints flags the bridge file between two clusters, not the leaves", () => { + const root = mkdtempSync(join(tmpdir(), "forge-rank-")); + // a ↔ bridge ↔ b : removing bridge.js disconnects a.js from b.js. + writeFileSync(join(root, "a.js"), 'import "./bridge.js";\nexport const a = 1;\n'); + writeFileSync(join(root, "bridge.js"), 'import "./b.js";\nexport const bridge = 1;\n'); + writeFileSync(join(root, "b.js"), "export const b = 1;\n"); + const points = chokepoints(importGraph(root)); + assert.deepEqual( + points.map((p) => p.file), + ["bridge.js"], + "only the articulation point is a chokepoint", + ); + assert.ok(points[0].splits >= 1, "it splits off at least one subtree"); +}); + +test("history weighs files named by lesson globs and summary file lists; empty ledger → zeros", () => { + const lesson = mintClaim({ + kind: "lesson", + body: { + correctedBehavior: "never edit generated files by hand", + trigger: { + action: "edit", + files: ["src/gen/*.js"], + keywords: [], + symbols: [], + }, + whatWentWrong: "hand-edited a generated file", + }, + scope: { level: "repo" }, + provenance: { author: "amina" }, + t: 10, + }).claim; + lesson.evidence = [ + { + oracle: "test.run", + result: "confirm", + ref: "test:unit", + t: 10, + author: "sami", + }, + ]; + const summary = mintClaim({ + kind: "summary", + body: { files: ["src/app.js"], text: "fixed the login flow" }, + scope: { level: "repo" }, + provenance: { author: "amina" }, + t: 10, + }).claim; + const files = ["src/gen/out.js", "src/app.js", "src/quiet.js"]; + const h = history([lesson, summary], files, 10); + assert.ok(h.get("src/gen/out.js").weight > 0, "glob-matched file carries lesson weight"); + assert.equal(h.get("src/gen/out.js").hits, 1); + assert.ok(h.get("src/app.js").weight > 0, "summary-listed file carries weight"); + assert.equal(h.get("src/quiet.js").weight, 0, "unnamed file carries none"); + const empty = history([], files, 10); + assert.ok( + files.every((f) => empty.get(f).weight === 0), + "no claims → all zeros (fail-open)", + ); +}); + +test("rankReport without an atlas reports built:false", () => { + const root = mkdtempSync(join(tmpdir(), "forge-rank-")); + assert.deepEqual(rankReport(root), { built: false }); +}); + +test("rankReport ranks a historied file above an equally-central clean one (the hazard join)", () => { + const root = mkdtempSync(join(tmpdir(), "forge-rank-")); + // Two symmetric leaf files import the same hub — identical centrality by construction. + writeFileSync(join(root, "hub.js"), "export const hub = 1;\n"); + writeFileSync(join(root, "left.js"), 'import "./hub.js";\nexport function leftFn(){}\n'); + writeFileSync(join(root, "right.js"), 'import "./hub.js";\nexport function rightFn(){}\n'); + build({ root }); + // A verified lesson names left.js — the join must break the structural tie. + const minted = mintClaim({ + kind: "lesson", + body: { + correctedBehavior: "guard the left path", + trigger: { + action: "edit", + files: ["left.js"], + keywords: [], + symbols: [], + }, + whatWentWrong: "left.js regressed twice", + }, + scope: { level: "repo" }, + provenance: { author: "amina" }, + t: 1, + }); + putClaim(repoLedger(root), minted.claim); + appendEvidence(repoLedger(root), minted.claim.id, { + oracle: "test.run", + result: "confirm", + ref: "test:left", + t: 1, + author: "sami", + }); + const r = rankReport(root, { top: 10 }); + const left = r.topFiles.find((f) => f.file === "left.js"); + const right = r.topFiles.find((f) => f.file === "right.js"); + assert.ok(left && right, "both leaves are in the report"); + assert.equal(left.score, right.score, "structure alone cannot tell them apart"); + assert.ok(left.history > 0 && right.history === 0, "history is the discriminator"); + assert.ok(left.hazard > right.hazard, "the historied file carries the higher hazard"); + assert.ok( + r.topFiles.findIndex((f) => f.file === "left.js") < + r.topFiles.findIndex((f) => f.file === "right.js"), + "hazard ordering surfaces the historied file first", + ); +});