Skip to content

Analysis of model_fusion Recommendations

Jump to bottom
Chris Sweet edited this page Jun 12, 2026 · 2 revisions

type: analysis up: "Lessons-Learned-From-model_fusion" extends: "Lessons-Learned-From-model_fusion" related:

Analysis of model_fusion Recommendations

An assessment of the 12 implementation recommendations in Lessons-Learned-From-model_fusion, filed in response to the user's question "do you agree with the assessment, and what parts would bring the greatest value?" after the same-day refresh of the field report against HEAD 88f4624 (2026-06-12).

Filing meta-note. This page uses type: analysis, which is not in the SCHEMA's declared type list (concept | entity | source-summary | synthesis | index | comparison | untyped). Filing it anyway is the point: recommendation #4 in Lessons-Learned-From-model_fusion argues for adding analysis and decision page types precisely because query-driven assessments like this one have nowhere clean to land. This page is empirical evidence for recommendation #4 — it exists because the destination type is missing. Until #4 ships, the page violates SCHEMA in the way the field report predicts.

Headline

Largely agree with the field report's diagnostic frame. The 12 recommendations are not equal in value or cost; the field report's own priority section gets two of the top three right (#1, #4) but underweights #8 and overweights #2. Four items I'd push back on or reframe.

High-value cluster (do these first, in order)

1. SessionStart auto-load — #1

Single highest-leverage change, as the field report argues. The current SessionStart hook tells the agent the wiki exists; loading the index and recent log entries gives the agent what the wiki actually says. Without this, the wiki is RAG-on-demand, not memory.

  • Cost: a few thousand tokens per session start.
  • Payoff: the wiki functions as memory in the sense the Wiki-LLM-Pattern page claims it does. Every other improvement is conditional on this one.
  • Friction: the hook scaffold already ships; the delta is small.
  • One pushback on the field report's framing: "default on" is right in spirit, but "two of three inspected projects had the existing hook off" is also evidence that opt-in friction was too high, not that defaults must flip. Either way, ship it.

2. analysis and decision page types — #4

Without destination types, Discipline-Gates fire correctly but the agent has nowhere clean to file query-driven content. Two SCHEMA entries plus section templates is a cheap edit with disproportionate effect.

  • Cost: SCHEMA addition (in both create-mode heredoc and update-mode append per Lesson-Parallel-File-Drift), a section template per type, acknowledgment in the verification gate.
  • Payoff: unlocks the "user asked X, we discussed, we decided" class of content.
  • Pairs with #1: the auto-loaded index becomes more useful as more answers land as analysis pages instead of being dropped.
  • This page itself is the dogfood case (see filing meta-note above).

3. Claude-memory vs wiki boundary in CLAUDE.md — #8

Cheapest item on the entire list. A 5-line stanza in CLAUDE.md.template declaring which content goes to Claude-memory (user identity, preferences, workflow style, cross-project guidance) versus the wiki (project-specific knowledge, synthesis, decisions, experiment results). The field report's priority section underweights this; it's the lowest-cost item with measurable benefit.

  • Cost: a few minutes.
  • Payoff: ambiguous filing decisions stop dropping content into neither place.
  • Why I'd promote it over #2: the field report puts UserPromptSubmit (#2) third; I'd put #8 third because it ships in a day and benefits every project immediately.

4. Finish reciprocal-edge enforcement — #9(b)

PR #16 shipped the vocabulary at wiki/Edge-Types.md.template (12 forward + 12 inverse predicates). The remaining wiring is small:

  • Add the inverse predicate names to wiki/init-wiki.sh's generated SCHEMA page-type list so derived projects' frontmatter validates them.
  • Declare the inverse pairs in the SHACL spec so the Knowledge-Graph-Pipeline materialisation (PR #14, already shipped) actually produces the inverse triples during build.
  • The PostToolUse write-time check the field report proposes is genuinely nice, but the SHACL declaration is more important: the KG's materialisation pass becomes mechanically complete the moment the spec declares the inverses. The hook is then a secondary catch.

Closes the gap Limitation-Inverse-Edges-Missing names as template issue #3.

Items I'd push back on or reframe

#10 Project-typed SCHEMA at instantiate time — defer or skip

"Research wiki", "documentation wiki", "personal knowledge" are not as crisp as the field report suggests. Many real projects span multiple categories. Forking the base SCHEMA per project type risks exactly the variant proliferation Governance warns about, but escalated to the SCHEMA layer where drift is hardest to fix.

  • Better path: keep one SCHEMA. Let derived projects extend with project-specific types via the existing typed-edge mechanism. Project type can be a tag, not a SCHEMA fork.

#6 PostToolUse records compounding ratio — right goal, wrong mechanism

The measurement principle is correct (the pattern's value claim of "5–15 pages per source" needs an empirical check), but a PostToolUse shell hook lacks the session and related-pages context to compute the ratio cleanly.

  • Better path: the Knowledge-Graph-Pipeline (shipped PR #14) already has the wiki structure as RDF. Combining it with git log gives a per-session "pages touched together" graph offline. Compute the compounding ratio as part of the Lint operation, surface it in /wiki-lint output. Same metric, no shell-hook fragility.

#11 CLAUDE.md re-injection mid-session — overlaps with auto-compact

Claude Code's auto-compact already handles some of the "instructions buried under context" problem the field report describes. A periodic re-injection hook competes with auto-compact's own context-management logic and risks redundant chrome.

  • Worth experimenting with topic-shift-triggered re-injection, but not a clear win. Lower priority than the field report's "next priority" framing suggests.

#2 UserPromptSubmit synthesis-question hook — false-positive risk

The discipline of "before answering 'why X', check if there's a page on X" is real. The detection mechanism is brittle. "Why", "compare", "should we" appear in many non-synthesis prompts. Injecting a <system-reminder> on every match generates noise that becomes background, defeating the point.

  • Mitigations: scope to longer multi-clause prompts only; apply only when no recent wiki read has happened in the session; or require two synthesis verbs together.
  • Net: ship with conservative detection or skip until #1 + #4 are in place. The compounding effect of "wiki state already in context (#1) + clean destination type (#4)" may make this hook unnecessary.

Items I largely accept without modification

  • #3 Stop hook for unfiled substantive content — agree on the problem; the proposed fix has similar false-positive risk to #2 but is worth shipping conservatively after #1 + #4.
  • #5 primary_wiki + cross-wiki rules — specific to multi-wiki users; cheap to add; doesn't harm single-wiki users.
  • #7 Lint as forcing schedule — agree, low cost (extend the SessionStart hook from #1).
  • #12 Filing-footer in chat transcript — agree on transparency value; opt-in default would be wiser than opt-out.

Suggested implementation order

  1. #8 — same day, no risk
  2. #1 — small delta on existing hook scaffold
  3. #4 — SCHEMA addition + section templates (and this page becomes valid)
  4. #9(b) — finish what PR #16 started
  5. #7 — extend the SessionStart hook from step 2
  6. Then evaluate #2, #3, #12 once 1-5 are in place — some may be obviated

#10, #11, #6 (as a hook) deferred or reframed.

See also

Clone this wiki locally