Skip to content

Evidence and hierarchy (issue #19)#22

Merged
mxriverlynn merged 4 commits into
mainfrom
canonical-evidence-resource
May 28, 2026
Merged

Evidence and hierarchy (issue #19)#22
mxriverlynn merged 4 commits into
mainfrom
canonical-evidence-resource

Conversation

@mxriverlynn
Copy link
Copy Markdown
Collaborator

@mxriverlynn mxriverlynn commented May 28, 2026

Summary

This PR extracts the evidence-based reasoning rule out of /research and into a canonical reference that every judgment-producing skill and reviewing agent loads at runtime, so the trust-class vocabulary and corroboration gate apply uniformly across the plugin instead of being one skill's private convention.

  • Adds a third foundational mechanic to the plugin alongside sizing and YAGNI: an explicit posture on what counts as evidence (codebase / web / provided trust classes), how to characterize its strength (proximity heuristic, corroboration gate scoped to web sources), and what to do when no evidence exists at all (label and defer with a reopen trigger, same pattern YAGNI uses for deferred items).
  • Scope is documentation-only — no runtime code changes, no SKILL frontmatter or allowed-tools shifts. Every changed file is markdown: one new canonical rule under plugin/references/, one new operator-facing long-form doc, durable research backing the rule, and a sweep of seven SKILL.md files plus four long-form agent docs adding a single-line pointer to the rule.
  • The interesting design call is the asymmetry: the corroboration gate fires on web sources but not on codebase findings. A single path:line citation stands; a single Stack Overflow answer driving a recommendation does not. Worth weighing whether that asymmetry holds up at the seams (codebase-vs-web disagreement, provided-material handling).
  • Reviewer attention is best spent on the rule's wording and the boundary with YAGNI, not on the per-skill pointer edits, which are mechanical.

What to look at first

  • The asymmetry between web and codebase corroboration in plugin/references/evidence-rule.md — Principle 2 explicitly scopes the gate to web sources and defers extending it to codebase evidence. The rationale is that a single file path is the current state of the system, but it is worth pushing on whether that holds when the codebase finding is itself a single grep hit in unfamiliar code.
  • The YAGNI-vs-evidence boundary in evidence-rule.md ("When the rule and YAGNI both apply") and the same section in docs/evidence.md. The claim is that YAGNI gates inclusion and this rule characterizes quality once included, and that the two do not collapse. Worth checking the seam holds at edge cases like a YAGNI-passing item whose only evidence is a single uncorroborated blog post.
  • The proximity-inversion conditions (formal methods, regulatory, pre-incident observation of intended behavior). These are the carve-outs that prevent the principle from being misread as "running code always beats docs." Confirm the three named contexts cover the cases that come up in /research and /investigate today.
  • The 728-line research report under docs/research/evidence-hierarchy.md is the durable backing for the rule but is not on the critical path for review; a skim of its conclusions is sufficient unless something in the rule looks wrong.

Files of interest

  • plugin/references/evidence-rule.md — the new canonical rule consumed at runtime by every evidence-aware skill and agent; this is the file that matters most.
  • docs/evidence.md — the operator-facing long-form companion; verify it stays consistent with the canonical rule and follows the writing-voice profile.
  • docs/research/evidence-hierarchy.md — the research that justifies the principles, the trust classes, and the scoped corroboration gate.
  • plugin/skills/research/SKILL.md — the representative skill update; /research is where the vocabulary originated, so its diff shows the pattern other skills follow.
  • docs/plans/canonical-evidence-resource/feature-implementation-plan.md — the plan that drove the change, useful if a reviewer wants to check scope decisions against intent.

Strict-mode research report surveying six disciplines (medicine,
historiography, law, intelligence analysis, journalism, academic
research) and software-engineering practice on hierarchies of evidence
confidence, written as input for the canonical evidence-based source
requested in #19. Recommends O7 (extract /research's existing
trust-class and corroboration primitives into a YAGNI-style canonical
pair) over a full composite framework, with the source-proximity
ordering framed as directional rather than strict.
Small-team plan-implementation pass (PM + junior-developer +
information-architect). Single round; spec-maturity gate did not trip;
all open questions resolved deterministically. Plan ships the
canonical pair (docs/evidence.md + plugin/references/evidence-rule.md),
extracts /research's trust-class block to cite the canonical source,
cross-links the YAGNI pair, and retrofits 11 behavioral-consumer
files. Defers GRADE labels, Admiralty axes, strict source-tier
ordering, the corroboration-gate codebase extension, and the 7
descriptive-only single-occurrence files under YAGNI.
Add docs/evidence.md and plugin/references/evidence-rule.md as the
canonical pair defining what "evidence-based" means in Han, paralleling
the YAGNI canonical pair. The rule file (74 lines, under the 80-line
context-hygiene cap) names three structural principles
(proximity-to-origin as a directional heuristic, corroboration scoped
to web sources, explicit no-evidence labeling) and extracts the
trust-class vocabulary (codebase / web / provided) from /research,
which now cites the canonical pair instead of defining the vocabulary
inline.

Cross-link the YAGNI pair and the evidence pair in both directions so
the inclusion gate (YAGNI: is there any evidence?) and the quality
characterization (evidence-rule: how strong is it?) stay distinguishable.

Retrofit 11 behavioral consumer files to cite the canonical pair
alongside their existing yagni-rule citations. Defer 7 descriptive-only
single-occurrence files under YAGNI per D-11 in the implementation
plan. Defer GRADE labels, Admiralty axes, strict source-tier ordering,
and the corroboration-gate codebase extension under the same plan.

See docs/plans/canonical-evidence-resource/feature-implementation-plan.md
for the full plan, decision log, and YAGNI ledger.

Closes #19.
Threads the new docs/evidence.md reference and the companion evidence-rule
through the operator-facing surface: README, CONTRIBUTING, concepts, and the
long-form skill and agent docs whose SKILL.md or agent definition picked up
the canonical evidence rule in this branch. Adds Evidence as the third
foundational mechanic alongside Sizing and YAGNI in concepts.md, and ensures
every long-form doc for a touched entity links the rule.
@mxriverlynn mxriverlynn marked this pull request as ready for review May 28, 2026 19:59
@mxriverlynn mxriverlynn merged commit a68db1b into main May 28, 2026
@mxriverlynn mxriverlynn deleted the canonical-evidence-resource branch May 28, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant