Evidence and hierarchy (issue #19)#22
Merged
Merged
Conversation
Strict-mode research report surveying six disciplines (medicine, historiography, law, intelligence analysis, journalism, academic research) and software-engineering practice on hierarchies of evidence confidence, written as input for the canonical evidence-based source requested in #19. Recommends O7 (extract /research's existing trust-class and corroboration primitives into a YAGNI-style canonical pair) over a full composite framework, with the source-proximity ordering framed as directional rather than strict.
Small-team plan-implementation pass (PM + junior-developer + information-architect). Single round; spec-maturity gate did not trip; all open questions resolved deterministically. Plan ships the canonical pair (docs/evidence.md + plugin/references/evidence-rule.md), extracts /research's trust-class block to cite the canonical source, cross-links the YAGNI pair, and retrofits 11 behavioral-consumer files. Defers GRADE labels, Admiralty axes, strict source-tier ordering, the corroboration-gate codebase extension, and the 7 descriptive-only single-occurrence files under YAGNI.
Add docs/evidence.md and plugin/references/evidence-rule.md as the canonical pair defining what "evidence-based" means in Han, paralleling the YAGNI canonical pair. The rule file (74 lines, under the 80-line context-hygiene cap) names three structural principles (proximity-to-origin as a directional heuristic, corroboration scoped to web sources, explicit no-evidence labeling) and extracts the trust-class vocabulary (codebase / web / provided) from /research, which now cites the canonical pair instead of defining the vocabulary inline. Cross-link the YAGNI pair and the evidence pair in both directions so the inclusion gate (YAGNI: is there any evidence?) and the quality characterization (evidence-rule: how strong is it?) stay distinguishable. Retrofit 11 behavioral consumer files to cite the canonical pair alongside their existing yagni-rule citations. Defer 7 descriptive-only single-occurrence files under YAGNI per D-11 in the implementation plan. Defer GRADE labels, Admiralty axes, strict source-tier ordering, and the corroboration-gate codebase extension under the same plan. See docs/plans/canonical-evidence-resource/feature-implementation-plan.md for the full plan, decision log, and YAGNI ledger. Closes #19.
Threads the new docs/evidence.md reference and the companion evidence-rule through the operator-facing surface: README, CONTRIBUTING, concepts, and the long-form skill and agent docs whose SKILL.md or agent definition picked up the canonical evidence rule in this branch. Adds Evidence as the third foundational mechanic alongside Sizing and YAGNI in concepts.md, and ensures every long-form doc for a touched entity links the rule.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR extracts the evidence-based reasoning rule out of
/researchand into a canonical reference that every judgment-producing skill and reviewing agent loads at runtime, so the trust-class vocabulary and corroboration gate apply uniformly across the plugin instead of being one skill's private convention.allowed-toolsshifts. Every changed file is markdown: one new canonical rule underplugin/references/, one new operator-facing long-form doc, durable research backing the rule, and a sweep of seven SKILL.md files plus four long-form agent docs adding a single-line pointer to the rule.path:linecitation stands; a single Stack Overflow answer driving a recommendation does not. Worth weighing whether that asymmetry holds up at the seams (codebase-vs-web disagreement, provided-material handling).What to look at first
plugin/references/evidence-rule.md— Principle 2 explicitly scopes the gate to web sources and defers extending it to codebase evidence. The rationale is that a single file path is the current state of the system, but it is worth pushing on whether that holds when the codebase finding is itself a single grep hit in unfamiliar code.evidence-rule.md("When the rule and YAGNI both apply") and the same section indocs/evidence.md. The claim is that YAGNI gates inclusion and this rule characterizes quality once included, and that the two do not collapse. Worth checking the seam holds at edge cases like a YAGNI-passing item whose only evidence is a single uncorroborated blog post./researchand/investigatetoday.docs/research/evidence-hierarchy.mdis the durable backing for the rule but is not on the critical path for review; a skim of its conclusions is sufficient unless something in the rule looks wrong.Files of interest
plugin/references/evidence-rule.md— the new canonical rule consumed at runtime by every evidence-aware skill and agent; this is the file that matters most.docs/evidence.md— the operator-facing long-form companion; verify it stays consistent with the canonical rule and follows the writing-voice profile.docs/research/evidence-hierarchy.md— the research that justifies the principles, the trust classes, and the scoped corroboration gate.plugin/skills/research/SKILL.md— the representative skill update;/researchis where the vocabulary originated, so its diff shows the pattern other skills follow.docs/plans/canonical-evidence-resource/feature-implementation-plan.md— the plan that drove the change, useful if a reviewer wants to check scope decisions against intent.