Add integration test for directive rule example fixtures - #274
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #274 +/- ##
=======================================
Coverage 95.85% 95.85%
=======================================
Files 190 190
Lines 24555 24571 +16
=======================================
+ Hits 23537 23553 +16
Misses 624 624
Partials 394 394
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an integration test that enforces every generated-section directive rule (catalog, include, toc, build) ships canonical bad/, good/, and (when fixable) fixed/ example folders. Validation-only bad fixtures opt out via a new bad/.nofix marker. Fixtures and skill docs are updated to align with this contract.
Changes:
- New
TestDirectiveRulesHaveExamplesintegration test enforcing fixture folder structure for allgensection.Directiverules, with a.nofixopt-out for non-fixable bad cases. - MDS019 catalog: add
fixed/folder (default + data files), updatebad/default.mdand README to includerow:parameter and corrected expected output. - Add
.nofixmarkers for MDS021 (cycle detection) and MDS039 (validation-only diagnostics); updatemarkdown-auditskill docs to point Claude at example folders rather than paraphrasing directive syntax.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| internal/integration/directive_examples_test.go | New test enforcing presence of bad/good/fixed example folders for directive rules. |
| internal/rules/MDS039-build/bad/.nofix | Opt-out marker listing bad fixtures that cannot be auto-fixed. |
| internal/rules/MDS021-include/bad/.nofix | Opt-out marker for the cycle-detection bad fixture. |
| internal/rules/MDS019-catalog/README.md | Updated bad example output to reflect new row: parameter. |
| internal/rules/MDS019-catalog/bad/default.md | Adds explicit row: parameter to the catalog directive. |
| internal/rules/MDS019-catalog/fixed/default.md | New post-fix expected body for the bad fixture. |
| internal/rules/MDS019-catalog/fixed/data/alpha.md | Supporting file referenced by the catalog glob. |
| internal/rules/MDS019-catalog/fixed/data/beta.md | Supporting file referenced by the catalog glob. |
| .claude/skills/markdown-audit/SKILL.md | Instructs Claude to consult rule example folders for directive syntax. |
| .claude/skills/markdown-audit/patterns.md | Replaces inline directive snippets with pointers to canonical example folders; expands schema-flavor guidance. |
| editors/claude-code-audit/skills/markdown-audit/SKILL.md | Mirrors the SKILL.md update for the audit plugin. |
jeduden
force-pushed
the
claude/add-rule-examples-0oMsm
branch
from
May 14, 2026 07:55
b34fba5 to
5f1948b
Compare
jeduden
force-pushed
the
claude/add-rule-examples-0oMsm
branch
from
May 14, 2026 08:19
001fbc6 to
1ba6e05
Compare
Every rule that implements gensection.Directive (MDS019 catalog, MDS021 include, MDS038 toc, MDS039 build) must ship bad/, good/, and — when fixable — fixed/ example folders. A new integration test in internal/integration/directive_examples_test.go walks rule.All() and fails when any of those folders are missing or when bad/ has a fix-able entry with no matching fixed/ counterpart. Non-fixable bad fixtures (cycle detection, build validation) opt out via a bad/.nofix sentinel file. Backfills MDS019-catalog/fixed/ so the loop tests the catalog fix path, and adjusts the MDS019 bad fixture so the post-fix body satisfies the default-enabled blank-line-around-lists rule. The markdown-audit skill (.claude/skills/markdown-audit/) now points at these example folders as the single source of truth for the <?catalog?> and <?include?> fix recipes. Check 6 (kind without schema) gains an inline-schema vs proto.md section that links to the four MDS020 example flavors so the audit reader picks the right shape per kind. https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
The bad/good folders under each rule test diagnostics — bad/* must fire a diagnostic, good/* must pass all rules. That layer is the wrong shape for "what does the hand-maintained anti-pattern look like before someone adopts this directive?", which is what the markdown-audit skill needs to show users. Each directive rule (MDS019 catalog, MDS021 include, MDS038 toc, MDS039 build) now ships a separate pattern/bad/ and pattern/good/ pair: - pattern/bad — the user-authored Markdown someone would write today without the directive (hand-maintained index, duplicated section, hand-maintained TOC, hand-maintained build output snippet). - pattern/good — the same content rewritten with the directive, plus any sibling files the directive needs. The four directive rule READMEs grew a ## Pattern section that <?include?>s the new fixtures. A new directive-rule-readme kind (with proto at internal/rules/directive-proto.md) layers a stricter schema over rule-readme that requires that Pattern section with Without/With subsections. Until kind-schema composition lands (plan/156_kind-schema-composition.md), the directive-rule-readme schema duplicates rule-readme's structure and the four directive READMEs are excluded from the rule-readme glob so only one schema resolves per file. TestDirectiveRulesHaveExamples now also requires pattern/bad/*.md and pattern/good/*.md for every gensection.Directive rule. The markdown-audit skill patterns.md + SKILL.md point at pattern/ as the canonical before/after pair. https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
Files the use case (a "Directive rules" listing inside internal/rules/index.md that selects on a nature property) and the proposed surface (a where: parameter on the catalog directive that reuses the CUE expression matcher from mdsmith list query). https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
Copilot flagged that <?include?> on pattern/bad and pattern/good rewrote relative link targets so the displayed paths (e.g. pattern/bad/data/alpha.md) did not match the source files (data/alpha.md). The rewrite is stable so mdsmith fix did not flag the body as stale, but the cosmetic mismatch confuses a reader trying to copy the snippet verbatim. Drop the include directives from the ## Pattern sections. The snippets in each directive README now match the source pattern/ files character-for-character, and the prose points to pattern/bad and pattern/good as the canonical folders the integration test still enforces. Also add bad/.nofix on MDS019 for the new dotdot.md fixture from main (#275). The dotdot escape is a validation-only diagnostic with no auto-fix. https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
Copilot flagged that the <?include?> for bad/default.md
in the ## Examples section displayed paths with a `bad/`
prefix (`[data/alpha.md](bad/data/alpha.md)`), while the
source fixture uses `[data/alpha.md](data/alpha.md)`.
The displayed body IS what `mdsmith fix` produces — the
include rule's adjustLinks rewrites relative link
targets so they resolve from the host file's directory.
The body is stable, not stale.
But the cosmetic mismatch makes the example misleading,
and the row template `{filename}` placeholder gets the
same rewrite applied. Replace both <?include?> blocks
with inline snippets that match the fixture files
character-for-character; link to good/default.md and
bad/default.md by path for readers who want the source.
https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
Extend <?catalog?> with a `where:` parameter that evaluates a CUE expression against each matched file's parsed front matter, dropping non-matching files before sort and render. Reuses the matcher from internal/query/ so the catalog and `mdsmith list query` accept the same grammar. Invalid CUE expressions emit an MDS019 diagnostic on the directive's opening line; missing fields and type mismatches silently exclude the file, matching list-query semantics. Covers tasks 1, 5, and 6 from plan/157. https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
Add a required `nature` front-matter key to the rule-readme schema (internal/rules/proto.md) and the directive-rule-readme schema (internal/rules/directive-proto.md). The field labels each rule's kind so the catalog directive can filter listings by it. Allowed values: directive, generator, content, style, structure. Backfill `nature:` on every MDS rule README and add a fixture under MDS020 (bad/nature-missing.md) that exercises the new schema requirement. Covers tasks 2 and 3 from plan/157. https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
Uses the new catalog where: filter (plan 157 task 1) and the nature front-matter property (plan 157 tasks 2 + 3) to surface a filtered listing of just the four directive rules. Closes the remaining acceptance criterion in plan/157. https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
Replace the hand-maintained list of four directive rule pattern folders with a <?catalog?> directive that selects every rule README with nature: directive in its front matter. The list now stays in sync with the rule catalog automatically — a new directive rule starts showing up once its README declares nature: directive. https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP
jeduden
force-pushed
the
claude/add-rule-examples-0oMsm
branch
from
May 14, 2026 08:44
6f1a5a6 to
4fb4bbc
Compare
jeduden
pushed a commit
that referenced
this pull request
May 14, 2026
Main introduced `nature:` as a required README front-matter field (#274). The four new content-constraint rules all describe their constraint as `content`. Plan 162's background section now notes that `nature:` is orthogonal to `Category()` — `nature:` is the kind of check, `Category()` is the document target. https://claude.ai/code/session_01Lh9smMq7STmiYqZxwtAsNh
jeduden
pushed a commit
that referenced
this pull request
May 14, 2026
Main introduced `nature:` as a required README front-matter field (#274). The four new content-constraint rules all describe their constraint as `content`. Plan 162's background section now notes that `nature:` is orthogonal to `Category()` — `nature:` is the kind of check, `Category()` is the document target. https://claude.ai/code/session_01Lh9smMq7STmiYqZxwtAsNh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enforce that every rule providing a generated-section directive ships canonical before/after example fixtures. Add a new integration test that validates the presence and consistency of
bad/,good/, andfixed/example folders for all directive rules.Changes
New integration test (
internal/integration/directive_examples_test.go):TestDirectiveRulesHaveExamplesvalidates that each directive rule has required example foldersrequireExampleDirensuresbad/andgood/folders exist with.mdfilesrequireFixedMatchesBadenforces 1:1 mapping betweenbad/andfixed/examples for fixable rulesisNoFixBadsupports opt-out viabad/.nofixmarker file for validation-only fixtures (e.g., cycle detection, missing required params)Example fixtures and markers:
internal/rules/MDS019-catalog/fixed/default.mdwith post-fix bodyinternal/rules/MDS019-catalog/fixed/data/{alpha,beta}.mdsupporting filesinternal/rules/MDS019-catalog/bad/default.mdto includerow:parameterinternal/rules/MDS021-include/bad/.nofix(cycle detection cannot auto-fix)internal/rules/MDS039-build/bad/.nofix(validation-only diagnostics)Documentation updates:
.claude/skills/markdown-audit/patterns.mdto reference example folders as canonical source of truth for directive syntax instead of inline code blocks.claude/skills/markdown-audit/SKILL.mdandeditors/claude-code-audit/skills/markdown-audit/SKILL.mdto instruct Claude to read example folders rather than paraphrase directive syntax from memoryinternal/rules/MDS019-catalog/README.mdto reflect corrected example outputImplementation Details
The test uses
filepath.Globto locate rule fixture directories by pattern (internal/rules/<id>-*), ensuring a 1:1 mapping between rule IDs and fixture directories. The.nofixmarker file allows rules to opt out specific bad fixtures from the fixed/ requirement when they represent pure validation diagnostics that cannot be auto-corrected (e.g., cycle detection in include directives, missing required parameters in build directives).This makes the gap between rule documentation and actual directive syntax explicit and loud — the integration test fails when example folders go missing or fall out of sync.
https://claude.ai/code/session_01K7ZGjbkxFHTdTekshbN3kP