Skip to content

check-non-standard-chars skips .md files (and U+00D7), so gha's own README has 39 unflagged em-dashes #322

Description

@d-morrison

Noticed while working #321. Not blocking that PR (whose own added lines are clean); filing so it isn't lost.

What's wrong

check-non-standard-chars/check-non-standard-chars.py scans only two extensions:

extensions = ['.qmd', '.R']   # line 93

so Markdown is never scanned. The consequence is visible in this repo:

$ grep -c $'' README.md
39

39 em-dashes in README.md, and the chars / check-chars selftest job reports
success on every run. CHANGELOG.md, REVDEPS.md, changelog.d/*.md, and the
examples/ stubs' comments are equally unscanned.

Markdown files are source too: version-controlled, diffed, and rendered, so they
carry the same smart-quote / copy-paste corruption and whole-paragraph-reflow
diff problems .qmd and .R do. The ai-config corpus's
shared/coding/ascii-punctuation-in-source.md states the rule as covering every
tracked source file, .md explicitly included, and names extending this very
check as the enforcement follow-up.

Second, smaller gap: the glyph set

NON_STANDARD_CHARS (line 25) covers six code points: curly quotes (U+2018,
U+2019, U+201C, U+201D) and en/em dashes (U+2013, U+2014). It has no entry for
the multiplication sign (U+00D7), which the same shared fragment names
alongside the others as a glyph that slips in from rendered-text copy-paste and
should be written as $\times$ / × in prose or × in a string
literal.

Proposed scope

  1. Add .md to the scanned extensions.
  2. Add U+00D7 to NON_STANDARD_CHARS.
  3. Make the extension list an input rather than a hard-coded literal, with
    the current set as the default, so a consumer can narrow or widen it without
    forking the script (per shared/coding/configurable-parameters.md).
  4. Clean up this repo's own existing violations in the same PR, or the newly
    widened check turns chars / check-chars red on main immediately. This is
    the reason to treat (4) as part of the change rather than a follow-up: 39
    hits in README.md alone, plus whatever the other .md files hold.
  5. Update the action's own description: (it says ".qmd and .R files"), the
    README/website tables, and website/reference/check-non-standard-chars.qmd,
    per the doc-sync checklist in CLAUDE.md.

Consumer impact

Widening the default scope will flag pre-existing prose in every consumer repo
that calls check-non-standard-chars.yml, not just this one. Worth deciding
whether the new extension list ships as the default (with a heads-up in the
changelog under ### Changed) or as an opt-in input that defaults to today's
.qmd/.R behavior. My inclination is the former, since the point of the rule
is that Markdown was never meant to be exempt, but that's a maintainer call.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions