v3.22.0

Added

Two pieces of enforcement. No new detection categories: the catalog stays at 61 and the engine at 46 types.

• Preservation validator (detector/validate.js, detector/validate.test.js). Edit mode writes to files, and until now the promises it makes were prose instructions to a model with nothing checking them. validate(original, rewritten) errors when a rewrite modifies a fenced code block, YAML frontmatter, a blockquote, a table cell, inline code, a URL, a file path, or the heading count and nesting, and when the rewrite ends with more flagged patterns than it started with. Warnings cover reworded headings, figures that vanished, and rewrites that drop more than 40% of the words. There is a CLI (node detector/validate.js before.md after.md) that exits 1 on any error. 23 tests, no dependencies.

• Two carve-outs, because a validator that fires on its own skill's instructions gets switched off. URLs are compared with AI tracking parameters stripped from both sides, since the skill tells you to strip them; heading text changing is a warning rather than an error, since the skill tells you to sentence-case Title Case headings and cut emoji from them.

• Self-scan and PROOF.md (scripts/self-scan.js). Scores this repo's documentation with this repo's detector and publishes both numbers: raw, which counts every pattern quoted as an example, and exempt, which applies the self-reference escape hatch SKILL.md has documented in prose since v1 and never implemented. Budgets gate the exempt column in CI and only move down.

• The scan found two gaps in our own work, both recorded in PROOF.md rather than quietly patched: the em-dash rule carves out list-item separators but not Keep-a-Changelog version headings (## [3.21.0] — 2026-07-30) or a bold lead term with a parenthetical before the dash, and roughly half of CHANGELOG.md's residual score is release notes enumerating the very words each new rule catches.

• "Never inject these" guardrails in SKILL.md, under Tone calibration. The instruction to put voice back on purpose has a predictable failure mode: the model installs a personality the author never had, trading one detectable register for a louder one. Seven additions are now out of bounds regardless of how the result scores: fake first person, manufactured stakes, forced contrarianism, performed candor, em-dash theatrics, staccato conversion, and invented specifics. The governing test is provenance: subtraction and sharpening are in scope, addition of stance, personality, or fact is not. These are constraints on the editor rather than detections on the text, which is why they sit with the rewrite instructions and do not change the catalog count. Adapted from isatimur/de-slop's guardrails.

• False-positive issue template (.github/ISSUE_TEMPLATE/false_positive.yml). A rule firing on human writing is the defect this project most wants reported, and the form collects what makes a report measurable rather than anecdotal: the shortest text that fires, the register, how the text was actually written, and whether it can become a public fixture. Register is the field that matters most, since false-positive rates differ sharply across blog, docs, academic, and chat prose.

• Human-control corpus and false-positive measurement (corpus/, scripts/corpus.js, scripts/fp-measure.js). This repo has always asserted things about false positives — the tiering exists to reduce them, the tolerance matrix relaxes rules per register, SKILL.md opens with "signals, not proof" — and had never measured one. Every document in the corpus was written by a person, so every flag on it is a false positive by construction: no labelling, no judge, no model in the loop. The corpus is hash-only; text is fetched into a gitignored cache or read from wherever it already lives, and only hashes and metadata are committed. Register is the unit of analysis, following patina's finding that false-positive rates ran from 4% to 34% across registers inside one language.

• First measurement: 0.0% false positives at every threshold across 560 paragraphs, Wilson 95% CI 0.0–0.7%, worst paragraph 11 out of 100. The corpus is nine public-domain works (1788–1907) plus 25 of the maintainer's own blog posts from 2019 to December 2022, read from pre-2023 web.archive.org captures rather than the live site. At the document-score level the detector does not fire on human prose. The same measurement against the current published versions of those posts also returned 0.0% across 628 paragraphs, so the result does not depend on which copy was measured.

• Provenance is verified, not assumed. The old site used compressed slugs (beveragetax, challengerfunnel) that do not match current URLs, so archived candidates were found by slug similarity and then confirmed by content: a capture is accepted only at 0.45+ Jaccard similarity against the current text. Genuine matches land between 0.76 and 0.98; four slug guesses scored below 0.17 and were rejected by that check rather than silently accepted. Nine posts with no verifiable pre-2023 capture were dropped rather than included on their current-site text. Worth recording separately: the median archived capture is only 0.92 similar to its currently published counterpart, so the live site's posts have been edited since, and measuring "pre-2023 writing" against them would have measured the wrong thing.

• The flag level says something else, and detect mode shows users flags rather than scores. On the maintainer's Wayback-verified pre-2023 posts, em-dash fires on 18.3% of paragraphs and tier1 on 12.5%. The Tier 1 words responsible are embrace (7), leverage and inflections (7), when it comes to (5), in order to (4), that said (4). Those are not AI tells in that text; they are a marketer's ordinary 2019 vocabulary, written years before the models existed. Both rates are slightly higher on the verified originals than on the current published versions, which is what later editing passes would do. Recorded in corpus/README.md as a decision to make rather than a defect to patch.

• Two defects surfaced and are filed rather than patched here: hedge-stack matches ordinary negation such as "could not possibly" (#69), and the em-dash flags on the public-domain leg are an artifact of era and Gutenberg transcription, so nineteenth-century text cannot test that rule at all.

• Corpus hygiene worth recording: two guest posts were excluded by byline, and three posts carrying "Looking back from 2025" retrospective inserts were dropped because their pre-LLM provenance is broken. The second was caught by reading the worst-scoring paragraphs, not by any check in the tooling.

• scripts/corpus.test.js covers the extraction helpers with 15 tests. A silent extraction bug would not crash anything; it would quietly change a published rate. Two tests exist purely to protect the measurement: em dashes must survive extraction, since the em-dash rule is scored against this corpus, and extraction must throw rather than return empty text, since an empty document would shrink the denominator without saying so.

• Tier 1 split into 1A frequency markers and 1B clarity edits. Tier 1 is defined by an empirical claim — these words "appear 5–20x more often in AI text than human text" — and several members could not plausibly meet it. in order to, utilize, serves as, features, boasts, commence, ascertain, and endeavor are wordiness and formality edits: worth making, but not evidence that a machine wrote the sentence. They now emit tier1-clarity, are weighted like Tier 2, and are excluded from the dense-AI-vocabulary signal, so a wordiness fix can no longer push a document toward an AI classification. The edit advice is unchanged for every word; what changed is what a flag claims. Detect mode reports the two bands separately.

• The measurement that prompted it. Against 257 paragraphs of the maintainer's verified pre-2023 writing, Tier 1 fired on 12.5%. Split, that is 8.9% markers and 3.5% clarity, with no paragraph triggering both. Roughly a quarter of Tier 1 hits on genuine human prose were wordiness being reported as an AI signal. commence and ascertain firing on the Federalist Papers and Faraday is the same problem from the formal-register end.

• The 5–20x claim is now labelled as inherited rather than measured. It traces to brandonwise/humanizer, which asserts the ratio in two places and publishes no method or dataset. SKILL.md says so plainly and commits to re-deriving the ratios once a machine-written corpus exists. The conflation of wordiness with frequency evidence is inherited too: in order to, utilize, and serves as are on that upstream list.

• Machine-written corpus, and the first true-positive rate this project has ever had. Two external datasets, neither generated by anyone with a stake in these numbers: RAID (Dugan et al. 2024, MIT — 11 model families, sampled by byte-range from an 11.8 GB CSV) and HC3 (Guo et al. 2023, CC-BY-SA-4.0 — paired human and ChatGPT answers to the same questions). Same hash-only design as the human half. scripts/csv-lite.js vendors a small RFC 4180 reader because the RAID generations contain commas, quotes, and newlines, and splitting on delimiters would silently corrupt the text being measured.

• The composite score does not separate the classes. ROC-AUC 0.501 at paragraph level pooled (HC3 0.554, RAID 0.451) and 0.623 at document level (HC3 0.654, RAID 0.599). The best operating point found costs 12.8% false positives to catch 27.7% of machine text. 0.5 is a coin flip.

• The 0–100 scale uses about a tenth of its range. No paragraph of either class scored above 11, so every threshold at or above 15 reports 0.0% on both sides, and SKILL.md's own band puts everything at or under 15 in "Minimal AI signals". Category weights run 2–12 and rawScore is divided by max(1, log2(words / 50)). This is a calibration defect rather than a detection failure, and it is the most fixable finding on the page.

• The discriminating signal is structural, not lexical. At document level uniformity fires on 2.1% of human text and 25.1% of machine text, a lift of 11.7x — the best discriminator in the engine by an order of magnitude. filler is 3.4x; chatbot, hedge-stack, and fnword-trigram-entropy are machine-only. The 112-entry vocabulary table has a lift of 0.9: it fires slightly more often on human writing than on machine writing. That is what NulightJens/humanizer-stack argues from StoryScope and what harshaneel/humanize reaches independently, and on this engine they look right.

• em-dash is inverted as an authorship signal, firing on 9.9% of human documents and 1.9% of machine ones (lift 0.2). It holds on both legs and is not a transcription artifact. Unchanged as writing advice; recorded because it points the wrong way as evidence.

• Sampling note worth keeping: evenly spaced byte offsets across RAID returned fewer model families at 40 windows than at 16, having resonated with the file's domain-then-model sort order and missed gpt4, chatgpt, and cohere entirely. Offsets now follow a golden-ratio low-discrepancy sequence, and the builder warns at build time when model, domain, or unit coverage falls short.

• Fixed the two defects the measurement found. hedge-stack allowed two words between the modal and the hedge adverb, so could not possibly and inverted questions like could a savage possibly both fired; it now allows at most one and never a negator (#69). The em-dash rule carved out list-item separators but not Keep-a-Changelog version headings (## [3.21.0] — 2026-07-30) or a bold lead term carrying a parenthetical (#67); both are carve-outs now. The version-heading pattern is deliberately narrow — a bracketed semver, a dash, an ISO date, nothing else — because SKILL.md applies the em-dash rule to headings too, and a prose dash in a heading still counts.

• Both fixes improved discrimination, measured on the corpus. hedge-stack went from firing on 0.6% of human units with a lift below 1 (it fired more on human text than machine text) to 0.2% with a lift of 2.2. em-dash false hits on human text dropped from 17.9% to 13.7%. Its lift stays inverted at 0.1, which is a finding about the signal rather than a bug in the rule.

Changed

• npm test now runs the preservation tests as well. npm run self-scan and npm run self-scan:check are new. CI runs the self-scan check as a step in the existing detector job rather than a second job, since branch protection pins required checks by job name.
• Edit mode's output format in SKILL.md now points at the validator as an optional mechanical check.