v0.5.1 — self-audit defect fixes (D1–D6)

What changed

This release fixes six defects found by auditing the tool's own template
against 25 real arXiv cs.CL LaTeX sources — the corpus and the full
coverage matrix are in case-studies/arxiv-corpus/.

• D1 — neutral factory config. CHECKS_CONFIG shipped pre-filled with
  one specific paper's config, so a generic audit fired that paper's checks
  against any document. It now ships empty: the per-paper categories
  (C1/C2/C3/C6) report not applicable; C4's self-cite sub-check stays
  silent without a prefix. The generic checks still run on built-in defaults.
• D2 — "clean" is not "did not run". The summary distinguishes
  checked / not applicable / error per category.
• D3 — one crashed check no longer aborts the audit. A crash degrades
  that category to [ERROR] and the rest still run; exit code 2 now also
  covers a crashed check.
• D4 — BibTeX key regex. @\w+\{([^,]+), swallowed newlines, so a key
  on its own line was captured with its surroundings and cite keys that
  existed in refs.bib were reported missing. Replaced by
  @\w+\{\s*([^,\s]+)\s*, with regression tests.
• D5 — raw docstring. No more SyntaxWarning on import.
• D6 — C1 observation failures are errors, not findings. A missing
  abstract no longer files as a HIGH finding.

Verification

• 240 tests passing, 1 skipped (pytest -m "not slow"), pyflakes clean.
• The regression-injection meta-test is 45/45 effective, including
  "full suite returns rc=0 after all restores".
• CI: 8 jobs green.

Reproducing the audit

The v0.5.0 findings are kept verbatim. audit_corpus.py restores the
shipped v0.5.0 config on a v0.5.1+ template, so the config-driven defects
(D1/D2/D3) replay as recorded; D4 and D6 live in the function bodies and
cannot be restored by configuration — the script and the case-study README
say so explicitly.

v0.4.0 — Plugin API

v0.4.0 — Plugin API: third-party audit checks via entry-points

TLDR. Third-party packages can now register audit checks via [tool.tmaudit.plugins] in pyproject.toml. tmaudit discovers them via importlib.metadata.entry_points, runs them via audit-all per paper, and caches per (paper, plugin, tex) fingerprint.

218 unit tests pass, 14/14 meta-test bugs caught, 44/44 community-files checks pass.

---

🎉 What's new

Finding + @check + entry-points

A user-side plugin is now just:

from tmaudit import Finding, check

@check(name='no-todo', severity='LOW',
       help_text='Flag TODO markers', version='0.1.0')
def flag_todos(tex, config=None):
    return [
        Finding(category='TODO', severity='LOW',
                message=f'TODO at line {i+1}', line=i+1)
        for i, line in enumerate(tex.splitlines())
        if 'TODO' in line
    ]

declared in your pyproject.toml:

[project.entry-points."tmaudit.plugins"]
no-todo = "my_plugin.checks:flag_todos"

Discovered automatically:

$ pip install -e ./my_plugin
$ tmaudit plugins list        # shows 'no-todo'
$ tmaudit plugins info no-todo
$ tmaudit plugins run no-todo --paper 5

audit-all + cache integration

Plugins run automatically when you tmaudit audit-all. Each plugin's findings are cached by (paper_n, plugin_name, plugin_version, tex_hash) so plugin updates and paper edits both invalidate.

Per-paper disable

'c11_plugins_disabled': ['no-todo'],   # in PAPER_CONFIGS

Example plugin ships in-tree

tmaudit_example_plugin/ (sibling directory) registers three demo checks:

• flag-todo-markers (MEDIUM): flags TODO / FIXME markers.
• flag-xxx-markers (LOW): flags XXX debug markers.
• flag-long-abstract (MEDIUM, configurable): flags abstracts > N words.

Install with pip install -e ./tmaudit_example_plugin and copy as a starting point.

Bug 13 + Bug 14 (meta-test)

• Bug 13: Plugin loader raises on a malformed entry-point.
• Bug 14: Per-paper c11_plugins_disabled is ignored.

Both caught by _check_all_regressions.py (now 14/14).

§14 engineering notes

~415 lines, 14 sub-sections documenting the design rationale: motivation, protocol, pyproject schema, discovery, lifecycle, CLI, per-paper override, cache integration, test plan, limitations, roadmap to v1.0, alternative designs, and implementation milestones.

---

📋 Highlights

1. Plugin API (the headline feature of v0.4.0).
2. tmaudit plugins list / info / run subcommands.
3. Per-paper c11_plugins_disabled config.
4. Plugin-fingerprint cache keys (plugin_cache_key).
5. Example plugin ships in-tree (tmaudit_example_plugin/).
6. Bug 13 + Bug 14 meta-tests (regression-injection discipline).
7. §14 engineering notes (415 lines, 14 sub-sections).
8. Paper 5 Bug 6 fix (multi-branch c2_section_pattern).

---

📦 Upgrade from v0.3.0

git pull
pip install -e .[dev]
# Try the example plugin:
pip install -e ./tmaudit_example_plugin
tmaudit plugins list
tmaudit audit-all
python _check_all_regressions.py   # expect 14/14 caught

See RELEASE_NOTES_v0.4.0.md for the full release notes and CHANGELOG.md for commit-by-commit history.

v0.3.0 — 10 audit categories, statistical rigor

v0.3.0 — 10 audit categories, statistical rigor

TLDR. Closes the v0.3.0 vision: all 10 audit categories (C1–C10) are implemented. Three new categories — C8 (statistical-power), C9 (figure-caption), C10 (reproducibility) — bring scientific-rigor checks alongside the long-standing stylistic-rigor checks.

176 tests pass, 12/12 meta-test bugs caught, 27 commits since v0.1.1.

---

🔬 What's new

C8 — Statistical-power audit

For each c8_claimed_effects entry (per-paper opt-in config), the auditor:

• Re-derives Cohen's d from the matching \begin{tabular} row.
  A mismatch |d_actual - d_claimed| > 0.10 emits a HIGH finding.
• Computes post-hoc power via Φ(|d|·√(n/2) − z_{α/2}).
  Below 0.50 → underpowered (MED, recommend n×4).
  Above 0.99 with small claimed d → suspiciously overpowered (MED).
• Scans for "significantly different" claims in the body.
  Missing p-value within 200 chars → MED.
  Internally inconsistent (d < 0.10 with p < 0.001) → MED.

C9 — Figure-caption audit

Four sub-checks (sub-checks 1, 2, 4 always run; sub-check 3 opt-in):

• Caption exists (HIGH): figure without \caption{...} is undocumented.
• Caption placement (MED): per IEEE/ACM/TMLR convention,
  \caption should appear BELOW \includegraphics.
• Caption content (MED, opt-in): each caption must mention at least
  one of the per-paper c9_figure_keywords.
• Figure referenced (MED): every \label{fig:...} must be
  referenced by \ref / \autoref / \cref in the body.

C10 — Reproducibility audit

Three sub-checks (sub-checks 1, 3 always run; sub-check 2 opt-in):

• Availability statement (HIGH if absent, MED if future-tense):
  the paper must say where the code/data can be obtained.
• Statement consistency (MED, opt-in via c10_reproducibility_claims):
  SOTA claim coupled with "we do not release" is flagged.
• Reproducibility metadata (LOW): missing hyperparameters, random
  seed, hardware spec, or library version.

---

📋 Highlights

1. 10 audit categories (C1..C10) — closes the v0.3.0 vision.
2. 176 unit tests pass (was 122 in v0.2.0). Test counts: C8 +18,
   C9 +16, C10 +20.
3. 12/12 meta-test bugs caught — every category now has a
   regression-injection test that re-introduces a specific bug and
   asserts that the targeted test fails (catches the regression).
4. 3 new per-paper opt-in configs: c8_claimed_effects,
   c9_figure_keywords, c10_reproducibility_claims. Empty list
   disables the corresponding sub-checks.
5. §12 + §13 engineering notes document the design rationale
   (~1100 lines in engineering_notes_verify_template.md).
6. Bug 6 anchor refresh — paper_configs.py Paper 5's
   c2_section_pattern now correctly accepts \subsection{Statistical Protocol} (not just \section{Power analysis}).

---

📦 Upgrade from v0.2.0

git clone https://github.com/aidless/tmaudit.git
cd tmaudit
git checkout v0.3.0
pip install -e .[dev]
python gen_verify_scripts.py --paper 1   # regenerate per-paper verify
python _check_all_regressions.py          # confirm 12/12 caught

c8_claimed_effects, c9_figure_keywords, and c10_reproducibility_claims
are now part of CHECKS_CONFIG. Empty defaults are written for Papers 2/4;
example entries exist for Papers 1/3/5 (C8) and Papers 1/5 (C9 / C10).

See RELEASE_NOTES_v0.3.0.md for the full release notes and
CHANGELOG.md for the commit-by-commit history.