2.21.5

2.21.5 (2026-09-21)

Fix

• fix: Add tests to validate citations and update citations (#5478)

• fix: Add tests to vaidate citations and update citations

A little less than 100 cases, where found. I manually went though these.

If a case was found missing (even if just by surface features then i replaced it with a google scholar reference, ACL, etc.. If you came up a second time I wrote an exception.

• Updated citation that did not have a match, these error were various (most are mild a few are bad):
  • shortened journal articles
  • incorrectly referencing the repo
  • draft citations (where no paper existed), e.g. "Extending the Massive Text Embedding Benchmark to French"
  • some where using github handles as author names
  • where arxiv id wsa incorrect
  • master thesis that have since been taken down but published as an article (wtf!)
• Deleted a few citations, these include due to:
  • where articles (e.g. this) were taken down
  • where I couldn't reconstruct the article (this on seems to refer to the full conference proceedings
  • where they refer to it refers to the wrong article (here)
• updated description for bibtex citations

Please take care to review this in detail.

• fix duplicates

• fix citations

• clean up

• lint

• format

• add cit for tateoba

• remove none citations

• re-add swebench cit

• add timeout

• fail on unverified

• add from comments

• add missing doi

• added missing doi

• remove non-paper citations from models

• update uv.lock

• more fixes

• finished remaining

• format

• Modify commit message to skip CI for citation cache

Add [skip ci] to citation cache update commit message.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

• ci: split citation checks and restore monthly refresh

• ci: stop pinning uv to 0.10.12 in citation_check.yml

Reproduced the 45-minute hang from PR #5478's citation-check-pr run:
uv sync --group test --frozen with uv 0.10.12 deadlocks right after
creating the virtualenv, with zero output, matching the CI log exactly.
uv 0.11.22 fixed a resolver deadlock ("more deadlock-resistant concurrent
hashmap"); test.yml doesn't pin a version and gets 0.12.17, which installs
in seconds. Drop the pin so citation checks track the same unpinned,
patched uv version as test.yml.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

• fix: drop unverifiable misc citation from NQTablesRetrieval

The doshi2026tableir @misc entry has no DOI or arXiv id, so refaudit
can't confirm it and a title-only lookup false-matched an unrelated
Crossref record, failing the citation check. Keep only the
herzig-etal-2021-open citation, which the dataset is adapted from and
verifies cleanly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

• ci: split install and test steps in model_loading.yml, use --frozen

Same root cause as the reference-models hang: uv sync without --frozen
triggers a full universal re-lock (recomputing metadata across every
platform/Python version for all 80 extras), which stalled in CI for the
extract-and-run job. Add install-for-model-load-test to run uv sync --frozen, separate from running the test itself, matching the
install-for-tests/test split used elsewhere.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> (da37120)