Repository navigation
This release corrects claims in v1.0.0 that the data does not support, and makes every published figure reproducible from the script.
Corrections to v1.0.0
- "No mixed granularities, no silent scheme drift" was false. The collection holds 17 distinct
cefr_levelstrings: the six bands,A1+throughC2+in three corpora, a bareB, and four separate null markers (EMPTY,NA,Unassessable,unrated). This is now Finding 4. - "Three corpora repeat their own texts under conflicting levels" — it is ten. Only
cople2_ptandelg_cefr_nlare structural; the rest are a handful of rows.caes_eswas wrongly listed among them. - "No cross-corpus overlap anywhere in thirteen languages" rested on an approximate sweep that compares only the six languages holding more than one corpus, skips 16.7% of rows under 80 characters, and has about 64% recall at its own threshold. Replaced with an exhaustive inversion over every text.
- "
langis correct on every row" overstated a metadata-consistency check. No language identifier was run. - "673 rows under 20 characters" was 310.
- 13,611 duplicate rows were presented as a leakage figure; 77% are
caes_esshort-utterance coincidences that are not a defect.
Reproducibility
Figures cited in v1.0.0 but never emitted — the 130 exact matches, none ×214, the elg_cefr_nl multiplicity, the parquet file count, and "every other field identical" — are now printed by the script. FINDINGS.md states that any figure not in run_output.txt is a defect in the document, and that now holds.
New checks
cefr_level inventory · exhaustive cross-corpus inversion · field identity across repeated rows · short and placeholder rows · parquet file count per dataset · row license field against each dataset card.
New findings
- The label space is not one scheme — 17 distinct level strings, four different ways of recording no label.
- The row
licensefield disagrees with the dataset card in 13 of 24 corpora. Six differ substantively, and in three the rows state weaker terms than the card:icle500_enrows sayCC0 1.0 (Public Domain)againstcc-by-nc-4.0,learn_welsh_cysayspublic, andzaebuc_ardrops the NC term.
Finding 1 strengthened
Containment now reports injectivity and the similarity distribution, answering the obvious objection that a 0.5 threshold might be matching shared exam boilerplate. It is not: the match is one-to-one onto 324 distinct texts with no collisions, median similarity 0.97, and only 16 of 324 below 0.7. 130 are identical. If only the exact matches are admitted, the finding stands on those alone.
Fixed
The dataset-card comparison reported "0 disagree" when it had reached no cards at all. It now reports unreachable cards and draws no conclusion for them.