v1.1.0 — corrected findings, full reproducibility

This release corrects claims in v1.0.0 that the data does not support, and makes every published figure reproducible from the script.

Corrections to v1.0.0

• "No mixed granularities, no silent scheme drift" was false. The collection holds 17 distinct cefr_level strings: the six bands, A1+ through C2+ in three corpora, a bare B, and four separate null markers (EMPTY, NA, Unassessable, unrated). This is now Finding 4.
• "Three corpora repeat their own texts under conflicting levels" — it is ten. Only cople2_pt and elg_cefr_nl are structural; the rest are a handful of rows. caes_es was wrongly listed among them.
• "No cross-corpus overlap anywhere in thirteen languages" rested on an approximate sweep that compares only the six languages holding more than one corpus, skips 16.7% of rows under 80 characters, and has about 64% recall at its own threshold. Replaced with an exhaustive inversion over every text.
• "lang is correct on every row" overstated a metadata-consistency check. No language identifier was run.
• "673 rows under 20 characters" was 310.
• 13,611 duplicate rows were presented as a leakage figure; 77% are caes_es short-utterance coincidences that are not a defect.

Reproducibility

Figures cited in v1.0.0 but never emitted — the 130 exact matches, none ×214, the elg_cefr_nl multiplicity, the parquet file count, and "every other field identical" — are now printed by the script. FINDINGS.md states that any figure not in run_output.txt is a defect in the document, and that now holds.

New checks

cefr_level inventory · exhaustive cross-corpus inversion · field identity across repeated rows · short and placeholder rows · parquet file count per dataset · row license field against each dataset card.

New findings

• The label space is not one scheme — 17 distinct level strings, four different ways of recording no label.
• The row license field disagrees with the dataset card in 13 of 24 corpora. Six differ substantively, and in three the rows state weaker terms than the card: icle500_en rows say CC0 1.0 (Public Domain) against cc-by-nc-4.0, learn_welsh_cy says public, and zaebuc_ar drops the NC term.

Finding 1 strengthened

Containment now reports injectivity and the similarity distribution, answering the obvious objection that a 0.5 threshold might be matching shared exam boilerplate. It is not: the match is one-to-one onto 324 distinct texts with no collisions, median similarity 0.97, and only 16 of 324 below 0.7. 130 are identical. If only the exact matches are admitted, the finding stands on those alone.

Fixed

The dataset-card comparison reported "0 disagree" when it had reached no cards at all. It now reports unreachable cards and draws no conclusion for them.

v1.0.0 — a provenance and redundancy check over the public UniversalCEFR collection

Applies declare-then-test to the 24 public corpora of the UniversalCEFR
collection — 13 languages, 65,037 rows. The method: find the source a dataset
declares, then test the data against that declaration rather than trusting it.

Most of the collection is clean

One schema across all 24 corpora, lang correct on every one of the 65,037
rows, and no cross-corpus overlap anywhere in thirteen languages — with a single
exception. Every corpus already carries a source_name field, so the
declaration convention is implemented; what had not been done is testing it.

Findings

• cambridge_exams_en is 99.7% contained in elg_cefr_en. 324 of 325 texts,
  with the CEFR labels agreeing on 324 of 324; 130 are byte-identical after
  normalising whitespace and case. The two declare different repositories,
  different papers and different licences, so agreement between them reads as
  corroboration and is not.
• cople2_pt — 471 texts, each present exactly twice, as (A1,A2) ×236,
  (B1,B2) ×163, (C1,C2) ×72, every other field identical: a band expanded into
  one row per sub-level.
• elg_cefr_nl — 1,195 texts stored as 3,596 rows, mostly triples over
  contiguous levels, consistent with per-rater judgements stored as texts.
• caes_es — the placeholder string none appears 214 times across five
  different levels.
• ComplexityMT — listed in the organisation, no data files.
• Collection-wide: 13,611 of 65,037 rows duplicate another row, so a random
  split leaks.

What this cannot determine

Whether any of it originates upstream or in the conversion to the UniversalCEFR
format. From outside the two are indistinguishable; the maintainers can tell
immediately.

Reproducing

python3 universalcefr_check.py --download
python3 universalcefr_check.py

Roughly four minutes. Hashing is Blake2b rather than Python's per-process salted
hash(), so pair counts are stable across runs. The script downloads the
published exports and redistributes no data — each corpus keeps its own
CC BY-NC / CC BY-NC-SA licence.