v1.0.0 — a provenance and redundancy check over the public UniversalCEFR collection

Applies declare-then-test to the 24 public corpora of the UniversalCEFR
collection — 13 languages, 65,037 rows. The method: find the source a dataset
declares, then test the data against that declaration rather than trusting it.

Most of the collection is clean

One schema across all 24 corpora, lang correct on every one of the 65,037
rows, and no cross-corpus overlap anywhere in thirteen languages — with a single
exception. Every corpus already carries a source_name field, so the
declaration convention is implemented; what had not been done is testing it.

Findings

• cambridge_exams_en is 99.7% contained in elg_cefr_en. 324 of 325 texts,
  with the CEFR labels agreeing on 324 of 324; 130 are byte-identical after
  normalising whitespace and case. The two declare different repositories,
  different papers and different licences, so agreement between them reads as
  corroboration and is not.
• cople2_pt — 471 texts, each present exactly twice, as (A1,A2) ×236,
  (B1,B2) ×163, (C1,C2) ×72, every other field identical: a band expanded into
  one row per sub-level.
• elg_cefr_nl — 1,195 texts stored as 3,596 rows, mostly triples over
  contiguous levels, consistent with per-rater judgements stored as texts.
• caes_es — the placeholder string none appears 214 times across five
  different levels.
• ComplexityMT — listed in the organisation, no data files.
• Collection-wide: 13,611 of 65,037 rows duplicate another row, so a random
  split leaks.

What this cannot determine

Whether any of it originates upstream or in the conversion to the UniversalCEFR
format. From outside the two are indistinguishable; the maintainers can tell
immediately.

Reproducing

python3 universalcefr_check.py --download
python3 universalcefr_check.py

Roughly four minutes. Hashing is Blake2b rather than Python's per-process salted
hash(), so pair counts are stable across runs. The script downloads the
published exports and redistributes no data — each corpus keeps its own
CC BY-NC / CC BY-NC-SA licence.