LexiGrade

A reproducible 6,000-word benchmark and CPU baseline for predicting English Speller Database (ESDB) size bands from word spelling.

LexiGrade v2 is independently derived from the open-licensed English Speller Database / SCOWL project. It includes equivalent JSONL and CSV releases, a deterministic exporter, a group-aware evaluation split, a TF-IDF logistic-regression baseline, a CLI, FastAPI service, Docker packaging, tests, and experiment reports.

The primary target is esdb_size_band: 35, 40, 50, 60-65, 70, or 80. An ESDB size is a dictionary-inclusion threshold, not an observed word frequency, learner score, or language-proficiency level.

cefr_proxy is only a secondary, heuristic alias derived from the ESDB band. It is not official CEFR data, is not validated against learners, and must not be used to assess a person's language ability. LexiGrade is an experimental research and software-demonstration project, not a production decision system.

What is included

• exactly 6,000 lowercase ASCII English headwords, 1,000 per size band;
• single words matching [a-z]+, globally restricted to 3–24 characters;
• true minimum ESDB size after the documented dialect/headword filters, computed before tag-based eligibility and banding;
• at most one sampled word from each ESDB cluster;
• deterministic SHA-256 selection and a cluster-aware holdout;
• shared word/character TF-IDF features with ESDB-band and difficulty heads;
• predictions, confusion matrices, calibration diagnostics, and a model card;
• optional explicit PyTorch Transformer training;
• dataset, manifest, full upstream notice, and baseline model bundled in the wheel.

There are no translations, example sentences, learner records, book/page metadata, private source paths, or user data in the release.

Published CPU baseline

The fixed cluster-aware split contains 4,800 training records and 1,200 holdout records. The model was trained with Python 3.11.9 and scikit-learn 1.9.0 and receives only the isolated word spelling.

Task	Accuracy	Macro-F1	ECE
ESDB size band (primary)	0.250833	0.248922	0.042284
Difficulty tier (secondary)	0.321667	0.293977	0.054427

These results are an internal reproducible floor, not external validation. ECE is a diagnostic computed on the holdout; the published logistic-regression probabilities have not been post-hoc calibrated.

Model card · Primary confusion matrix · Primary calibration diagnostic

Quickstart

Install from a checkout

git clone https://github.com/DenisGeide/LexiGrade.git
cd LexiGrade
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .

PowerShell activation:

.\.venv\Scripts\Activate.ps1

The installed package contains the published dataset, manifest, license notice, and baseline model. A repository-relative data or model path is not required for ordinary inference.

Inspect or export the bundled data

lexigrade data-info
lexigrade data-export --format jsonl --output lexigrade_v2.jsonl
lexigrade data-export --format csv --output lexigrade_v2.csv

Predict

lexigrade predict apple consistency interdisciplinary

Each result reports the predicted esdb_size_band first. cefr_proxy, difficulty_tier, and raw logistic-regression probabilities are secondary convenience outputs. The probabilities have not been post-hoc calibrated.

Run the API

lexigrade serve --host 0.0.0.0 --port 8000

Open http://localhost:8000/docs, or send lowercase ASCII words:

curl -X POST http://localhost:8000/v1/predict \
  -H "Content-Type: application/json" \
  -d '{"words":["apple","consistency","interdisciplinary"]}'

A prediction has this shape (probability maps abbreviated):

{
  "text": "consistency",
  "esdb_size_band": "60-65",
  "cefr_proxy": "B2-proxy",
  "band_confidence": 0.271149,
  "band_probabilities": {
    "35": 0.103428,
    "40": 0.179520,
    "50": 0.227671,
    "60-65": 0.271149,
    "70": 0.131491,
    "80": 0.086740
  },
  "difficulty_tier": 4,
  "difficulty_confidence": 0.351907,
  "difficulty_probabilities": {
    "1": 0.073320,
    "2": 0.185121,
    "3": 0.273719,
    "4": 0.351907,
    "5": 0.115933
  }
}

This example was produced by the bundled v2 baseline. One prediction does not describe aggregate evaluation quality; use the published holdout report for that.

Dataset

Property	Value
Version	2.0.0
Records	6,000
Sampling	1,000 records per ESDB band
Primary target	esdb_size_band
Bands	35, 40, 50, 60-65, 70, 80
Text policy	one lowercase ASCII word, 3–24 characters
Cluster policy	at most one selected record per ESDB cluster
Formats	JSONL and CSV
Upstream snapshot	en-wl/wordlist@1e5b7d3

The six bands are deliberately balanced for classification experiments. Their distribution does not estimate the prevalence of English vocabulary in the wild. Selection order is the stable digest SHA-256("lexigrade-v2" + NUL + word), followed by the word as a tie-breaker. There are no class-specific word-length windows.

The secondary aliases are deterministic:

ESDB size band	cefr_proxy	difficulty_tier
35	A1-proxy	1
40	A2-proxy	2
50	B1-proxy	3
60-65	B2-proxy	4
70	C1-proxy	5
80	C2-proxy	5

These aliases are labels for experiments, not evidence of CEFR alignment.

Read the dataset README, data card, manifest and checksums, and data terms before redistributing the data.

Train and reproduce

python -m pip install -c constraints/reproducibility-py311.txt -e ".[test]"
lexigrade train \
  --data data/lexigrade_v2.jsonl \
  --seed 42 \
  --test-size 0.2 \
  --max-features 40000 \
  --bootstrap-samples 500

The training command writes the model and an auditable report:

artifacts/lexigrade_baseline.joblib
reports/baseline/
├── MODEL_CARD.md
├── artifact_manifest.json
├── metrics.json
├── predictions.csv
├── split.json
├── band_confusion_matrix.{csv,svg}
├── difficulty_confusion_matrix.{csv,svg}
├── band_calibration.svg
└── difficulty_calibration.svg

The model receives only word text. It does not receive ESDB size, band, cluster, proxy, or difficulty fields as input features. The holdout is deterministic, stratified by esdb_size_band, and grouped by split_group.

See reproducibility and experiment tracking.

Optional Transformer experiment

The optional neural module exposes the full PyTorch optimization loop rather than hiding it behind a trainer abstraction:

python -m pip install -e ".[transformer]"
python -m lexigrade.transformer_train \
  --config configs/distilbert_multitask.json

It is an experiment template, not a claim that a Transformer is better than the published CPU baseline. See Transformer training.

Docker

docker compose up --build

The API is available at http://localhost:8000. The image uses the model bundled inside the installed package and runs as an unprivileged user. To test a trusted external model, set LEXIGRADE_MODEL_PATH explicitly.

Project layout

LexiGrade/
├── assets/                       # Technical dataset/report SVGs
├── configs/                      # Versioned neural experiment config
├── constraints/                  # Reproducibility constraints
├── data/                         # Public v2 JSONL/CSV release
├── docs/                         # Data, model, and reproducibility notes
├── licenses/SCOWL_COPYRIGHT.txt  # Full upstream notice
├── reports/baseline/             # Fixed holdout artifacts
├── scripts/                      # Pinned-source deterministic exporter
├── src/lexigrade/
│   └── resources/                # Data, notice, manifest, and model in wheel
└── tests/                        # CPU-only checks

Limitations

• ESDB size reflects spelling-dictionary scope, not corpus frequency.
• A spelling-only model can learn morphology and length shortcuts without meaning.
• The balanced sample is artificial and should not be treated as a natural prior.
• The release covers lowercase ASCII headwords and omits phrases, punctuation, capitalization, non-ASCII spellings, senses, and context.
• The holdout is an internal deterministic split, not an external validation set.
• Raw model confidence may shift outside the release distribution.
• cefr_proxy is a naming convenience only; no CEFR calibration or learner validation has been performed.

Licenses and attribution

• Code: Apache License 2.0.
• Upstream word/database material: the ESDB terms reproduced in full at licenses/SCOWL_COPYRIGHT.txt.
• LexiGrade selection, stable IDs, derived annotations, and release metadata: CC BY 4.0, only to the extent those elements are copyrightable and owned by the LexiGrade author.

CC BY 4.0 does not replace or weaken the upstream ESDB terms. Retain the full upstream notice when copying or redistributing the dataset.