Roadmap

A phase-by-phase plan for growing the profilers from two standalone CLIs into a complete text-leveling toolkit for the EFL classroom — and, ultimately, the shared analysis engine behind the RubricMaker grading platform.

The through-line is one teacher question: "is this text right for this class, and if not, what has to change?" Each phase moves closer to answering it in one command, and closer to feature parity with what RubricMaker already does inside a full app (see RubricMaker Alignment).

Status legend — ✅ shipped · 🔨 in progress · 📋 planned · 💭 idea

This is a direction of travel, not a dated commitment. Phases are ordered by dependency: each builds on the one before.

Phase 0 — Foundations ✅ (shipped)

Everything that exists today.

✅ Vocabulary profiler — CEFR (A1–C2), AWL, and NAWL banding, deterministic word-list matching, dependency-free Python 3.
✅ Grammar profiler — ~70 constructions across every major family, mapped to CEFR via the CEFR-J Grammar Profile, rule-based over a spaCy parse (no AI).
✅ File formats — .txt, .md, .docx, .pdf, and stdin.
✅ Two output modes — colour pretty for the terminal, json for scripts, auto-selected by whether output is a TTY.
✅ Claude Code plugins & skills — ask for a level in a Cowork session.
✅ Rebuilt, provenance-documented word lists and a citable grammar profile.

Outcome: you can profile any single text two ways, by hand or in chat.

Phase 1 — One text, one verdict ✅ (shipped)

Make the two profilers answer as one, and answer against a target.

✅ Unified difficulty report — text_report.py runs both profilers and prints one combined summary (vocabulary band + grammatical range + a blended estimatedLevel), instead of two separate invocations.
✅ --target-level B1 — pass the class's level and the report flags what exceeds it: words above B1, constructions above B1, and a one-line verdict ("on level" / "reaches B2 — pre-teach 6 words, 2 structures").
✅ Coverage figure — "a B1 learner will already know ~92% of the running words," the single number teachers ask for most.
✅ Readability line — Flesch–Kincaid / Flesch Reading Ease, reported alongside — never instead of — the CEFR bands.
✅ Pre-teaching exports — --export md|csv|flashcards write the above-target words and structures as a handout, spreadsheet, or RubricMaker-importable flashcard deck; --cloze turns the examples into fill-the-gap sentences that paste straight into a RubricMaker fill-the-gap question.
✅ Free dictionary enrichment — decks ship with plain definitions, phonetics and part-of-speech from the Free Dictionary API (no key), backed by a persistent JSON lookup cache; --pre-enrich primes the cache for a whole class's word list in one polite, rate-limited pass. Offline or on a miss, the deck falls back to in-context sentences.
✅ OLP-EN-CEFRJ wordlist merge — build_wordlists.py gap-fills the CEFR lists from the open CEFR-J + Octanove profiles and emits levels.json, so CEFR levels need no dictionary API.

RubricMaker parallel: its document analysis already reports vocabulary and grammar on one uploaded essay — Phase 1 gives the CLI the same combined view, and its deck import now reads the CLI's enriched output directly.

Phase 2 — A class set, not one text ✅ (shipped)

Scale from one text to the folder of candidate readings a teacher actually has.

✅ Batch mode — --file accepts a directory, a glob (** for recursion), or a single file; every supported text (.txt/.md/.docx/.pdf) is profiled in one run, with unreadable files skipped — never fatal.
✅ CSV / summary report — one row per text (filename, typical & reached vocabulary band, grammar range, % above target, fits verdict), ready for a spreadsheet.
✅ Rank & filter — --sort ranks the set by level (or band / word count / name); --min-level/--max-level keep only the texts in the band — so "which of these 20 articles suits B1?" is exactly class_profile.py --file essays/ --max-level B1.
✅ Several classes at once — --targets A2,B1,B2 shows each text's fits / %-above verdict for every level side by side in one run.
✅ Aggregate distribution — the pooled CEFR distribution over the whole set (typical band, 90%-coverage band, off-list share) — the dashboard's headline chart, from the command line.
✅ Vocabulary-list export by band — --export-vocab dumps the distinct words in each CEFR band to CSV (vocab-A1.csv … vocab-C2.csv, plus vocab-off-list.csv), each row a word with its running occurrences and the texts it appears in — ready-made pre-teaching lists and glossaries.
✅ Pre-teaching exports for the whole set — --export md|csv|flashcards writes one handout / worksheet / RubricMaker deck per text (reusing the Phase 1 writers), plus a combined class-wide deck with a level-keyed markdown index of every above-target word and the texts it came from (per level under --targets), and a set-level summary handout.
✅ --pre-enrich — prime the dictionary cache from the whole folder's distinct vocabulary in one polite, rate-limited pass before exporting decks.
✅ Plugin & skill — class-profile ships as a Claude Code plugin and skill like the other three tools, so "which of these suits B1?" works in a session.
✅ Sample folder + CI golden checks — the repo bundles sample-readings/ (an A1→C2 gradient) and CI asserts the full CSV / JSON / export / band contracts against it end to end.

Why now: Use Cases 5 and 10 are one-command operations now — what used to be shell loops is first-class.

RubricMaker parallel: directly mirrors RubricMaker's Vocabulary Profile dashboard, which aggregates a class's texts into a CEFR distribution and exports vocabulary lists by CEFR band to CSV — Phase 2 produces the same artifact from the command line, plus ranking/filtering the dashboard doesn't have.

Phase 3 — From analysis to adaptation 🔨 (in progress)

Stop only describing difficulty and start helping change it — the heart of the original VocabKitchen ("adjust a text to a target level").

✅ Above-target highlighting — the report lists every word and structure above the target level, coloured in pretty mode, so a rewriter sees the offenders at a glance (Use Case 6). Already delivered by text_report.py --target-level and the band-tinted vocabulary view; the class profile ranks the offenders across a whole set.
✅ Simpler-synonym suggestions — text_report.py --suggest suggests a lower-band alternative (purchase → buy (A1)) for each above-target word in the bundled curated list (WordLists/synonyms.csv, validated against levels.json by build_wordlists.py --check) — inline in the terminal, in JSON (aboveTarget.words[i].suggestion), and as a Simpler alternative column in the --export md handout.
✅ Grammar gap report — text_report.py --gap-report lists the target-level constructions the text does not use yet — the "introduce these structures" checklist for graded-reader authors who need to add, not just remove — grouped by category in the terminal, in JSON (grammarGap.missing), and as a Constructions to introduce section in the --export md handout. Needs the grammar side; the class profile reuses the same per-text handouts.
✅ Vocabulary interleaving — class_profile.py --interleave builds a spaced-introduction schedule across a folder of readings: each reading introduces at most --new-words-per-reading new above-target words (overflow is deferred to the next reading with room), words that recur later are flagged for spaced review, and words absent for two or more readings are marked due — written as a <set>-interleave-<LEVEL>.md|csv plan next to the handouts, so repeated readings introduce new words in a spaced pattern instead of all at once.
✅ Watch mode — text_report.py --watch keeps re-profiling the --file input whenever it changes on disk (polls every second; --watch 0.2 for faster) until Ctrl-C — the tight edit → re-check loop: save the graded reader and the report updates on the spot.

Why now: Phase 1 knows the target and Phase 2 handles many texts, so adaptation aids are the natural next step and the highest-value one for materials writers — all five Phase 3 items are now shipped.

RubricMaker parallel: RubricMaker's grammar linker checks target grammar per-criterion; the CLI's gap report is the authoring-side twin — what's missing versus what's present.

Phase 4 — Standards & framework alignment 📋

Speak the frameworks teachers are held accountable to, not just raw CEFR.

✅ Curriculum checklist — profile a text (or unit) against a supplied checklist file (sections [vocabulary] and [grammar]) and get a pass/fail coverage report: each required word present or missing (with its CEFR band when recognised), each required construction used or not, curriculum.pass in JSON, a terminal checklist, and a Curriculum checklist section in the --export md handout — threaded through class_profile.py --export md so a folder run shows which texts cover the unit's requirements, and through --export csv as a folder-level coverage grid (one row per text, one column per required item) — the same grid is mirrored as a Curriculum coverage section in the set-level summary handout so --export md shows the grid too, and rides in the JSON report as curriculumCoverage (items × rows × cells) so scripts can consume the pass/fail matrix without CSV parsing. The checklist file itself is schema-validated before any profiling: unknown section headers (e.g. [grammer]) fail fast with a did you mean hint, malformed headers error, and empty required sections — plus grammar items that don't resolve against the construction list (typos like second conditinal get a did you mean hint) — warn before any profiling (Use Case 7).
✅ Cambridge English exam mapping — text_report.py --cambridge reports the matching Cambridge Qualification (A2 Key, B1 Preliminary, B2 First, C1 Advanced, C2 Proficiency) next to each band the report actually shows — vocabulary typical/reaches, grammar typical/reaches, estimated level — as an opt-in, in JSON (cambridge), the terminal, and the --export md handout.
✅ CEFR Can-Do framing — text_report.py --cando expresses a text's demands as Can-Do descriptors, the language rubrics and self-assessment already use: one descriptor per band the report actually shows (reading/listening, interaction, spoken/written production) — opt-in, in JSON (cando), the terminal, and the --export md handout. With --target-level, each dimension also reports aboveTarget: the descriptors the text demands beyond what the class is expected to do yet (every level strictly above the target up to the text's own band), shown as an Above the {target} target block — the same diff threads through class_profile.py --export md so a folder run shows every text's Can-Do demands (and their gap against the class level) per handout, with --export flashcards writing the demands as a combined Can-Do reference deck in the RubricMaker import shape — under --targets each class level gets its own deck, demands measured against that level — and --cando-diff aggregating the set's shared above-target demands into the summary handout — which descriptors the texts demand together, most-common first (or, with --cando-diff-sort band, ordered by the CEFR ladder ascending to see which demand levels to tackle in order).
✅ Pluggable vocabulary profiles — first-class support for swapping in Oxford 3000/5000 or the Octanove C1/C2 profile via the existing one-word-per-line format, for teams standardised on a licensed list. Shipped: --profile on all three CLIs swaps the bundled CEFR lists for a directory of A1.txt..C2.txt (file = level, replaces the lists) or a single recognition-list file (levels from the bundled levels.json index, --profile-level fallback, words outside it off-list); the active profile rides in the report payload as vocabulary.profile (contract 1.4).
📋 Profile semantics hardening — a profile run changes what "level" means, and the payload doesn't say so: with a single-file recognition profile, every word the bundled levels.json doesn't know lands in one fallback band (default B1) and words outside the team list are all off-list, so coverage, typical, and the blended estimatedLevel/verdict mean "relative to the team's list", not CEFR — a teacher could misread a profile run's "suits B1?" verdict as a standard CEFR verdict. Separately, a --profile directory with no A1.txt..C2.txt files is silently accepted (all-empty bands → 100% off-list, exit 0). Fix (small, medium value): reject a profile directory with no level files; when a profile is active, tag the payload (a profile note in the verdict or a vocabulary.profileContext flag) so consumers know the semantics changed.

RubricMaker parallel: RubricMaker offers the exact same Cambridge English exam mapping and links criteria to standards (CCSS/NGSS, Dutch kerndoelen, CEFR Can-Do). Phase 4 makes the CLI produce output in those same frameworks, so a text profiled on the command line and a criterion graded in the app describe proficiency identically.

Phase 5 — The analysis engine behind RubricMaker 🔨

Close the loop: make these profilers the authoritative text-analysis backend RubricMaker calls, so both tools share one CEFR taxonomy and one implementation.

✅ Stable library / JSON contract — a documented, versioned output schema and an importable entry point (not just a CLI), so RubricMaker's document analysis (vocabulary & grammar checking on uploaded essays) can call the profilers directly instead of maintaining a parallel implementation. Shipped: the shared analysis engine (analysis.py) — word lists + grammar engine loaded once, one payload builder — is imported by both CLIs, so a single text and a whole folder now produce byte-identical report shapes (a folder handout is the same artifact as the single-text report), and python3 -c "import analysis" is the entry point the app can build on. The engine is fully self-contained (readability, target flagging, Cambridge/Can-Do mapping and the curriculum checklist live in it; text_report.py re-exports them), and the payload is a versioned contract: every report carries schemaVersion (1.3), the JSON Schema is checked in at analysis.schema.json (byte-equal to analysis.payload_schema(), asserted in tests + CI, and printable via --schema on either CLI).
✅ Per-criterion grammar pass/fail — emit results in the shape RubricMaker's grammar linker consumes for its per-criterion, apply-as-comment breakdown. Shipped: every grammar-enabled payload carries grammarCriteria — one entry per registered construction (id/name/category/level), a used/not used status with pass/fail, and count + up to two examples when used, ladder-ordered — so a comment can be attached per criterion without re-deriving anything. The apply-as-comment reference implementation is shipped too: analysis.grammar_comments turns the criteria into one rubric comment per construction (Uses the … — E.g. "…" with the detected span as evidence / Doesn't use the … yet), carried in the payload as grammarComments under --comments and rendered as a Rubric comments section in the --export md handouts — per text in a folder run too. The vocabulary half of the pass is shipped too: analysis.vocabulary_comments turns the above-target words into one comment each (Above B1: "anticipate" (B2) — used 1×. E.g. "…", with the simpler synonym appended under --suggest), carried as vocabComments and rendered as a Vocabulary comments section alongside the grammar half — so --comments covers the whole report. Under a targetLevel the grammar half is filtered to the class level: at/below constructions keep their pass/fail comments (kind "rubric"), used above-target constructions become "pre-teach" notes (Uses the … — above the B1 target: pre-teach or rewrite.), and unused above-target ones drop — a B1 class isn't expected to produce C2 structures. Every pre-teach note carries a rewrite suggestion from the curated WordLists/structure-rewrites.csv (validated by build_wordlists.py --check against the construction registry): Rewrite: swap for a past simple or a present modal ("would have passed" → "passed"). The folder surface completes the picture: --export md adds a Demand scan table to the set summary (per-text above-target word + pre-teach structure counts, the whole-class demand at a glance), and --export flashcards --comments writes a combined rubric-comment deck — one per --targets level, so the same construction is a rubric card at its own level and a pre-teach card above it.
✅ Feed the Vocabulary Profile dashboard — batch-profile a class's essays and hand back the per-student CEFR vocabulary distribution the dashboard renders. Shipped: every class_profile JSON row now carries the per-student CEFR distribution (per-level counts + percentages, total, typical, coverage), so a class's essays become per-student band spreads without re-deriving from the pooled aggregate.
✅ Single source of truth for grammar — the CEFR-J Grammar Profile bundled here becomes the shared taxonomy both tools cite, so a construction is levelled the same way whether it's detected in a reading (CLI) or an essay (app). Shipped: grammar_profile --taxonomy exports the full construction registry (id/name/category/level/cefrjCode, levelled exactly as detection resolves them); GrammarProfile/taxonomy.json is the checked-in document both tools cite, guarded byte-equal by tests and CI.
✅ Free lookup swap in RubricMaker — replace cambridgeApi.lookupWord (the paid Cambridge Dictionary API) with the free stack the CLI now uses: CEFR levels from the bundled CEFR-J / levels.json, definitions + phonetics from the Free Dictionary API, sharing the same cache format so teacher lookups and deck exports use one warmed cache. Shipped: dictionary.py is the shared layered lookup — the cambridgeApi.lookupWord replacement (dictionary.lookup_word), levels always from levels.json (never the API), definitions/phonetics/POS from the Free Dictionary API with the bundled WordNet as the offline fallback, all through one JSON cache with results tagged by source.
✅ Offline dictionary fallback — bundle Open English WordNet (CC BY 4.0) so definitions ship even with no network, writing results into the same lookup cache — the final layer that makes enrichment fully offline-capable. Shipped: WordLists/dictionary/wordnet.json (built by build_dictionary.py, CC BY 4.0) answers when the Free Dictionary API is unreachable or misses, so definitions ship with no network; deck exports fall back to WordNet glosses before the in-text sentence, on both CLIs.
✅ Import RubricMaker vocabulary lists into --pre-enrich — read a class's vocabulary-list export (CSV/JSON) directly, so the cache is primed from the app's own data without reformatting. Shipped: --pre-enrich reads a class's RubricMaker CSV/JSON vocabulary-list export directly (word column / words array), on both CLIs.
📋 Harden the shared dictionary cache — transient offline failures must never be persisted as definitive misses. The Phase 5 rewrite routes every lookup through one write, so when the API is unreachable and WordNet doesn't know the word, the entry lands in the cache as None — byte-identical to a definitive 404 miss. pre_enrich_words flips to offline mode after the first network error, so one unreachable API None-caches every remaining word in a class list, and later online runs answer "cache-miss" without ever touching the API. The "network errors are never cached" docstring is now false; dictionary.lookup_word has the milder variant (offline misses persisted as source: "offline" dicts, never retried). Impact: silent, permanent degradation of the one shared cache the whole lookup stack is built on — a single offline pre-enrich poisons teacher lookups and deck exports alike. Fix (tiny): never persist source == "offline" entries; prefer the API over cached WordNet/offline entries when back online (source-aware reads) so WordNet answers upgrade to real definitions + phonetics; add the regression test — offline pre-enrich, then an online export, must still hit the API.
💭 Invert the dictionary dependency — dictionary.py (the shared stack) must not import a CLI: today it imports text_report at module level while both CLIs import dictionary only inside functions (four+ sites) precisely to dodge a cycle — the module graph is dictionary → text_report → analysis → {vocab_profile, grammar_profile}. Impact: the function-level import dictionary sprinkles are a smell, any future top-level import from a CLI creates a cycle, and the "single importable entry point for RubricMaker" story is muddier than the docs claim. Fix (medium, low risk): move the fetcher + cache primitives (lookup_dictionary, parse_dictionary_entry, _DictNetworkError, cache load/save, default_dictionary_cache_path) out of text_report.py into the shared layer so it's dependency-free and the fallback chain, cache, and read_word_list live in one place — the mock-HTTP test harness in test_text_report.py already covers this surface.
💭 Rate-limit resilience for --pre-enrich — lookup_dictionary folds 429 and 5xx into _DictNetworkError, which pre-enrich treats as "offline" — so one rate-limit response mid-way through a 200-word class list stops the entire pass. Fix (small–medium): retry with backoff, or soft-fail only after N consecutive errors, so a single 429 doesn't abort the marquee Phase 5 flow against a hobby-host API.

Why last: it depends on the combined report (Phase 1), the framework alignment (Phase 4), and a proven detection surface — but it's the destination that makes the whole roadmap coherent: one leveling engine, two front ends (a CLI for materials prep, a web app for grading).

Phase 6 — Beyond the command line 📋

The engine installable anywhere: package the shared analysis core so any front end — RubricMaker, a future standalone GUI, a notebook — imports one contract, with the performance and data hygiene that implies.

📋 Package the suite as a vocabkitchen package — stop being five sibling scripts and become a package. Every module imports its siblings by bare name (all must sit on sys.path together), the plugins are symlink farms CI has to dereference at install time, and each plugin bundle carries a full copy of WordLists — now including the 10 MB wordnet.json, three times over. A real package (pyproject.toml, console entry points) behind thin one-line script shims (from vocabkitchen.cli import main; main()) would: kill the symlink/dereference fragility and the deferred-import hackery, give RubricMaker one real contract (pip install vocabkitchen), let the payload schema ride as package metadata (the byte-equality guard stays — it's great), and make the plugins thin wrappers around installed commands. Tradeoff: the README's beloved "no install, just python3 X.py" story — survivable via the shims. Effort: large — the Phase 6 change that compounds every other one here.
💭 Performance & data hygiene — memoize load_level_index: vocab_profile.load_level_index re-parses the 3.5 MB levels.json on every call (deck export, pre-enrich, lookup paths — several times per run), and the codebase already has the _wordnet_cache pattern and tracks perf baselines (perf-*.json in CI), so this is a 5-line win in the project's own value system. And watch the 10 MB wordnet.json: it's parsed eagerly into a big Python dict on the first fallback miss and copied into every plugin install — a gzip + lazy per-initial-letter load (or JSONL) would cut memory and repo weight, but it's a real format change, worth doing only as part of the packaging work.
💭 Audio-backed flashcards — a new direction nobody's considered: parse_dictionary_entry already receives phonetics[].audio URLs from the Free Dictionary API and drops them. For a flashcard tool aimed at learners, audio on the card back is the kind of feature that changes perceived quality. It needs a RubricMaker importer column (or an audio-manifest sidecar export) and has no offline path — so it's a "when the app catches up" idea, but the freshest branch this diff opens.

Why now: Phase 5 made the engine one importable thing; Phase 6 makes it one installable thing — the enabler for the roadmap's "one leveling engine, two front ends" and the basis for the standalone GUI Frontend research that sits after the phases.

How the phases build on each other

Phase 0  two profilers, one text            ✅ shipped
   │
Phase 1  one combined verdict + target      ✅ shipped
   │
Phase 2  many texts, CSV, by-band export    ✅ shipped  → answers "which of these?"
   │
Phase 3  highlight + suggest + gaps ✅    🔨  → answers "what do I change?"
   │
Phase 4  Cambridge / curriculum / Can-Do     → answers "in my framework"
   │
Phase 5  shared engine for RubricMaker       → one taxonomy, two front ends
   │
Phase 6  installable engine, beyond the CLI  📋  → answers "pip install it"

Already-shipped groundwork that makes this realistic: a data-driven design (word lists and the grammar profile are swappable files, not hard-coded), a stable JSON output the tests freeze, and a no-AI, rule-based core that is deterministic and auditable — the same principles RubricMaker's own analysis is built on.

After the phases — a GUI front end 💭

Everything above stops at one importable, installable engine. Sitting after the phases — not as one of them — is the standalone GUI: a separate front end over the same engine for teachers who don't want a terminal at all. The viability research (engine-fit evidence, toolkit options, packaging reality, effort estimates, recommended sequencing) lives on GUI Frontend, deliberately placed at the very tail end of this plan: it builds on Phase 5's importable engine and Phase 6's packaging, and it changes none of what the phases deliver.

Vocabkitchen CLI

Home
Use Cases
Roadmap
RubricMaker Alignment
GUI Frontend — after the phases

Reference

Related

RubricMaker

Navigation Menu

Roadmap

Roadmap

Phase 0 — Foundations ✅ (shipped)

Phase 1 — One text, one verdict ✅ (shipped)

Phase 2 — A class set, not one text ✅ (shipped)

Phase 3 — From analysis to adaptation 🔨 (in progress)

Phase 4 — Standards & framework alignment 📋

Phase 5 — The analysis engine behind RubricMaker 🔨

Phase 6 — Beyond the command line 📋

How the phases build on each other

After the phases — a GUI front end 💭

See also

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Vocabkitchen CLI

Clone this wiki locally