wordhoard 0.1.0

wordhoard 0.1.0

An open, multilingual, frequency-ranked dictionary built entirely from
openly-licensed sources — lemma, part of speech, corpus frequency (rank +
count), and, where the language allows, noun gender, inflection, verb
conjugation, and a CEFR estimate.

Languages & coverage

NOUN-only gender coverage (proper nouns excluded — they lack grammatical gender
in these lexicons); scripts/report.py:

lang	words	nouns	gender%	core g% (top-5k)	plural%	verb conj%
de	69,877	27,596	97%	99%	67%	39%
fr	66,814	29,355	97%	100%	47%	26%
es	56,638	22,918	95%	99%	59%	18%
it	68,063	29,028	92%	98%	41%	15%
en	85,142	31,756	— (no grammatical gender)	—	74%	58%

What's in this release

• wordhoard.db — the canonical SQLite dataset (word + word_form).
  wordhoard.db.gz is the same file gzipped (~34 MB) for a smaller download.
• wordhoard-csv-v0.1.0.zip — one CSV per language, one row per (lemma,
  POS) with a ;-joined forms column. Self-contained, for quick inspection.

Schema and column semantics are documented in the README data dictionary.

Highlights since the German-only prototype

• French, Spanish, Italian, English builds alongside German.
• Lemma/POS correction on all five languages: a kaikki known-inflection
  lexicon fixes spaCy's context-free single-token tagging (e.g. German lege
  folds into legen/VERB); genuinely ambiguous surfaces are left untouched.
• Verb conjugation tables with real features (pres.3sg, part.perf,
  impf.1pl, …) for all five languages.

Known limitations

See DATASET.md —
notably: CEFR is a frequency proxy for non-German; some tail POS noise remains
on genuinely ambiguous surfaces (by design); and Romance verb forms may include
stress-accented/dialectal spelling variants inherited from Wiktionary.

License

Dataset: CC-BY-SA-4.0 (attribution in
NOTICE.md). Code:
MIT. No Goethe content is redistributed — it is a calibration input only.