mGSL

marmooo's General Service List (English). mGSL provides a huge list with less noise and accurate statistics.

Demo

• Vocabee (mGSL based English vocabulary training app)
• graded-enja-corpus

Installation

• install marmooo/google-ngram-small-en licensed under the CC-BY-4.0
• install AGID licensed under the AGID License
• install kujirahand/EJDICT licensed under the CC0 1.0
• install datasets/world-cities licensed under the CC-BY 3.0
• install smashew/NameDatabases licensed under the Unlicense
• install LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words licensed under the CC-BY 4.0
• install RobertJGabriel/Google-profanity-words licensed under the Apache 2.0
• install fangpsh/Abbreviations licensed under the Apache 2.0
• install adambom/dictionary licensed under the MIT and Project Gutenberg License
• install umpirsky/country-list licensed under the MIT
• install umpirsky/language-list licensed under the MIT

Build

1. bash install.sh
2. deno run --allow-read --allow-write build1.js # listup
3. deno run --allow-read --allow-write build2.js # lemmatization
4. deno run --allow-read --allow-write build3.js # remove bad words

License

CC-BY-SA 4.0

Attribution

• vendor/ancdic.tsv is modified ANC Frequency Dictionary
• vendor/booqs/*.csv are BooQs licensed under the CC-BY-SA 4.0
• vendor/ngsl/*.xlsx are NGSL licensed under the CC-BY-SA 4.0