English word frequency list exported from wordfreq

This is a list of 25000 English words and their frequencies, extracted from the excellent wordfreq library. The list is presented in JSON format, with one large array holding many smaller arrays with two elements, the first of which is the word and the second of which is the logarithm of the word's frequency. (To recover the original frequency fraction, use math.exp().)

I made the list because the wordfreq library has a dependency without binary builds on particular platforms, and it's probably easier to distribute this list to my students than it is to show them how to install a C++ compiler. Hopefully other people will benefit from the list as well.

Also included in this repository is the (very simple) Python script that I used to produce this list.

NOTE: No effort was made to filter this list to remove words that might be offensive or inappropriate.

Sample code

Some quick examples of using the list in Python. This code loads the word list, and prints out a word's frequency:

wordlist = json.load(open("wordfreq-en-25000-log.json"))
lookup = dict(wordlist)
lookup['allison'] # -12.0892

This code shows the word's frequency rank:

wordlist = json.load(open("wordfreq-en-25000-log.json"))
lookup = {row[0]: i for i, row in enumerate(wordlist)}
lookup['allison'] # 10171

This code picks words at random from the list, weighted by frequency (requires numpy):

import numpy as np
wordlist = json.load(open("wordfreq-en-25000-log.json"))
words = [w[0] for w in wordlist]
probs = np.exp([w[1] for w in wordlist])
picked = np.random.choice(word_arr[:,0], p=probs/np.sum(probs), size=14)
print(" ".join(picked))
# example output:
# grow does as cause surprised morning several way said eliminating about september western league

License

In accordance with wordfreq's license, this data is made available under a Creative Commons Attribution-ShareAlike 4.0 license. Please see wordfreq's attributions and documentation for more information about the data and how it was made.

The little script I used to make the list is hereby released into the public domain.