BYU's Corpus of Contemporary American English
560 million words / 220,000 texts. US, 1990-2017. Best coverage of all types of genres (informal to formal): spoken, fiction, magazines, newspaper, academic. The most widely-used corpus of English. The Corpus of Contemporary American English (COCA) is the largest freely-available corpus of English, and the only large and balanced corpus of American English. COCA is probably the most widely-used corpus of English, and it is related to many other corpora of English that we have created, which offer unparalleled insight into variation in English.
This corpus was acquired from Mark Davies using the above URL in April, 2018.
This corpus is available in three formats: Database (db),
Word/lemma/POS (wlp), and plain text (text). The text is partitioned
into different genres (spoken, fiction, magazine, newspaper,
academic). The 2012-2015 files contain updates to the original corpus
(for all genres) Note that 10 tokens every 200 tokens have been
replaced by @ @ @ @ @ @ @ @ @ @ to get around copyright
restrictions.
A campus-wide (academic multi-user) license was purchased for this corpus by Noah Smith and Yejin Choi (via Dallas Card and Eunsol Choi).
To be granted access to the BYU corpora, you must read and agree to the terms of the license (below).
After reading the license (pay careful attention to the distribution restrictions), contact one of the nlp-corpora maintainers to be granted access to the set of BYU corpora:
- Suchin Gururangan --- sg01@cs.washington.edu
(The following is a copy of LICENSE.txt from the original corpus.)
Restrictions on use of the corpora
You must agree to these restrictions in order to obtain the data , or else obtain a waiver from us for a particular point listed below.
[ Data ]
-
In no case can substantial amounts of the full-text data (typically, a total of 50,000 words or more) be distributed outside the organization listed on the license agreement. For example, you cannot create a large word list or set of n-grams, and then distribute this to others, and you could not copy 70,000 words from different texts and then place this on a website where users from outside your organization would have access to the data.
-
The data cannot be placed on a network (including the Internet), unless access to the data is limited (via restricted login, password, etc) just to those from the organization listed on the license agreement. Academic Single-User licenses do not allow the data to be distributed over a network.
-
In addition to the full-text data itself, #2 also applies to derived frequency, collocates, n-grams, concordance and similar data that is based on the corpus.
-
If portions of the derived data is made available to others, it cannot include substantial portions of the the raw frequency of words (e.g. the word occurs 3,403 times in the corpus) or the rank order (e.g. it is the 304th most common words). (Note: it is acceptable to use the frequency data to place words and phrases in "frequency bands", e.g. words 1-1000, 1001-3000, 3001-10,000, etc. However, there should not be more than about 20 frequency bands in your application.)
[ License ]
-
Academic Single User license: the data can only be accessed by one single person. Otherwise, you will need an Academic Multi-User license (ACAD-2+).
-
Academic licenses: are only valid for one campus. So if you are part of a research group, for example, with members at universities X, Y, and Z, they all need to purchase the data separately.
-
Academic licenses: you can not use the data to create software or products that will be sold to others.
-
Academic licenses: students in your undergraduate classes cannot have access to substantial portions of the data (e.g. 50,000 words or more). Graduate students can have access to the data for work on theses and dissertations. The data is primarily intended for use in research, not teaching. If you need corpus data for undergraduate classes, please use the standard web interface for the corpora.
-
Academic Multi-User and Commercial licenses: supervisors will make best efforts to ensure that other employees or students who have access to the data are aware of these restrictions.
-
Commercial license: large companies with employees at several different sites (especially different countries) may need to contact us for a special license.
[ General ]
-
Any publications or products that are based on the data should contain a reference to the source of the data: http://www.corpusdata.org.
-
Note that a small, unique change will be made to each set of data, and this will serve as a "fingerprint" to identify you as the unique source of the datasets that you download. Automated Google searches are run daily to find copies of the data on the Web. If we find the data online and it is the data that was sent to you (and we will be able to determine that is the case), then you will be required to contact the administrators for that website, to have the data removed.