Skip to content

Commit eaa8f7e

Browse files
authored
Update README.md
1 parent deba783 commit eaa8f7e

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
# The Pile™
22

3-
The Pile is (going to be) the world's largest diverse open source language modeling data set. We are currently developing Version 1, with an ultimate goal of 1 TiB of English text.
3+
The Pile is a large, diverse, open source language modelling data set that consists of many smaller datasets combined together. The objective is to obtain text from as many modalities as possible to ensure that models trained using The Pile will have much broader generalization abilities. We are currently developing Version 1, with an ultimate goal of 1 TiB of English text. After the completion of Version 1, our next goal is a fully-multilingual, 10TiB text dataset.
4+
5+
The Pile is currently still under development.
6+
47

58
| Component | Size |Weight|Epochs|Mean Document Size|
69
|-----------------|----------|------|-----:|------------------|

0 commit comments

Comments
 (0)