Skip to content
/ pile-explorer Public

For exploring the data and documenting its limitations

License

MIT license
4 stars 3 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

EleutherAI/pile-explorer

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits

CODEOWNERS

CODEOWNERS

 
 

LICENSE

LICENSE

 
 

README.md

README.md

 
 

commands.txt

commands.txt

 
 

profanity_analysis.py

profanity_analysis.py

 
 

topic_analysis_01.py

topic_analysis_01.py

 
 

topic_analysis_02.py

topic_analysis_02.py

 
 

topic_analysis_03.py

topic_analysis_03.py

 
 

Repository files navigation

Exploring the Pile

This repository contains code for exploring the Pile and documenting its limitations

Language Modeling Data Format

The data in the Pile is stored in the lm_dataformat. This repository is designed to be used on data stored in that format. For documentation, see the linked repository.

About

For exploring the data and documenting its limitations

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

4 stars

Watchers

4 watching

Forks

3 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages