GitHub - yifan/c4-qcri: replicate c4 dateset process without Apache Beam, use Slurm instead

yifan / c4-qcri Public

replicate c4 dateset process without Apache Beam, use Slurm instead

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 87 Commits
ar-badwords.txt		ar-badwords.txt
c4-filter.py		c4-filter.py
c4-qcri.sh		c4-qcri.sh
download_books.py		download_books.py
pack.sh		pack.sh
parallel.sh		parallel.sh
requirements.txt		requirements.txt
sampling_domains.py		sampling_domains.py
split_wet_file.py		split_wet_file.py