New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

数据分发有考虑过使用S3进行存储和提供下载吗 #40

Open

chinoll opened this issue Nov 11, 2023 · 1 comment

chinoll commented Nov 11, 2023

考虑提供和commoncrawl一样的下载方式吗

xclimbing commented Nov 14, 2023 •

edited

考虑提供和commoncrawl一样的下载方式吗

目前数据分发有三个渠道：百度网盘和威力同步提供目前搜集到的所有数据（包括清洗过及未清洗的原始数据）；而清洗过的数据会通过huggingface分发。详见官方主页 (https://mnbvc.253874.net/) 相关说明。

MNBVC是一个公益项目，目前人力和资源都有限，将来是否提供其他分发方式，待讨论。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment