🕷️ The pipeline for the OSCAR corpus
-
Updated
Dec 18, 2023 - Rust
🕷️ The pipeline for the OSCAR corpus
Eventually a search engine, but currently a filtering pipeline for HTML and soon WARC files.
Add a description, image, and links to the common-crawl topic page so that developers can more easily learn about it.
To associate your repository with the common-crawl topic, visit your repo's landing page and select "manage topics."