Eventually a search engine, but currently a filtering pipeline for HTML and soon WARC files.
-
Updated
May 31, 2022 - Rust
Eventually a search engine, but currently a filtering pipeline for HTML and soon WARC files.
⚙️ A Rust library for reading and writing WARC files
🗄️ A simple CLI for converting WARC to Parquet.
Add a description, image, and links to the warc topic page so that developers can more easily learn about it.
To associate your repository with the warc topic, visit your repo's landing page and select "manage topics."