Skip to content

Demo corpus: arXiv CS abstracts (~20k)

Choose a tag to compare

View all tags
@qikijames qikijames released this 20 Jul 03:04
· 135 commits to main since this release
demo-data-v1
6f228c3
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The opt-in demo corpus for just demo-data — not a software release.

One zstd parquet, ~10 MB: 20,000 arXiv computer-science titles+abstracts (metadata only, CC0), harvested via arXiv's OAI-PMH with a 2022+ datestamp floor so the corpus skews to modern ML (top categories cs.LG / cs.CV / cs.CL). Columns: id, title, abstract, authors, categories, primary_category, published, updated.

Consumed by deploy/compose/demo-data/load_arxiv.py (the default DEMO_DATA_URL); regenerate with deploy/compose/demo-data/build_arxiv_slice.py. See docs: Demo corpus.

Thank you to arXiv for use of its open access interoperability.

Assets 4
Loading