Skip to content

CorpusMind v0.1.20 — Full Reference Corpora (BNC Baby, BAWE, Leipzig)

Choose a tag to compare

View all tags
@waleedmandour waleedmandour released this 24 Jul 09:14
· 25 commits to main since this release
v0.1.20
85ac10e

Phase 2: Full Reference Corpora

New Full Corpus Downloads

  • BNC Baby (4M words, BNC User Licence) — 4 subcorpora: academic, fiction, news, spoken
  • BAWE (6.5M words, CC BY-NC-SA 3.0) — academic English with discipline metadata
  • Leipzig English News 10K (CC BY 4.0) — full sentences with NLP pipeline
  • Leipzig Arabic News 10K (CC BY 4.0) — Arabic news with NLP pipeline

New Endpoint

POST /reference-corpora/{name}/download-full — downloads archive, extracts text, creates Corpus row, ingests through NLP pipeline, tags documents with genre metadata

UI Improvements

  • Full Corpus vs Frequency List badge in BundledReferences
  • Auto-detection of format (full_corpus calls download-full endpoint)

Verified

  • ruff: 0 errors
  • pytest: 24/24 passed
  • typecheck: clean
  • build: clean

Full Changelog: v0.1.19...v0.1.20

Assets 9
Loading