Skip to content

Releases

pby-v1: Project Ben-Yehuda public-domain Hebrew lines

Choose a tag to compare

@shaypal5 shaypal5 released this 06 Nov 16:37
8411075

Public-domain Hebrew lines from Project Ben-Yehuda, built by htext pby build. This is v1, at these pins:

  • PBY dump: projectbenyehuda/public_domain_dump @ 5e4277f, VERSION 2026-03.
  • Wrap check and S10b length profile: hocrsyngen @ 88c61af.
  • Parameters: as_of 2026, seed 15, at most 200 lines per work.

Credit: Project Ben-Yehuda volunteers.

Corpus card: data/pby/v1/CORPUS_CARD.md (corpus_card.json).

Asset Lines Contents
pby_lines_il_plain_v1.txt 1,627,470 public domain in Israel, plain
pby_lines_il_niqqud_v1.txt 401,492 public domain in Israel, pointed
pby_lines_il_us_plain_v1.txt 70,251 also public domain in the US, plain
pby_lines_il_us_niqqud_v1.txt 10,050 also public domain in the US, pointed
pby_line_index.v1.jsonl.gz 2,028,962 rows per il line: sha256, PBY work id, subset, genre, period, il_us

The corpus card, SHA256SUMS, the work-level exclusion list and the gzipped rights index are in git under data/pby/v1/ at this tag (commit 8411075). Verify the downloaded assets with:

sha256sum -c --ignore-missing data/pby/v1/SHA256SUMS

Each corpus file is a --text-corpus input for hocrsyngen, which records it as external:sha256:<hex>.

Licences. The outputs are CC0-1.0. The source texts are public domain, as PBY's dump licence states. pd_israel and pd_us are the project's reading of the law, not legal advice; see docs/pby_corpus.md.