Skip to content

knesset-laws-v2: Knesset laws as published (K1–8), decontaminated

Choose a tag to compare

@shaypal5 shaypal5 released this 20 Nov 02:13
e48d0bc

Zenodo: 10.5281/zenodo.24384730 (all versions: 10.5281/zenodo.24384729). Hugging Face: HeOCR/htext-knesset-laws, the same files.

The enacted laws of the first eight Knessets (1949–1977), as printed in Reshumot, cut into lines. This is v2, part of the htext-cc0 release (decision 0022), and it replaces v1 as the default. Laws are not protected by copyright in Israel (Copyright Act 2007, section 6).

What changed from v1. v2 is v1 minus every line that shares any 6-word sequence with HeOCR's evaluation sets (12 private development, test and held-out targets), and minus every line of the 32 laws where a line shares a long run with them. Every other line is the same as in v1. HeOCR's evaluation ran its own independent check on this lines file and found 0 overlap. See KNOWN_ISSUES.md for v1.

Text quality: OCR layer. The text comes from the OCR layer of the Knesset's DOC files, filtered against a lexicon built from the pby v2 transcriptions, with look-alike OCR confusions dropped. These lines never feed a lexicon. The eye check is inherited from v1: of v1's 100 fresh random lines checked against the scans, 94 are in v2, and 9 of them have an error (9.6%), with 9 wrong words of 428 (2.1%).

223,617 lines from 1,972 laws. Card, rights index and checksums: data/knesset_laws/v2/.

Pins
knesset_laws_lines_plain_v2.txt 9853052 sha256:790099a2e64c2e391957a2c9f306c6a8c0addd087cfd8431666165f5b4f65de1
knesset_laws_line_index.v1.jsonl.gz 9106157 sha256:103049ccc3c2a35f3abb3ae24a53fd8a4137d9be3db00ef10581b8adb5cafaca

Credit: The Knesset of Israel (Knesset database, laws as published in Reshumot), https://main.knesset.gov.il