Skip to content

Releases

v0.6.0

Choose a tag to compare

@maxmelichov maxmelichov released this 14 Nov 13:36
93a82fa

Self-contained weights, pinned per release.

One file, nothing else

G2P() needs only model.onnx (or model_int8.onnx). The vocabularies, per-letter constraints and cascade conditioning all travel inside the ONNX metadata.

from renikud_onnx import G2P

g2p = G2P()                               # downloads model.onnx on first use
g2p = G2P(filename="model_int8.onnx")     # 4× smaller, within ~0.3 pt overall
print(g2p.phonemize("שלום לכולם"))         # ʃalˈom lekulˈam

Pinned model revision

Each release now downloads the exact Hugging Face commit it was tested against (DEFAULT_HF_REVISION), so an installed version keeps loading the same weights however the model repo evolves. Pass G2P(revision="main") to opt into the latest upload. (#4, #5)

New weights

  • 93.3% word accuracy on 126k words of unseen podcast speech; per-category numbers are in the README.
  • The attested-reading rescorer is removed; the new weights fold in what it did. On 24 hand-checked pointed words the default decode goes from 13/24 to 17/24 segments right.
  • niqqud="use" still constrains the exact-MAP decode: 24/24 segments on that set, and 7/7 against the reference readings on the pointed possessive-suffix words.

Breaking changes

G2P(tau=..., mask=..., inventory_sources=...) and rescoring_enabled are removed.

Install

pip install 'renikud-plus[cpu]'   # or [gpu]

v0.5.0

Choose a tag to compare

@maxmelichov maxmelichov released this 29 Aug 22:06

Hebrew G2P with the full decode stack, and pointed input that finally counts.

Vowelized (pointed) input

Niqqud in the input used to tokenize to [UNK] and leak the raw combining marks into the IPA. It is now handled in two modes:

  • niqqud="strip" (default) — points are dropped, so pointed text decodes byte-identically to the same text unpointed.
  • niqqud="use" — points are read as evidence. Each vowel sign, dagesh qal and shin/sin dot is pinned into the exact-MAP energy as a hard constraint, so the model predicts only what the pointing leaves open.
g2p = G2P(niqqud="use")
g2p.phonemize("סֵפֶר")   # sˈefeʁ   book
g2p.phonemize("סַפָּר")   # sˈapaʁ   barber
g2p.phonemize("סָפַר")   # sˈafaʁ   counted
g2p.phonemize("סִפֵּר")   # sˈipeʁ   told

All four are sˈefeʁ when stripping — the skeleton ספר cannot tell them apart. Qamats and shva stay ambiguous by design (qamats qatan shares its sign; shva na vs nah), and a constraint that would leave a letter with no legal reading is dropped, so sloppy pointing degrades to the normal decode. Niqqud does not mark stress, so stress stays the model's own call.

On 24 hand-checked pointed words: 24/24 vowels and consonants right (13/24 stripping), 19/24 including stress.

Decoding

  • Exact-MAP cascade decode, on by default: consonant, vowel and stress decoded jointly under E(c,v,s) = log P(c) + log P(v|c) + log P(s|c,v), with per-letter legality and "stress needs a vowel" as hard constraints. This removes the stranded-stress-mark class — a mark can no longer land on a letter with no vowel.
  • Attested-reading rescorer, on whenever datastore.json sits beside the weights — which it now does, since the automatic download fetches it too.
  • Force lexicon — G2P(..., lexicon={...}) or a TSV path.
  • Long inputs past the encoder's 2,046-character window are split on sentence, then comma, then word boundaries, with a raw-Hebrew-in-output backstop.

Packaging

  • The Hebrew number front end moved to hebrew-num2words and is now a dependency rather than ~560 bundled lines. Output is unchanged and the API is re-exported.
  • G2P() also accepts session_options, repo_id, filename and cache_dir.
  • Includes 0.4.0, which was never published: ONNX Runtime is no longer a hard dependency — install renikud-plus[cpu] or renikud-plus[gpu], or bring your own build.

Full notes in CHANGELOG.md.

v0.3.0

Choose a tag to compare

@maxmelichov maxmelichov released this 17 Jul 10:46

Install

pip install renikud-plus

Usage

from renikud_onnx import G2P

g2p = G2P()  # downloads model.onnx from Hugging Face on first use
print(g2p.phonemize("שלום עולם"))
print(g2p.vocalize("שלום עולם"))

ONNX weights (~310 MB) are no longer a separate download step — they are cached via huggingface_hub.

v0.2.0

Choose a tag to compare

@maxmelichov maxmelichov released this 17 Jul 10:44

Summary

First PyPI release of ReNikud Plus as renikud-plus.

Install:

pip install renikud-plus
hf download notmax123/RenikudPlus model.onnx --local-dir .

Usage is unchanged:

from renikud_onnx import G2P
g2p = G2P("model.onnx")
print(g2p.phonemize("שלום עולם"))
print(g2p.vocalize("שלום עולם"))

Notes

  • Distribution name: renikud-plus (import still renikud_onnx)
  • ONNX weights are not bundled; download from Hugging Face