Skip to content

ReDefiNER 1.0.7

Choose a tag to compare

@Jungroo-AI Jungroo-AI released this 02 Nov 09:52
· 20 commits to main since this release

Define what counts as an entity. Run it locally. A 0.8B model that extracts and labels exact spans using your plain-English rulebook.

pip install redefiner

Weights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks · Changelog

Same weights, and the same outputs as 1.0.6 for every valid input.

transformers 5.19 to 5.22

  • 1.0.6 needed transformers 5.19. transformers 5.20 changed how the rotary embedding is created and 5.21 how the Gated DeltaNet layers read their cache; the package now works with both. Outputs are identical across 5.19, 5.20, 5.21 and 5.22, on CPU and on an NVIDIA A10 (with and without flash-linear-attention).
  • transformers 5.21 and later need Python 3.11 or newer; on Python 3.10, pip installs transformers 5.20.
  • The weekly model tests run with the lowest and the highest transformers the package allows, and with the newest release.

Clearer errors

  • Wrong input types raise a TypeError that says what was expected. extract(None, rulebook) now says "texts must be a string or a list of strings, not NoneType" instead of "'NoneType' object is not iterable", and extract(text, None) says "a rulebook must be rulebook text or a dict, not NoneType" instead of an AttributeError. The same goes for label spans that aren't lists, a list of texts holding a non-string, and bytes.
  • An empty rulebook is reported as empty, not as "no types found".

Command line

  • python -m redefiner runs the command line, like redefiner.

Docs

  • The Quickstart explains threshold and confidence up front: threshold decides whether text counts as an entity at all, confidence also includes how sure the model is of the type, so spans below 0.5 confidence still come back. It suggests choosing a confidence cutoff on 20 to 50 labelled texts of your own.
  • The Development section's download command works. In 1.0.6 it gave two patterns to one --exclude, and hf download read the second one as a filename.