Repository navigation
ReDefiNER 1.0.7
Define what counts as an entity. Run it locally. A 0.8B model that extracts and labels exact spans using your plain-English rulebook.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks · Changelog
Same weights, and the same outputs as 1.0.6 for every valid input.
transformers 5.19 to 5.22
- 1.0.6 needed transformers 5.19. transformers 5.20 changed how the rotary embedding is created and 5.21 how the Gated DeltaNet layers read their cache; the package now works with both. Outputs are identical across 5.19, 5.20, 5.21 and 5.22, on CPU and on an NVIDIA A10 (with and without flash-linear-attention).
- transformers 5.21 and later need Python 3.11 or newer; on Python 3.10, pip installs transformers 5.20.
- The weekly model tests run with the lowest and the highest transformers the package allows, and with the newest release.
Clearer errors
- Wrong input types raise a
TypeErrorthat says what was expected.extract(None, rulebook)now says "texts must be a string or a list of strings, not NoneType" instead of "'NoneType' object is not iterable", andextract(text, None)says "a rulebook must be rulebook text or a dict, not NoneType" instead of anAttributeError. The same goes forlabelspans that aren't lists, a list of texts holding a non-string, and bytes. - An empty rulebook is reported as empty, not as "no types found".
Command line
python -m redefinerruns the command line, likeredefiner.
Docs
- The Quickstart explains
thresholdandconfidenceup front:thresholddecides whether text counts as an entity at all,confidencealso includes how sure the model is of the type, so spans below 0.5 confidence still come back. It suggests choosing a confidence cutoff on 20 to 50 labelled texts of your own. - The Development section's download command works. In 1.0.6 it gave two patterns to one
--exclude, andhf downloadread the second one as a filename.