Repository navigation
ReDefiNER 1.0.2
A 0.8B model for entity extraction and span labeling, guided by the rulebook you write.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks
A documentation release: same weights and the same outputs as 1.0.1.
- Confidence vs threshold explained. An extracted span's
confidenceis its span probability times its type probability, computed withoutnone_bias, whilethresholdfilters on the span probability alone, so a returned span can score below 0.5. Inlabel,confidenceis the chosen type's probability among the rulebook's types. Also in theextractandlabeldocstrings. - Device default corrected.
Tagger.from_pretraineduses a GPU when one is available; passdevice="cpu"to force the CPU. - Limits and results up front. "At a glance" lists the limits (rulebooks tested up to about 10,000 tokens, text read 256 tokens at a time, spans up to 16 tokens) and headline benchmark results, including where the model trails.
- Install requirements next to
pip install: Python 3.10+, a CUDA build of PyTorch for GPUs,redefiner[gpu]on Linux, and the CPU-only PyTorch index for Linux without a GPU. - S1MB comparison now includes Jev 1.13.0 and Fastino GLiDE from the leaderboard.
- README layout. Three examples stay visible and six are collapsed, so the benchmarks come sooner.