Skip to content

ReDefiNER 1.0.2

Choose a tag to compare

@Jungroo-AI Jungroo-AI released this 29 Oct 10:26
· 34 commits to main since this release

A 0.8B model for entity extraction and span labeling, guided by the rulebook you write.

pip install redefiner

Weights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks

A documentation release: same weights and the same outputs as 1.0.1.

  • Confidence vs threshold explained. An extracted span's confidence is its span probability times its type probability, computed without none_bias, while threshold filters on the span probability alone, so a returned span can score below 0.5. In label, confidence is the chosen type's probability among the rulebook's types. Also in the extract and label docstrings.
  • Device default corrected. Tagger.from_pretrained uses a GPU when one is available; pass device="cpu" to force the CPU.
  • Limits and results up front. "At a glance" lists the limits (rulebooks tested up to about 10,000 tokens, text read 256 tokens at a time, spans up to 16 tokens) and headline benchmark results, including where the model trails.
  • Install requirements next to pip install: Python 3.10+, a CUDA build of PyTorch for GPUs, redefiner[gpu] on Linux, and the CPU-only PyTorch index for Linux without a GPU.
  • S1MB comparison now includes Jev 1.13.0 and Fastino GLiDE from the leaderboard.
  • README layout. Three examples stay visible and six are collapsed, so the benchmarks come sooner.