Skip to content

ReDefiNER 1.0.3

Choose a tag to compare

@Jungroo-AI Jungroo-AI released this 30 Oct 05:13
· 32 commits to main since this release

A 0.8B model for entity extraction and span labeling, guided by the rulebook you write.

pip install redefiner

Weights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks

Fixes from an end-to-end test of 1.0.1. Same weights; every well-formed rulebook gives exactly the same output (all 476 rulebooks in the repo encode identically).

Rulebooks

  • A line written as a type header that isn't [TYPE] NAME (letters, digits and underscores), such as [TYPE] DATE-OF-BIRTH, [TYPE]EMAIL or [type] CITY, is now an error that quotes the line. Before, the type was silently dropped and its text folded into the type above.
  • Windows line endings and a byte-order mark are removed before reading. Before, a rulebook saved on Windows could be read in the wrong layout and give different answers.
  • A rulebook that mixes the two layouts ([RULES] on its own line plus a [NOT_AN_ENTITY] section) is an error; a rulebook dict without "classes", or a class without "name", gets a clear message.

Python API and server

  • A malformed span is a ValueError, so the server answers 400 instead of 500.
  • allow_none and snap must be JSON true or false (the string "false" used to count as true); threshold and none_bias must be numbers.
  • Tagger.from_pretrained("./weigths") raises FileNotFoundError instead of trying a Hugging Face repo called ./weigths.
  • extract never returns a whitespace-only span.

Packaging and docs

  • The source package no longer includes tests that can't run outside the repository.
  • Python 3.13 is tested in CI.
  • README: the two rulebook layouts, M2 and Linux speed and memory (peak about 2–2.6 GB), a note that the server has no authentication, and fixes to the pronoun guidance; the benchmark page says which sha256 is which.