Repository navigation
Releases
ReDefiNER 1.0.10
Same weights, and the same outputs as 1.0.9 for every valid input.
redefiner serve: POST /extract and /label can leave out"rulebook"when the served model has a default rulebook (a pointer such asjungroo-ai/redefiner-pii); the model's default rulebook is used. Without a default rulebook, a missing rulebook is still a 400 error.
pip install -U redefiner
redefiner serve --model jungroo-ai/redefiner-pii
curl -s localhost:8000/extract -d '{"texts": ["Email Priya Raman at priya.raman@example.com"]}'
ReDefiNER 1.0.9
Same weights. With default settings, the same outputs as 1.0.8 for every valid input.
max_span_tokens: let extract() and label() return spans longer than the trained 16 tokens, such as spaced card
numbers and IBANs (Tagger.from_pretrained(..., max_span_tokens=64)ortagger.set_max_span_tokens(64)). Longer
spans reuse the trained width-16 setting; no weights change. On the release's reproduce sets, limits of 32, 48 and 64
gave the same predictions as 16.- Pointer folders and repos: a config.json with
"format": "pointer"names the release to load the weights from,
plus a default rulebook and its settings (span limit,none_bias,threshold).jungroo-ai/redefiner-piiis one. extract()andlabel()take no rulebook when the Tagger has a default one (from a pointer).redefiner redact FILE: masks what a rulebook defines in text files and writes an audit file of what was masked.
--rules,--none-bias,--min-confidenceand--max-span-tokenschange the rulebook and settings.- Docs:
benchmarks/pii/(PII Masking Benchmark and RedactionBench results, both PII rulebooks, a rerun script)
and a PII section in the README and model card. - Docs: the README says what the name means (ReDefine + NER) and that the model is non-generative, and the span
limit is described as 16 tokens by default, longer withmax_span_tokens.
PII setup: jungroo-ai/redefiner-pii. PII benchmark results: benchmarks/pii.
pip install --upgrade redefiner
ReDefiNER 1.0.8
A 0.8B model that extracts the entities you define in a plain-English rulebook. It runs on a CPU and is free under Apache 2.0.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks · Changelog
Docs and package metadata only. Same weights, and the same outputs as 1.0.7 for every valid input.
- A one-sentence description on PyPI, GitHub and the model card.
- Links to jungroo.com/redefiner, for help with your own data, in the README, the model card and the demo.
- A training-data section in the README and the model card.
- PyPI's project links include the company page.
Need it on your own data? We write and test rulebooks with you and run ReDefiNER on your own servers: jungroo.com/redefiner or hello@jungroo.com.
ReDefiNER 1.0.7
Define what counts as an entity. Run it locally. A 0.8B model that extracts and labels exact spans using your plain-English rulebook.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks · Changelog
Same weights, and the same outputs as 1.0.6 for every valid input.
transformers 5.19 to 5.22
- 1.0.6 needed transformers 5.19. transformers 5.20 changed how the rotary embedding is created and 5.21 how the Gated DeltaNet layers read their cache; the package now works with both. Outputs are identical across 5.19, 5.20, 5.21 and 5.22, on CPU and on an NVIDIA A10 (with and without flash-linear-attention).
- transformers 5.21 and later need Python 3.11 or newer; on Python 3.10, pip installs transformers 5.20.
- The weekly model tests run with the lowest and the highest transformers the package allows, and with the newest release.
Clearer errors
- Wrong input types raise a
TypeErrorthat says what was expected.extract(None, rulebook)now says "texts must be a string or a list of strings, not NoneType" instead of "'NoneType' object is not iterable", andextract(text, None)says "a rulebook must be rulebook text or a dict, not NoneType" instead of anAttributeError. The same goes forlabelspans that aren't lists, a list of texts holding a non-string, and bytes. - An empty rulebook is reported as empty, not as "no types found".
Command line
python -m redefinerruns the command line, likeredefiner.
Docs
- The Quickstart explains
thresholdandconfidenceup front:thresholddecides whether text counts as an entity at all,confidencealso includes how sure the model is of the type, so spans below 0.5 confidence still come back. It suggests choosing a confidence cutoff on 20 to 50 labelled texts of your own. - The Development section's download command works. In 1.0.6 it gave two patterns to one
--exclude, andhf downloadread the second one as a filename.
ReDefiNER 1.0.6
Define what counts as an entity. Run it locally. A 0.8B model that extracts and labels exact spans using your plain-English rulebook.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks · Changelog
Same weights. Same outputs as 1.0.5, except label(..., snap=True) on a span that ends with a space or is only whitespace.
Label mode
snap=Truetrims spaces from both ends of a span before widening it to whole tokens. Before, a span with a trailing space was widened into the next token: "Ada Lovelace " became "Ada Lovelace correspond", and could get a different label. Withsnap=True, a span of only whitespace now comes back unsupported.- The docs say what snapping does and doesn't do: it widens a span to whole tokens, not words, so a snapped span can still end inside a word ("Ada Lo" becomes "Ada Lov"), and part of a word that starts and ends on token boundaries is labeled as given, with no flag. Also in the
labeldocstring andllms.txt.
PyPI page
- Images show and links work: every README link is absolute, and a test keeps it that way.
Docs
- The headline gives each gap: about 5 F1 behind Claude Haiku 5.5 (73.1) and 4.3 behind Gemini 3.1 Pro (72.4) on CrossNER AI.
- The key-value tables have column headers. Limits say what long texts mean: each part is read on its own, so labels can differ across a document. The Release row separates the weights (1.0) from the package (1.0.6).
CHANGELOG.md, also linked from PyPI.
Package and CI
- Python 3.14 is tested in CI and listed on PyPI.
- The weekly model tests run with the pinned transformers and with the newest release, so a transformers change that breaks the model code shows up before the pin is widened.
- The Development section downloads the model files without the benchmark outputs and images.
ReDefiNER 1.0.5
Define what counts as an entity. Run it locally. A 0.8B model that extracts and labels exact spans using your plain-English rulebook.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks
Docs and package metadata only. Same weights and the same outputs as 1.0.4.
Frontier comparison
benchmarks/frontier/: ReDefiNER against Claude Opus 5.5, Claude Opus 4.8, GPT-5.4, Claude Haiku 5.5 and Gemini 3.1 Pro, given the same rulebooks, on CrossNER AI (extraction) and S1MB Few-NERD (labeling). ReDefiNER is 4 to 11 F1 behind on extraction and 10 to 15 points behind on labeling, and costs far less per request: about 85 times less than Claude Haiku 5.5 on an RTX A5000 GPU, and about 9 times less on a 2-vCPU server with no GPU, with the rulebook cached. Time and cost per request, and how each was measured, are on the page.
README and model card
- New headline and description, with the two-rulebook example right under them.
- A shorter "At a glance" and Quickstart: install, one extraction call and a label call first; install notes, the rule comparison and the confidence arithmetic in expandable sections.
- Benchmark notes that say what each number covers, including why CrossNER AI is 68.1 in the frontier table (seed-0 type order) and 66.7 in the GoLLIE table (mean over three orders).
- The Speed section adds the 2-vCPU server timings and marks the CPU timings as extract mode.
Package
- PyPI summary, authors (Jungroo AI Labs) and keywords (pii, privacy, on-premise, cpu) updated.
ReDefiNER 1.0.4
A generalist 0.8B model for entity extraction and span labeling, guided by the rulebook you write.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks
Fixes from an end-to-end test of 1.0.3. Same weights; every rulebook with one section per type gives exactly the same output (all 490 rulebooks in the repo encode identically).
Rulebooks
- A type defined twice (two
[TYPE] PERSONsections, or a dict that repeats a class name) is now aValueErrornaming the type and its lines. Before, it was accepted silently, and when another type sat between the two sections their labels were swapped: with PERSON, ORG, PERSON, "Tim Cook" came out as ORG at 0.99 confidence. This applied toextractandlabel. - A dict's
"closing"rules are written under a second[RULES]line, as the docs, the demo anddocs/templates/blank.txtalready did. On CrossNER AI, closing rules scored 75.98 F1 with the header and 75.64 without. No rulebook in the repo has closing rules, so none changes.
Device
tagger.devicesays where the model runs (cpu,cuda:0), andrepr(tagger)shows it.- With the default device, a
RuntimeWarningwhen the model falls back to CPU on a machine with an NVIDIA driver: a CPU-only PyTorch build, or a CUDA build the driver can't run. Passdevice="cpu"to silence it. redefiner serveprints the device at startup, andGET /infoincludes it.
Docs
- README: a "Rule lines" section. One rule per line (wrapping long rules onto continuation lines lowered CrossNER AI F1 by 0.8 and 0.5); rule IDs such as
G1orPE1are optional (removing them changed F1 by −0.02 and +0.6); a rule belongs to the section it sits in; quoted strings under a type are read as examples of it. Type names are case-sensitive. docs/writing-rules.mdquotes its example rules as the single lines they are in the benchmark rulebooks.llms.txt: a short summary of the API and rulebook format for AI assistants and coding agents.
ReDefiNER 1.0.3
A 0.8B model for entity extraction and span labeling, guided by the rulebook you write.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks
Fixes from an end-to-end test of 1.0.1. Same weights; every well-formed rulebook gives exactly the same output (all 476 rulebooks in the repo encode identically).
Rulebooks
- A line written as a type header that isn't
[TYPE] NAME(letters, digits and underscores), such as[TYPE] DATE-OF-BIRTH,[TYPE]EMAILor[type] CITY, is now an error that quotes the line. Before, the type was silently dropped and its text folded into the type above. - Windows line endings and a byte-order mark are removed before reading. Before, a rulebook saved on Windows could be read in the wrong layout and give different answers.
- A rulebook that mixes the two layouts (
[RULES]on its own line plus a[NOT_AN_ENTITY]section) is an error; a rulebook dict without"classes", or a class without"name", gets a clear message.
Python API and server
- A malformed span is a
ValueError, so the server answers 400 instead of 500. allow_noneandsnapmust be JSONtrueorfalse(the string"false"used to count as true);thresholdandnone_biasmust be numbers.Tagger.from_pretrained("./weigths")raisesFileNotFoundErrorinstead of trying a Hugging Face repo called./weigths.extractnever returns a whitespace-only span.
Packaging and docs
- The source package no longer includes tests that can't run outside the repository.
- Python 3.13 is tested in CI.
- README: the two rulebook layouts, M2 and Linux speed and memory (peak about 2–2.6 GB), a note that the server has no authentication, and fixes to the pronoun guidance; the benchmark page says which sha256 is which.
ReDefiNER 1.0.2
A 0.8B model for entity extraction and span labeling, guided by the rulebook you write.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks
A documentation release: same weights and the same outputs as 1.0.1.
- Confidence vs threshold explained. An extracted span's
confidenceis its span probability times its type probability, computed withoutnone_bias, whilethresholdfilters on the span probability alone, so a returned span can score below 0.5. Inlabel,confidenceis the chosen type's probability among the rulebook's types. Also in theextractandlabeldocstrings. - Device default corrected.
Tagger.from_pretraineduses a GPU when one is available; passdevice="cpu"to force the CPU. - Limits and results up front. "At a glance" lists the limits (rulebooks tested up to about 10,000 tokens, text read 256 tokens at a time, spans up to 16 tokens) and headline benchmark results, including where the model trails.
- Install requirements next to
pip install: Python 3.10+, a CUDA build of PyTorch for GPUs,redefiner[gpu]on Linux, and the CPU-only PyTorch index for Linux without a GPU. - S1MB comparison now includes Jev 1.13.0 and Fastino GLiDE from the leaderboard.
- README layout. Three examples stay visible and six are collapsed, so the benchmarks come sooner.
ReDefiNER 1.0.1
A 0.8B model for entity extraction and span labeling, guided by the rulebook you write.
pip install redefinerWeights: jungroo-ai/redefiner-0.8b · Demo: jungroo-ai/redefiner-demo · Benchmarks
Changes since 1.0.0 (same weights, same outputs):
Tagger.from_pretraineddownloads only the 9 files it needs from the Hub, not the benchmarks and docs stored in the model repo (157 files before).- On CPU, transformers no longer warns that the run is "much slower" without
causal_conv1dandflash-linear-attention. Those are CUDA kernels; the CPU path is the intended one. - README: an HTTP server section with a working request for
/extractand/label.