Skip to content

v2.1.0

Choose a tag to compare

@github-actions github-actions released this 02 Oct 11:24
· 18 commits to main since this release

🚀 Key Features & Performance

  • New Default LLM Engine: Transitioned to Qwen2.5-7B-Instruct-Q4_K_M. Working in tandem with GLiNER and Regex, the pipeline now achieves an outstanding ~91.4% Recall on the ai4privacy benchmark.
  • Hyper-Aggressive PII Detection: The system now successfully extracts complex, multi-word entities (addresses, fragmented dates) and outperforms standard datasets by catching unannotated context-sensitive PII.

🛠️ Architecture & Core Improvements

  • Token Exhaustion Prevention: Redesigned the LLM prompt to implement a "Silent Ignore" rule. The model now strictly outputs JSON only for sensitive words, drastically reducing generation time, eliminating token limit crashes, and preventing JSON parsing errors.
  • Benchmarking Suite: Added an advanced evaluation script to calculate Precision, Recall, and F1-Score locally, including logic to handle partial dataset annotations.