Repository navigation
v1.3.0
🚀 Precision Tuning & Prompt Optimization
In this update, I focused on refining the synergy between the GLiNER extraction engine and the LLM validation layer to prevent data leakage and improve overall processing speed.
✨ Key Changes & Improvements
- Smart LLM Delegation (Threshold Tuning): During extensive benchmarking, I noticed that local LLMs sometimes tend to "over-correct" and drop valid PII redactions, risking data leaks. To solve this, I introduced a low confidence threshold at
0.75.- If GLiNER is enough confident (
> 0.75) that a word is sensitive, it bypasses the LLM entirely. - The LLM is now exclusively triggered as a fallback to resolve truly ambiguous cases (
<= 0.75). This change improved recall by 8–10%.
- If GLiNER is enough confident (
- Prompt Overhaul for Small Models: The internal prompts have been rewritten. I removed repetitive instructions and restructured the logic (Role ➔ Rules ➔ Output Format ➔ Data) to be perfectly optimized for small-parameter models (1B to 7B). This prevents context-loss, speeds up inference, and drastically improves the model's adherence to the strict JSON output schema.
Update your local environment to experience a faster and more secure redaction pipeline! 🌊