v0.4.3 — Citation Coherence, BM25 Pipeline Refinements & Admin Valves

Highlights

The v0.4 line rewires what gets read, what gets cited, and how it gets organised — same total context size, far better signal density, and citations that actually work in the UI.

• 🎯 Pages are ranked and budgeted by relevance. BM25 scores every fetched page against your question. The most relevant pages get up to 3× the default character allowance; marginal pages shrink; off-topic noise (a Python tutorial that snuck into an autonomous-vehicles query) is dropped entirely.
• 🔗 Citations that always resolve. Inline [N] markers in the model's reply now map 1:1 to the source pills in the OWUI sidebar — no more dangling [3], no more [REF]…[/REF] improvisations from models like Mistral, no more pool-only snippets cited with invented slugs.
• 🧠 Better-structured answers across small and reasoning models. Tested on Mistral 24B, Gemma3 27B, and Qwen3 thinking. Replies are noticeably more comprehensive after the prompt was restructured around how transformer attention actually behaves on long contexts.
• 🎛️ New admin valve inject_snippet_pool (default ON) — flip OFF to keep the LLM context tight to fully-fetched pages only.

In practice

For a ??:10 <question>, the two most relevant pages can each carry 8–12k characters of real body text while marginally-related ones shrink to ~200-char snippets and irrelevant ones disappear — instead of everyone getting a flat 4k slice. The answer you get back is noticeably more focused, and every cited source is one click away in the OWUI sidebar.

Other changes / fixes

Budget preservation across drops, surplus reclamation, reasoning-model reply-length accounting, prompt internals, User-Agent rotation pool expansion (20 → 40), debug-gated stats line, and more — see the full CHANGELOG for the complete list.

Upgrade

Drop the new easysearch.py into your OWUI Functions panel. No config migration needed: existing valves keep their values; the new inject_snippet_pool defaults to ON (matches v0.4.2 behaviour but with BM25 filtering).

v0.3.5

What's Changed

Bug Fixes

• fix: clamp search results per query to API hard cap — Brave Search API rejects count > 20 with 422 for all plans. Added max_results_per_query valve (default: 20) to prevent this. Closes #3.
• fix: guard against race condition in DebugService.emit and outlet — concurrent requests on the shared Filter singleton could reset self.ctx = None mid-execution, causing 'NoneType' has no attribute 'model' crashes. outlet() and DebugService.emit() now capture a local reference to ctx. Closes #1.
• fix: await Users.get_user_by_id calls (PR #4 by @jaybill) — missing await was returning coroutines instead of user objects, causing 'coroutine' has no attribute 'role'. Closes #5.

Security

• Wrap search results in XML tags — raw web content is now enclosed in <search_results> tags with an explicit untrusted-data instruction to mitigate indirect prompt injection from malicious pages. Addresses #2.

Contributors

• @jaybill — PR #4

v0.3.1

• Fixed an issue where context-aware queries (??) would lose the original conversation language by implementing a strict Language Anchor using msg_list[-2].
• Fixed a bug in _parse_trigger that caused single-language modifiers (e.g., ??:de) to ignore the response_lang constraint.
• Introduced a Pre-Fetch Blacklist in _process_results to safely skip .pdf, .docx, .zip and other non-HTML binaries before establishing an HTTP connection.
• Upgraded the text cleaner with an Aggressive Scrubber regex to annihilate C0/C1 control codes, \ufffd replacements, and Zero-Width/Bi-Directional characters from dirty DOM elements.
• Fixed a bug in the Gap-Filler logic that caused the total source count to exceed the requested target.

v0.3.0

• Added oversampling pool injection: all retrieved snippets are now fed to the LLM for massive signal density.
• Enhanced System Prompt to enforce prioritization of Snippets over noisy/empty Scraped Content.
• Implemented dual-language syntax ??:src>dest for decoupled search/response.

v0.2.8 - First Release

EasySearch v0.2.8: High-Performance Web Search Filter - First Release
An intelligent, context-aware web search filter for Open WebUI. EasySearch bypasses noisy standard web scrapers, utilizing parallel fetching, structural HTML cleaning, and dynamic context-awareness to feed your LLM only the highest quality data.