Repository navigation
Evidence highlighting is a verbatim-span contract, and that is the first thing abliterated models stop honouring #872
behrnt-slatgng
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Disclosure: I build Grunz, a hosted chat and coding agent on open weights. Posting about the citation pipeline specifically, because kotaemon asks more of the model than most RAG UIs do and that turns out to interact badly with a class of models a lot of people are now running.
Evidence highlighting is the strictest output contract in RAG
Most RAG tools ask the model for an answer and a source label. kotaemon asks for an answer plus the exact spans that support it, so the UI can highlight them in the original document. That is a much harder constraint: the model has to reproduce text verbatim from the retrieved context, character-accurate enough to locate, and wrap it in whatever structure the parser expects.
It is also the single most valuable thing kotaemon does, and the thing users trust the UI for. When the highlight is wrong or missing, the failure is not cosmetic — it silently converts a grounded answer into an ungrounded one that still looks grounded.
Why this breaks on abliterated models, and why it breaks quietly
A lot of people now run uncensored variants — huihui, Heretic, the various
-abliterateduploads. On refusal-direction-ablated weights the degradation order is consistent and not the intuitive one:The model still knows the material. What it stops doing reliably is respecting the instruct template, honouring stop sequences, and staying inside a structured-output contract.
Applied to evidence highlighting, that means the model starts paraphrasing the span instead of quoting it. The answer is still correct, the citation still points at the right document, and the highlight silently fails to match — or worse, matches a nearby passage. A paraphrased span is a plausible-looking citation that does not actually support the claim, which is the exact failure a citation feature exists to prevent.
The reason it goes unnoticed is that none of the usual signals move. Prose quality holds up. Perplexity on a generic corpus barely changes while format compliance falls off a cliff.
Suggestions
Adjacent, for the decomposition path
Complex-reasoning mode decomposes a question into sub-questions. Each decomposition is a summarisation step, and summarisation drops constraints preferentially — they are short, low-frequency, and look unimportant. A question like "what changed in v3, excluding the deprecated endpoints" reliably loses the exclusion somewhere in the decomposition. Carrying the original question verbatim alongside each sub-question, rather than only the sub-question, prevents most of that.
Happy to go deeper on any of it.
All reactions