Repository navigation
When the context overflows, the grounding instructions are what gets evicted first #15163
behrnt-slatgng
started this conversation in
Ideas / Requests / Suggestions
Replies: 1 comment 1 reply
|
Agreed that the grounding policy is not a degradable part of the prompt. I’d also make each budget decision observable: configured versus observed context limit, serialized prompt size, which chunks were kept or dropped, and a hard failure if the grounding instructions cannot fit. That makes the failure testable. A small regression suite could deliberately overfill the context with plausible but irrelevant chunks and verify that the answer either cites only the retained material or returns the configured insufficient-evidence response. If it still produces an uncited answer, that is a policy failure—not just a retrieval miss. |
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Disclosure: I build Grunz, a hosted chat and coding agent on open weights. Different product entirely — posting because enterprise search has a failure mode that consumer chat does not, and it is caused by something nobody instruments.
The system prompt is the first thing evicted, and in enterprise search that is the part that matters
The context window advertised on a model card is a property of the weights. The window an endpoint actually serves is whatever the provider configured. Most hosted endpoints serve around 32K tokens regardless of the advertised figure — a few reach 256K, and I have not found one genuinely serving the 1M numbers that get quoted.
It fails silently. No 413, no truncation notice. The front of the context is evicted and the model answers confidently from what remains.
Onyx is unusually exposed because of what it puts at the front. A typical assistant prompt carries the instructions that make the answer trustworthy: answer only from the retrieved documents, cite every claim, say explicitly when the documents do not cover it, respect these persona constraints. Then come the retrieved chunks, then the history, then the question.
Pile enough retrieved context in and the instructions are what falls off the edge. What the user sees is not an error — it is an assistant that stops citing, stops saying "I don't know", and starts answering from parametric memory in a tone indistinguishable from a grounded answer. In a company knowledge base that is the worst possible failure, because the whole value proposition is that the answer came from your documents.
It is also easy to hit accidentally: raising the number of retrieved chunks is the obvious knob for "the answer missed something", and it is the same knob that pushes the instructions out.
Things that would help
If anyone is pointing Onyx at self-hosted open weights
Two things worth knowing:
Abliterated variants undermine grounding specifically. On refusal-ablated models the degradation order is: instruction-following and output-format adherence degrade first, knowledge and reasoning later. Since "answer only from the provided context" and "cite your source" are instructions, an ablated model is measurably more willing to answer from parametric memory while presenting it as document-grounded. Prose quality holds up, so it passes every casual evaluation.
Perplexity will not warn you. It barely moves while format compliance falls off a cliff. If you are evaluating a model for this, send N requests with an explicit output contract — a required citation format, a required "not in the provided context" response — and score how many come back well-formed, separately from whether the answer is right.
Happy to go deeper on any of it.
All reactions