-
Notifications
You must be signed in to change notification settings - Fork 0
Local LLM Limitations: Entity Contagion & Drift
Local LLMs can produce strong narrative prose while still struggling with entity consistency, lexical stability, long-context retrieval, instruction adherence, and local inference constraints. These limitations vary by model, quantization method, inference backend, prompt structure, and decoding configuration.
This page describes five commonly observed failure classes and explains what can be stated about them without treating speculative internal mechanisms as established facts.
During multi-character storytelling, a model may incorrectly transfer an attribute, possession, relationship, action, or role from one entity to another. This observable failure is sometimes informally described as entity contagion, entity bleeding, or feature bleeding.
-
Example: The prompt defines Chris as the guitarist and Mike as a childhood friend who does not play music. After a scene set at a music venue, the model generates:
“Mike adjusted the strap of his guitar and smiled at Valerie as she approached.”
The output assigns Chris's established role to Mike. The error does not prove that a literal “floating trait” moved inside the model; that language is only a conceptual description of the observed result.
The following diagram illustrates the externally visible failure. It is not a representation of measured internal model activations.
flowchart TD
A[Established canon: Chris is the guitarist] --> C[Longer narrative context]
B[Established canon: Mike is not a musician] --> C
C --> D[Mike becomes the active subject near music-related text]
D --> E[Generation assigns the guitar-related action to Mike]
E --> F[Entity-consistency failure]
- No built-in entity database: A base language model does not maintain a transactional character database. It generates each next token from the supplied context and its learned parameters.
- Context-sensitive retrieval: Character facts may be retrieved less reliably when they are distant, surrounded by competing details, or expressed ambiguously.
- Learned associations: Patterns learned during training can influence a continuation when the supplied constraints are weak, conflicting, or insufficiently salient.
- No automatic semantic verification: Autoregressive generation does not inherently include a separate pass that compares every generated attribute with an external source of truth.
- Model scale is not a guarantee: Larger parameter counts can improve capability, but do not guarantee consistent entity tracking in every prompt or generation.
- Explicit Entity Definitions: Keep character roles, possessions, relationships, and exclusions compact, explicit, and unambiguous.
- External Canonical State: Maintain important character facts outside the model and include the relevant state in each generation request.
- Context Pruning and Summarization: Limit unrelated raw history and summarize older events so that active entity constraints remain easy to locate.
- Semantic Validation: Run a separate validation pass that compares candidate responses with explicit character constraints before accepting them. Regexes can detect narrowly defined text patterns, but are not sufficient for general semantic consistency.
Aggressive weight quantization reduces numerical precision and can increase generation errors. Depending on the model, quantization method, inference backend, and decoding configuration, this may appear as malformed wording, inappropriate token choices, repetition, or less coherent prose.
-
Example: Instead of generating “She furrowed her eyebrows in frustration,” a heavily quantized model might generate:
“She furrowed her highbrows in frustration” or “She furrowed her eyebrows wish.”
Quantization approximates model weights at lower precision. Its effects propagate through the network and cannot generally be attributed to a specific change in subword-embedding distances or to tokens being mathematically “adjacent.”
- Evaluate Quantizations Empirically: Compare candidate quantizations with representative narrative prompts; quality differences depend on the model and quantization method.
- Use Higher Precision When Justified: A higher-precision model representation may reduce quantization-related degradation, but requires more memory and compute and does not guarantee error-free prose.
- Tune Decoding Settings: Adjust temperature, Top-P, Top-K, Min-P, and repetition controls for the selected model. These settings can change output stability, but cannot repair approximation error in quantized weights.
- Optional Post-Processing: Use spelling, grammar, or semantic review only when the risk of changing intentional prose is acceptable. Regex filters are suitable only for narrowly defined patterns; broader correction requires a dictionary, grammar tool, or separate language-model pass.
Models can retrieve information less reliably when it appears deep inside a long context, particularly when relevant information is surrounded by large amounts of unrelated text. This position-dependent degradation is commonly described as the Lost in the Middle problem.
Needle in a Haystack evaluations measure related retrieval behavior by placing a specific fact—the “needle”—at different positions inside a much larger context and testing whether the model can retrieve it. Performance varies by model, prompt structure, information position, and context length. Support for a large context window does not guarantee that every part of that window will be used equally reliably.
- Example: Important rules, character details, or plot events established many turns earlier may be ignored or recalled incorrectly when they appear deep inside the assembled context.
- Dynamic Context Pruning: Remove irrelevant or redundant raw turns instead of filling the entire available context window.
- Layered Summaries: Condense older events into compact, task-relevant summaries while retaining a short window of recent raw turns.
- Canonical State: Store important structured facts separately rather than relying exclusively on retrieval from narrative prose.
- Prompt Placement: Keep critical instructions and currently relevant facts in clear, stable prompt sections. Repeating them may help in some models, but contradictory or excessive duplication can also reduce reliability.
During extended roleplay or storytelling sessions, accumulated narrative context can reduce the model's adherence to earlier instructions. Constraints such as “Never speak as the user” may be followed less consistently as the prompt grows and the generated narrative establishes competing patterns.
- Example: The model may begin writing dialogue for the user's protagonist or stop following required character formatting.
System prompts influence probabilistic generation; they are not compiled program rules. This does not imply a measurable point at which instructions literally “collapse,” and drift cannot be explained solely as narrative text overpowering a system prompt.
- Reassemble Active Instructions: Include the current rules in every stateless request instead of assuming that the model retains persistent state between calls.
- Keep Rules Consistent: Use concise instructions and avoid conflicting or duplicated constraints.
- Semantic Validation: Check generated content against explicit rules before accepting or displaying it.
- Structured Output Enforcement: Use constrained decoding for syntactic requirements such as valid JSON or a fixed grammar. Constrained syntax does not generally guarantee semantic rules such as perspective, character ownership, or factual consistency.
- User-Triggered Recovery: Provide a retry, reset, or undo workflow for visible drift that passes a cleanly assembled prompt to the model again.
As prompt size approaches the configured context limit, KV-cache memory consumption grows with sequence length and prompt prefill becomes more expensive. On memory-constrained systems, this can cause latency spikes, reduced generation speed, backend-specific memory offloading, or swapping.
- Example: A background memory update that overlaps with user-facing generation may cause both requests to compete for local GPU, VRAM, unified memory, or CPU resources.
A larger context can also make relevant information harder to retrieve, but repetitive output, rambling, or abrupt cutoffs do not have one universal cause. They may result from model behavior, retrieval degradation, decoding settings, output-token limits, stop sequences, or backend constraints.
- Bound Raw History: Limit unnecessary raw conversation history and retain only the most relevant recent turns.
- Summarize Older Context: Represent older events through compact summaries or structured state rather than continually expanding the raw prompt.
- Choose an Appropriate Context Size: Configure a context length that fits the available hardware and the requirements of the selected model.
- Serialize Local Inference Work: Avoid concurrent background and foreground model requests when they would exceed available compute or memory.
- Backend-Specific KV-Cache Optimization: Use KV-cache quantization or offloading only when supported by the selected backend and after measuring its memory, speed, and quality trade-offs.
These failure classes describe observable behavior, not directly measured internal model states. Their severity depends on the model, prompt, context, quantization, backend, and decoding configuration.