Skip to content

Local LLM Limitations: Entity Contagion & Drift

Jump to bottom
Justus Brugman edited this page Aug 22, 2026 · 10 revisions

Local LLM Limitations: Entity Contagion & Drift

Local LLMs can produce strong narrative prose while still struggling with entity consistency, lexical stability, long-context retrieval, instruction adherence, and local inference constraints. These limitations vary by model, quantization method, inference backend, prompt structure, and decoding configuration.

This page describes five commonly observed failure classes, explains what can be stated about them without treating speculative internal mechanisms as established facts, and lists practical mitigations.

1. Entity Contagion & Feature Bleeding

The Problem

During multi-character storytelling, a model may incorrectly transfer an attribute, possession, relationship, action, or role from one entity to another. This observable failure is sometimes informally described as entity contagion, entity bleeding, or feature bleeding.

  • Example: The prompt defines Chris as the guitarist and Mike as a childhood friend who does not play music. After a scene set at a music venue, the model generates:

    “Mike adjusted the strap of his guitar and smiled at Valerie as she approached.”

The output assigns Chris's established role to Mike. The error does not prove that a literal “floating trait” moved inside the model; that language is only a conceptual description of the observed result.

Conceptual Drift Pipeline

The following diagram illustrates the externally visible failure. It is not a representation of measured internal model activations.

flowchart TD
    A[Established canon: Chris is the guitarist] --> C[Longer narrative context]
    B[Established canon: Mike is not a musician] --> C
    C --> D[Mike becomes the active subject near music-related text]
    D --> E[Generation assigns the guitar-related action to Mike]
    E --> F[Entity-consistency failure]
Loading

Why It Can Happen

  • No built-in entity database: A base language model does not maintain a transactional character database. It generates each next token from the supplied context and its learned parameters.
  • Context-sensitive retrieval: Character facts may be retrieved less reliably when they are distant, surrounded by competing details, or expressed ambiguously.
  • Learned associations: Patterns learned during training can influence a continuation when the supplied constraints are weak, conflicting, or insufficiently salient.
  • No automatic semantic verification: Autoregressive generation does not inherently include a separate pass that compares every generated attribute with an external source of truth.
  • Model scale is not a guarantee: Larger parameter counts can improve capability, but do not guarantee consistent entity tracking in every prompt or generation.

Potential Mitigations

  • Keep character definitions explicit, compact, and unambiguous.
  • Maintain canonical state outside the model and include the relevant state in each request.
  • Limit unrelated raw history and summarize older events.
  • Validate candidate responses against explicit character constraints before displaying or persisting them.
  • Prefer semantic validation over regexes for relationship, ownership, role, or continuity checks; regexes are suitable only for narrowly defined textual patterns.

Storyteller Coverage

Storyteller includes fixed_protagonists.yml and canonical-state.yaml in the combined system message used for story generation. When validation is enabled, ResponseGuard performs a synchronous validator pass against the supplied rules and fixed-protagonist constraints and can return an ALLOW or REPLACE decision.

The validator does not currently receive canonical-state.yaml. It can therefore enforce explicit rules and fixed-protagonist constraints, but cannot directly validate every fact stored only in canonical state.

2. Lexical Corruption & Quantization Noise

The Problem

Aggressive weight quantization reduces numerical precision and can increase generation errors. Depending on the model, quantization method, inference backend, and decoding configuration, this may appear as malformed wording, inappropriate token choices, repetition, or less coherent prose.

  • Example: Instead of generating “She furrowed her eyebrows in frustration,” a heavily quantized model might generate:

    “She furrowed her highbrows in frustration” or “She furrowed her eyebrows wish.”

Quantization approximates model weights at lower precision. Its effects propagate through the network and cannot generally be attributed to a specific change in subword-embedding distances or to tokens being mathematically “adjacent.”

Potential Mitigations

  • Evaluate quantizations empirically with representative prompts; quality differences depend on the model and quantization method.
  • Use a higher-precision model representation when the measured quality improvement justifies the additional memory and compute cost.
  • Tune decoding settings for the selected model, while recognizing that sampling changes do not repair quantization error in the weights.
  • Apply spelling, grammar, or semantic post-processing only when its risk of altering intentional prose is acceptable.

Storyteller Coverage

Storyteller does not attempt to detect or correct lexical errors in generated story text.

ResponseSanitizer trims responses and decodes visible JSON-style escape sequences, but does not perform spelling, grammar, dictionary, or semantic correction. The validator checks explicit story rules and fixed-protagonist constraints; it is not designed to detect lexical corruption.

Any lexical degradation introduced by the selected model, quantization method, inference backend, or decoding configuration may therefore remain present in the final response.

3. Context Degradation (“Lost in the Middle”)

The Problem

Models can retrieve information less reliably when it appears deep inside a long context, particularly when relevant information is surrounded by large amounts of unrelated text. This position-dependent degradation is commonly described as the Lost in the Middle problem.

Needle in a Haystack evaluations measure related retrieval behavior by placing a specific fact—the “needle”—at different positions inside a much larger context and testing whether the model can retrieve it. Performance varies by model, prompt structure, information position, and context length. Support for a large context window does not guarantee that every part of that window will be used equally reliably.

  • Example: Important rules, character details, or plot events established many turns earlier may be ignored or recalled incorrectly when they appear deep inside the assembled context.

Potential Mitigations

  • Remove irrelevant or redundant raw history.
  • Summarize older events into compact, task-relevant memory layers.
  • Maintain important structured facts in canonical state rather than relying exclusively on narrative recall.
  • Keep the most relevant recent exchanges available as raw turns.
  • Avoid contradictory copies of the same rule across multiple prompt sections.

Storyteller Coverage

Storyteller avoids sending the complete raw conversation history to the model. Instead, it assembles each story request from a bounded window of recent raw turns, periodically generated memory summaries, canonical state, fixed-protagonist definitions, and the active system instructions.

  • Memory Hierarchy: Older events are periodically condensed into recent-summary.md, summary.md, and canonical-state.yaml after their configured batch thresholds are reached. These asynchronous derived-memory updates may temporarily lag behind the latest persisted turn.
  • Single System Message Stacking: Active instructions, fixed protagonists, canonical state, and summaries are assembled into one first system message. This provides a stable prompt structure, but does not guarantee that every included fact will be retrieved or followed.
  • Recent Raw-Turn Window: Only the most recent chat.maxRecentTurns story turns are included as raw conversation messages. Older turns remain persisted in history.json but are represented to the model through the derived-memory layers.

4. Instruction Drift & Guardrail Degradation

The Problem

During extended roleplay or storytelling sessions, accumulated narrative context can reduce the model's adherence to earlier instructions. Constraints such as “Never speak as the user” may be followed less consistently as the prompt grows and the generated narrative establishes competing patterns.

  • Example: The model may begin writing dialogue for the user's protagonist or stop following required character formatting.

System prompts influence probabilistic generation; they are not compiled program rules. This does not imply a measurable point at which instructions literally “collapse,” and drift cannot be explained solely as narrative text overpowering a system prompt.

Potential Mitigations

  • Assemble the active rules on every stateless request instead of assuming persistent model state.
  • Keep instructions concise, consistent, and free of conflicting duplicates.
  • Validate generated content against explicit semantic rules before accepting it.
  • Use constrained decoding for syntactic requirements such as valid JSON or a fixed grammar.
  • Do not treat grammar-constrained output as a guarantee of semantic rules such as perspective, character ownership, or factual consistency.

Storyteller Coverage: ⚠️ Best-Effort Mitigation

Storyteller uses several best-effort safeguards to detect, limit, and recover from drift:

  1. Periodic Cache-Buster Requests (cacheBuster.interval): After every N persisted story turns (default: 5), Storyteller sends a transient reset request with a unique token prepended to the system prompt. The response is discarded, and the request is not added to history.json.

    The changed prefix prevents exact-prefix cache reuse for that individual request on cache-sensitive OpenAI-compatible backends. It does not flush the backend's cache, modify persistent model state, or guarantee that later requests will follow the rules more closely.

  2. User-Triggered Resets: Ctrl-W sends the same transient reset request without modifying history.json. When a persisted turn is available, Ctrl-U removes the most recent turn, sends the reset request, and restores the removed user prompt to the input buffer for editing and retrying.

  3. Structured Validator Output: When validation is enabled, ValidationClient requests a constrained JSON Schema response containing an ALLOW or REPLACE decision.

    With validation.outputMode=auto, schema-constrained output is used when the backend supports it. If the backend rejects structured output, Storyteller falls back to tolerant text parsing for the remainder of the active session. Invalid, empty, or unparseable validator responses are handled using the configured fail-closed response.

These safeguards reduce the impact of drift but cannot guarantee that it will not occur. Schema enforcement constrains only the validator's decision format, not the original story generation.

5. Context Window Saturation & Performance Degradation

The Problem

As prompt size approaches the configured context limit, KV-cache memory consumption grows with sequence length and prompt prefill becomes more expensive. On memory-constrained systems, this can cause latency spikes, reduced generation speed, backend-specific memory offloading, or swapping.

  • Example: A background memory update that overlaps with user-facing generation may cause both requests to compete for local GPU, VRAM, unified memory, or CPU resources.

A larger context can also make relevant information harder to retrieve, but repetitive output, rambling, or abrupt cutoffs do not have one universal cause. They may result from model behavior, retrieval degradation, decoding settings, output-token limits, stop sequences, or backend constraints.

Potential Mitigations

  • Limit unnecessary raw history and redundant prompt content.
  • Summarize older context while retaining a short window of recent raw turns.
  • Configure a context size appropriate for the available hardware and model.
  • Use KV-cache quantization only when supported by the selected backend and after measuring its quality and performance trade-offs.
  • Serialize background inference jobs when concurrent local requests would exceed available compute or memory.

Storyteller Coverage

Storyteller reduces prompt growth and serializes derived-memory work, but does not impose a hard token limit on the complete assembled prompt.

  1. Raw-History Bounding: chat.maxRecentTurns (default: 2) restricts the number of raw narrative turns included in foreground story requests. This reduces prompt growth but does not limit the combined size of system instructions, fixed-protagonist definitions, canonical state, and summaries.

  2. Sequential Derived-Memory Queue (DerivedMemoryTaskQueue): After a story turn completes, Storyteller checks whether the long-term summary, recent summary, or canonical state requires an update. Eligible jobs are submitted to a single-threaded daemon queue, preventing derived-memory inference calls from running concurrently with one another.

  3. Asynchronous Background Updates: Eligible memory updates run asynchronously and sequentially, so they do not block completion of the story turn that scheduled them. A background update may still overlap with a later foreground story request when both use the same inference backend.

Summary

These failure classes describe observable behavior, not directly measured internal model states. Their severity depends on the model, prompt, context, quantization, backend, and decoding configuration. Application-level memory shaping, validation, constrained output, and workload scheduling can reduce their impact, but cannot provide absolute correctness guarantees.

Clone this wiki locally