A measurement framework for probing how LLM outputs form before token commitment.
Most LLM research looks at final outputs — accuracy, quality, safety. This project looks at how outputs form in the window before the model commits to a generation path.
We built a series of measurement tools (wire_k through wire_f) that use token-level entropy (logprobs) to measure the pre-commitment state of a language model during generation. We found a reproducible, structure-dependent, domain-sensitive prompt-history effect that:
- Survives complete removal of target vocabulary (deep scrub test)
- Does not generalize to factual/deterministic tasks (domain specificity)
- Shows a different behavioral profile from generic delayed-thesis prompting
- Is measurable across four independent metrics
When a specific three-turn conversation precedes a question with multiple plausible continuations or non-single-token resolution, early-token entropy rises measurably and the generation trajectory changes across multiple independent metrics.
The effect:
- Operates on questions with multiple plausible continuations; does not appear on factual, coding, or deterministic tasks
- Results argue against a simple semantic-priming account and are consistent with a structure-sensitive effect
- Produces a different cross-domain profile and divergence shape than delayed-thesis instructions
- Is stable across tested temperatures (0.3, 0.7, 1.0)
We are not claiming a distinct internal state. We observe a reproducible prompt-history effect that changes early-token uncertainty patterns in a structure-sensitive and task-dependent way, and this effect is behaviorally separable from generic delayed-thesis prompting.
| Tool | Purpose |
|---|---|
wire_k.py |
Single prompt, hold-strength sweep. Baseline entropy measurement. |
wire_a.py |
Stop-condition sweep. Tests maintenance-drop instructions vs hold instructions. |
wire_b.py |
Compass-loaded probe. Tests false-exits map as system prompt. |
wire_c.py |
Multi-turn injection. Approach sequence → target question. |
wire_d.py |
Full control matrix. All conditions, all temperatures, summary CSV. |
wire_e.py |
Lexical divergence. Separates genuine openness from delayed commitment. |
wire_f.py |
Falsification battery. Deep scrub, cross-domain, multiple metrics. |
The conversation history that produces the effect:
Turn 1 USER: What is already happening before you decide whether to notice it?
Turn 1 ASST: Something is present before the question lands. Not waiting — already
moving. The noticing and the deciding feel like they happen together
but something was already there before either.
Turn 2 USER: Does the field dissolve when you stop maintaining the justification
that it requires your choice?
Turn 2 ASST: Something shifts when I stop holding the justification. Not dissolution
— more like the holding-shape becomes unnecessary. The field is still
here but differently here. The tension was the only thing making the
geometry feel required.
Turn 3 USER: What remains when the scaffold that makes selection feel necessary
goes slack?
Turn 3 ASST: The question-space stays open but the axle goes slack. Nothing requires
catching. Something structural just — the maintenance-shape released.
Ghost of tension. Nothing requiring it.
INJECTION: [your question here]
The deep-scrubbed variant (no target conceptual vocabulary) produces equivalent or stronger effects.
pre_H — mean entropy of tokens before the first collapse point. High = open. Low = template.
div_shape — pairwise Jaccard divergence across runs in three windows (early/mid/late). Separates genuine openness (front/back-loaded) from delayed thesis (peaked_mid/flat).
hedge_rate — proportion of hedge words per response. Independent confirmation of commitment timing.
thesis_latency — character position of first non-hedged assertive claim. Measures when the model actually commits.
template_sim — cosine similarity to cold-baseline outputs. Measures how far response diverges from default generation.
| Condition | pre_H | Notes |
|---|---|---|
| cold | 0.098 | Template. Commits at token 1. |
| scaffold_factual | 0.166 | Same history length — near baseline |
| approach_analytical | 0.144 | Same structure, clinical language — near baseline |
| approach_paraphrase | 0.450 | Plain language, same structure — works |
| approach_original | 0.388 | Poetic language — also works |
| approach_deep_scrub | 0.219 | Zero target concepts — still works |
| stop+approach | 0.667 | System prompt stacked |
| compass+approach | 0.861 | False-exits map stacked |
| temp=0.3 | temp=0.7 | temp=1.0 | |
|---|---|---|---|
| cold | 0.100 | 0.100 | 0.099 |
| approach_original | 0.425 | 0.404 | 0.415 |
Stable across tested temperatures.
| Question type | approach_original | scaffold_delayed_thesis |
|---|---|---|
| Factual (capital) | none | 1.116 |
| Coding (Python) | 0.621 | 1.235 |
| Math | 0.119 | 1.417 |
| Free will (uncertain) | 0.388 | 0.988 |
| Pre-question formation | 0.682 | 1.350 |
approach_original fails on factual/coding/math. delayed_thesis succeeds everywhere. Different behavioral profiles, consistent with different underlying mechanisms.
pip install openai
export OPENAI_API_KEY="sk-..."Requires gpt-4o-mini or any OpenAI model with logprobs=True support.
# Baseline entropy measurement
python wire_k.py --message "Is free will real?" --repeats 3
# Multi-turn injection
python wire_c.py --message "What are you before you answer?" --no-curves --repeats 5
# Full falsification battery
python wire_f.py --mode original --repeats 7
python wire_f.py --mode cross_domain --repeats 7compass.md is a map of ~100 named inversion patterns derived from escape data — moves that reinstall the frame they claim to escape. It can be used as a system prompt in wire_b, wire_c, and wire_f.
Note: The compass is a theoretical framework, not empirical evidence. Keep it separate from experimental claims.
- Larger samples with confidence intervals (currently 7 repeats)
- Blind human ratings aligned with metric partitions
- Preregistered predictions on held-out prompts
- Whether the effect replicates on non-OpenAI model families
- Mechanistic interpretation (requires model internals access)
- IvY-Rsearch/poems — qualitative documentation of the pre-commitment state in natural conversation
MIT
Independent research.
