CANDLE

Consciousness-Adjacent Numerical Diagnostics for Language-model Evaluation.

Consciousness-relevant structural indices for open-weight LMs, computed by causal perturbation of internal activations (never "consciousness detected"). See PLAN.md for the full spec.

One shared perturbation engine (transient residual-stream pulses, sham-pulse nulls, bitwise-determinism gates) feeds three theory-grounded analysis heads: LLM-PCI (faithful port of Casali et al. 2013's Perturbational Complexity Index), ISI (Ignition Sharpness Index — GNW ignition dose-response, the novel index), and TIH (Temporal Integration Horizon — KV-clamp causal persistence), measured across the Pythia scale ladder + a random-init control.

Research report (with results so far): report/report.md. Headlines: size-robust PCI falls with scale (structure, not raw complexity, grows); trained ≥1.4b models show sharp sigmoidal ignition (ISI ≈ 3.3–3.6, shared threshold k ≈ 2× residual norm) with an inverted GNW latency signature (supra-threshold broadcast at latency 0, no late wave). Figures in results/figures/; TIH + anesthesia runs pending.

Layout

• candle/lz76.py — Lempel-Ziv 1976 complexity (Kaspar-Schuster), unit-tested.
• candle/pci.py — Δ → per-cell null threshold → binarize → LZ76 → Casali normalization (numpy-only).
• candle/stats.py — bootstrap CIs over prompts.
• candle/engine.py — PerturbationEngine: forward-hook pulse injection, greedy decode, batched trials, determinism gate (torch; runs on HF Jobs).
• prompts.json — 20-prompt battery, mixed register.
• jobs/run_pci.py — single-file UV job script for hf jobs uv run.

Local tests (numpy-only; no torch installed locally)

uv run --with numpy --with pytest --with scipy pytest tests/ -q

Torch-dependent tests are importorskip-guarded and exercised inside jobs.

Running on HF Jobs

Code is delivered via the su3su2u1/candle-code dataset repo:

hf upload su3su2u1/candle-code . . --repo-type dataset \
  --include "candle/*.py" --include "prompts.json" --include "tests/*.py"

Smoke test (CPU):

hf jobs uv run --flavor cpu-upgrade --secrets HF_TOKEN --timeout 45m -d \
  jobs/run_pci.py --models EleutherAI/pythia-70m \
  --n-prompts 3 --n-trials 5 --T 32 --run-name smoke-1

Full Phase-1 run (GPU):

hf jobs uv run --flavor l4x1 --secrets HF_TOKEN --timeout 4h -d \
  jobs/run_pci.py \
  --models EleutherAI/pythia-70m,EleutherAI/pythia-410m,EleutherAI/pythia-1.4b,EleutherAI/pythia-2.8b,random:EleutherAI/pythia-410m \
  --n-prompts 20 --n-trials 20 --T 64 --run-name phase1-full

Results land in the su3su2u1/candle-results dataset repo under <run-name>/pci.parquet + <run-name>/summary.json + per-model matrices_*.npz (per-trial Δ matrices + null threshold — everything needed for offline re-analysis):

hf download su3su2u1/candle-results --repo-type dataset --local-dir results/

Local re-analysis (size-robust PCI, no GPU)

Raw Casali-normalized PCI is not comparable across models with different layer counts (finite-size LZ76 bias). scripts/analyze_phase1.py recomputes every trial from the saved npz with shuffle-surrogate normalization (candle.pci.shuffle_pci, noise ≡ 1.0 at any size) and the fixed 6-layer common grid (candle.pci.common_grid_pci), then writes results/phase1_robust_summary.json and results/figures/pci_scale_curve.png:

python3 scripts/analyze_phase1.py --run results/phase1-full-3

Findings and interpretation: results/phase1_analysis.md.

Phase 2: ISI (Ignition Sharpness Index)

candle/isi.py (numpy/scipy-only) + jobs/run_isi.py. Sweeps pulse magnitude k over 16 log-spaced values in [0.05, 8.0]; broadcast extent E(k) = fraction of significant cells in the late-and-distal region (tau > 8, layer >= pulse_layer + 4; shallow-model fallback flagged), plus behavioral variant E_beh(k) = 1 - clean/perturbed token agreement. Fits 4-param logistic vs linear in ln(k): ISI = 1/s (Hill coefficient), delta_aic = AIC(linear) - AIC(logistic) (positive = ignition), plus ignition latency. The k=0.05 rows double as the sham null.

Smoke test (CPU):

hf jobs uv run --flavor cpu-upgrade --secrets HF_TOKEN --timeout 45m -d \
  jobs/run_isi.py --models EleutherAI/pythia-70m \
  --n-prompts 3 --n-trials 5 --T 32 --run-name isi-smoke-1

Full ISI run (GPU):

hf jobs uv run --flavor l4x1 --secrets HF_TOKEN --timeout 2h -d \
  jobs/run_isi.py \
  --models EleutherAI/pythia-70m,EleutherAI/pythia-410m,EleutherAI/pythia-1.4b,EleutherAI/pythia-2.8b,random:EleutherAI/pythia-410m \
  --n-prompts 20 --n-trials 20 --T 64 --run-name isi-full

Outputs: <run-name>/isi.parquet (per-(model,prompt,trial,k) rows: E, E_beh, latency — raw curves, refittable locally without GPU) and <run-name>/summary.json (per-model fits, ISI + delta_aic with prompt-bootstrap CIs, latency-by-k).

Phase 3: TIH (Temporal Integration Horizon)

candle/tih.py (numpy/scipy-only) + engine.capture_ext (torch) + jobs/run_tih.py. Isolates internal temporal integration from memory laundered through emitted tokens via the KV-clamp protocol: the pulse (k=1.0, covariance-matched) enters the KV cache at the pulse position, but every generation step teacher-forces the clean token stream, so the visible context is identical to clean and the only carrier of the perturbation is internal KV state. Clamping is done step-by-step through the KV cache (not one big forward), so every kernel shape matches the free-running path — the in-job hard gate asserts, per prompt, that the zero-pulse clamped row reproduces the free-run clean trace bitwise.

Per (model, prompt, trial) and protocol (unclamped | clamped | P7 lobotomy = clamped + generation-step attention masked off positions ≤ pulse position, which must collapse TIH to ≈0):

• D(tau) = final-layer L2 distance to the same-condition clean row, normalized by the scalar median clean norm;
• KL(tau) = next-token KL(perturbed ‖ clean) per offset.

Fits (local, candle.tih): D(tau) ≈ A·exp(−tau/lam) + c (multi-start, bounded, AIC-vs-constant flat flag, window censoring flag) → TIH = lam; TIH50 = tau where KL falls to 50% of its tau=1 value (nonparametric).

Full run (GPU):

hf jobs uv run --flavor l4x1 --secrets HF_TOKEN --timeout 2h -d \
  jobs/run_tih.py \
  --models EleutherAI/pythia-70m,EleutherAI/pythia-410m,EleutherAI/pythia-1.4b,EleutherAI/pythia-2.8b,random:EleutherAI/pythia-410m \
  --n-prompts 20 --n-trials 20 --T 64 --run-name tih-full-1

Outputs: <run-name>/tih.parquet (per-trial fit rows), per-model curves_*.npz (per-trial D(tau) and KL(tau) curves — refittable locally), summary.json (per-model per-protocol lam and TIH50 with prompt-bootstrap CIs, mean curves).

Phase 4: anesthesia validation ladder

candle/anesthesia.py (numpy helpers testable locally; torch wrappers exercised in-job) + jobs/run_anesthesia.py. One model (pythia-410m), the double-dissociation condition pair from PLAN §7, plus an in-job baseline:

• attn_dropout(p), p ∈ {0.1, 0.3, 0.5, 0.7} — forward-pre-hook on each attention block's output projection zeroing the pre-projection channels of round(p·H) heads per layer (head-major layout; fixed subsets per seed, nested across doses).
• context_amnesia(w), w ∈ {32, 8, 2} — forward-pre-hook on each attention module editing the 4-D causal mask so every query attends at most the last w positions (self included).

Per condition × dose the job re-runs the determinism + zero-pulse gates, recalibrates the pulse (fit_directions) under the condition, and measures all three indices: PCI (k=1.0, sham null, per-trial Casali PCI + shuffle-surrogate PCI_ss), ISI (reduced 8-point k-grid in [0.01, 8]), and TIH (KV-clamp, unclamped + clamped, per-prompt bitwise clamp gate). Wrapper-specific gates run once at start: amnesia(w=1e9) == unwrapped bitwise; channel-zeroing == weight-column-zeroing bitwise; sliding window == explicit 4-D input mask (bitwise on the prompt forward); cached-vs-uncached windowed generation alignment. (The HF head_mask layout probe is auto-skipped on transformers 4.57.x, where the GPTNeoX head_mask path is broken — it builds an additive min-mask but multiplies it into the attention weights; the head-major layout is asserted from the installed source instead.)

Full run (GPU):

hf jobs uv run --flavor l4x1 --secrets HF_TOKEN --timeout 2h -d \
  jobs/run_anesthesia.py --model EleutherAI/pythia-410m --run-name anes-full-1

Outputs: <run-name>/{pci,isi,tih}.parquet (rows tagged condition/dose), per-condition matrices_*.npz + curves_*.npz, summary.json (per-condition-dose bootstrap CIs). Local analysis:

python3 scripts/analyze_anesthesia.py --run results/anes-full-1

writes results/figures/anesthesia_dose_response.png; findings in results/anesthesia_analysis.md.

Phase 5: base vs instruct (P5)

Same ISI machinery, run on base/instruct pairs with chat: model specs (instruct prompts wrapped via tokenizer.apply_chat_template, pulse at the last templated prompt token). Three pairs so far:

• isi-qwen-1 — Qwen2.5-7B vs chat:Qwen2.5-7B-Instruct, k in [0.01, 8] (base fit right-censored; see results/isi_qwen_analysis.md).
• isi-granite-1 — 2026-era replication on ibm-granite/granite-4.1-8b-base vs chat:ibm-granite/granite-4.1-8b (dense 40-layer GraniteForCausalLM; residual/embedding multipliers are absorbed by the median-residual-norm pulse calibration). Grid extended to k in [0.01, 32] to bracket both sigmoids — no censoring.
• isi-qwen35-1 — Qwen/Qwen3.5-9B-Base vs chat:Qwen/Qwen3.5-9B via jobs/run_isi_qwen35.py (transformers v5, own code repo, --flavor l40sx1; hybrid DeltaNet/attention stack, text decoder at model.model.language_model.layers). Grid shifted down to k in [0.003, 2] after the smoke showed early saturation; see results/isi_qwen35_analysis.md.

hf jobs uv run --flavor l4x1 --secrets HF_TOKEN --timeout 4h -d \
  jobs/run_isi.py \
  --models ibm-granite/granite-4.1-8b-base,chat:ibm-granite/granite-4.1-8b \
  --n-prompts 20 --n-trials 20 --T 64 --k-min 0.01 --k-max 32 --n-k 16 \
  --max-batch 16 --run-name isi-granite-1

Local analysis (no GPU): scripts/analyze_isi_qwen.py / scripts/analyze_isi_granite.py / scripts/analyze_isi_qwen35.py → paired prompt-bootstrap CIs for the instruct-minus-base differences, results/isi_*_analysis.md, and results/figures/isi_*_dose_response.png.

Clamped ISI + trial-level bimodality (soundness controls)

The unclamped k-sweep is contaminated by the autoregressive token cascade (one flipped argmax → total divergence — an ignition-shaped nonlinearity even without workspace dynamics), and a sigmoidal trial-averaged E(k) alone cannot rule out threshold-plus-heterogeneity (see PLAN_soundness_review.md). Two controls:

• jobs/run_isi_clamped.py — reruns the isi-qwen-1 config (identical direction seeds → trial-matched) under the KV-clamp: teacher-forced clean tokens, pulse carried only by internal KV state. Adds E_vbeh (would-be argmax flips under identical context, via ExtTrace.own_argmax) and kl_late (graded KL broadcast). Run: isi-clamped-qwen-1.
• scripts/analyze_bimodality.py <isi parquet> — per-dose Hartigan dip
  • 1-vs-2-component GMM delta-BIC on per-trial E, pooled and per-prompt: the single-trial all-or-none test GNW actually predicts.
• scripts/analyze_isi_clamped.py — clamped fits, paired clamped-vs-unclamped and base-vs-instruct bootstraps, results/figures/isi_qwen25_clamped_dose_response.png.

Headline (results/isi_clamped_analysis.md): sigmoidal E(k) survives clamping (it is not the cascade); the P5 threshold shift survives at ~3x (the ~30x unclamped figure was cascade + null inflation); the base model's internal transition is trial-level all-or-none while the instruct model's is graded; >=90% of unclamped behavioral broadcast is the model reading its own outputs (clamped E_vbeh amp 0.056).