v0.2-fulldepth: full-depth J-lens (20 prompts, 63 layers, intervention-grade)

Full-depth Jacobian lens for Qwen3.6-27B-4bit (MLX): 20 prompts x 63 layers (J_0..J_62), fp16 npz (~3.3 GB), fitted in 2.75 h on an M4 Pro / 64 GB.

What's new vs v0.1-demo (12 prompts, 23 late layers)

1. Full depth. All 63 source layers, enabling early/mid-layer readouts and workspace-level analyses, not just late-layer demos.
2. Exact analytic branch Jacobians (analytic_attn.py): attention (FA softmax + GDN recurrence) and MLP branches assembled analytically with input norms folded per-position — verified to ~1e-8 against mx.vjp on synthetic layers, and 1.4-2.1% vs the exact 5120-VJP reference on real activations (the residual is the single remaining branch-product junction; the old hybrid carried 3 such junctions at 1.8-2.6%). ~7-14x faster than the hybrid it replaces.
3. Chain indexing fix. The previous chain evaluated layer l's Jacobian at its own output acts[l] and saved the product under l — an off-by-one that left an extra layer-l factor in every J_l (33-49% rel error on a toy-chain ground truth; tests/test_chain_indexing.py). J_l now correctly transports acts[l] (what lens.transport and the interventions use). v0.1-demo carries the off-by-one; prefer this release.
4. Decay/write-gate gradients included. The custom Metal backward kernel (v4) now computes dg/dbeta (verified vs the batched ops BPTT to ~3e-7, incl. saturated gates), restoring the x->g/beta paths that were previously zeroed — measured at 4.9-7.5% of ||M_l|| (scripts/measure_gbeta_gap.py). The lens is intervention-grade, not just readout-grade.

Usage

# viewer
uv run python -m jlens_qwen.serve --lens data/lens/full_depth_analytic.npz
# readout sanity
uv run python scripts/workspace_range.py --lens data/lens/full_depth_analytic.npz

Fit convention: J_l = position-averaged, future-summed d(final_norm)/d(acts[l]) over 32-token prompts (positions 4..30), averaged over 20 prompts.

Readout sanity (currency prompt, top-5 per layer)

• L24-L48: Italy / Italian / 意大利 (intermediate concept, visible from L24 — v0.1's late-layer lens could not see it before L40)
• L54: 欧元 / Currency / euro (transition)
• L58-L62: euro / Euro / 欧元 — matching the model's answer (Euro)

Download (split asset — GitHub caps release files at 2GB)

# download both parts, then reassemble:
cat jlens-qwen3.6-27b-4bit-20prompt-63layer.npz.part-* > data/lens/full_depth_analytic.npz
shasum -a 256 data/lens/full_depth_analytic.npz
# expect: 0a5e0917f2747683eff05d2554d59ca5f25452420165fc27567ea5cfcbe6e9d7

v0.1-demo: pre-fitted J-lens (12 prompts, 23 layers)

⚠️ Superseded by v0.2-fulldepth. v0.1 fits only 12 prompts / 23 late layers and predates two correctness fixes shipped in v0.2 — a chain-multiply off-by-one and the GDN decay-gate (g/β) gradient paths. Prefer v0.2 for any new work.

Pre-fitted J-lens for immediate use

This release includes a pre-fitted J-lens so you can run the visualizer
without spending hours fitting your own.

What's in this lens (underfit / demo version)

• Model: mlx-community/Qwen3.6-27B-4bit
• Prompts: 12 (WikiText-103 + c4)
• Layers: 23 (L40–L62, late/workspace range)
• Fit method: VJP-based chain-multiply, 32-token sequences
• File: jlens-qwen3.6-27b-4bit-12prompt-23layer.npz (1.1 GB)

Quality — what to expect

This is a demo-quality (underfit) lens. It is sufficient to explore
the J-space visualizer and reproduce the core readout findings, but it is
not research-grade:

• ✅ Readouts work on factual prompts: "currency" → "Italian" (L52) →
  "euro" (L57+) on the boot-shaped-country prompt, matching the paper's
  pattern.
• ⚠️ Mid-layer noise: readouts in L47–L53 contain artifacts (____
  tokens) due to the low prompt count.
• ⚠️ Interventions are weak: steering/spiking changes the J-lens
  readout but barely changes the model's output. The J-lens vectors are
  small with only 12 prompts.
• ⚠️ No early layers: L0–L39 are not covered. The server falls back
  to the logit lens (J=I) there, which produces garbage before ~L55.

A better version is coming

A full-depth, 20-prompt lens is being fitted right now (all 64
layers, L0–L62). It will be published as a separate release asset
(v0.2-fulldepth) when the fit completes (~10 hours from now). That
version will have:

• Readouts at ALL layers (including the early/mid workspace range).
• Less noise (20 prompts vs 12).
• The ability to run the paper's workspace-level experiments (census,
  ablation, reportability).

An even better version (100+ prompts, analytic attention assembly) is
planned — see PERFORMANCE.md in the repo for the optimization roadmap.

How to use this lens

# Download the lens
gh release download v0.1-demo --repo WeZZard/jlens-qwen36 \
  --pattern '*.npz' --dir data/lens/

# Rename to the default path
mv data/lens/jlens-qwen3.6-27b-4bit-12prompt-23layer.npz data/lens/lens.npz

# Start the server
uv run python -m uvicorn jlens_qwen.serve:app --host 127.0.0.1 --port 8765

# Open http://127.0.0.1:8765/

Or fit your own

# Default VVP fit (25 late layers, ~5-8h):
uv run python scripts/run_fit.py --n-prompts 20 --n-layers 25 --layer-start 40

# Hybrid analytic fit (full 64-layer depth, ~10-12h):
# See README.md for the one-liner.