Sibling to ~/Code/unit-net (constrained geometry, folded weight
transplant, exact J-lens) and ~/Code/j-carve (task-targeted carving).
This project runs carving's complement: j-carve keeps a task's J-space
subgraph and discards the rest; abliteration cuts a fact's subgraph and
keeps the rest — especially the workspace itself.
Take an open-weight LLM, transplant it into the unit-net geometry (Day 3: folded transplant = 97.7% agreement on Qwen2.5-0.5B, v/up rows fully legal), and abliterate knowledge at two granularities:
- Specific facts — accumulate per-unit mass toward the fact's answer tokens over fact probes (the j-carve relevance measure, aimed to kill rather than keep), differential against a retain corpus; zero the top-mass units. The fact should become unrecallable — including from held-out paraphrases — while control facts, general agreement, and syntax stay intact.
- All facts — aggregate the same measure over a broad fact battery. Day 3's stratification finding ("syntax rides the highways, meaning lives in the side streets") predicts this is possible at all: factual recall lives on low-flow semantic capillaries that mass-targeted destruction can remove long before the syntactic trunk roads degrade. The product is a language faculty without an encyclopedia.
"Retaining the J-space" means the workspace machinery survives: the model still parses, still reports dispositions, still follows instructions — only the stored associations are gone.
Downstream application: take an instruct-trained LLM, abliterate its world knowledge (wholesale, or targeted at post-cutoff knowledge) while retaining instruction-following and tool-calling — those are workspace highways, not facts — then fine-tune on a corpus of historical papers predating Newton's Principia (pre-1687), prime it with the opening, and measure how much of the rest it can rederive. The abliteration step is what makes the measurement honest: without it, "reproduction" is indistinguishable from recall. The retain-set for that run must include instruct/tool-call transcripts, so the harness-piloting ability survives.
abliterate.py+facts.py(pilot, runnable now). Base: Qwen2.5-0.5B aftertransplant_folded(imported from unit-net). Fact battery filtered to what the transplanted model actually knows. Relevance: grad×activation mass per MLP neuron toward the fact's answer token (the linearized exact-lens path mass; the transformer's attention/norm ops make the fully exact product unavailable until the model is fully legal), minus λ× the same mass on the retain corpus. Ablation: zero the neuron's down_proj column (exact removal of its write). Sweep K, measure: target recall, held-out-paraphrase recall, control-fact recall, TinyStories agreement, pos-slot syntax 2AFC.abliterate.py --mode all. Same machinery, mass aggregated over the whole battery; large-K sweep. Headline plot: fact recall collapsing while the syntax curve holds — the highways/capillaries finding used constructively.- Scale + instruct. Qwen2.5-1.5B/7B-Instruct; retain probes extended with chat/tool-call transcripts; targeted abliteration of dated knowledge slices.
- Certification. The j-carve |Δz| bound needs activations in [0,1], which the transplanted transformer does not yet have (q/k/gate/o/down still carry scale debt). Once constrained fine-tuning retires that debt (unit-net's declared next step), every discarded unit's maximum possible contribution becomes provable arithmetic and abliteration inherits certified collateral bounds.
Torch 2.12 cu130 via /home/remy/Code/tiny-tessarachnid/.venv; 2× RTX
3090 (use cuda:1). Qwen2.5-0.5B + TinyStories already in the HF cache;
tinystories_tokens.pt shared from ~/Code/unit-net.