control-vector: add projective apply mode (h -= alpha*(h.d)d) - #1
Open
msuiche wants to merge 7 commits into
Open
control-vector: add projective apply mode (h -= alpha*(h.d)d)#1msuiche wants to merge 7 commits into
msuiche wants to merge 7 commits into
Conversation
Control vectors have only ever been additive here: build_cvec() does
h += v. Refusal-ablation work measures a different operation on the same
container -- h -= alpha*(h.d)d, deleting the component along a direction
rather than steering towards it.
The two are not a scale factor apart, and that is the problem worth
solving. A projective direction loaded by the additive path raises no
error and produces wrong output: it pushes every token along the refusal
axis instead of removing that component. Tensor names, dtype and shapes
are all valid, so nothing downstream notices. Measured on stories260K
with identical direction data, add and project differ by 5.13 max logit.
So the operation travels with the file, as gguf key `dspark.mode`, and an
unrecognised value is fatal -- there is nothing safe to fall back to.
Absent key means add, which is what every control vector written before
this key existed is.
include/llama.h llama_cvec_apply_mode; llama_set_adapter_cvec_ex
llama-adapter.cpp projective branch; unit-norm warning
llama-context.{h,cpp} plumb mode/alpha; old entry point wraps as ADD/1.0
common/common.{h,cpp} read mode/alpha_default/hook_point; refuse mixing
The apply is mul_mat/repeat/mul/scale/sub rather than out_prod, since
those are supported on every backend. mul_mat contracts over the
embedding axis to give one coefficient per token; mul with a [1,n_tokens]
operand broadcasts it back, which is the rank-1 outer product.
Four things the loader refuses rather than guesses at:
- unknown dspark.mode
- dspark.hook_point naming a hook we do not apply at. The same direction
applied after attn.wo_b instead of the post-layer residual measured 9x
weaker, so this is a correctness field, not documentation.
- combining a project-mode file with any other vector. The existing
loader sums across --control-vector arguments, which is meaningful for
additive vectors only; two projections do not compose into a projection
along the sum of their directions.
- two direction tensors for the same layer in project mode.
alpha stays a separate parameter and is never folded into the data:
projection is quadratic in the direction's norm, so scaling d by s scales
the removal by s^2. --control-vector-scaled's strength therefore becomes
alpha for project-mode files instead of a data multiplier.
llama_set_adapter_cvec() keeps its signature and delegates with
(ADD, 1.0), so existing callers and additive vectors are unaffected.
Tests: test-cvec-project checks the op composition numerically against a
hand-computed projection -- a transposed operand or a broadcast along the
wrong axis still yields a same-shaped tensor and plausible output, so
shape checks are not enough. test-cvec-model checks logits on a real
model, including that alpha=0 is bit-identical to no vector at all. That
control is there because the vLLM port of this shipped a bug where the
alpha=0 arm traced a graph with the op removed and cached it, which made
the entire ablation look like a no-op.
The 1-based naming invites two readings and we shipped the wrong one.
common_control_vector_load_one() stores direction.N at data offset
(N-1)*n_embd, which reads like "N-1 is the layer". But
llama_adapter_cvec::apply() then fills tensors[il] from offset
(il-1)*n_embd, so the two -1s cancel: tensors[il] holds direction.il and
apply_to() uses it at graph layer il.
test-cvec-layer-map settles it without reading the code again: put one
direction in one slot, sweep single-layer il_start/il_end ranges, and see
which layer responds. direction.3 responds at layer 3 and nowhere else.
This matters because the wrong reading does not fail, it degrades.
Adjacent-layer refusal directions have cosine similarity 0.83-0.91, so a
vector shifted one layer up the stack still ablates, still answers
coherently, and still passes any smoke test. It cost us a wrong exporter
that no in-runtime measurement could have caught, since our own export and
import cancelled each other out.
test-cvec-inspect reads a real vector through this loader and prints mode,
alpha, populated slots, resolved layer ids and per-direction norms. No
model needed, so a 300B-class model's control vector can be checked on a
laptop -- which is how the off-by-one surfaced: it reported layers 11..39
for a file derived from layers 10..38.
Worth noting for anyone comparing: llama.cpp's own cvector-generator names
the difference measured at layer il as direction.{il+1} (mean.hpp:18,
pca.hpp:305), so upstream's generator and applier sit one layer apart. Not
touching that -- existing vectors depend on the applier's behaviour, and
the applier is what decides where a distributed file takes effect.
Metal does not implement GGML_OP_OUT_PROD -- only CPU and CUDA do (ggml-cuda.cu:4969). The one-op form would split the graph and fall back to CPU on Apple silicon, so the five-op composition is required rather than merely preferred. The comment said the wrong thing.
The project-mode duplicate check probed whether the destination row was still zero, which misjudges a direction that happens to be zero in the probed positions. Track filled layer indices in a set instead. Fixture and assertion added: cv-duplayer.gguf carries direction.2 and direction.02 -- distinct gguf tensor names, same parsed layer index, since gguf names must be unique. Verified the check fires rather than assuming it.
Project mode takes alpha from --control-vector-scaled and otherwise from the file's dspark.alpha_default. The choice was keyed on "strength != 1.0", which cannot tell an explicit "-scaled f:1.0" from the 1.0 that --control-vector fills in by default -- so asking for alpha=1.0 got silently replaced by the file's 4.0, a 4x difference in ablation strength. Adds strength_set to common_control_vector_load_info, set only by --control-vector-scaled. Both directions asserted: explicit 1.0 wins, unset falls through to the file.
A project-mode file may carry dspark.layer_ids_zero_based. It is informational -- the direction.<N> tensor names are what get applied -- so when the two disagree the file was produced by a broken exporter and would steer the wrong layers. This is not hypothetical. An exporter emitted direction.11..39 for directions derived at layers 10..38 while that field still read 10..38. The file was self-inconsistent, and a reader that only looks at the names cannot tell. Worse, the resulting one-layer shift does not fail: adjacent layers' refusal directions have cosine similarity 0.83-0.91, so the vector still ablates and still answers coherently. Refuse it instead, reporting both ranges. Two fixtures: cv-shifted.gguf reproduces the bug, cv-consistent.gguf is the same file done correctly, so the test proves the check fires without false-positiving. Also verified against the real 300B-class artifact and its pre-fix counterpart.
Measured over the 29-layer vector the range is 0.555-0.979 (mean 0.863), not the 0.83-0.91 quoted. Both comments use it to explain why a one-layer shift degrades instead of failing; that still holds, but the true spread is wider at the low end than stated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a projective apply mode to the existing control vector path, beside the
additive one it has always had.
Why
build_cvec()doesh += v— it steers towards a direction. Refusal-ablationwork measures a different operation on the same container:
These are not a scale factor apart, and that gap is the problem worth solving. A
projective direction loaded by the additive path raises no error and produces
wrong output: it pushes every token along the refusal axis instead of removing
that component. Tensor names, dtype and shapes are all valid, so nothing
downstream notices. Measured on
stories260Kwith identical direction data, thetwo differ by 5.13 max logit.
So the operation travels with the file as gguf key
dspark.mode, and anunrecognised value is fatal — there is nothing safe to fall back to. Absent key
means
add, which is what every control vector written before this key existed is.Motivating result: on a 304B-class MoE, a 478 KB projective control vector
reaches 0% refusal on four held-out offensive-security suites, matching a
re-uploaded abliterated checkpoint of the same model. Runtime-toggleable via
alpha, no weights redistributed.What changed
include/llama.hllama_cvec_apply_modeenum;llama_set_adapter_cvec_ex()src/llama-adapter.{h,cpp}apply_to();mode/alphastate; unit-norm warningsrc/llama-context.{h,cpp}common/common.{h,cpp}dspark.mode/alpha_default/hook_point; refuse mixingcommon/arg.cpp--control-vector-scaledwas actually passedtests/llama_set_adapter_cvec()keeps its exact signature and delegates with(ADD, 1.0f), so every existing caller and every pre-existing additive vector isunaffected. That is asserted, not assumed.
The apply
ggml_tensor * coef = ggml_mul_mat(ctx, layer_dir, cur); // [1, n_tokens] ggml_tensor * proj = ggml_mul(ctx, ggml_repeat(ctx, layer_dir, cur), coef); cur = ggml_sub(ctx, cur, ggml_scale(ctx, proj, alpha));mul_matcontracts over the embedding axis for one coefficient per token;mulagainst a
[1, n_tokens]operand broadcasts it back, giving the rank-1 outerproduct.
ggml_out_prodwould do this in one op, but Metal does not implementGGML_OP_OUT_PROD(only CPU and CUDA do,ggml-cuda.cu:4969), so it would splitthe graph and fall back to CPU on Apple silicon. These five ops run natively
everywhere.
Things the loader refuses rather than guesses at
dspark.modedspark.hook_pointnaming a hook we do not apply at. The same direction appliedafter
attn.wo_binstead of the post-layer residual measured 9x weaker, sothis is a correctness field, not documentation.
across
--control-vectorarguments, which is meaningful for additive vectorsonly — two projections do not compose into a projection along the sum of their
directions, and the sum of two unit vectors is not a unit vector.
alphais never folded into the data: projection is quadratic in the direction'snorm, so scaling
dbysscales the removal bys^2.--control-vector-scaled's strength becomesalphafor project-mode files. Thatneeded
strength_seton the load info, because keying onstrength != 1.0cannottell an explicit
-scaled f:1.0from the 1.0 that--control-vectorfills in,and silently replacing a requested 1.0 with a file's 4.0 is a 4x difference in
ablation strength.
Tests
test-cvec-project— the op composition against a hand-computed projection.Shape checks are not enough: a transposed operand or a broadcast along the wrong
axis still yields a correctly shaped tensor and plausible output. Asserts
(h'.d)=0at alpha=1, elementwise equality at alpha=4, bit-exact identity atalpha=0, orthogonal tokens untouched, idempotence at alpha=1. Runs in ctest, no
model needed.
test-cvec-model— logits on a real model. 27 assertions including:That last one matters: it fails if the new mixing check is too broad. And the
alpha=0 bit-identity check exists because a separate port of this shipped a bug
where the alpha=0 control traced a graph with the op removed and cached it, making
the entire ablation read as a no-op.
test-cvec-layer-map— pinsdirection.Nto a graph layer by measurement.The 1-based naming invites two readings:
common_control_vector_load_one()storesdirection.Nat offset(N-1)*n_embd, which reads like "N-1 is the layer", butapply()then fillstensors[il]from(il-1)*n_embd, so the two-1s cancel anddirection.Nlands at layerN. This test puts one direction in one slot andsweeps single-layer ranges to see which layer responds.
It is here because reading the load function alone gave the wrong answer and
produced a wrong exporter elsewhere. A one-layer shift does not fail, it
degrades — adjacent layers' refusal directions have cosine similarity 0.83-0.91,
so a shifted stack still ablates, still answers coherently, and still passes a
smoke test.
test-cvec-inspect— prints mode, alpha, populated slots, resolved layer ids andper-direction norms for any control vector, without loading the model it belongs
to. This is what surfaced the off-by-one above, by reporting layers 11-39 for a
file derived from layers 10-38.
Existing tests: 8/8 pass (
test-arg-parser,test-sampling,test-chat*,test-json-schema-to-grammar).Note on cvector-generator
Not touched, but worth recording: it names the difference measured at layer
ilas
direction.{il+1}(mean.hpp:18,pca.hpp:305), so the generator and theapplier sit one layer apart. Whether that is deliberate or an old off-by-one, the
applier is what decides where a distributed file takes effect, so this PR matches
the applier and changes neither.
Scope
Vectors are checkpoint-specific; no cross-model transfer is claimed.
rank > 1isexpressible in the format but not implemented here, and measured no better than
rank-1 in the work this comes from.