You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Directional steering (DS4-style activation steering) for Qwen3.8 Flash Next
#5461
I ported the directional steering idea from antirez/ds4 to oMLX and measured it on Qwen3.8 Flash Next. It is a new subsystem (~5.8k lines including tests, data, docs and 11 locales), so per CONTRIBUTING I'd like to check interest and approach before opening a PR.
What it does. A "pack" holds one direction per layer, learned from paired prompts (e.g. succinct vs verbose requests). At inference the hook shifts each layer's attention-branch output along that direction, with a strength you can change live in the model settings. No fine-tuning; nothing changes unless a pack is set.
Results MacBook Pro 14" M4 Max 128 GB, Qwen3.8 Flash Next oQ4e+MTP, 100-pair verbosity pack (DS4 prompts), temperature 0:
strength -0.5
strength -1
Answer length
-24 … -30%
-46 … -50%
KL on neutral text (gate ≤ 0.05)
0.021
0.045
NLL change
-0.3%
+0.2%
Repetition / valid JSON
lower / unchanged
lower / unchanged
Scope
Decode cost
prompt + first 32 tokens (default)
+0.2–0.4% (within noise)
every token
~5%
MTP draft acceptance: -2.1 to -2.3 points.
Try it
omlx steer build MODEL --good-file good.txt --bad-file bad.txt --name terse # or --preset verbosity
omlx steer sweep MODEL --pack terse --scales=-1,-0.5,0
omlx steer verify MODEL --pack terse --quick
or Model settings → Experimental → Directional steering.
Scope. New omlx/steering package; model settings fields (strength/scope change live, no reload; the prompt-cache key is salted so steered and unsteered prompts never share blocks); admin routes (build/verify jobs, pack list); omlx steer CLI; settings UI in 11 locales; docs. Only qwen4_exp is enabled; other architectures are refused with a clear message.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Hi!
I ported the directional steering idea from antirez/ds4 to oMLX and measured it on Qwen3.8 Flash Next. It is a new subsystem (~5.8k lines including tests, data, docs and 11 locales), so per CONTRIBUTING I'd like to check interest and approach before opening a PR.
What it does. A "pack" holds one direction per layer, learned from paired prompts (e.g. succinct vs verbose requests). At inference the hook shifts each layer's attention-branch output along that direction, with a strength you can change live in the model settings. No fine-tuning; nothing changes unless a pack is set.
Results MacBook Pro 14" M4 Max 128 GB, Qwen3.8 Flash Next oQ4e+MTP, 100-pair verbosity pack (DS4 prompts), temperature 0:
MTP draft acceptance: -2.1 to -2.3 points.
Try it
or Model settings → Experimental → Directional steering.
Scope. New
omlx/steeringpackage; model settings fields (strength/scope change live, no reload; the prompt-cache key is salted so steered and unsteered prompts never share blocks); admin routes (build/verify jobs, pack list);omlx steerCLI; settings UI in 11 locales; docs. Onlyqwen4_expis enabled; other architectures are refused with a clear message.Branch: https://github.com/undici77/omlx/tree/steering-experimental
Questions
All reactions