Skip to content

fix: reject malformed scaled steering entries with 4xx#29

Merged
RhizoNymph merged 1 commit into
feat/prefill-steeringfrom
fix/steering-malformed-entry-validation
Mar 31, 2026
Merged

fix: reject malformed scaled steering entries with 4xx#29
RhizoNymph merged 1 commit into
feat/prefill-steeringfrom
fix/steering-malformed-entry-validation

Conversation

@RhizoNymph

Copy link
Copy Markdown
Owner

Summary

Malformed dict-style steering layer entries (e.g. {"vector": [...]} without "scale", or {"scale": 2.0} without "vector") caused a KeyError that bubbled up as a 500 Internal Server Error. This PR fixes the issue in two places:

  • vllm/config/steering_types.py: normalize_layer_entry() now validates that both "vector" and "scale" keys are present in dict entries before accessing them. Missing keys raise a ValueError with a clear message listing exactly which keys are absent.

  • vllm/entrypoints/serve/steering/api_router.py: The three _normalize_spec() calls that happened before the main try block are now wrapped in their own try/except catching KeyError, TypeError, and ValueError, returning a 400 Bad Request with a descriptive error message.

Four new unit tests in tests/v1/test_steering_types.py cover the missing-key scenarios (missing "scale", missing "vector", missing both, and verifying the error message content).

What changed and why

The root cause was that normalize_layer_entry blindly did entry["vector"] and entry["scale"] on dict entries without checking the keys existed first. Additionally, the normalization step in the API router happened outside any error handling, so even if the function raised a proper error it would still surface as a 500.

Both issues are fixed: the function now validates up front and raises ValueError (not KeyError), and the router catches normalization errors and returns 400.

@RhizoNymph
RhizoNymph merged commit 5100a10 into feat/prefill-steering Mar 31, 2026
@RhizoNymph
RhizoNymph deleted the fix/steering-malformed-entry-validation branch April 2, 2026 06:02
RhizoNymph added a commit that referenced this pull request Apr 15, 2026
commit acf39ab1a8b7790be65ca3ceca623cd1b148569c
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 19:35:30 2026 -0700

    feat: gate global steering changes behind steering api key if set

commit 58ab6e83a55f6092470ff0b97535d03b17e9e999
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 18:50:40 2026 -0700

    remove agent overview doc

commit 3d77e12e1dc2685bbe7945878b0c60c006acbb68
Merge: ada55957f 3b08fb305
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Apr 14 17:09:56 2026 -0700

    Merge pull request #83 from RhizoNymph/refactor/steering-mixin

    refactor(steering): extract SteeringModelRunnerMixin to match LoRA pattern

commit ada55957f721d48848a5391efb2f4b0dd908e72d
Merge: 9a00f0f08 0b325140b
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Apr 14 16:58:05 2026 -0700

    Merge pull request #81 from RhizoNymph/docs/steering-update

    docs(steering): refresh supported-scope and runtime-boundaries

commit 9a00f0f083e25058f7c66108457a78e21c590ec5
Merge: 5ffa1cd69 0eba71d29
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Apr 14 16:57:15 2026 -0700

    Merge pull request #82 from RhizoNymph/refactor/steering-modules-router-merge

    refactor(steering): consolidate modules router under serve/steering/

commit 5ffa1cd6984fb48d04633f950e7808d31234d221
Merge: 52f85006f 548684983
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Apr 14 16:49:55 2026 -0700

    Merge pull request #80 from RhizoNymph/chore/rename-steering-phase-tests

    chore(steering): rename phase1..phase5 test prefixes to semantic names

commit 52f85006fd34715af8154237e703a85f50ef8d73
Merge: 95ffa52f3 49649a584
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Apr 14 16:48:29 2026 -0700

    Merge pull request #79 from RhizoNymph/chore/steering-sampling-params-imports

    chore(steering): hoist sampling_params inline imports

commit 3b08fb3059bb7042b434ec6f9ae40f136472f30f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 15:56:35 2026 -0700

    refactor(steering): extract SteeringModelRunnerMixin to match LoRA pattern

    Co-authored-by: Claude

commit 0eba71d2901616c54b3cdf1108a87e687a1fd535
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 15:00:50 2026 -0700

    refactor(steering): fold serve/steering_modules/ into serve/steering/

    Co-authored-by: Claude

commit 548684983d31bc2d192068326d8c83b8715cda63
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 14:06:57 2026 -0700

    chore(steering): rename phase1..phase5 test prefixes to semantic names

    Co-authored-by: Claude

commit 0b325140bbe60f00bbeee26e964ecec9ef654a23
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 14:04:14 2026 -0700

    docs(steering): refresh supported-scope and runtime-boundaries sections

commit 49649a58483c49f05f7b4b9dbba53c787105e93b
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 14:03:25 2026 -0700

    chore(steering): hoist sampling_params inline imports to module level

commit 95ffa52f355f74adfc9dfb207f7460cf05a085e4
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 13 13:41:55 2026 -0700

    perf(steering): batch per-layer table writes via index_copy_

commit 27a9fa4bc0226e98c66c6103c955dd62e63d191f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 13 13:38:14 2026 -0700

    perf(steering): skip populate when state is unchanged via dirty flag

commit 6d727d4f64eedf2ed39639892876c9bbc11537a6
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 13 02:07:31 2026 -0700

    perf(steering): make hash_steering_config 14x faster and cache it

commit 37a9446eea958eac8026963c441eb48f8abb02d9
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Apr 12 16:15:54 2026 -0700

    perf: short-circuit steering buffer updates when no state is active

commit b871782c147af79d7af78dc484b13d5853b47153
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Apr 12 15:48:24 2026 -0700

    fix: don't create steering manager at all when steering not enabled

commit d7db597e9b0aafec12e72c84889f1ddd3c1f3c49
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 9 01:04:34 2026 -0700

    chore: remove implementation docs

commit 2e20d1b9bf897ad593aa075793317cd8431cf2f2
Merge: 897aa5e3e 346383689
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 8 21:16:40 2026 -0700

    Merge pull request #77 from RhizoNymph/fix/remove-vestigial-vector-buffers

    fix: remove vestigial steering_vector_{hook} GPU buffers

commit 897aa5e3ee85680be7ea0a221eda9ff33789f12f
Merge: 85024d4ca 012dcf9f3
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 8 21:14:56 2026 -0700

    Merge pull request #75 from RhizoNymph/docs/remove-vector-buffer-refs

    docs: remove steering_vector_{hook} buffer references

commit 85024d4ca4776645e8d0e2c2c88abfd2073c1782
Merge: 9bcf5ce42 224c95539
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 8 21:14:14 2026 -0700

    Merge pull request #76 from RhizoNymph/fix/steering-transition-priority

    fix: prioritize prefill→decode transitions over new-request deferrals

commit 34638368923a706bab519b7b0b0cd13d6276ca52
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 21:11:33 2026 -0700

    fix: remove vestigial steering_vector_{hook} GPU buffers from Phase 1

    Closes #74

    Co-authored-by: Claude

commit 224c955390fff1f2098266018e6e59a95dafcd0c
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 21:09:55 2026 -0700

    fix: prioritize prefill→decode transitions over new-request deferrals

    Co-authored-by: Claude

commit 012dcf9f3be502fda884fba20c9ce3d9feab7d7f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 21:06:54 2026 -0700

    docs: remove steering_vector_{hook} buffer references

    Part of #74 — the vector buffers are being removed from the code.

    Co-authored-by: Claude

commit 9bcf5ce42b3e98e1b0beefe01aa2c2ffebba405a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 19:28:48 2026 -0700

    phase 5 of new models

commit a8cc4349303695d48dce7a09fdeebf722a180b3c
Merge: 183425b41 ff3bf0039
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 8 17:24:56 2026 -0700

    Merge pull request #71 from RhizoNymph/feat/named-steering-vectors

    feat: named steering vector modules

commit ff3bf00392efa6f575ce8a8b62510561d0ea2558
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 16:37:19 2026 -0700

    reject malformed json bodies instead of silently skipping them

commit 173c88b4a440e63caa2d8ecbdcefc3f3bce8085c
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 16:10:29 2026 -0700

    validate named steering entries

commit 183425b41a16442c123ef0a0171ea3aed0dfd098
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 15:56:13 2026 -0700

    feat: more steering models

commit eba7fd616e02e8f2d79d1a56732fd1cc72c37266
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 8 15:44:50 2026 -0700

    fix: gate named vector registry on actually enabling steering

commit b642ba6a42a12ceb4ca45046396f49714528da0b
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 23:41:37 2026 -0700

    feat: phase 1 of adding new models

commit 3a43d1cc1299b5a621d9b476a39112ecd1daf207
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 23:40:59 2026 -0700

    add register steering buffers helper

commit 15b49e31cbf7e7e072dfbde575c8a6bf6b75db16
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 23:36:15 2026 -0700

    feat: wire up named steering module registry, CLI args, and docs

    Co-authored-by: Claude

commit e9bf1d7455a0a6f06231e04e75c6af6c51e60712
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 23:31:05 2026 -0700

    feat: add steering modules CLI args and HTTP API endpoints

    Co-authored-by: Claude

commit cb3f0d418d2553afcc320a32d9e394dd2fc076fb
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 21:18:38 2026 -0700

    feat: add steering_name field and serving handler resolution

commit 48935407f5cdd564e1ab5fc24c16392b2d72c38f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 21:19:15 2026 -0700

    feat: add steering module registry and merge_steering_specs helper

commit 1e50302281f4f2c55481121617cb3f308bd8f055
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 16:00:19 2026 -0700

    Add steering docs

commit e27a7cced6a0a8da91c66d6045d4bbe4b24baa6e
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 7 15:43:30 2026 -0700

    fix: use httpx for async test and don't fail steering manager test
    without available cuda

commit 9a1c7e0badc9dd96a875a43628f07e732ab3560c
Merge: b1b5b93b1 5575ae564
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Apr 7 09:06:42 2026 -0700

    Merge pull request #70 from RhizoNymph/feat/prefill-steering

    feat: prefill/decode phase-aware steering vectors

commit 5575ae56469a4ef6b4fd0e57e226b30e97e5efce
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 6 17:05:35 2026 -0700

    fix: APC streaming semantics

commit b10dd23f57301f01796794c6869fb9968086777c
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 6 16:20:06 2026 -0700

    fix: keep steering prefix-cache keys consistent during deferred
    registration

commit 2aa6ac4a0adc16896cc1f562019989f8fef441f0
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 6 16:08:05 2026 -0700

    chore: update docs

commit f9c5245f18e876f8e22e25b1a27258ef7bfefb3f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 6 16:07:51 2026 -0700

    fix: remove post_mlp_post_ln hook point and rename post_mlp_pre_ln to
    post_mlp

commit b72b74c0e501b6341fd823d8318d025d5393421f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Apr 6 16:05:53 2026 -0700

    fix: update prompt length before recomputing block hashes to fix
    prefix-cache key misses

commit 03d11ebf5fceac2d9c2676e21d2907e421578cce
Merge: 650aaee77 cc205f6ff
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Apr 5 18:51:00 2026 -0700

    Merge pull request #69 from RhizoNymph/fix/steering-streaming-hash-mismatch

    fix: remove redundant _reset call from streaming steering path

commit cc205f6ff9dab721720aaf998e306f5f08a72116
Merge: 1de87974e 650aaee77
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Apr 5 18:50:50 2026 -0700

    Merge branch 'feat/prefill-steering' into fix/steering-streaming-hash-mismatch

    Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

commit 650aaee772d0d81ed4482b55b580533c1adf9159
Merge: e89f20aeb 4b3bb798e
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Apr 5 18:50:12 2026 -0700

    Merge pull request #68 from RhizoNymph/fix/steering-gpu-resident-vectors

    fix: store per-request steering vectors on GPU to avoid CPU->GPU copy

commit e89f20aeb20d69ea68bf3951284542e0c46d7405
Merge: 806e39cad 96d73fba5
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Apr 5 18:50:01 2026 -0700

    Merge pull request #67 from RhizoNymph/fix/steering-reject-extra-keys

    fix: reject extra keys in scaled steering entries

commit 1de87974e68cf455bea2ce9ab941b24040653d44
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Apr 5 18:37:37 2026 -0700

    fix: remove redundant _reset call from streaming steering path

    Co-authored-by: Claude

commit 4b3bb798e8604de998cfef30a74955b8090c9f8e
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Apr 5 18:30:32 2026 -0700

    fix: store per-request steering vectors on GPU to avoid CPU->GPU copy each step

commit 96d73fba5018d133ba239f2e23fd038f788b32a1
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Apr 5 18:30:24 2026 -0700

    fix: reject extra keys in scaled steering entries

commit 806e39cad4cb5ad9ff768d2d173d95312ab78bbc
Merge: 54b0ccb9b bed8e050f
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Apr 5 13:28:12 2026 -0700

    Merge pull request #65 from RhizoNymph/test/steering-retry-and-replay-coverage

    test: cover steering transition deferral and pending globals replay

commit bed8e050fdb1b6bbe6c43f86b775c220ec247841
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Apr 5 13:21:38 2026 -0700

    test: cover steering transition deferral and pending globals replay

    Closes two test-coverage gaps identified in the post-merge audit of the
    steering feature:

    1. TestTransitionUnderCapacity exercises the bookkeeping contract of
       _handle_steering_transition when decode registration fails due to
       capacity. Verifies the deferred entry is appended with phase="decode",
       _req_steering_phase[req_id] is set to "decode" regardless, and
       get_row_for_config falls back to row 2 until the next retry succeeds.

    2. TestPendingGlobalsReplay verifies that phase-labeled entries captured
       into _pending_steering_globals before SteeringManager lazy init land
       in the matching global_{base,prefill,decode}_vectors dicts rather
       than collapsing into a single tier, including end-to-end via
       populate_steering_tables.

    Also drops two unused-variable / nested-if lint issues in
    TestPreemptionResumption that were blocking pre-commit on this file.

    Co-authored-by: Claude

commit 54b0ccb9b1405c3feb388222388becc328fae091
Merge: 735fc4d91 416c9b388
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Apr 4 17:26:36 2026 -0700

    Merge pull request #64 from RhizoNymph/fix/steering-preempt-resume-config-leak

    fix: reset steering phase on preemption-resumption to prevent refcount leak

commit 416c9b38849fe2ed88b46a82345e32f7499d20b4
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Apr 4 16:33:53 2026 -0700

    fix: reset steering phase on preemption-resumption to prevent refcount leak

    When a request with per-request steering is preempted (e.g. via
    reset_prefix_cache() which fires on every global steering update) and
    later resumed, num_computed_tokens is reset to 0 but the model runner's
    _req_steering_phase and the SteeringManager's registered configs are
    NOT cleaned up. On resumption the request re-prefills using the global
    prefill row (not its per-request row), and the prefill->decode
    transition double-registers the decode config -> permanent row leak
    on finish.

    Add _reset_steering_for_resumption helper that releases the stale
    decode config, drops any stale deferred entries, and re-registers the
    prefill config. Call it from both the v1 cached-reqs resumption path
    (guarded by resumed_from_preemption) and the v2 _update_streaming_request
    path.

    Co-authored-by: Claude

commit 735fc4d9183af796a556ca31937b9a3b7d162fef
Merge: 7d566afc7 cde991759
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Apr 4 14:42:58 2026 -0700

    Merge pull request #62 from RhizoNymph/fix/steering-api-empty-hook

    fix: drop empty hook entries in steering _normalize_spec

commit 7d566afc7bb163838da34957ddd8786c5766190b
Merge: 6c56b5aa7 8ed116494
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Apr 4 14:32:36 2026 -0700

    Merge pull request #61 from RhizoNymph/fix/steering-manager-phase-validation

    fix: validate phase in populate_steering_tables

commit 6c56b5aa7dfaab2834aae67d0cc8d284af831630
Merge: b8f783c89 729f196f7
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Apr 4 14:31:59 2026 -0700

    Merge pull request #60 from RhizoNymph/fix/steering-prefill-decode-cross-validation

    fix: validate prefill/decode steering dimensions without base

commit cde991759948516e5afe1be2b4dd9664a19619a8
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Apr 4 13:37:59 2026 -0700

    fix: drop empty hook entries in steering _normalize_spec

    Co-authored-by: Claude

commit 8ed116494e6c30dd9696e64ef8560af8dd8696b2
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Apr 4 13:36:57 2026 -0700

    fix: validate phase in populate_steering_tables

    Co-authored-by: Claude

commit 729f196f757e8fe88e979ae07cc211e0fdcef8dd
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Apr 4 13:35:05 2026 -0700

    fix: validate overlapping prefill/decode steering dimensions without base

    Co-authored-by: Claude <noreply@anthropic.com>

commit b8f783c898a9ab2b8c66807be72da8b071b8cd23
Merge: d4e78d422 546d8e14c
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Apr 3 23:23:38 2026 -0700

    Merge pull request #58 from RhizoNymph/fix/steering-hash-invalidation

    fix: invalidate cached steering hashes on streaming sampling_params update

commit d4e78d4226c5354f61d927d23c2440221ce0adc9
Merge: 2fddb12f4 9377c7141
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Apr 3 22:38:04 2026 -0700

    Merge pull request #57 from RhizoNymph/fix/steering-reject-when-disabled

    fix: reject per-request steering when enable_steering is off

commit 546d8e14c92a543168a9dda37e6858a28a416a70
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Apr 3 21:34:06 2026 -0700

    fix: invalidate cached steering hashes on streaming sampling_params update

    Co-authored-by: Claude

commit 9377c7141c95b51cf8eb92bb3d0d2633674e6eec
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Apr 3 21:32:23 2026 -0700

    fix: reject per-request steering when enable_steering is off

    Co-authored-by: Claude

commit 2fddb12f4e86f813bdeba582973524d48fa77a76
Merge: a681974aa 5959d8fe3
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Apr 3 16:52:06 2026 -0700

    Merge pull request #56 from RhizoNymph/fix/steering-streaming-hashes

    fix: refresh steering hashes when updating streaming requests

commit a681974aaae738cafe54b883166b5fc063c59724
Merge: 4dc95f529 3526be8be
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Apr 3 16:51:29 2026 -0700

    Merge pull request #55 from RhizoNymph/fix/steering-forbid-extra

    fix: reject unknown fields in SetSteeringRequest

commit 5959d8fe325beb50a7294e123f4744f74b08ed0d
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Apr 3 15:40:08 2026 -0700

    fix: refresh steering hashes when updating streaming requests

    Co-authored-by: Claude

commit 3526be8be85624eb4506000ac540e9c958ba9c26
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Apr 3 13:19:23 2026 -0700

    fix: reject unknown fields in SetSteeringRequest to catch legacy scales

    Co-authored-by: Claude

commit 4dc95f52958069c4c431b7c0ebf1308d6b5ff1fe
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Apr 3 01:05:58 2026 -0700

    fix: fail on stale cache after steering change and relax decode-only admission

    Co-authored-by: Claude

commit e2ef338ed7bfe52bcabd343f2cdcd82f7faca9c7
Merge: da2fec961 6288714c5
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 23:25:29 2026 -0700

    Merge pull request #54 from RhizoNymph/fix/steering-atomic-replace

    fix: make steering replace=true atomic via single worker RPC

commit da2fec96126507d48e3d5fa3b588697f6238e8de
Merge: ee488edac bb24d9947
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 23:22:06 2026 -0700

    Merge pull request #53 from RhizoNymph/fix/steering-openai-plumbing

    fix: expose full steering API in OpenAI protocol models

commit 6288714c5d3ff3da62d0736a766b4974d501b1f3
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 23:07:33 2026 -0700

    fix: make steering replace=true atomic via single worker RPC

    Co-authored-by: Claude

commit bb24d99473b12a367a962f25e5c77de36a2b5d7d
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 23:05:25 2026 -0700

    fix: expose full steering API surface in OpenAI protocol models

    Co-authored-by: Claude

commit ee488edacddfb8da257320656769d978a0db92d2
Merge: 56f66ad7b 442e9eba8
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 18:34:01 2026 -0700

    Merge pull request #51 from RhizoNymph/fix/steering-request-attrs

    fix: remove unused steering vector attributes from Request

commit 56f66ad7b7e31fafad1b5b4e0c9e04bfc51debf9
Merge: 46e522e0f 9b2997d1c
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 18:29:20 2026 -0700

    Merge pull request #49 from RhizoNymph/fix/steering-scheduler-capacity

    fix: add decode-only capacity check and transition prediction for WAITING requests

commit 46e522e0f5fbd007b8c1b1dff6029f37481f341a
Merge: ece7238fb 7623fbe3f
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 18:26:51 2026 -0700

    Merge pull request #48 from RhizoNymph/fix/steering-model-runner-guards

    fix: add error handling for steering registration and fix backfill refcounting

commit ece7238fb087b3d498868085fa8841c47ef18292
Merge: 9ae9bcd37 6e6deadb1
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 18:25:48 2026 -0700

    Merge pull request #47 from RhizoNymph/fix/steering-device-mismatch

    fix: resolve GPU+CPU device mismatch in steering table population

commit 9ae9bcd37ac590c8f830af39bcc316fe55fe135a
Merge: 5d19f7f10 e6a9ffaf1
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 18:24:01 2026 -0700

    Merge pull request #50 from RhizoNymph/fix/steering-replace-clear

    fix: honor replace=True in set_steering_vectors when no new vectors provided

commit 5d19f7f10f2d53aad4759bfbfd24e5a0639a6a4d
Merge: 000d00f48 dd79f0844
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Thu Apr 2 18:22:54 2026 -0700

    Merge pull request #52 from RhizoNymph/fix/steering-cache-key-phase

    fix: disambiguate boundary block cache keys for prefill vs decode steering

commit 442e9eba84ba37a7e0439e43dd89509915326a17
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 00:31:03 2026 -0700

    fix: remove unused steering vector attributes from Request to prevent memory waste

    Co-authored-by: Claude

commit e6a9ffaf1031efd4b99fb8c7eac3f852f604a87b
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 00:29:30 2026 -0700

    fix: honor replace=True in set_steering_vectors when no new vectors provided

    Co-authored-by: Claude

commit dd79f084480a4092c5493303bb7ebf2940ab55e5
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 00:29:00 2026 -0700

    fix: disambiguate boundary block cache keys for prefill vs decode steering

    Co-authored-by: Claude

commit 9b2997d1cb33475567c791c3a03624972e0806c5
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 00:27:06 2026 -0700

    fix: add decode-only capacity check and transition prediction for WAITING requests

    Co-authored-by: Claude

commit 7623fbe3fb789b1776e1f0c4d26e70ed143de71a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 00:27:05 2026 -0700

    fix: add error handling for steering registration and fix backfill refcounting

    Co-authored-by: Claude

commit 6e6deadb1e9fd56a13fd72ed8adb22acb1979317
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Apr 2 00:27:04 2026 -0700

    fix: resolve GPU+CPU device mismatch in steering table population

    Co-authored-by: Claude

commit 000d00f48bbba901b7d4fa682c75eb6439be9db4
Merge: dfed3d2bf f9130cf07
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 1 18:34:56 2026 -0700

    Merge pull request #45 from RhizoNymph/fix/steering-lazy-init-capacity

    fix: gracefully handle capacity exhaustion during steering lazy init

commit f9130cf07d50a9ff0c02d89e4e5f22fb4c373d53
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 1 16:24:56 2026 -0700

    fix: defer steering registrations on capacity exhaustion instead of dropping

    Replace the silent fallback-to-global approach with a proper deferral
    mechanism.  Failed registrations during lazy init and prefill-to-decode
    transitions are now appended to a unified
    `_pending_steering_registrations` list that is retried with priority
    before new admissions on every step.

    The retry loop also filters stale entries:
    - Entries for finished requests are dropped (prevents row leaks).
    - Entries whose request changed phase are dropped (e.g. prefill
      deferred but request already transitioned to decode).

    This subsumes the decode-only `_pending_decode_registrations` list
    and the row-leak fix from PR #46.

commit 251cabe638c7b33d2dcbf27e1da9754dd2e63369
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 1 01:16:28 2026 -0700

    fix: gracefully handle capacity exhaustion during steering lazy init

commit dfed3d2bfd1557d7c8c3925dd5de8224c1e9e91d
Merge: 51672b8f1 41135ae57
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 1 12:53:25 2026 -0700

    Merge pull request #44 from RhizoNymph/fix/steering-swap-states-hash

    fix: swap steering hashes in InputBatch.swap_states()

commit 51672b8f1ef164970470580e380534f85ec3b35a
Merge: 47365775f f2638d4b7
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 1 12:36:32 2026 -0700

    Merge pull request #43 from RhizoNymph/fix/steering-dimension-cross-validation

    fix: validate matching dimensions between base and phase steering vectors

commit 41135ae5709c78d4639a0b0148967b491a38894e
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 1 01:16:27 2026 -0700

    fix: swap steering hashes in InputBatch.swap_states()

    Co-authored-by: Claude

commit f2638d4b7f2f6a9136c11362f08b89b9873d42f3
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 1 01:14:35 2026 -0700

    fix: validate matching dimensions between base and phase steering vectors

commit 47365775f6c7310d6381fb5495777950fd729997
Merge: 981ad5ac8 ebeff444f
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 23:59:48 2026 -0700

    Merge pull request #42 from RhizoNymph/docs/steering-bugfix-round2

    docs: update steering docs for deferred cleanup and pre-init status

commit 981ad5ac85d2c93c3ae1f356f148ea097350481d
Merge: 7431847f1 f06f5db30
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 23:59:35 2026 -0700

    Merge pull request #41 from RhizoNymph/fix/steering-status-pending-globals

    fix: report pending phase-specific globals in steering status

commit 7431847f1cd2647ba96829395b2037f2317f520f
Merge: f04aaf36d e1a6a2ef5
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 23:59:12 2026 -0700

    Merge pull request #40 from RhizoNymph/fix/steering-deferred-decode-leak

    fix: track req_id in deferred decode registrations to prevent leak

commit f04aaf36d143c040b2b652403de3f0449a5d3d62
Merge: 7e41715be e9d8f110c
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 23:58:37 2026 -0700

    Merge pull request #39 from RhizoNymph/fix/steering-boundary-block-hash

    fix: include decode hash in boundary block cache keys

commit ebeff444f4bd59751824305843481a4e92178ece
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 23:34:00 2026 -0700

    docs: update steering docs for deferred cleanup and pre-init status

    Co-authored-by: Claude

commit e9d8f110c39446cd8c4e9bcbaddc91c82236962f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 23:29:38 2026 -0700

    fix: include decode hash in boundary block cache keys

    Co-authored-by: Claude

commit f06f5db3000d6fb29d21219538cc4d9c43030e04
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 23:27:52 2026 -0700

    fix: report pending phase-specific globals in steering status

commit e1a6a2ef5da949d4636d98779cb146b803cb376b
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 23:26:48 2026 -0700

    fix: track req_id in deferred decode registrations to prevent leak

commit 7e41715be28060d2ebb7dd97bff0aef72cf04b27
Merge: 5edfb74f2 0c7e77579
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 19:44:19 2026 -0700

    Merge pull request #38 from RhizoNymph/docs/steering-bugfix-updates

    docs: update steering docs for bugfix PRs #35-#37

commit 5edfb74f2d3657093e316c5fe673cf58e6a1ce99
Merge: a7bdd3fe1 2f0a0c960
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 19:44:01 2026 -0700

    Merge pull request #36 from RhizoNymph/fix/steering-scheduler-transition-capacity

    fix: transition-aware steering capacity and decode-start admission

commit a7bdd3fe1ad5f2a3c4e25a67d4da9b1617a34992
Merge: 2830adbf0 b2d8d475c
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 19:08:27 2026 -0700

    Merge pull request #35 from RhizoNymph/fix/steering-worker-status-validation

    fix: remove double validation and add phase-specific norms to steering status

commit 2830adbf014067736c04afdc1ddfdb11c898009c
Merge: c01e6dce4 5cb702a2a
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 19:01:21 2026 -0700

    Merge pull request #37 from RhizoNymph/fix/steering-model-runner-transition

    fix: graceful steering transition + phase-tracked release

commit 0c7e77579e3cab4e5cdc6987e43a376481bb647d
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 18:58:04 2026 -0700

    docs: update steering docs for transition-aware capacity, deferred registration, and phase-specific norms

commit 5cb702a2aa253defaa567fbda72e8c2e26650267
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 18:30:11 2026 -0700

    fix: graceful steering transition with deferred registration and phase tracking

    Co-authored-by: Claude

commit b2d8d475c0010910fd2b680d9460dc9caa715708
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 17:59:07 2026 -0700

    fix: remove double validation and add phase-specific norms to steering status

    Co-authored-by: Claude

commit 2f0a0c96055e7330687ad1e101059eb3fa311040
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 17:58:31 2026 -0700

    fix: transition-aware steering capacity counting and decode-start admission check

    Co-authored-by: Claude

commit c01e6dce4032a00a962d1b257b3f907e327f5b54
Merge: 5ab47798d 84eeb8b21
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 01:05:21 2026 -0700

    Merge pull request #34 from RhizoNymph/fix/steering-decode-cache-block-hash

    fix: include decode steering hash in generated cache block keys

commit 5ab47798d2d6a6778e61610ddde0fcaad2f80240
Merge: c3bda6dc5 2b8a2a6ec
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 01:04:14 2026 -0700

    Merge pull request #32 from RhizoNymph/fix/steering-scheduler-single-phase-capacity

    fix: scheduler steering capacity counts only active phase

commit c3bda6dc5f8da7e9dcae49faba5c3b69aed6a67a
Merge: dd9e5b185 35c55f9eb
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Mar 31 01:02:59 2026 -0700

    Merge pull request #33 from RhizoNymph/fix/steering-lazy-init-decode-registration

    fix: register decode steering configs during lazy manager init

commit 84eeb8b21d7435d91655d6a19aa7b33f7de16800
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 31 00:01:30 2026 -0700

    fix: include decode steering hash in generated cache block keys

commit 35c55f9eb69bca17dd3f0c80830b419921bef073
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Mar 30 23:42:06 2026 -0700

    fix: register decode steering configs during lazy manager init

commit 2b8a2a6eccde4b10f8565bbc62e13c8024a85327
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Mar 30 23:41:46 2026 -0700

    fix: scheduler steering capacity counts only active phase

commit dd9e5b1852d053f31535ec75c72d973095b07424
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Mar 30 23:09:17 2026 -0700

    docs: update steering docs for PR #29, #30, #31 post-merge fixes

commit 63089a37e19e877625de97ad69e62dbf8f0f4b42
Merge: dbe8d1880 dcdf60110
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Mon Mar 30 23:00:16 2026 -0700

    Merge pull request #31 from RhizoNymph/fix/steering-buffer-overwrite

    fix: stop writing phase-specific globals to shared steering buffers

commit dbe8d18809e04fcfd7f959d90a4f7d2eed95adac
Merge: 5100a10d4 bd69fae0b
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Mon Mar 30 22:59:26 2026 -0700

    Merge pull request #30 from RhizoNymph/fix/steering-decode-start-registration

    fix: register decode steering config when request starts past prefill

commit 5100a10d4a0d95229c3b6587484b1a122bc08ae2
Merge: 8d1c1fe47 3de429a5a
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Mon Mar 30 22:57:48 2026 -0700

    Merge pull request #29 from RhizoNymph/fix/steering-malformed-entry-validation

    fix: reject malformed scaled steering entries with 4xx

commit dcdf60110fd6e89e8dd9a3128a8b3a71049de610
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 15:29:56 2026 -0700

    fix: stop writing phase-specific globals to shared steering buffers

commit bd69fae0b51f0c2e75f323955d62265b4623db9e
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 15:13:54 2026 -0700

    fix: register decode steering config when request starts past prefill

commit 3de429a5a84be3952759555b49ca01f22b05e37f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 14:40:26 2026 -0700

    fix: reject malformed scaled steering entries with 4xx instead of 500

commit 8d1c1fe47dac02e3b3f07ef324967a969d66790d
Merge: 9d8d7a4e9 4f7f350dd
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Mar 29 14:13:33 2026 -0700

    Merge pull request #27 from RhizoNymph/fix/steering-clear-pending-globals

    fix: clear pending steering globals on clear_steering_vectors

commit 9d8d7a4e99566fb6c445ef46de477110d0bf5f0c
Merge: c57ff54da d6efa42ce
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Mar 29 14:11:58 2026 -0700

    Merge pull request #28 from RhizoNymph/fix/steering-scheduler-phase-capacity

    fix: count steering capacity by (hash, phase) pairs to match worker row allocation

commit 4f7f350dd603817a7a0bbda59431220197550988
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 12:08:15 2026 -0700

    fix: clear pending steering globals on clear_steering_vectors to prevent stale replay

commit d6efa42ce8806af8846600a0ec6d39dfc5609952
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 12:07:57 2026 -0700

    fix: count steering capacity by (hash, phase) pairs to match worker row allocation

commit c57ff54daf3f93216b8d609b9719ed38e918339e
Merge: 849fa0577 2f3b14e09
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Mar 29 11:27:02 2026 -0700

    Merge pull request #25 from RhizoNymph/fix/steering-pending-globals-queue

    fix: preserve phase-specific global vectors set before steering manager init

commit 2f3b14e094f4f85932b7650819b3cb0e35716583
Merge: 55a9da811 849fa0577
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Mar 29 11:26:51 2026 -0700

    Merge branch 'feat/prefill-steering' into fix/steering-pending-globals-queue

    Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

commit 849fa0577a9ca55e201830e59d351140414c21c6
Merge: 018a7adcc 229ce88f3
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Mar 29 11:24:05 2026 -0700

    Merge pull request #24 from RhizoNymph/fix/steering-scheduler-phase-detection

    fix: use num_prompt_tokens for steering phase detection in scheduler

commit 018a7adcc417fa3cd8647954752fbade1cdbb31c
Merge: 007de86fc 91b89b480
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sun Mar 29 11:16:11 2026 -0700

    Merge pull request #26 from RhizoNymph/fix/steering-phase-keyed-rows

    fix: key steering table rows by (config_hash, phase) to prevent cross-phase aliasing

commit 91b89b4805a7a5a3464cd2d605211f4d8e2c1189
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 11:11:10 2026 -0700

    fix: key steering table rows by (config_hash, phase) to prevent cross-phase aliasing

commit 55a9da811b0ae47e46816eb8588bf234d8130835
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 00:38:21 2026 -0700

    fix: queue phase-specific global vectors for replay on steering manager lazy init

commit 229ce88f3392682b44cc69f7b5ca12e827acd14d
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 00:35:41 2026 -0700

    fix: use num_prompt_tokens for steering phase detection in scheduler

commit 007de86fc362ed6d86dc4bbebbf5e5a41581851e
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sun Mar 29 00:13:51 2026 -0700

    docs: update steering docs for prefill steering (Phase 4)

    - ROADMAP.md: Mark Phase 4 as DONE with full implementation summary
    - STEERING.md: Update to three-tier model, phase-aware table layout,
      remove stale "decode-only" language, update key files table
    - MVP_IMPLEMENTATION.md: Add Phase 4 lessons learned (prefix cache,
      lazy init race condition, phase detection, additive composition,
      co-located scales, scheduler dual-hash admission, test subtleties)
    - API.md: Update to three-tier format, co-located scales, remove
      old scales field, update examples

commit 8f055b92c5f82d1283a2b186619bcb2b128cf22a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 23:59:30 2026 -0700

    fix: register steering configs on first-step manager init + fix additive test

    The SteeringManager is lazily created in _update_steering_buffers, but
    config registration happens in _update_states which runs before. On the
    first request, registration was silently skipped because the manager
    didn't exist yet. Fix: after creating the manager, scan the batch and
    register any pending configs.

    Also fix test_additive_composition which incorrectly assumed base(250) +
    decode(250) equals base(500) — steering_vectors affects both phases, so
    prefill effective vectors differed between the two approaches.

commit f15d55b8d0748a38b4e5786c09afa6430d9d28e7
Merge: 9c1dca9f5 2917406a5
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 23:21:56 2026 -0700

    Merge pull request #23 from RhizoNymph/feat/prefill-steering-tests

    test: add prefill steering integration tests

commit 2917406a5d1e53de9fbf4db4fbbc1adde087777a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 22:59:26 2026 -0700

    test: add prefill steering integration tests and fix existing test API calls

    Co-authored-by: Claude

commit 9c1dca9f547e52defa4782326a2b6777e7940f8f
Merge: 2be915751 45100595f
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 21:43:26 2026 -0700

    Merge pull request #21 from RhizoNymph/feat/prefill-steering-manager

    feat: phase-aware SteeringManager and GPU model runner

commit 45100595fc03a46d5407edab6240971e39b3d2ed
Merge: 1beb47cc2 2be915751
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 21:35:19 2026 -0700

    Merge branch 'feat/prefill-steering' into feat/prefill-steering-manager

    Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

commit 2be91575101afb56f9952093c4c40b5f5ce00174
Merge: a286bd9d6 ec9e660b3
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 21:34:15 2026 -0700

    Merge pull request #22 from RhizoNymph/feat/prefill-steering-scheduler

    feat: prefill steering scheduler, API, and worker (WS4)

commit ec9e660b307566bba43e5230cd2321e437010510
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 21:33:10 2026 -0700

    feat: wire prefix cache invalidation for prefill-affecting steering changes

    Replace the TODO with actual calls to engine.reset_prefix_cache() when:
    - Global base or prefill vectors are set (affects prefill KV cache)
    - replace=True is used (clears all tiers including prefill)
    - All steering is cleared via /v1/steering/clear

commit a286bd9d660ad473232f9b60a0f04706494375a5
Merge: 67b6d9d65 89e056fb6
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 21:25:44 2026 -0700

    Merge pull request #19 from RhizoNymph/feat/prefill-steering-cache-keys

    feat: prefill steering prefix cache key integration

commit 89e056fb6bf5884135ef2a26c7e935da5c988b85
Merge: 68cce443a 67b6d9d65
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 21:22:09 2026 -0700

    Merge branch 'feat/prefill-steering' into feat/prefill-steering-cache-keys

    Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

commit 67b6d9d653da3c84fae431aad0a14e06b77ce85e
Merge: b1b5b93b1 46665f131
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 21:18:16 2026 -0700

    Merge pull request #20 from RhizoNymph/feat/prefill-steering-types

    feat: three-tier steering types with prefill/decode separation

commit b7d55a044a2fde9ae18c587711d8ddac6b03b9a6
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 21:06:52 2026 -0700

    feat: implement prefill steering scheduler, API, and worker changes

    - Scheduler: dual-hash admission control tracks union of active hashes
      (prefill hash for prefill-phase requests, decode hash for decode-phase).
      New requests must have capacity for both their prefill and decode hashes.
    - API router: three-tier global steering (vectors, prefill_vectors,
      decode_vectors) with co-located scale normalization. Removed old
      scales field. Uses kwargs-based collective_rpc for worker calls.
    - Worker: three-tier set_steering_vectors with base/prefill/decode tiers.
      Extracted validation, buffer application, and manager notification into
      separate helper methods. Supports replace=True across all tiers.
    - Tests: comprehensive dual-hash scheduler admission tests, three-tier
      API router tests with co-located scale format, and three-tier worker
      tests including replace and cross-tier validation.
    - Docs: updated steering.md with dual-hash admission details and
      three-tier HTTP API examples.

    Co-authored-by: Claude

commit 1beb47cc2be30910c88e5b2aeba266a6e6c65036
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 21:01:12 2026 -0700

    feat: phase-aware SteeringManager and GPU model runner integration

    Co-authored-by: Claude

commit 46665f131579bb24b50b0786d2424e9cd28a45fe
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 20:41:31 2026 -0700

    feat: add three-tier steering types with prefill/decode phase separation

    Co-authored-by: Claude

commit 68cce443a84085f7f591e00e9967d1a62fed5f93
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 20:38:47 2026 -0700

    feat: add prefill steering config hash to prefix cache block keys

    Add _gen_steering_extra_hash_keys() to include per-request prefill
    steering config hashes in block hash extra keys. This ensures that
    blocks computed under different prefill steering produce different
    cache entries, with zero impact when steering is not used.

    Co-authored-by: Claude

commit b1b5b93b15ad8bc2bcea10a786b9270398c66b3e
Merge: 03868735b 21c6acbb4
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Sat Mar 28 18:41:07 2026 -0700

    Merge pull request #18 from RhizoNymph/feat/steering-hook-points

    feat: Phase 3 — multiple steering intervention points

commit 21c6acbb4b55f6cb5244fb67522514660ca2abfa
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 18:36:12 2026 -0700

    docs: add Phase 3 lessons learned to implementation notes

    Key findings:
    - Splitting ops are not required for buffer correctness — custom op
      opacity is sufficient, graph partitioning is a separate concern
    - Inline tensor math triggers AOT autograd pickle failures due to
      Enum WeakValueDictionary in traced graph metadata
    - Always-allocate beats opt-in when memory is trivial vs model size
    - Hook points at same residual state exist for SAE position matching
    - Breaking backward compat cleanly beats merging logic

    Co-authored-by: Claude

commit 15f34a6d006cd509fe10297e007a9a658697d5c6
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 18:33:51 2026 -0700

    docs: update steering docs for simplified hook point design

    - Remove references to --steering-hook-points CLI flag
    - Remove references to steering_hook_vectors (legacy separate field)
    - Document that all 4 hook points are always allocated unconditionally
    - Document that apply_steering is NOT a splitting op (zero graph partitions)
    - Update all API examples to use the hook-point-aware steering_vectors format

    Co-authored-by: Claude

commit 6837fabf7a76a86eebdb0546a7c03b52ec1e2204
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 18:26:07 2026 -0700

    chore: remove --steering-hook-points CLI flag and config field

    All 4 hook point buffers are always allocated unconditionally.
    The memory cost is trivial (~11 MB at max_configs=16 for 26 layers)
    and there are no graph partitions, so there is no reason to control
    which hook points are active.

    Co-authored-by: Claude

commit 5369eead8d2575202add671ba975ccffbc9b1be3
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 17:49:26 2026 -0700

    feat: remove graph partitions, always allocate all hook point buffers

    - Remove apply_steering from splitting_ops — the custom op is still
      opaque to the tracer (preventing constant-folding) but no longer
      partitions the compiled graph
    - Always allocate all 4 hook point buffers unconditionally (~11 MB for
      26 layers at max_steering_configs=16) — no hasattr guards needed
    - Remove active_hook_points parameter from Gemma3DecoderLayer

    Co-authored-by: Claude

commit 45dbf1fef9bc0e0a3e3d5f2a2aef033703199cfc
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 16:14:47 2026 -0700

    refactor: remove backward-compat steering_vectors, require hook points

    steering_vectors is now always dict[str, dict[int, list[float]]]
    (hook_point → layer → vector). Removes the old flat format, the
    separate steering_hook_vectors field, and all merging logic.

    Co-authored-by: Claude

commit 3d4022dd3f3f795bc2ebe48c5cb618cfece31e59
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 15:05:39 2026 -0700

    docs: update steering docs for Phase 3 hook points

    Co-authored-by: Claude

commit cf969cb1bdf8e1e38cdfebdb6ed16e6304a73c9c
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 28 14:57:47 2026 -0700

    fix: restore hook-point-aware api_router and protocol after merge

    The infrastructure agent's PR overwrote the API agent's changes to
    api_router.py and protocol.py. This restores:
    - hook_vectors field on SetSteeringRequest
    - vectors default_factory=dict (no longer required since hook_vectors
      can supply vectors)
    - Hook-point-aware set_steering that wraps legacy vectors under
      post_mlp_pre_ln before passing to workers
    - Updated tests to match new API format

    Co-authored-by: Claude

commit 11b0da7f4de47509e7ff794f90bc41616478d986
Merge: 5fd0e3929 8fbe4ddb4
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 23:29:20 2026 -0700

    Merge pull request #15 from RhizoNymph/feat/steering-hook-points-manager-runner-worker

    feat: multi-hook-point support for SteeringManager, ModelRunner, WorkerBase

commit 8fbe4ddb4381681807fc6d02949f5a1387ec6b74
Merge: 1c8717e30 5fd0e3929
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 23:29:05 2026 -0700

    Merge branch 'feat/steering-hook-points' into feat/steering-hook-points-manager-runner-worker

    Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

commit 5fd0e39291aebc935402364a942d8cacf1b87cd8
Merge: c80df1880 ee599fed1
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 23:27:46 2026 -0700

    Merge pull request #16 from RhizoNymph/feat/steering-hook-points-api

    feat: add steering_hook_vectors API support

commit c80df1880ea3ff2960474885a37b11480c8baa1c
Merge: 54e3357d2 119f87a2e
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 23:25:17 2026 -0700

    Merge pull request #17 from RhizoNymph/feat/steering-gemma3-hook-points

    feat: update Gemma3 model for multiple steering hook points

commit ee599fed1209f693691f4edf1392dfa745f4f63a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 21:44:19 2026 -0700

    docs: update steering feature doc for hook-point API support

commit ffb372c34ae61ba81c1820951453f41a2b6455a0
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 21:43:20 2026 -0700

    feat: add steering_hook_vectors support to API layer

    Add steering_hook_vectors field across the API layer to support
    multiple hook points for activation steering vectors.

    Changes:
    - SamplingParams: new steering_hook_vectors field with validation
      that checks hook point names against VALID_HOOK_POINT_NAMES and
      reuses _validate_layer_vector_dict for inner dict validation
    - Request: new steering_hook_vectors field populated from
      sampling_params, steering_config_hash updated to build canonical
      representation merging both steering_vectors and
      steering_hook_vectors
    - ChatCompletionRequest/CompletionRequest: new steering_hook_vectors
      field passed through to SamplingParams.from_optional
    - SetSteeringRequest: new hook_vectors field, vectors field now
      defaults to empty dict for backward compatibility
    - api_router set_steering: builds hook-point-aware dict merging
      legacy vectors (as post_mlp_pre_ln) with explicit hook_vectors,
      validates hook point names, passes new format to collective_rpc

commit 1c8717e30ea7ebe65d726f384c05921b71bb7f98
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 21:37:33 2026 -0700

    feat: update SteeringManager, ModelRunner, WorkerBase for multi-hook-point steering

    Co-authored-by: Claude

commit 119f87a2e2f1894dcac27cb042ee9fa59a02d16b
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 21:29:40 2026 -0700

    feat: update Gemma3 model to support multiple steering hook points

    Co-authored-by: Claude

commit 54e3357d26d440f423fe2f2a818193572b849c73
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 21:21:16 2026 -0700

    feat: add SteeringHookPoint enum and --steering-hook-points CLI arg

    Co-authored-by: Claude

commit 03868735bd343e1943f02ba7e3ace08f9e718a1f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 20:11:52 2026 -0700

    docs: add per-request steering lessons learned to implementation notes

commit 2d8278f1f792ab8416b8affd70f69de280ba248a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 19:21:58 2026 -0700

    fix: steering hash overflow, add concurrent CUDA graph integration test

    - Mask steering_config_hash to fit np.int64 (was overflowing with
      full 64-bit unsigned SHA-256 hash)
    - Add test_per_request_steering_concurrent_with_cuda_graphs: sends
      3 requests (unsteered, +500, -500) in same batch, verifies per-request
      steering works with torch.compile + CUDA graphs active, then verifies
      unsteered determinism on a second batch

commit 6bae6a4ad51baebc1d7575e320241cdd2b5bfa25
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 18:40:38 2026 -0700

    docs: update steering docs for per-request implementation

commit 0265546ffca239b75ca32147f5dcb9bbb4063de5
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 18:34:12 2026 -0700

    docs: add OVERVIEW.md and steering feature documentation

commit 4c57a2d31b2c647491dff7f1339cd81ad1fe9adf
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 18:12:01 2026 -0700

    test: add scheduler admission, worker steering, and integration tests

commit 066839d90d3ce9c547c1fbfc9fdf9478ed4fc131
Merge: 72b7b0c68 142432b7c
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 17:58:51 2026 -0700

    Merge pull request #13 from RhizoNymph/feat/per-request-steering-runner

    feat: integrate SteeringManager into model runner and worker_base

commit 72b7b0c6899515ce25da7846e46281f24378ea3a
Merge: b2e770b7d 2e914cfc7
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 17:58:21 2026 -0700

    Merge pull request #11 from RhizoNymph/feat/per-request-steering-tests

    test: SteeringManager and indexed-gather op unit tests

commit 142432b7cd911ee029ae13a3b4d29c2e713f12b7
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 17:51:50 2026 -0700

    feat: integrate SteeringManager into model runner and worker_base

commit b2e770b7d38de65611808cb45ffd9c3d3c24e60f
Merge: 5d9c9f015 23198f1c0
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 17:43:22 2026 -0700

    Merge pull request #12 from RhizoNymph/feat/per-request-steering-manager

    feat: SteeringManager, InputBatch tracking, scheduler admission control

commit 23198f1c085115e3935edd8c4dfe6c8574c84fe7
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 17:24:10 2026 -0700

    feat: add SteeringManager, InputBatch tracking, and scheduler admission control

commit 2e914cfc75f20e5b90b1801e7953aff75da585ad
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 17:02:33 2026 -0700

    test: add SteeringManager and indexed-gather op unit tests

commit 5d9c9f015dd9d21c372594ac2ab237091f3e52ee
Merge: 9e535b083 2066a7e1c
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 16:47:55 2026 -0700

    Merge pull request #9 from RhizoNymph/feat/per-request-steering-config

    feat: add SteeringConfig, SamplingParams field, and API protocol support

commit 9e535b08395e3dd40e17f6bb754a24e93432af28
Merge: 602474f3c 10c95f5d4
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Fri Mar 27 16:46:03 2026 -0700

    Merge pull request #10 from RhizoNymph/feat/per-request-steering-op

    feat: update steering op to indexed gather with table/index buffers

commit 2066a7e1c90fcca42816bb15e00180b9283698bb
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 16:04:51 2026 -0700

    feat: add steering config, SamplingParams field, and API protocol support

commit 10c95f5d48ea45d213d2020717970467b7c93ba4
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 16:10:09 2026 -0700

    feat: update steering op to indexed gather, add table/index buffers to Gemma3

commit 602474f3c947718c34b57029b45d7717c61dd2f9
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 14:08:39 2026 -0700

    remove todo doc for mvp

commit 7619f8ae91273cf022892d1543a5bace29346784
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Fri Mar 27 14:08:28 2026 -0700

    fix: change injection point to raw residual stream not post layernorm

commit 98408699f80226539d908b3bb690af8f6e853d9e
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Mar 26 22:47:14 2026 -0700

    fix: decode-only steering with torch.compile and CUDA graphs

    Replace ForwardContext-based num_decode_tokens read (constant-folded by
    torch.compile) with persistent per-layer mask buffers updated in-place
    by the model runner before each forward pass.

    - Custom op takes 3 args (hidden, vec, mask) and is a splitting op
    - Each decoder layer registers a steering_decode_mask buffer
    - Model runner discovers steerable layers and updates masks from
      scheduler output (decode tokens = 1 scheduled token, ordered first)
    - Add num_decode_tokens to TritonAttentionMetadata and
      FlashAttentionMetadata
    - Increase test steering magnitude for dummy-weight models
    - Add prefix cache reset after clearing steering in E2E test

    Co-authored-by: Claude

commit c19a78587d29270c901c4fd66453a87eef049e8b
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Mar 26 13:36:55 2026 -0700

    docs/update temporary steering docs

commit d51bd507c693f10372676030b3b97985c47ff561
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Mar 26 00:41:13 2026 -0700

    fix: typed steering exceptions, remove GET lock, cache steerable layers

    Co-authored-by: Claude

commit 6c93d113d849d6cf474a6a8c90b617c37cab609a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Thu Mar 26 00:20:18 2026 -0700

    fix: cast steering vector dtype to match hidden states

    Co-authored-by: Claude

commit 531ad064402b875e1d8d11f3062e2c0165db61b2
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Mar 25 18:45:38 2026 -0700

    fix: register steering as custom op for torch.compile safety

    Co-authored-by: Claude

commit 5c4ec89f2fbb90e6aa4912b2d27c2684f01c609f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Mar 25 18:43:51 2026 -0700

    fix: return clear error for empty steering vectors request

    Co-authored-by: Claude

commit e76810451eb76e8dc5d63ada1964f9dc1f98c423
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 24 18:44:25 2026 -0700

    temporary steering docs

commit 1f2c5140c5b7fde10e7cffd4ba452207a115ba4f
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Mar 24 18:15:36 2026 -0700

    add steering unit tests, fix test bugs

    - Add 65 unit tests across 4 files covering: SetSteeringRequest
      protocol validation, get_num_decode_tokens forward context utility,
      WorkerBase steering methods (set/clear/status with mock model,
      validation errors, edge cases), and API router endpoints (mock
      engine, all HTTP status codes, replace flag, attach/detach).
    - Fix test_steering.py: use llm.llm.generate() to get RequestOutput
      and compare only completion token_ids, not the prompt+output
      concatenation that VllmRunner.generate() returns.
    - Fix test_api_router.py: reset module-level _steering_lock between
      tests to prevent cross-test lock state leakage.
    - Fix test_worker_steering.py: delete no-op assertion that parsed as
      `assert True` due to operator precedence.

commit 36a071e006e09890c479559880858c6dff174b97
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Mar 23 22:12:03 2026 -0700

    fix steering: unify API, remove redundant runner methods, fix test

    - Remove set_steering_vectors and clear_steering_vectors from
      GPUModelRunner — these had opposite semantics from the canonical
      WorkerBase versions (zeroed non-specified layers vs leaving them
      untouched) and were unreachable from the HTTP API path.
    - Move import math/torch to module level in worker_base.py.
    - Rewrite test to use string-based collective_rpc dispatching to
      WorkerBase methods instead of calling GPUModelRunner directly via
      lambda, making it compatible with the V2 model runner and
      exercising the same code path as the HTTP API.

commit 77becfe0c9726bb5e651ee48c917bf738544bf36
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Mar 23 22:11:52 2026 -0700

    fix steering: extract decode-token utility, clean up API router

    - Move _get_num_decode_tokens from Gemma3DecoderLayer to a shared
      get_num_decode_tokens() utility in forward_context.py so future
      steerable models can reuse it without reimplementing fragile
      attn_metadata navigation.
    - Collapse duplicated except (ValueError, RuntimeError) / except
      Exception blocks in the steering API router into a single handler.
    - Add replace field to SetSteeringRequest for atomic clear+set within
      one lock acquisition, eliminating the observable empty-steering
      window when swapping profiles via separate /clear + /set calls.
    - Acquire _steering_lock in GET /v1/steering to prevent reading
      partially-applied state during concurrent mutations.

commit 56cdb4bb97a493034caf0b9f4fe24bb84cc45d62
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Mon Mar 23 15:54:12 2026 -0700

    steering api

commit 339a3a1bdec305d4a74a8be7be3862d8c54d569a
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Sat Mar 21 19:57:18 2026 -0700

    decode only steering
RhizoNymph added a commit that referenced this pull request Apr 16, 2026
commit 0e462d5e5c2d3c57c91980ca9cba9b8ca811154e
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 15 02:16:45 2026 -0700

    fix: use getattr for steering args in api_server init_app_state

commit 85594fb7dba189f9a3b8f067f5ca6df54f06f83d
Merge: d912217f8 3c2f98ec1
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Wed Apr 15 00:26:53 2026 -0700

    Merge pull request #88 from RhizoNymph/feat/steering-more-plain-models

    test: steering fixtures + layer_idx discovery fix for new models

commit 3c2f98ec1eb8d793f38a4e3e49f7f5ae7638b508
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Wed Apr 15 00:12:04 2026 -0700

    test: add steering fixtures for new decoder families

commit d912217f8fe1c31fc2aeac617b13cbed7bb333cf
Merge: 28306b439 2ff317c9b
Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
Date:   Tue Apr 14 23:05:44 2026 -0700

    Merge pull request #84 from RhizoNymph/feat/steering-more-plain-models

    feat: wire steering hooks into seven more decoder models

commit 2ff317c9b41265d096677d2feb723bf29c8321ae
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 21:09:13 2026 -0700

    feat: wire steering hooks into seven more decoder models

commit 28306b4398df9d6210acc340f1c2aa7bc15f6bf1
Author: RhizoNymph <quantnymph@gmail.com>
Date:   Tue Apr 14 19:46:40 2026 -0700

    Squashed commit of the following:

    commit acf39ab1a8b7790be65ca3ceca623cd1b148569c
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 14 19:35:30 2026 -0700

        feat: gate global steering changes behind steering api key if set

    commit 58ab6e83a55f6092470ff0b97535d03b17e9e999
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 14 18:50:40 2026 -0700

        remove agent overview doc

    commit 3d77e12e1dc2685bbe7945878b0c60c006acbb68
    Merge: ada55957f 3b08fb305
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Apr 14 17:09:56 2026 -0700

        Merge pull request #83 from RhizoNymph/refactor/steering-mixin

        refactor(steering): extract SteeringModelRunnerMixin to match LoRA pattern

    commit ada55957f721d48848a5391efb2f4b0dd908e72d
    Merge: 9a00f0f08 0b325140b
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Apr 14 16:58:05 2026 -0700

        Merge pull request #81 from RhizoNymph/docs/steering-update

        docs(steering): refresh supported-scope and runtime-boundaries

    commit 9a00f0f083e25058f7c66108457a78e21c590ec5
    Merge: 5ffa1cd69 0eba71d29
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Apr 14 16:57:15 2026 -0700

        Merge pull request #82 from RhizoNymph/refactor/steering-modules-router-merge

        refactor(steering): consolidate modules router under serve/steering/

    commit 5ffa1cd6984fb48d04633f950e7808d31234d221
    Merge: 52f85006f 548684983
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Apr 14 16:49:55 2026 -0700

        Merge pull request #80 from RhizoNymph/chore/rename-steering-phase-tests

        chore(steering): rename phase1..phase5 test prefixes to semantic names

    commit 52f85006fd34715af8154237e703a85f50ef8d73
    Merge: 95ffa52f3 49649a584
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Apr 14 16:48:29 2026 -0700

        Merge pull request #79 from RhizoNymph/chore/steering-sampling-params-imports

        chore(steering): hoist sampling_params inline imports

    commit 3b08fb3059bb7042b434ec6f9ae40f136472f30f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 14 15:56:35 2026 -0700

        refactor(steering): extract SteeringModelRunnerMixin to match LoRA pattern

        Co-authored-by: Claude

    commit 0eba71d2901616c54b3cdf1108a87e687a1fd535
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 14 15:00:50 2026 -0700

        refactor(steering): fold serve/steering_modules/ into serve/steering/

        Co-authored-by: Claude

    commit 548684983d31bc2d192068326d8c83b8715cda63
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 14 14:06:57 2026 -0700

        chore(steering): rename phase1..phase5 test prefixes to semantic names

        Co-authored-by: Claude

    commit 0b325140bbe60f00bbeee26e964ecec9ef654a23
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 14 14:04:14 2026 -0700

        docs(steering): refresh supported-scope and runtime-boundaries sections

    commit 49649a58483c49f05f7b4b9dbba53c787105e93b
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 14 14:03:25 2026 -0700

        chore(steering): hoist sampling_params inline imports to module level

    commit 95ffa52f355f74adfc9dfb207f7460cf05a085e4
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 13 13:41:55 2026 -0700

        perf(steering): batch per-layer table writes via index_copy_

    commit 27a9fa4bc0226e98c66c6103c955dd62e63d191f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 13 13:38:14 2026 -0700

        perf(steering): skip populate when state is unchanged via dirty flag

    commit 6d727d4f64eedf2ed39639892876c9bbc11537a6
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 13 02:07:31 2026 -0700

        perf(steering): make hash_steering_config 14x faster and cache it

    commit 37a9446eea958eac8026963c441eb48f8abb02d9
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Apr 12 16:15:54 2026 -0700

        perf: short-circuit steering buffer updates when no state is active

    commit b871782c147af79d7af78dc484b13d5853b47153
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Apr 12 15:48:24 2026 -0700

        fix: don't create steering manager at all when steering not enabled

    commit d7db597e9b0aafec12e72c84889f1ddd3c1f3c49
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 9 01:04:34 2026 -0700

        chore: remove implementation docs

    commit 2e20d1b9bf897ad593aa075793317cd8431cf2f2
    Merge: 897aa5e3e 346383689
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Wed Apr 8 21:16:40 2026 -0700

        Merge pull request #77 from RhizoNymph/fix/remove-vestigial-vector-buffers

        fix: remove vestigial steering_vector_{hook} GPU buffers

    commit 897aa5e3ee85680be7ea0a221eda9ff33789f12f
    Merge: 85024d4ca 012dcf9f3
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Wed Apr 8 21:14:56 2026 -0700

        Merge pull request #75 from RhizoNymph/docs/remove-vector-buffer-refs

        docs: remove steering_vector_{hook} buffer references

    commit 85024d4ca4776645e8d0e2c2c88abfd2073c1782
    Merge: 9bcf5ce42 224c95539
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Wed Apr 8 21:14:14 2026 -0700

        Merge pull request #76 from RhizoNymph/fix/steering-transition-priority

        fix: prioritize prefill→decode transitions over new-request deferrals

    commit 34638368923a706bab519b7b0b0cd13d6276ca52
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 21:11:33 2026 -0700

        fix: remove vestigial steering_vector_{hook} GPU buffers from Phase 1

        Closes #74

        Co-authored-by: Claude

    commit 224c955390fff1f2098266018e6e59a95dafcd0c
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 21:09:55 2026 -0700

        fix: prioritize prefill→decode transitions over new-request deferrals

        Co-authored-by: Claude

    commit 012dcf9f3be502fda884fba20c9ce3d9feab7d7f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 21:06:54 2026 -0700

        docs: remove steering_vector_{hook} buffer references

        Part of #74 — the vector buffers are being removed from the code.

        Co-authored-by: Claude

    commit 9bcf5ce42b3e98e1b0beefe01aa2c2ffebba405a
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 19:28:48 2026 -0700

        phase 5 of new models

    commit a8cc4349303695d48dce7a09fdeebf722a180b3c
    Merge: 183425b41 ff3bf0039
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Wed Apr 8 17:24:56 2026 -0700

        Merge pull request #71 from RhizoNymph/feat/named-steering-vectors

        feat: named steering vector modules

    commit ff3bf00392efa6f575ce8a8b62510561d0ea2558
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 16:37:19 2026 -0700

        reject malformed json bodies instead of silently skipping them

    commit 173c88b4a440e63caa2d8ecbdcefc3f3bce8085c
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 16:10:29 2026 -0700

        validate named steering entries

    commit 183425b41a16442c123ef0a0171ea3aed0dfd098
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 15:56:13 2026 -0700

        feat: more steering models

    commit eba7fd616e02e8f2d79d1a56732fd1cc72c37266
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 8 15:44:50 2026 -0700

        fix: gate named vector registry on actually enabling steering

    commit b642ba6a42a12ceb4ca45046396f49714528da0b
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 23:41:37 2026 -0700

        feat: phase 1 of adding new models

    commit 3a43d1cc1299b5a621d9b476a39112ecd1daf207
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 23:40:59 2026 -0700

        add register steering buffers helper

    commit 15b49e31cbf7e7e072dfbde575c8a6bf6b75db16
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 23:36:15 2026 -0700

        feat: wire up named steering module registry, CLI args, and docs

        Co-authored-by: Claude

    commit e9bf1d7455a0a6f06231e04e75c6af6c51e60712
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 23:31:05 2026 -0700

        feat: add steering modules CLI args and HTTP API endpoints

        Co-authored-by: Claude

    commit cb3f0d418d2553afcc320a32d9e394dd2fc076fb
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 21:18:38 2026 -0700

        feat: add steering_name field and serving handler resolution

    commit 48935407f5cdd564e1ab5fc24c16392b2d72c38f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 21:19:15 2026 -0700

        feat: add steering module registry and merge_steering_specs helper

    commit 1e50302281f4f2c55481121617cb3f308bd8f055
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 16:00:19 2026 -0700

        Add steering docs

    commit e27a7cced6a0a8da91c66d6045d4bbe4b24baa6e
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Apr 7 15:43:30 2026 -0700

        fix: use httpx for async test and don't fail steering manager test
        without available cuda

    commit 9a1c7e0badc9dd96a875a43628f07e732ab3560c
    Merge: b1b5b93b1 5575ae564
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Apr 7 09:06:42 2026 -0700

        Merge pull request #70 from RhizoNymph/feat/prefill-steering

        feat: prefill/decode phase-aware steering vectors

    commit 5575ae56469a4ef6b4fd0e57e226b30e97e5efce
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 6 17:05:35 2026 -0700

        fix: APC streaming semantics

    commit b10dd23f57301f01796794c6869fb9968086777c
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 6 16:20:06 2026 -0700

        fix: keep steering prefix-cache keys consistent during deferred
        registration

    commit 2aa6ac4a0adc16896cc1f562019989f8fef441f0
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 6 16:08:05 2026 -0700

        chore: update docs

    commit f9c5245f18e876f8e22e25b1a27258ef7bfefb3f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 6 16:07:51 2026 -0700

        fix: remove post_mlp_post_ln hook point and rename post_mlp_pre_ln to
        post_mlp

    commit b72b74c0e501b6341fd823d8318d025d5393421f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Apr 6 16:05:53 2026 -0700

        fix: update prompt length before recomputing block hashes to fix
        prefix-cache key misses

    commit 03d11ebf5fceac2d9c2676e21d2907e421578cce
    Merge: 650aaee77 cc205f6ff
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Apr 5 18:51:00 2026 -0700

        Merge pull request #69 from RhizoNymph/fix/steering-streaming-hash-mismatch

        fix: remove redundant _reset call from streaming steering path

    commit cc205f6ff9dab721720aaf998e306f5f08a72116
    Merge: 1de87974e 650aaee77
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Apr 5 18:50:50 2026 -0700

        Merge branch 'feat/prefill-steering' into fix/steering-streaming-hash-mismatch

        Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

    commit 650aaee772d0d81ed4482b55b580533c1adf9159
    Merge: e89f20aeb 4b3bb798e
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Apr 5 18:50:12 2026 -0700

        Merge pull request #68 from RhizoNymph/fix/steering-gpu-resident-vectors

        fix: store per-request steering vectors on GPU to avoid CPU->GPU copy

    commit e89f20aeb20d69ea68bf3951284542e0c46d7405
    Merge: 806e39cad 96d73fba5
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Apr 5 18:50:01 2026 -0700

        Merge pull request #67 from RhizoNymph/fix/steering-reject-extra-keys

        fix: reject extra keys in scaled steering entries

    commit 1de87974e68cf455bea2ce9ab941b24040653d44
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Apr 5 18:37:37 2026 -0700

        fix: remove redundant _reset call from streaming steering path

        Co-authored-by: Claude

    commit 4b3bb798e8604de998cfef30a74955b8090c9f8e
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Apr 5 18:30:32 2026 -0700

        fix: store per-request steering vectors on GPU to avoid CPU->GPU copy each step

    commit 96d73fba5018d133ba239f2e23fd038f788b32a1
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Apr 5 18:30:24 2026 -0700

        fix: reject extra keys in scaled steering entries

    commit 806e39cad4cb5ad9ff768d2d173d95312ab78bbc
    Merge: 54b0ccb9b bed8e050f
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Apr 5 13:28:12 2026 -0700

        Merge pull request #65 from RhizoNymph/test/steering-retry-and-replay-coverage

        test: cover steering transition deferral and pending globals replay

    commit bed8e050fdb1b6bbe6c43f86b775c220ec247841
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Apr 5 13:21:38 2026 -0700

        test: cover steering transition deferral and pending globals replay

        Closes two test-coverage gaps identified in the post-merge audit of the
        steering feature:

        1. TestTransitionUnderCapacity exercises the bookkeeping contract of
           _handle_steering_transition when decode registration fails due to
           capacity. Verifies the deferred entry is appended with phase="decode",
           _req_steering_phase[req_id] is set to "decode" regardless, and
           get_row_for_config falls back to row 2 until the next retry succeeds.

        2. TestPendingGlobalsReplay verifies that phase-labeled entries captured
           into _pending_steering_globals before SteeringManager lazy init land
           in the matching global_{base,prefill,decode}_vectors dicts rather
           than collapsing into a single tier, including end-to-end via
           populate_steering_tables.

        Also drops two unused-variable / nested-if lint issues in
        TestPreemptionResumption that were blocking pre-commit on this file.

        Co-authored-by: Claude

    commit 54b0ccb9b1405c3feb388222388becc328fae091
    Merge: 735fc4d91 416c9b388
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Apr 4 17:26:36 2026 -0700

        Merge pull request #64 from RhizoNymph/fix/steering-preempt-resume-config-leak

        fix: reset steering phase on preemption-resumption to prevent refcount leak

    commit 416c9b38849fe2ed88b46a82345e32f7499d20b4
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Apr 4 16:33:53 2026 -0700

        fix: reset steering phase on preemption-resumption to prevent refcount leak

        When a request with per-request steering is preempted (e.g. via
        reset_prefix_cache() which fires on every global steering update) and
        later resumed, num_computed_tokens is reset to 0 but the model runner's
        _req_steering_phase and the SteeringManager's registered configs are
        NOT cleaned up. On resumption the request re-prefills using the global
        prefill row (not its per-request row), and the prefill->decode
        transition double-registers the decode config -> permanent row leak
        on finish.

        Add _reset_steering_for_resumption helper that releases the stale
        decode config, drops any stale deferred entries, and re-registers the
        prefill config. Call it from both the v1 cached-reqs resumption path
        (guarded by resumed_from_preemption) and the v2 _update_streaming_request
        path.

        Co-authored-by: Claude

    commit 735fc4d9183af796a556ca31937b9a3b7d162fef
    Merge: 7d566afc7 cde991759
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Apr 4 14:42:58 2026 -0700

        Merge pull request #62 from RhizoNymph/fix/steering-api-empty-hook

        fix: drop empty hook entries in steering _normalize_spec

    commit 7d566afc7bb163838da34957ddd8786c5766190b
    Merge: 6c56b5aa7 8ed116494
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Apr 4 14:32:36 2026 -0700

        Merge pull request #61 from RhizoNymph/fix/steering-manager-phase-validation

        fix: validate phase in populate_steering_tables

    commit 6c56b5aa7dfaab2834aae67d0cc8d284af831630
    Merge: b8f783c89 729f196f7
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Apr 4 14:31:59 2026 -0700

        Merge pull request #60 from RhizoNymph/fix/steering-prefill-decode-cross-validation

        fix: validate prefill/decode steering dimensions without base

    commit cde991759948516e5afe1be2b4dd9664a19619a8
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Apr 4 13:37:59 2026 -0700

        fix: drop empty hook entries in steering _normalize_spec

        Co-authored-by: Claude

    commit 8ed116494e6c30dd9696e64ef8560af8dd8696b2
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Apr 4 13:36:57 2026 -0700

        fix: validate phase in populate_steering_tables

        Co-authored-by: Claude

    commit 729f196f757e8fe88e979ae07cc211e0fdcef8dd
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Apr 4 13:35:05 2026 -0700

        fix: validate overlapping prefill/decode steering dimensions without base

        Co-authored-by: Claude <noreply@anthropic.com>

    commit b8f783c898a9ab2b8c66807be72da8b071b8cd23
    Merge: d4e78d422 546d8e14c
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Apr 3 23:23:38 2026 -0700

        Merge pull request #58 from RhizoNymph/fix/steering-hash-invalidation

        fix: invalidate cached steering hashes on streaming sampling_params update

    commit d4e78d4226c5354f61d927d23c2440221ce0adc9
    Merge: 2fddb12f4 9377c7141
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Apr 3 22:38:04 2026 -0700

        Merge pull request #57 from RhizoNymph/fix/steering-reject-when-disabled

        fix: reject per-request steering when enable_steering is off

    commit 546d8e14c92a543168a9dda37e6858a28a416a70
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Apr 3 21:34:06 2026 -0700

        fix: invalidate cached steering hashes on streaming sampling_params update

        Co-authored-by: Claude

    commit 9377c7141c95b51cf8eb92bb3d0d2633674e6eec
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Apr 3 21:32:23 2026 -0700

        fix: reject per-request steering when enable_steering is off

        Co-authored-by: Claude

    commit 2fddb12f4e86f813bdeba582973524d48fa77a76
    Merge: a681974aa 5959d8fe3
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Apr 3 16:52:06 2026 -0700

        Merge pull request #56 from RhizoNymph/fix/steering-streaming-hashes

        fix: refresh steering hashes when updating streaming requests

    commit a681974aaae738cafe54b883166b5fc063c59724
    Merge: 4dc95f529 3526be8be
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Apr 3 16:51:29 2026 -0700

        Merge pull request #55 from RhizoNymph/fix/steering-forbid-extra

        fix: reject unknown fields in SetSteeringRequest

    commit 5959d8fe325beb50a7294e123f4744f74b08ed0d
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Apr 3 15:40:08 2026 -0700

        fix: refresh steering hashes when updating streaming requests

        Co-authored-by: Claude

    commit 3526be8be85624eb4506000ac540e9c958ba9c26
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Apr 3 13:19:23 2026 -0700

        fix: reject unknown fields in SetSteeringRequest to catch legacy scales

        Co-authored-by: Claude

    commit 4dc95f52958069c4c431b7c0ebf1308d6b5ff1fe
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Apr 3 01:05:58 2026 -0700

        fix: fail on stale cache after steering change and relax decode-only admission

        Co-authored-by: Claude

    commit e2ef338ed7bfe52bcabd343f2cdcd82f7faca9c7
    Merge: da2fec961 6288714c5
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 23:25:29 2026 -0700

        Merge pull request #54 from RhizoNymph/fix/steering-atomic-replace

        fix: make steering replace=true atomic via single worker RPC

    commit da2fec96126507d48e3d5fa3b588697f6238e8de
    Merge: ee488edac bb24d9947
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 23:22:06 2026 -0700

        Merge pull request #53 from RhizoNymph/fix/steering-openai-plumbing

        fix: expose full steering API in OpenAI protocol models

    commit 6288714c5d3ff3da62d0736a766b4974d501b1f3
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 23:07:33 2026 -0700

        fix: make steering replace=true atomic via single worker RPC

        Co-authored-by: Claude

    commit bb24d99473b12a367a962f25e5c77de36a2b5d7d
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 23:05:25 2026 -0700

        fix: expose full steering API surface in OpenAI protocol models

        Co-authored-by: Claude

    commit ee488edacddfb8da257320656769d978a0db92d2
    Merge: 56f66ad7b 442e9eba8
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 18:34:01 2026 -0700

        Merge pull request #51 from RhizoNymph/fix/steering-request-attrs

        fix: remove unused steering vector attributes from Request

    commit 56f66ad7b7e31fafad1b5b4e0c9e04bfc51debf9
    Merge: 46e522e0f 9b2997d1c
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 18:29:20 2026 -0700

        Merge pull request #49 from RhizoNymph/fix/steering-scheduler-capacity

        fix: add decode-only capacity check and transition prediction for WAITING requests

    commit 46e522e0f5fbd007b8c1b1dff6029f37481f341a
    Merge: ece7238fb 7623fbe3f
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 18:26:51 2026 -0700

        Merge pull request #48 from RhizoNymph/fix/steering-model-runner-guards

        fix: add error handling for steering registration and fix backfill refcounting

    commit ece7238fb087b3d498868085fa8841c47ef18292
    Merge: 9ae9bcd37 6e6deadb1
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 18:25:48 2026 -0700

        Merge pull request #47 from RhizoNymph/fix/steering-device-mismatch

        fix: resolve GPU+CPU device mismatch in steering table population

    commit 9ae9bcd37ac590c8f830af39bcc316fe55fe135a
    Merge: 5d19f7f10 e6a9ffaf1
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 18:24:01 2026 -0700

        Merge pull request #50 from RhizoNymph/fix/steering-replace-clear

        fix: honor replace=True in set_steering_vectors when no new vectors provided

    commit 5d19f7f10f2d53aad4759bfbfd24e5a0639a6a4d
    Merge: 000d00f48 dd79f0844
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Thu Apr 2 18:22:54 2026 -0700

        Merge pull request #52 from RhizoNymph/fix/steering-cache-key-phase

        fix: disambiguate boundary block cache keys for prefill vs decode steering

    commit 442e9eba84ba37a7e0439e43dd89509915326a17
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 00:31:03 2026 -0700

        fix: remove unused steering vector attributes from Request to prevent memory waste

        Co-authored-by: Claude

    commit e6a9ffaf1031efd4b99fb8c7eac3f852f604a87b
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 00:29:30 2026 -0700

        fix: honor replace=True in set_steering_vectors when no new vectors provided

        Co-authored-by: Claude

    commit dd79f084480a4092c5493303bb7ebf2940ab55e5
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 00:29:00 2026 -0700

        fix: disambiguate boundary block cache keys for prefill vs decode steering

        Co-authored-by: Claude

    commit 9b2997d1cb33475567c791c3a03624972e0806c5
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 00:27:06 2026 -0700

        fix: add decode-only capacity check and transition prediction for WAITING requests

        Co-authored-by: Claude

    commit 7623fbe3fb789b1776e1f0c4d26e70ed143de71a
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 00:27:05 2026 -0700

        fix: add error handling for steering registration and fix backfill refcounting

        Co-authored-by: Claude

    commit 6e6deadb1e9fd56a13fd72ed8adb22acb1979317
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Apr 2 00:27:04 2026 -0700

        fix: resolve GPU+CPU device mismatch in steering table population

        Co-authored-by: Claude

    commit 000d00f48bbba901b7d4fa682c75eb6439be9db4
    Merge: dfed3d2bf f9130cf07
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Wed Apr 1 18:34:56 2026 -0700

        Merge pull request #45 from RhizoNymph/fix/steering-lazy-init-capacity

        fix: gracefully handle capacity exhaustion during steering lazy init

    commit f9130cf07d50a9ff0c02d89e4e5f22fb4c373d53
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 1 16:24:56 2026 -0700

        fix: defer steering registrations on capacity exhaustion instead of dropping

        Replace the silent fallback-to-global approach with a proper deferral
        mechanism.  Failed registrations during lazy init and prefill-to-decode
        transitions are now appended to a unified
        `_pending_steering_registrations` list that is retried with priority
        before new admissions on every step.

        The retry loop also filters stale entries:
        - Entries for finished requests are dropped (prevents row leaks).
        - Entries whose request changed phase are dropped (e.g. prefill
          deferred but request already transitioned to decode).

        This subsumes the decode-only `_pending_decode_registrations` list
        and the row-leak fix from PR #46.

    commit 251cabe638c7b33d2dcbf27e1da9754dd2e63369
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 1 01:16:28 2026 -0700

        fix: gracefully handle capacity exhaustion during steering lazy init

    commit dfed3d2bfd1557d7c8c3925dd5de8224c1e9e91d
    Merge: 51672b8f1 41135ae57
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Wed Apr 1 12:53:25 2026 -0700

        Merge pull request #44 from RhizoNymph/fix/steering-swap-states-hash

        fix: swap steering hashes in InputBatch.swap_states()

    commit 51672b8f1ef164970470580e380534f85ec3b35a
    Merge: 47365775f f2638d4b7
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Wed Apr 1 12:36:32 2026 -0700

        Merge pull request #43 from RhizoNymph/fix/steering-dimension-cross-validation

        fix: validate matching dimensions between base and phase steering vectors

    commit 41135ae5709c78d4639a0b0148967b491a38894e
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 1 01:16:27 2026 -0700

        fix: swap steering hashes in InputBatch.swap_states()

        Co-authored-by: Claude

    commit f2638d4b7f2f6a9136c11362f08b89b9873d42f3
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Apr 1 01:14:35 2026 -0700

        fix: validate matching dimensions between base and phase steering vectors

    commit 47365775f6c7310d6381fb5495777950fd729997
    Merge: 981ad5ac8 ebeff444f
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 23:59:48 2026 -0700

        Merge pull request #42 from RhizoNymph/docs/steering-bugfix-round2

        docs: update steering docs for deferred cleanup and pre-init status

    commit 981ad5ac85d2c93c3ae1f356f148ea097350481d
    Merge: 7431847f1 f06f5db30
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 23:59:35 2026 -0700

        Merge pull request #41 from RhizoNymph/fix/steering-status-pending-globals

        fix: report pending phase-specific globals in steering status

    commit 7431847f1cd2647ba96829395b2037f2317f520f
    Merge: f04aaf36d e1a6a2ef5
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 23:59:12 2026 -0700

        Merge pull request #40 from RhizoNymph/fix/steering-deferred-decode-leak

        fix: track req_id in deferred decode registrations to prevent leak

    commit f04aaf36d143c040b2b652403de3f0449a5d3d62
    Merge: 7e41715be e9d8f110c
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 23:58:37 2026 -0700

        Merge pull request #39 from RhizoNymph/fix/steering-boundary-block-hash

        fix: include decode hash in boundary block cache keys

    commit ebeff444f4bd59751824305843481a4e92178ece
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 23:34:00 2026 -0700

        docs: update steering docs for deferred cleanup and pre-init status

        Co-authored-by: Claude

    commit e9d8f110c39446cd8c4e9bcbaddc91c82236962f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 23:29:38 2026 -0700

        fix: include decode hash in boundary block cache keys

        Co-authored-by: Claude

    commit f06f5db3000d6fb29d21219538cc4d9c43030e04
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 23:27:52 2026 -0700

        fix: report pending phase-specific globals in steering status

    commit e1a6a2ef5da949d4636d98779cb146b803cb376b
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 23:26:48 2026 -0700

        fix: track req_id in deferred decode registrations to prevent leak

    commit 7e41715be28060d2ebb7dd97bff0aef72cf04b27
    Merge: 5edfb74f2 0c7e77579
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 19:44:19 2026 -0700

        Merge pull request #38 from RhizoNymph/docs/steering-bugfix-updates

        docs: update steering docs for bugfix PRs #35-#37

    commit 5edfb74f2d3657093e316c5fe673cf58e6a1ce99
    Merge: a7bdd3fe1 2f0a0c960
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 19:44:01 2026 -0700

        Merge pull request #36 from RhizoNymph/fix/steering-scheduler-transition-capacity

        fix: transition-aware steering capacity and decode-start admission

    commit a7bdd3fe1ad5f2a3c4e25a67d4da9b1617a34992
    Merge: 2830adbf0 b2d8d475c
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 19:08:27 2026 -0700

        Merge pull request #35 from RhizoNymph/fix/steering-worker-status-validation

        fix: remove double validation and add phase-specific norms to steering status

    commit 2830adbf014067736c04afdc1ddfdb11c898009c
    Merge: c01e6dce4 5cb702a2a
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 19:01:21 2026 -0700

        Merge pull request #37 from RhizoNymph/fix/steering-model-runner-transition

        fix: graceful steering transition + phase-tracked release

    commit 0c7e77579e3cab4e5cdc6987e43a376481bb647d
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 18:58:04 2026 -0700

        docs: update steering docs for transition-aware capacity, deferred registration, and phase-specific norms

    commit 5cb702a2aa253defaa567fbda72e8c2e26650267
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 18:30:11 2026 -0700

        fix: graceful steering transition with deferred registration and phase tracking

        Co-authored-by: Claude

    commit b2d8d475c0010910fd2b680d9460dc9caa715708
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 17:59:07 2026 -0700

        fix: remove double validation and add phase-specific norms to steering status

        Co-authored-by: Claude

    commit 2f0a0c96055e7330687ad1e101059eb3fa311040
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 17:58:31 2026 -0700

        fix: transition-aware steering capacity counting and decode-start admission check

        Co-authored-by: Claude

    commit c01e6dce4032a00a962d1b257b3f907e327f5b54
    Merge: 5ab47798d 84eeb8b21
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 01:05:21 2026 -0700

        Merge pull request #34 from RhizoNymph/fix/steering-decode-cache-block-hash

        fix: include decode steering hash in generated cache block keys

    commit 5ab47798d2d6a6778e61610ddde0fcaad2f80240
    Merge: c3bda6dc5 2b8a2a6ec
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 01:04:14 2026 -0700

        Merge pull request #32 from RhizoNymph/fix/steering-scheduler-single-phase-capacity

        fix: scheduler steering capacity counts only active phase

    commit c3bda6dc5f8da7e9dcae49faba5c3b69aed6a67a
    Merge: dd9e5b185 35c55f9eb
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Tue Mar 31 01:02:59 2026 -0700

        Merge pull request #33 from RhizoNymph/fix/steering-lazy-init-decode-registration

        fix: register decode steering configs during lazy manager init

    commit 84eeb8b21d7435d91655d6a19aa7b33f7de16800
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 31 00:01:30 2026 -0700

        fix: include decode steering hash in generated cache block keys

    commit 35c55f9eb69bca17dd3f0c80830b419921bef073
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Mar 30 23:42:06 2026 -0700

        fix: register decode steering configs during lazy manager init

    commit 2b8a2a6eccde4b10f8565bbc62e13c8024a85327
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Mar 30 23:41:46 2026 -0700

        fix: scheduler steering capacity counts only active phase

    commit dd9e5b1852d053f31535ec75c72d973095b07424
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Mar 30 23:09:17 2026 -0700

        docs: update steering docs for PR #29, #30, #31 post-merge fixes

    commit 63089a37e19e877625de97ad69e62dbf8f0f4b42
    Merge: dbe8d1880 dcdf60110
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Mon Mar 30 23:00:16 2026 -0700

        Merge pull request #31 from RhizoNymph/fix/steering-buffer-overwrite

        fix: stop writing phase-specific globals to shared steering buffers

    commit dbe8d18809e04fcfd7f959d90a4f7d2eed95adac
    Merge: 5100a10d4 bd69fae0b
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Mon Mar 30 22:59:26 2026 -0700

        Merge pull request #30 from RhizoNymph/fix/steering-decode-start-registration

        fix: register decode steering config when request starts past prefill

    commit 5100a10d4a0d95229c3b6587484b1a122bc08ae2
    Merge: 8d1c1fe47 3de429a5a
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Mon Mar 30 22:57:48 2026 -0700

        Merge pull request #29 from RhizoNymph/fix/steering-malformed-entry-validation

        fix: reject malformed scaled steering entries with 4xx

    commit dcdf60110fd6e89e8dd9a3128a8b3a71049de610
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 15:29:56 2026 -0700

        fix: stop writing phase-specific globals to shared steering buffers

    commit bd69fae0b51f0c2e75f323955d62265b4623db9e
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 15:13:54 2026 -0700

        fix: register decode steering config when request starts past prefill

    commit 3de429a5a84be3952759555b49ca01f22b05e37f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 14:40:26 2026 -0700

        fix: reject malformed scaled steering entries with 4xx instead of 500

    commit 8d1c1fe47dac02e3b3f07ef324967a969d66790d
    Merge: 9d8d7a4e9 4f7f350dd
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Mar 29 14:13:33 2026 -0700

        Merge pull request #27 from RhizoNymph/fix/steering-clear-pending-globals

        fix: clear pending steering globals on clear_steering_vectors

    commit 9d8d7a4e99566fb6c445ef46de477110d0bf5f0c
    Merge: c57ff54da d6efa42ce
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Mar 29 14:11:58 2026 -0700

        Merge pull request #28 from RhizoNymph/fix/steering-scheduler-phase-capacity

        fix: count steering capacity by (hash, phase) pairs to match worker row allocation

    commit 4f7f350dd603817a7a0bbda59431220197550988
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 12:08:15 2026 -0700

        fix: clear pending steering globals on clear_steering_vectors to prevent stale replay

    commit d6efa42ce8806af8846600a0ec6d39dfc5609952
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 12:07:57 2026 -0700

        fix: count steering capacity by (hash, phase) pairs to match worker row allocation

    commit c57ff54daf3f93216b8d609b9719ed38e918339e
    Merge: 849fa0577 2f3b14e09
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Mar 29 11:27:02 2026 -0700

        Merge pull request #25 from RhizoNymph/fix/steering-pending-globals-queue

        fix: preserve phase-specific global vectors set before steering manager init

    commit 2f3b14e094f4f85932b7650819b3cb0e35716583
    Merge: 55a9da811 849fa0577
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Mar 29 11:26:51 2026 -0700

        Merge branch 'feat/prefill-steering' into fix/steering-pending-globals-queue

        Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

    commit 849fa0577a9ca55e201830e59d351140414c21c6
    Merge: 018a7adcc 229ce88f3
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Mar 29 11:24:05 2026 -0700

        Merge pull request #24 from RhizoNymph/fix/steering-scheduler-phase-detection

        fix: use num_prompt_tokens for steering phase detection in scheduler

    commit 018a7adcc417fa3cd8647954752fbade1cdbb31c
    Merge: 007de86fc 91b89b480
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sun Mar 29 11:16:11 2026 -0700

        Merge pull request #26 from RhizoNymph/fix/steering-phase-keyed-rows

        fix: key steering table rows by (config_hash, phase) to prevent cross-phase aliasing

    commit 91b89b4805a7a5a3464cd2d605211f4d8e2c1189
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 11:11:10 2026 -0700

        fix: key steering table rows by (config_hash, phase) to prevent cross-phase aliasing

    commit 55a9da811b0ae47e46816eb8588bf234d8130835
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 00:38:21 2026 -0700

        fix: queue phase-specific global vectors for replay on steering manager lazy init

    commit 229ce88f3392682b44cc69f7b5ca12e827acd14d
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 00:35:41 2026 -0700

        fix: use num_prompt_tokens for steering phase detection in scheduler

    commit 007de86fc362ed6d86dc4bbebbf5e5a41581851e
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sun Mar 29 00:13:51 2026 -0700

        docs: update steering docs for prefill steering (Phase 4)

        - ROADMAP.md: Mark Phase 4 as DONE with full implementation summary
        - STEERING.md: Update to three-tier model, phase-aware table layout,
          remove stale "decode-only" language, update key files table
        - MVP_IMPLEMENTATION.md: Add Phase 4 lessons learned (prefix cache,
          lazy init race condition, phase detection, additive composition,
          co-located scales, scheduler dual-hash admission, test subtleties)
        - API.md: Update to three-tier format, co-located scales, remove
          old scales field, update examples

    commit 8f055b92c5f82d1283a2b186619bcb2b128cf22a
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 23:59:30 2026 -0700

        fix: register steering configs on first-step manager init + fix additive test

        The SteeringManager is lazily created in _update_steering_buffers, but
        config registration happens in _update_states which runs before. On the
        first request, registration was silently skipped because the manager
        didn't exist yet. Fix: after creating the manager, scan the batch and
        register any pending configs.

        Also fix test_additive_composition which incorrectly assumed base(250) +
        decode(250) equals base(500) — steering_vectors affects both phases, so
        prefill effective vectors differed between the two approaches.

    commit f15d55b8d0748a38b4e5786c09afa6430d9d28e7
    Merge: 9c1dca9f5 2917406a5
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 23:21:56 2026 -0700

        Merge pull request #23 from RhizoNymph/feat/prefill-steering-tests

        test: add prefill steering integration tests

    commit 2917406a5d1e53de9fbf4db4fbbc1adde087777a
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 22:59:26 2026 -0700

        test: add prefill steering integration tests and fix existing test API calls

        Co-authored-by: Claude

    commit 9c1dca9f547e52defa4782326a2b6777e7940f8f
    Merge: 2be915751 45100595f
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 21:43:26 2026 -0700

        Merge pull request #21 from RhizoNymph/feat/prefill-steering-manager

        feat: phase-aware SteeringManager and GPU model runner

    commit 45100595fc03a46d5407edab6240971e39b3d2ed
    Merge: 1beb47cc2 2be915751
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 21:35:19 2026 -0700

        Merge branch 'feat/prefill-steering' into feat/prefill-steering-manager

        Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

    commit 2be91575101afb56f9952093c4c40b5f5ce00174
    Merge: a286bd9d6 ec9e660b3
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 21:34:15 2026 -0700

        Merge pull request #22 from RhizoNymph/feat/prefill-steering-scheduler

        feat: prefill steering scheduler, API, and worker (WS4)

    commit ec9e660b307566bba43e5230cd2321e437010510
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 21:33:10 2026 -0700

        feat: wire prefix cache invalidation for prefill-affecting steering changes

        Replace the TODO with actual calls to engine.reset_prefix_cache() when:
        - Global base or prefill vectors are set (affects prefill KV cache)
        - replace=True is used (clears all tiers including prefill)
        - All steering is cleared via /v1/steering/clear

    commit a286bd9d660ad473232f9b60a0f04706494375a5
    Merge: 67b6d9d65 89e056fb6
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 21:25:44 2026 -0700

        Merge pull request #19 from RhizoNymph/feat/prefill-steering-cache-keys

        feat: prefill steering prefix cache key integration

    commit 89e056fb6bf5884135ef2a26c7e935da5c988b85
    Merge: 68cce443a 67b6d9d65
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 21:22:09 2026 -0700

        Merge branch 'feat/prefill-steering' into feat/prefill-steering-cache-keys

        Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

    commit 67b6d9d653da3c84fae431aad0a14e06b77ce85e
    Merge: b1b5b93b1 46665f131
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 21:18:16 2026 -0700

        Merge pull request #20 from RhizoNymph/feat/prefill-steering-types

        feat: three-tier steering types with prefill/decode separation

    commit b7d55a044a2fde9ae18c587711d8ddac6b03b9a6
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 21:06:52 2026 -0700

        feat: implement prefill steering scheduler, API, and worker changes

        - Scheduler: dual-hash admission control tracks union of active hashes
          (prefill hash for prefill-phase requests, decode hash for decode-phase).
          New requests must have capacity for both their prefill and decode hashes.
        - API router: three-tier global steering (vectors, prefill_vectors,
          decode_vectors) with co-located scale normalization. Removed old
          scales field. Uses kwargs-based collective_rpc for worker calls.
        - Worker: three-tier set_steering_vectors with base/prefill/decode tiers.
          Extracted validation, buffer application, and manager notification into
          separate helper methods. Supports replace=True across all tiers.
        - Tests: comprehensive dual-hash scheduler admission tests, three-tier
          API router tests with co-located scale format, and three-tier worker
          tests including replace and cross-tier validation.
        - Docs: updated steering.md with dual-hash admission details and
          three-tier HTTP API examples.

        Co-authored-by: Claude

    commit 1beb47cc2be30910c88e5b2aeba266a6e6c65036
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 21:01:12 2026 -0700

        feat: phase-aware SteeringManager and GPU model runner integration

        Co-authored-by: Claude

    commit 46665f131579bb24b50b0786d2424e9cd28a45fe
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 20:41:31 2026 -0700

        feat: add three-tier steering types with prefill/decode phase separation

        Co-authored-by: Claude

    commit 68cce443a84085f7f591e00e9967d1a62fed5f93
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 20:38:47 2026 -0700

        feat: add prefill steering config hash to prefix cache block keys

        Add _gen_steering_extra_hash_keys() to include per-request prefill
        steering config hashes in block hash extra keys. This ensures that
        blocks computed under different prefill steering produce different
        cache entries, with zero impact when steering is not used.

        Co-authored-by: Claude

    commit b1b5b93b15ad8bc2bcea10a786b9270398c66b3e
    Merge: 03868735b 21c6acbb4
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Sat Mar 28 18:41:07 2026 -0700

        Merge pull request #18 from RhizoNymph/feat/steering-hook-points

        feat: Phase 3 — multiple steering intervention points

    commit 21c6acbb4b55f6cb5244fb67522514660ca2abfa
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 18:36:12 2026 -0700

        docs: add Phase 3 lessons learned to implementation notes

        Key findings:
        - Splitting ops are not required for buffer correctness — custom op
          opacity is sufficient, graph partitioning is a separate concern
        - Inline tensor math triggers AOT autograd pickle failures due to
          Enum WeakValueDictionary in traced graph metadata
        - Always-allocate beats opt-in when memory is trivial vs model size
        - Hook points at same residual state exist for SAE position matching
        - Breaking backward compat cleanly beats merging logic

        Co-authored-by: Claude

    commit 15f34a6d006cd509fe10297e007a9a658697d5c6
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 18:33:51 2026 -0700

        docs: update steering docs for simplified hook point design

        - Remove references to --steering-hook-points CLI flag
        - Remove references to steering_hook_vectors (legacy separate field)
        - Document that all 4 hook points are always allocated unconditionally
        - Document that apply_steering is NOT a splitting op (zero graph partitions)
        - Update all API examples to use the hook-point-aware steering_vectors format

        Co-authored-by: Claude

    commit 6837fabf7a76a86eebdb0546a7c03b52ec1e2204
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 18:26:07 2026 -0700

        chore: remove --steering-hook-points CLI flag and config field

        All 4 hook point buffers are always allocated unconditionally.
        The memory cost is trivial (~11 MB at max_configs=16 for 26 layers)
        and there are no graph partitions, so there is no reason to control
        which hook points are active.

        Co-authored-by: Claude

    commit 5369eead8d2575202add671ba975ccffbc9b1be3
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 17:49:26 2026 -0700

        feat: remove graph partitions, always allocate all hook point buffers

        - Remove apply_steering from splitting_ops — the custom op is still
          opaque to the tracer (preventing constant-folding) but no longer
          partitions the compiled graph
        - Always allocate all 4 hook point buffers unconditionally (~11 MB for
          26 layers at max_steering_configs=16) — no hasattr guards needed
        - Remove active_hook_points parameter from Gemma3DecoderLayer

        Co-authored-by: Claude

    commit 45dbf1fef9bc0e0a3e3d5f2a2aef033703199cfc
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 16:14:47 2026 -0700

        refactor: remove backward-compat steering_vectors, require hook points

        steering_vectors is now always dict[str, dict[int, list[float]]]
        (hook_point → layer → vector). Removes the old flat format, the
        separate steering_hook_vectors field, and all merging logic.

        Co-authored-by: Claude

    commit 3d4022dd3f3f795bc2ebe48c5cb618cfece31e59
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 15:05:39 2026 -0700

        docs: update steering docs for Phase 3 hook points

        Co-authored-by: Claude

    commit cf969cb1bdf8e1e38cdfebdb6ed16e6304a73c9c
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Sat Mar 28 14:57:47 2026 -0700

        fix: restore hook-point-aware api_router and protocol after merge

        The infrastructure agent's PR overwrote the API agent's changes to
        api_router.py and protocol.py. This restores:
        - hook_vectors field on SetSteeringRequest
        - vectors default_factory=dict (no longer required since hook_vectors
          can supply vectors)
        - Hook-point-aware set_steering that wraps legacy vectors under
          post_mlp_pre_ln before passing to workers
        - Updated tests to match new API format

        Co-authored-by: Claude

    commit 11b0da7f4de47509e7ff794f90bc41616478d986
    Merge: 5fd0e3929 8fbe4ddb4
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 23:29:20 2026 -0700

        Merge pull request #15 from RhizoNymph/feat/steering-hook-points-manager-runner-worker

        feat: multi-hook-point support for SteeringManager, ModelRunner, WorkerBase

    commit 8fbe4ddb4381681807fc6d02949f5a1387ec6b74
    Merge: 1c8717e30 5fd0e3929
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 23:29:05 2026 -0700

        Merge branch 'feat/steering-hook-points' into feat/steering-hook-points-manager-runner-worker

        Signed-off-by: Nymph <82485126+RhizoNymph@users.noreply.github.com>

    commit 5fd0e39291aebc935402364a942d8cacf1b87cd8
    Merge: c80df1880 ee599fed1
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 23:27:46 2026 -0700

        Merge pull request #16 from RhizoNymph/feat/steering-hook-points-api

        feat: add steering_hook_vectors API support

    commit c80df1880ea3ff2960474885a37b11480c8baa1c
    Merge: 54e3357d2 119f87a2e
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 23:25:17 2026 -0700

        Merge pull request #17 from RhizoNymph/feat/steering-gemma3-hook-points

        feat: update Gemma3 model for multiple steering hook points

    commit ee599fed1209f693691f4edf1392dfa745f4f63a
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 21:44:19 2026 -0700

        docs: update steering feature doc for hook-point API support

    commit ffb372c34ae61ba81c1820951453f41a2b6455a0
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 21:43:20 2026 -0700

        feat: add steering_hook_vectors support to API layer

        Add steering_hook_vectors field across the API layer to support
        multiple hook points for activation steering vectors.

        Changes:
        - SamplingParams: new steering_hook_vectors field with validation
          that checks hook point names against VALID_HOOK_POINT_NAMES and
          reuses _validate_layer_vector_dict for inner dict validation
        - Request: new steering_hook_vectors field populated from
          sampling_params, steering_config_hash updated to build canonical
          representation merging both steering_vectors and
          steering_hook_vectors
        - ChatCompletionRequest/CompletionRequest: new steering_hook_vectors
          field passed through to SamplingParams.from_optional
        - SetSteeringRequest: new hook_vectors field, vectors field now
          defaults to empty dict for backward compatibility
        - api_router set_steering: builds hook-point-aware dict merging
          legacy vectors (as post_mlp_pre_ln) with explicit hook_vectors,
          validates hook point names, passes new format to collective_rpc

    commit 1c8717e30ea7ebe65d726f384c05921b71bb7f98
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 21:37:33 2026 -0700

        feat: update SteeringManager, ModelRunner, WorkerBase for multi-hook-point steering

        Co-authored-by: Claude

    commit 119f87a2e2f1894dcac27cb042ee9fa59a02d16b
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 21:29:40 2026 -0700

        feat: update Gemma3 model to support multiple steering hook points

        Co-authored-by: Claude

    commit 54e3357d26d440f423fe2f2a818193572b849c73
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 21:21:16 2026 -0700

        feat: add SteeringHookPoint enum and --steering-hook-points CLI arg

        Co-authored-by: Claude

    commit 03868735bd343e1943f02ba7e3ace08f9e718a1f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 20:11:52 2026 -0700

        docs: add per-request steering lessons learned to implementation notes

    commit 2d8278f1f792ab8416b8affd70f69de280ba248a
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 19:21:58 2026 -0700

        fix: steering hash overflow, add concurrent CUDA graph integration test

        - Mask steering_config_hash to fit np.int64 (was overflowing with
          full 64-bit unsigned SHA-256 hash)
        - Add test_per_request_steering_concurrent_with_cuda_graphs: sends
          3 requests (unsteered, +500, -500) in same batch, verifies per-request
          steering works with torch.compile + CUDA graphs active, then verifies
          unsteered determinism on a second batch

    commit 6bae6a4ad51baebc1d7575e320241cdd2b5bfa25
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 18:40:38 2026 -0700

        docs: update steering docs for per-request implementation

    commit 0265546ffca239b75ca32147f5dcb9bbb4063de5
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 18:34:12 2026 -0700

        docs: add OVERVIEW.md and steering feature documentation

    commit 4c57a2d31b2c647491dff7f1339cd81ad1fe9adf
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 18:12:01 2026 -0700

        test: add scheduler admission, worker steering, and integration tests

    commit 066839d90d3ce9c547c1fbfc9fdf9478ed4fc131
    Merge: 72b7b0c68 142432b7c
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 17:58:51 2026 -0700

        Merge pull request #13 from RhizoNymph/feat/per-request-steering-runner

        feat: integrate SteeringManager into model runner and worker_base

    commit 72b7b0c6899515ce25da7846e46281f24378ea3a
    Merge: b2e770b7d 2e914cfc7
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 17:58:21 2026 -0700

        Merge pull request #11 from RhizoNymph/feat/per-request-steering-tests

        test: SteeringManager and indexed-gather op unit tests

    commit 142432b7cd911ee029ae13a3b4d29c2e713f12b7
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 17:51:50 2026 -0700

        feat: integrate SteeringManager into model runner and worker_base

    commit b2e770b7d38de65611808cb45ffd9c3d3c24e60f
    Merge: 5d9c9f015 23198f1c0
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 17:43:22 2026 -0700

        Merge pull request #12 from RhizoNymph/feat/per-request-steering-manager

        feat: SteeringManager, InputBatch tracking, scheduler admission control

    commit 23198f1c085115e3935edd8c4dfe6c8574c84fe7
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 17:24:10 2026 -0700

        feat: add SteeringManager, InputBatch tracking, and scheduler admission control

    commit 2e914cfc75f20e5b90b1801e7953aff75da585ad
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 17:02:33 2026 -0700

        test: add SteeringManager and indexed-gather op unit tests

    commit 5d9c9f015dd9d21c372594ac2ab237091f3e52ee
    Merge: 9e535b083 2066a7e1c
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 16:47:55 2026 -0700

        Merge pull request #9 from RhizoNymph/feat/per-request-steering-config

        feat: add SteeringConfig, SamplingParams field, and API protocol support

    commit 9e535b08395e3dd40e17f6bb754a24e93432af28
    Merge: 602474f3c 10c95f5d4
    Author: Nymph <82485126+RhizoNymph@users.noreply.github.com>
    Date:   Fri Mar 27 16:46:03 2026 -0700

        Merge pull request #10 from RhizoNymph/feat/per-request-steering-op

        feat: update steering op to indexed gather with table/index buffers

    commit 2066a7e1c90fcca42816bb15e00180b9283698bb
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 16:04:51 2026 -0700

        feat: add steering config, SamplingParams field, and API protocol support

    commit 10c95f5d48ea45d213d2020717970467b7c93ba4
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 16:10:09 2026 -0700

        feat: update steering op to indexed gather, add table/index buffers to Gemma3

    commit 602474f3c947718c34b57029b45d7717c61dd2f9
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 14:08:39 2026 -0700

        remove todo doc for mvp

    commit 7619f8ae91273cf022892d1543a5bace29346784
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Fri Mar 27 14:08:28 2026 -0700

        fix: change injection point to raw residual stream not post layernorm

    commit 98408699f80226539d908b3bb690af8f6e853d9e
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Mar 26 22:47:14 2026 -0700

        fix: decode-only steering with torch.compile and CUDA graphs

        Replace ForwardContext-based num_decode_tokens read (constant-folded by
        torch.compile) with persistent per-layer mask buffers updated in-place
        by the model runner before each forward pass.

        - Custom op takes 3 args (hidden, vec, mask) and is a splitting op
        - Each decoder layer registers a steering_decode_mask buffer
        - Model runner discovers steerable layers and updates masks from
          scheduler output (decode tokens = 1 scheduled token, ordered first)
        - Add num_decode_tokens to TritonAttentionMetadata and
          FlashAttentionMetadata
        - Increase test steering magnitude for dummy-weight models
        - Add prefix cache reset after clearing steering in E2E test

        Co-authored-by: Claude

    commit c19a78587d29270c901c4fd66453a87eef049e8b
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Mar 26 13:36:55 2026 -0700

        docs/update temporary steering docs

    commit d51bd507c693f10372676030b3b97985c47ff561
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Mar 26 00:41:13 2026 -0700

        fix: typed steering exceptions, remove GET lock, cache steerable layers

        Co-authored-by: Claude

    commit 6c93d113d849d6cf474a6a8c90b617c37cab609a
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Thu Mar 26 00:20:18 2026 -0700

        fix: cast steering vector dtype to match hidden states

        Co-authored-by: Claude

    commit 531ad064402b875e1d8d11f3062e2c0165db61b2
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Mar 25 18:45:38 2026 -0700

        fix: register steering as custom op for torch.compile safety

        Co-authored-by: Claude

    commit 5c4ec89f2fbb90e6aa4912b2d27c2684f01c609f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Wed Mar 25 18:43:51 2026 -0700

        fix: return clear error for empty steering vectors request

        Co-authored-by: Claude

    commit e76810451eb76e8dc5d63ada1964f9dc1f98c423
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 24 18:44:25 2026 -0700

        temporary steering docs

    commit 1f2c5140c5b7fde10e7cffd4ba452207a115ba4f
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Tue Mar 24 18:15:36 2026 -0700

        add steering unit tests, fix test bugs

        - Add 65 unit tests across 4 files covering: SetSteeringRequest
          protocol validation, get_num_decode_tokens forward context utility,
          WorkerBase steering methods (set/clear/status with mock model,
          validation errors, edge cases), and API router endpoints (mock
          engine, all HTTP status codes, replace flag, attach/detach).
        - Fix test_steering.py: use llm.llm.generate() to get RequestOutput
          and compare only completion token_ids, not the prompt+output
          concatenation that VllmRunner.generate() returns.
        - Fix test_api_router.py: reset module-level _steering_lock between
          tests to prevent cross-test lock state leakage.
        - Fix test_worker_steering.py: delete no-op assertion that parsed as
          `assert True` due to operator precedence.

    commit 36a071e006e09890c479559880858c6dff174b97
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Mar 23 22:12:03 2026 -0700

        fix steering: unify API, remove redundant runner methods, fix test

        - Remove set_steering_vectors and clear_steering_vectors from
          GPUModelRunner — these had opposite semantics from the canonical
          WorkerBase versions (zeroed non-specified layers vs leaving them
          untouched) and were unreachable from the HTTP API path.
        - Move import math/torch to module level in worker_base.py.
        - Rewrite test to use string-based collective_rpc dispatching to
          WorkerBase methods instead of calling GPUModelRunner directly via
          lambda, making it compatible with the V2 model runner and
          exercising the same code path as the HTTP API.

    commit 77becfe0c9726bb5e651ee48c917bf738544bf36
    Author: RhizoNymph <quantnymph@gmail.com>
    Date:   Mon Mar 23 22:11:52 2026 -0700

        fix steering: extract decode-token utility, clean up API router

        - Move _get_num_decode_tokens from Gemma3DecoderLayer to a shared
          get_num_decode_tokens() utility in forward_context.py so future
          steerable models can reuse it without reimplementing fragile
          attn_metadata navigation.
        - Collapse duplicated except (ValueError, RuntimeError) / except
     …
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant