Final deprecation prep - #1740
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Hook API & bridge correctness
Gated-hook centralization – check_hooks_to_add is the real gate and keys on the hook point's canonical identity, so aliased overrides that genuinely fire (BERT's hook_mlp_in) are no longer refused, filter sweeps warn-and-skip consistently across add_hook/hooks()/run_with_hooks, and the default caching sweep composes through hooks() without raising.
W_pos / W_E_pos bridge accessors – last WS-3 item; refuses T5's relative-bias table, slices OPT/BART's +2 positional rows to match the legacy converters.
hook_pattern write-capable on native bridges – the hook runs inside NativeAttention before the value matmul, so pattern edits genuinely re-weight the output instead of silently no-oping.
Deprecation warnings fire exactly once per entry point – from_pretrained suppresses the init warning, from_pretrained_no_processing warns at its own stack level so the default filter shows it, and the two classes that never warned on bare construction now do.
Legacy-model support on the bridge
boot_tl_legacy – loads legacy TransformerLens-format repos (NeelNanda/ArthurConmy/Baidicoot: TL config.json + .pth, file-based checkpoints, old-SoLU and minGPT layout converters ported in); SoLU-1L is bit-exact against the frozen goldens.
Native solu_ln – the SoLU activation plus the load-bearing mid-MLP LayerNorm, without which those checkpoints compute the wrong function.
Checkpoint plumbing – NeelNanda file-based labels restored to the registry (WS-6 regression), cfg.checkpoint_index/value stamped by both boots, negative indices accepted for legacy parity.
Wav2Vec2 adapter – reuses the HuBERT structure, routes the ForPreTraining checkpoints to their encoder, refuses direct pretraining-head wraps, and fixes encoder_output's block order for stable-LN encoders.
Tests re-anchored off live HookedTransformer
incl_bwd gradients – anchored on a new goldens gradients group (dataset revision bumped) plus an exact autograd self-consistency test.
Checkpoint-converter tests – run from frozen in-repo fixtures instead of building nine HookedTransformers.
BERT weight surface – all nineteen accessors certified on the bridge against raw HF parameters, succeeding the HookedEncoder-only suite.
Enc-dec stacking parity – oracle derived from raw HF T5 weights instead of a live HookedEncoderDecoder.
Demos
All ten Hooked-touching demos migrated – Main_Demo's checkpoint loop runs its original SoLU model via boot_tl_legacy; GPT_OSS's bespoke 60%-loader collapses into one boot call (its hand-built config was unfaithful — the rewrite fixes real divergence); Colab_Compatibility is reframed as the Legacy Compatibility Ledger tracking HookedTransformer-era models on the bridge; lit/model.py's token-id positional indexing fixed.
Re-executed where executable – committed output conventions preserved (stripped notebooks stay stripped), metadata byte-identical; Activation_Patching's committed gpt2 numbers reproduce byte-for-byte on the bridge.
Deprecation mechanics
HookedRootModule + HookPoint kept – scrubbed of HookedTransformer references, un-deprecated, pinned by a keeper-contract test; Main_Demo's arbitrary-module pedagogy survives 4.0 verbatim.
PEP-562 lazy legacy names – import transformer_lens loads none of the deprecated stack; a de-shadowing module class makes class-vs-submodule resolution order-independent; TYPE_CHECKING bindings keep mypy exact.
Protocol typing – tools/training.py against a new TrainableTransformerLensModel (standalone, exactly the surface the loop touches), SVDInterpreter against the base protocol; both shaped around beartype's getattr_static protocol checking.
filterwarnings policy synced – newer class warnings added, obsolete HookedRootModule entry removed.
Type of change
Checklist: