Skip to content

Final deprecation prep - #1740

Merged
jlarson4 merged 8 commits into
dev-4.xfrom
final-deprecation-prep
Sep 2, 2026
Merged

Final deprecation prep#1740
jlarson4 merged 8 commits into
dev-4.xfrom
final-deprecation-prep

Conversation

@jlarson4

@jlarson4 jlarson4 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Description

Hook API & bridge correctness

Gated-hook centralization – check_hooks_to_add is the real gate and keys on the hook point's canonical identity, so aliased overrides that genuinely fire (BERT's hook_mlp_in) are no longer refused, filter sweeps warn-and-skip consistently across add_hook/hooks()/run_with_hooks, and the default caching sweep composes through hooks() without raising.
W_pos / W_E_pos bridge accessors – last WS-3 item; refuses T5's relative-bias table, slices OPT/BART's +2 positional rows to match the legacy converters.
hook_pattern write-capable on native bridges – the hook runs inside NativeAttention before the value matmul, so pattern edits genuinely re-weight the output instead of silently no-oping.
Deprecation warnings fire exactly once per entry point – from_pretrained suppresses the init warning, from_pretrained_no_processing warns at its own stack level so the default filter shows it, and the two classes that never warned on bare construction now do.
Legacy-model support on the bridge

boot_tl_legacy – loads legacy TransformerLens-format repos (NeelNanda/ArthurConmy/Baidicoot: TL config.json + .pth, file-based checkpoints, old-SoLU and minGPT layout converters ported in); SoLU-1L is bit-exact against the frozen goldens.
Native solu_ln – the SoLU activation plus the load-bearing mid-MLP LayerNorm, without which those checkpoints compute the wrong function.
Checkpoint plumbing – NeelNanda file-based labels restored to the registry (WS-6 regression), cfg.checkpoint_index/value stamped by both boots, negative indices accepted for legacy parity.
Wav2Vec2 adapter – reuses the HuBERT structure, routes the ForPreTraining checkpoints to their encoder, refuses direct pretraining-head wraps, and fixes encoder_output's block order for stable-LN encoders.
Tests re-anchored off live HookedTransformer

incl_bwd gradients – anchored on a new goldens gradients group (dataset revision bumped) plus an exact autograd self-consistency test.
Checkpoint-converter tests – run from frozen in-repo fixtures instead of building nine HookedTransformers.
BERT weight surface – all nineteen accessors certified on the bridge against raw HF parameters, succeeding the HookedEncoder-only suite.
Enc-dec stacking parity – oracle derived from raw HF T5 weights instead of a live HookedEncoderDecoder.
Demos

All ten Hooked-touching demos migrated – Main_Demo's checkpoint loop runs its original SoLU model via boot_tl_legacy; GPT_OSS's bespoke 60%-loader collapses into one boot call (its hand-built config was unfaithful — the rewrite fixes real divergence); Colab_Compatibility is reframed as the Legacy Compatibility Ledger tracking HookedTransformer-era models on the bridge; lit/model.py's token-id positional indexing fixed.
Re-executed where executable – committed output conventions preserved (stripped notebooks stay stripped), metadata byte-identical; Activation_Patching's committed gpt2 numbers reproduce byte-for-byte on the bridge.
Deprecation mechanics

HookedRootModule + HookPoint kept – scrubbed of HookedTransformer references, un-deprecated, pinned by a keeper-contract test; Main_Demo's arbitrary-module pedagogy survives 4.0 verbatim.
PEP-562 lazy legacy names – import transformer_lens loads none of the deprecated stack; a de-shadowing module class makes class-vs-submodule resolution order-independent; TYPE_CHECKING bindings keep mypy exact.
Protocol typing – tools/training.py against a new TrainableTransformerLensModel (standalone, exactly the surface the loop touches), SVDInterpreter against the base protocol; both shaped around beartype's getattr_static protocol checking.
filterwarnings policy synced – newer class warnings added, obsolete HookedRootModule entry removed.

Type of change

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4
jlarson4 merged commit a96ead8 into dev-4.x Sep 2, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant