feat: add RWKV-7 (Goose) TransformerBridge adapter (RWKV7ForCausalLM)

[mukund1985]

Closes #1462.

Adds a TransformerBridge adapter for RWKV7ForCausalLM (RWKV-7 "Goose") from fla-org/flash-linear-attention, loaded via trust_remote_code=True.

Architecture overview

RWKV-7 is an attention-free recurrent model:

model.embeddings → model.layers[i] (RWKV7Block) → model.norm → lm_head

Each RWKV7Block contains:

• attn_norm — standard biased LayerNorm (NOT RMSNorm)
• attn — RWKV7Attention: generalized delta-rule time-mixing with matrix-valued state; 4-tuple return (y, v_first, state, ...)
• ffn_norm — fused add-and-norm when fuse_norm=True (default)
• ffn — RWKV7FeedForward: token-shift + sqrelu channel-mixing (not a gated MLP)

v_first (cross-block state) is managed internally by RWKV7Model.forward — no adapter handling required.

Adapter decisions

SSMBlockBridge for model.layers — delegates the whole recurrent block; exposes hook_in/hook_out on the residual stream. BlockBridge is not usable: its hook aliases hardcode a standard pre-norm flow that RWKV-7 does not follow.

SSM2MixerBridge for both attn and ffn sublayers — tuple-aware passthrough that correctly preserves attn's 4-tuple return (carrying v_first) and ffn's 2-tuple return without re-implementing either.

NormalizationBridge (biased LayerNorm) for attn_norm and ln_final — uses_rms_norm=False, normalization_type="LN".

ffn_norm as plain GeneralizedComponent — when fuse_norm=True, the block calls ffn_norm(hidden_states, residual, True), a fused 3-arg signature that NormalizationBridge's single-input contract cannot express.

head_dim excluded from _HF_PASSTHROUGH_ATTRS — read-only property on TransformerBridgeConfig (alias of d_head); a passthrough setattr would crash boot.

applicable_phases = [] — correctness covered by integration tests.

prepare_loading mirrors Raven's transformers v5 guards (_tied_weights_keys list→dict, _init_weights re-randomization guard), applied defensively via try/except.

Files changed

File	Change
transformer_lens/model_bridge/supported_architectures/rwkv7.py	New adapter
tests/unit/model_bridge/supported_architectures/test_rwkv7_adapter.py	31 unit tests (synthetic config)
tests/integration/model_bridge/test_rwkv7_adapter.py	Integration tests (gated: slow + skipif(CI) + importorskip("fla"))
transformer_lens/model_bridge/supported_architectures/__init__.py	Register RWKV7ArchitectureAdapter
transformer_lens/factories/architecture_adapter_factory.py	Register RWKV7ForCausalLM
transformer_lens/tools/model_registry/__init__.py	HF_SUPPORTED_ARCHITECTURES + CANONICAL_AUTHORS_BY_ARCH
transformer_lens/tools/model_registry/generate_report.py	Architecture description
transformer_lens/model_bridge/sources/_bridge_builder.py	RWKV-7 passthrough attrs

Testing

Unit tests (31) pass on synthetic config — no HF Hub access:

uv run pytest tests/unit/model_bridge/supported_architectures/test_rwkv7_adapter.py -v

Integration tests require flash-linear-attention (~191 MB checkpoint download):

pip install flash-linear-attention
uv run pytest tests/integration/model_bridge/test_rwkv7_adapter.py -v -m slow

[mukund1985]

@jlarson4 ready for review

[jlarson4]

Same note about reviewing the failing CI checks here @mukund1985. Thanks for taking both of these adapters on!

[jlarson4]

@mukund1985 Very similar comment here as on #1520. We cannot use the SSM-based components for non-SSM recurrent architectures. I recommend adopting the same OpaqueBlockBridge in place of the SSMBlockBridge as recommended in #1520. For your usage of SSM2MixerBridge, we should be able to replace it with a GeneralizedComponent without running into any issues. Let me know if you run into any issues with those adjustments.

[mukund1985]

Thanks for the review. Both points addressed in the latest commits:

• Replaced SSMBlockBridge with OpaqueBlockBridge for the model.layers block list (same refactor as feat: add RavenForCausalLM (Huginn) depth-recurrent adapter #1520).
• Replaced SSM2MixerBridge with GeneralizedComponent for the attn and ffn sublayers. Unit tests updated accordingly.
  Let me know if there is anything else to adjust.

[mukund1985]

@jlarson4 all CI checks are now green — ready for your re-review when you get a chance.

[jlarson4]

Looks great, thanks @mukund1985! Merging now