[Proposal] Add T5Gemma encoder-decoder adapter (T5GemmaForConditionalGeneration)

[jlarson4]

Proposal

Add a TransformerBridge adapter for T5GemmaForConditionalGeneration (Google T5Gemma), a recent encoder-decoder family built from decoder-only Gemma.

Motivation

T5Gemma is the one modern, actively-developed encoder-decoder line. It is made by converting pretrained Gemma 2/3 into an encoder-decoder, so it shares weight lineage with models TransformerLens already supports. That lets researchers compare a decoder-only model against its encoder-decoder counterpart on the same base, which decoder-only models alone cannot offer. Paired with a BART adapter, it keeps the library's encoder-decoder coverage current.

Gap scan (2026-06-18): 9 models, ~40K downloads. Downloads are low, but it would be good to support a modern T5 architecture for users who are interested in doing research in this area. Checkpoints go down to about 270M.

Pitch

Reuse both the existing T5 encoder-decoder pattern and the existing Gemma components.

• Claude Code users can scaffold with /add-model-support google/t5gemma-s-s-ul2.
• Register at the four sites listed in contributing.md.
• Verify smallest-first: google/t5gemma-s-s-ul2, then google/t5gemma-2b-2b-ul2.

Additional context

• Blog: https://developers.googleblog.com/en/t5gemma/
• HF docs: https://huggingface.co/docs/transformers/model_doc/t5gemma
• Found via the hf_scraper architecture-gaps pass (2026-06-18).

Checklist

• I have checked that there is no similar issue in the repo (required)

[SyedIshmumAhnaf]

Hi @jlarson4, I’d like to work on this if it’s available.

I’ve done an initial pass over the TransformerBridge path and this looks like a good fit: I’ll start from the existing T5 encoder-decoder adapter and registration pattern, then check what T5Gemma needs differently from the Gemma side rather than treating it as a completely new architecture. I’ll keep the first PR focused on adding T5GemmaForConditionalGeneration support, wiring the registry/loading path, and verifying against the smallest checkpoint first (google/t5gemma-s-s-ul2) before looking at the larger checkpoint.

Could you assign this to me?

[jlarson4]

@SyedIshmumAhnaf Can do! Let me know if you have any questions