Enable Gemma 4 E2B / E4B inference via vLLM RPA

[gagika]

Description

Enable Gemma 4 E2B / E4B inference via vLLM adapter by adding KV-shared layers wiring and a system_prompt flag to vllm_decode (required by the E-family -it checkpoints), and documents a verified inference recipe in Run_Gemma4.md.

Key changes

• gemma4_small.py — decoder layer returns the kernel-updated kv_cache (was dropped).
• decoders.py — KV-shared layers redirect to the donor's kv_caches slot via a layer→slot map; cache is written back per layer.
• attentions.py — sliding-window only on LOCAL_SLIDING layers; KV-shared layers no longer overwrite the donor's cache (update_kv_cache=False).
• vllm_decode.py / types.py — new system_prompt config knob, prepended as a system message when use_chat_template=True. Adapter registration + env flags moved from import time into main() (import side effects leaked into unrelated tests).
• Run_Gemma4.md — E2B / E4B recipe: system prompt, model-card sampling, full eos_token_id [1, 106, 50] stop-token set.

Tests

On v5p-8, e2b + e4b: cross-checked top-1 logits at greedy vs the native checkpoint — bit-identical. With the documented recipe, models generate coherent output and stop cleanly. CLI and Python API verified.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[github-actions]

🤖 Hi @gagika, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

[github-actions]

## 📋 Review Summary

This Pull Request successfully enables inference for Gemma 4 E2B/E4B models by implementing cross-layer KV sharing within the vLLM RPA (Ragged Paged Attention) path. The implementation is technically sound, handling the complex layer-to-slot mapping required for shared KV caches while maintaining compatibility with existing inference workflows.

🔍 General Feedback

• Robust KV-Sharing Implementation: The mapping logic in decoders.py correctly handles the redirection of shared layers to donor slots, ensuring efficient memory usage during TPU inference.
• Improved Attention Logic: The fix in attentions.py to restrict sliding window attention to LOCAL_SLIDING layers is a necessary correction for hybrid attention models like Gemma 4.
• Clear Documentation: The added recipes in Run_Gemma4.md provide essential guidance on system prompts and sampling parameters required for coherent output from these smaller checkpoints.

[codecov]

Codecov Report

❌ Patch coverage is 47.61905% with 11 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/maxtext/layers/decoders.py	0.00%	8 Missing ⚠️
src/maxtext/layers/attentions.py	0.00%	2 Missing ⚠️
src/maxtext/models/gemma4_small.py	90.90%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

🤖 Hi @gagika, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

[github-actions]

🤖 I'm sorry @gagika, but I was unable to process your request. Please see the logs for more details.

[github-actions]

🤖 Hi @gagika, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

[github-actions]

🤖 I'm sorry @gagika, but I was unable to process your request. Please see the logs for more details.

[khatwanimohit]

LGTM