Skip to content

Fix Gemma 3 safetensor loading#987

Merged
copybara-service[bot] merged 1 commit intomainfrom
test_858218632
Jan 20, 2026
Merged

Fix Gemma 3 safetensor loading#987
copybara-service[bot] merged 1 commit intomainfrom
test_858218632

Conversation

@copybara-service
Copy link

@copybara-service copybara-service bot commented Jan 19, 2026

Fix Gemma 3 safetensor loading

Loading Gemma 3 from safetensor weights fails due to two reasons:

  • Safetensor weights follow the pattern "language_model*". "language_model" was missing.
  • When we load weights, weights from the vision part of the model are automatically checked.
    Since they are absent from the key dict, an error is thrown.

Colab: https://colab.research.google.com/gist/abheesht17/9e85a2f7d08b60712f42bd1b44481ee6/bug-fix-gemma-3-loading-tunix.ipynb

Code snippet:

from tunix.models.gemma3 import params_safetensors as params_safetensors_lib
from tunix.models.gemma3 import model as gemma_lib
from flax import nnx

model_config = gemma_lib.ModelConfig.gemma3_4b_it()
MODEL_CP_PATH = local_model_path
gemma3 = params_safetensors_lib.create_model_from_safe_tensors(
    MODEL_CP_PATH, (model_config),
)

Error:

ValueError: Only one key should be found: [] for language_model.model.embed_tokens.weight

@copybara-service copybara-service bot changed the title Fix Gemma 3 loading from safetensor Fix Gemma 3 safetensor loading Jan 19, 2026
Loading Gemma 3 from safetensor weights fails due to two reasons:

- Safetensor weights follow the pattern "language_model*". "language_model" was missing.
- When we load weights, weights from the vision part of the model are automatically checked.
  Since they are absent from the key dict, an error is thrown.

Colab: https://colab.research.google.com/gist/abheesht17/9e85a2f7d08b60712f42bd1b44481ee6/bug-fix-gemma-3-loading-tunix.ipynb

Code snippet:

```
from tunix.models.gemma3 import params_safetensors as params_safetensors_lib
from tunix.models.gemma3 import model as gemma_lib
from flax import nnx

model_config = gemma_lib.ModelConfig.gemma3_4b_it()
MODEL_CP_PATH = local_model_path
gemma3 = params_safetensors_lib.create_model_from_safe_tensors(
    MODEL_CP_PATH, (model_config),
)
```

Error:

```
ValueError: Only one key should be found: [] for language_model.model.embed_tokens.weight
```
PiperOrigin-RevId: 858377286
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant