Add support for Gemma 3 270M

[msluszniak]

🚀 The feature, motivation and pitch

As in the title, users of both ExecuTorch and React Native ExecuTorch will heavily benefit from such addition to the set of available models, look here: software-mansion/react-native-executorch#566.

Alternatives

Support for any powerful yet very small model like Gemma 3 270M.

Additional context

No response

RFC (Optional)

No response

[kirklandsign]

cc @jackzhxng

[jackzhxng]

Hi @msluszniak, you can export Gemma3 270M with Optimum ExecuTorch

[msluszniak]

Thank you for the information :))

[msluszniak]

I'm reopening this issue as I cannot export it via optimum-executorch. After logging into HF CLI, installing optimum-executorch, and running the following command:

optimum-cli export executorch \
    --model "google/gemma-3-270m" \
    --task "text-generation" \
    --recipe "xnnpack" \
    --use_custom_sdpa \
    --use_custom_kv_cache \
    --qlinear 8da4w \
    --qembedding 8w \
    --output_dir="gemma3_270"

I get the following logs:

WARNING:torchao.kernel.intmm:Warning: Detected no triton, on systems without Triton certain kernels will not work
W1017 13:17:49.043000 91476 site-packages/torch/distributed/elastic/multiprocessing/redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs.
/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/executorch/exir/dialects/edge/_ops.py:9: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1.35k/1.35k [00:00<00:00, 2.00MB/s]
`torch_dtype` is deprecated! Use `dtype` instead!
model.safetensors: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 536M/536M [00:35<00:00, 15.0MB/s]
`torch_dtype` is deprecated! Use `dtype` instead!
WARNING:coremltools:scikit-learn version 1.7.2 is not supported. Minimum required version: 0.17. Maximum required version: 1.5.1. Disabling scikit-learn conversion API.
WARNING:coremltools:Torch version 2.8.0 has not been tested with coremltools. You may run into unexpected errors. Torch 2.5.0 is the most recent version that has been tested.
Traceback (most recent call last):
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/executorch/attentions/custom_sdpa.py", line 86, in get_custom_sdpa_for_ring_kv_cache
    from executorch.examples.models.llama.source_transformation.custom_kv_cache import (
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/executorch/examples/models/llama/__init__.py", line 7, in <module>
    from .model import Llama2Model
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/executorch/examples/models/llama/model.py", line 22, in <module>
    from executorch.extension.llm.export.config.llm_config import LlmConfig
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/executorch/extension/llm/export/__init__.py", line 7, in <module>
    from .builder import LLMEdgeManager
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/executorch/extension/llm/export/builder.py", line 37, in <module>
    from pytorch_tokenizers import get_tokenizer
ModuleNotFoundError: No module named 'pytorch_tokenizers'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/miniconda3/envs/optimum-executorch/bin/optimum-cli", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/commands/optimum_cli.py", line 208, in main
    service.run()
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/commands/export/executorch.py", line 232, in run
    main_export(
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/exporters/executorch/__main__.py", line 145, in main_export
    return export_to_executorch(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/exporters/executorch/convert.py", line 80, in export_to_executorch
    executorch_progs = recipe_func(model, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/exporters/executorch/recipes/xnnpack.py", line 109, in export_to_executorch_with_xnnpack
    exported_progs = model.export()
                     ^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/exporters/executorch/integrations.py", line 484, in export
    self._register_custom_attention(exportable_module)
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/exporters/executorch/integrations.py", line 462, in _register_custom_attention
    _custom_sdpa_for_ring_kv_cache = get_custom_sdpa_for_ring_kv_cache(exportable_module)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/optimum-executorch/lib/python3.11/site-packages/optimum/executorch/attentions/custom_sdpa.py", line 90, in get_custom_sdpa_for_ring_kv_cache
    raise ImportError(f"CustomRingKVCache not available in version {version.__version__} of ExecuTorch.")
ImportError: CustomRingKVCache not available in version 0.7.0 of ExecuTorch.

[jackzhxng]

@msluszniak how did you install Optimum?

[msluszniak]

I installed it from source, as it is described in README.md of Optimum ExecuTorch

[jackzhxng]

Ah, yes sorry ExecuTorch is planning on releasing 1.0 soon, the pip install may be broken until we bump it since this repo is moving pretty fast. Can you install with step 3 instead? python install_dev.py.

[msluszniak]

Sure thing, I'll return with an update ;)