Qualcomm AI Engine Direct - GA Static Gemma-2b-instruct

[DannyYuyang-quic]

Summary:

• e2e script for Gemma-2b-it in static llama version
• add model params file & model weight converter

Test plan

python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --temperature 0 --model_mode hybrid --max_seq_len 1024 --prefill_ar_len 128 --decoder_model gemma3-1b --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14459

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 New Failures, 4 Unrelated Failures

As of commit ba71192 with merge base b3f3111 ():

NEW FAILURES - The following jobs have failed:

• Build Presets / apple (macos) / build (gh)
  The process '/opt/homebrew/bin/git' failed with exit code 128
• Build Presets / apple (profiling) / build (gh)
• Build Presets / apple (pybind) / build (gh)
  The process '/opt/homebrew/bin/git' failed with exit code 128
• Build Windows Wheels / pytorch/executorch / upload / upload-wheel-py3_10-cpu (gh)
  Action 'https://api.github.com/repos/actions/download-artifact/tarball/d3f86a106a0bac45b974a628896c90dbdf5c8093' download has timed out. Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
• pull / test-qnn-wheel-packages-linux (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t d41c60d18f320dbeb9d7823db0d2525e1250f905a2f41c022a2e13a00562c532 /exec failed with exit code 1
• pull / test-qnn-wheel-packages-linux (3.11) / linux-job (gh)
  RuntimeError: Command docker exec -t 5a2292807a5d8588208a6d249fb12781c760234c22ab905cd877bba35d2e5ec5 /exec failed with exit code 1
• pull / test-qnn-wheel-packages-linux (3.12) / linux-job (gh)
  RuntimeError: Command docker exec -t 751c87f998cdb888f2e79332adb966b597ba8fa3d0016ce7ca0f26d014b9ee4b /exec failed with exit code 1
• pull / test-samsung-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t d08deaa3063ecb4ed71ea1004d940b5d06b49d42837391e7cf3af8d9d769e579 /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / unittest-editable / macos / macos-job (gh) (similar failure)
  exir/tests/test_quant_fusion_pass.py::TestQuantFusionPass::test_embedding_torchao

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-binary-size-linux-gcc / linux-job (gh) (trunk failure)
• pull / test-setup-linux-gcc / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• pull / unittest / macos / macos-job (gh) (trunk failure)
  exir/tests/test_quant_fusion_pass.py::TestQuantFusionPass::test_embedding_torchao

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[DannyYuyang-quic]

Hi @cccclai,
This is Gemma-2b-it in static llama version.
Please have a look.
Thanks!!

cc: @haowhsu-quic

[DannyYuyang-quic]

@pytorchbot label "release notes: qualcomm"

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D83020234.

[cccclai]

Looks like there are some lint failure. The qnn wheel package is due to torch/ao bump. It should be fixed soon-ish

[DannyYuyang-quic]

Looks like there are some lint failure. The qnn wheel package is due to torch/ao bump. It should be fixed soon-ish

Hi @cccclai, lint errors are fixed, thanks!

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D83020234.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D83020234.