Qualcomm AI Engine Direct - Scripts and accuracy improvement for Qwen3_0.6B/1.7B and Qwen 2.5_1.5B

[winskuo-quic]

Summary

• Adding static Qwen 2.5 - 1.5B to script.
• Adding static Qwen 3 0.6B/1.5B to script
• Adding back skip_advanced_requant.
• Adding prompt + special token for calibration, which helps certain models to improve accuracy.

Example Scripts:

python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -H haowhsu-linux -s 5f396958 -m SM8750 --prompt "How many r's in strawberries?" --temperature 0 --model_mode kv --max_seq_len 1024 --ptq 16a8w --decoder_model qwen3-0_6b --tasks wikitext --limit 1 --artifact ./qwen3-0_6b

Statistics on SM8750, seq_len=1024

qwen2 1.5B: ~34tok/sec. QNN on device PPL=9.4 (CPU FP=9.1)
qwen3 0.6B: ~56tok/sec. QNN on device PPL=16.8 (CPU FP=16.26)
qwen3 1.7B: ~14tok/sec. QNN on device PPL=14.1 (CPU FP=13.52)

Test plan

E2E in test_qnn_delegate.py

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13544

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit dde5562 with merge base 9359481 ():

NEW FAILURE - The following job has failed:

• pull / unittest-editable / linux / linux-job (gh)
  exir/tests/test_remove_unused_parameters_pass.py::TestRemoveUnusedParametersPass::test_remove_unused_parameters_nested_e2e_to_edge

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[winskuo-quic]

Hi @cccclai,
This PR is to improve accuracy for qwen3 0.6B/1.7B and enable qwen 2.5 1.5B.
The PPL score does not have much effect, which still closely align with FP CPU.
The main difference here is special tokens such as , are also feed as input during calibration, which makes the output string looks more similar to CPU FP's output.
Before, qwen3 basically could not hit eot and will repeat the same sentence. With this PR, we have tested a couple of prompts, and they can all hit eot condition.

Please have a look.
Thanks

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D80726276.