add qnn eval CI

[cccclai]

As title, add the script to prevent eval script regression. The test will calibrate qwen model and run eval, and test against threshold. If it's above the threshold, the test fail

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14528

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 58 Pending

As of commit b582450 with merge base 7b33035 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[kimishpatel]

what is the motivation to add this PR? In general I agree with protecting code, but it is not clear what kind of breakage we expect to prevent with this and enabling this on every pull request seems expensive.

[cccclai]

pull

It is super easy to use to evaluate accuracy with different modes and changes from qcom breaks it easily without tests

[kimishpatel]

discussed moving to trunk.yml if we cannot figure out how to make this specific job run only when changes are related to qualcomm

[cccclai]

Will follow up on narrowing down the path for the job