Qualcomm AI Engine Direct - Fix the regression of whisper model

[shewu-quic]

Summary:

• Resolve the Whisper model accuracy issue caused by upgrading the Transformers.
  • Modify decompose_sdpa.py to support kwargs "scale"
• fixed internal CI

cc: @haowhsu-quic , @winskuo-quic

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13062

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Failing to create ec2 instances -> lots of queueing

❌ 1 New Failure

As of commit 936997d with merge base 4197fc1 ():

NEW FAILURE - The following job has failed:

• pull / unittest-editable / macos / macos-job (gh)
  backends/xnnpack/test/ops/test_ceil.py::TestCeil::test_fp16_ceil

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[shewu-quic]

Hi @cccclai,
This PR is to fixed the accuracy issue of the whisper model when upgrade transformers. The primary issue is that the original sdpa does not account for the "scale" keyword argument, which is now used by the latest Whisper model.
https://github.com/huggingface/transformers/blob/d3b8627b56caa7ca8fac113c9f28d0256db0194d/src/transformers/models/whisper/modeling_whisper.py#L358

[shewu-quic]

@pytorchbot label "release notes: qualcomm"

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D79457707.

[cccclai]

Thanks!