Qualcomm AI Engine Direct - Fix mem_handel register twice issue

[shewu-quic]

Summary:

• Insert registered handle in pre_registered_handles_ map to avoid register multiple times for the same data_ptr

Background:
When running llama in lookahead mode using the same AR-N model for both the prompt processor and token generator. The input and output are the same, and the kv cache is shared between both components.. This causes a "register twice" error message from QNN when a shared buffer (Smart Mask) is used.

Error message:

[ERROR] [Qnn ExecuTorch]:  <E> Memory Handle duplicate found, matches Handle ID 0x2

[ERROR] [Qnn ExecuTorch]:  <E> Mem handle exists already

[ERROR] [Qnn ExecuTorch]:  <E> Failed to register memHandles

[ERROR] [Qnn ExecuTorch]:  <E> Failed to register memHandles

[ERROR] [Qnn ExecuTorch]:  <E> Failed to register mem with error 0x1f42

cc: @sxu , @haowhsu-quic

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13410

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (3 Unrelated Failures)

As of commit 9473642 with merge base 30a6f5e ():

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-buck / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[shewu-quic]

Hi @cccclai , @sxu ,

This PR is to fix register multiple times issue when using the same AR-N between both of prompt processor and token generator in lookahead mode.
Could you please help take a look?
Thanks

[facebook-github-bot]

@sxu has imported this pull request. If you are a Meta employee, you can view this in D80263917.

[sxu]

@shewu-quic Thanks for putting this up. However the situation you described (using same graph for two purposes) is not what we are doing, we have multiple graphs, and externally allocated ION buffers that we are trying to use as input for those graphs. The double registration error we are seeing comes from RegisterIonMem. I came up with #13421 which seems to fix the issue for us, however I don't quite understand the what the rest of the memory manager is for (the pre-register and the custom memory stuff), so I don't know if there's a better way to do it.

[cccclai]

Seems like there are some ongoing discussion, what do you think regarding this particular PR?

[shewu-quic]

Seems like there are some ongoing discussion, what do you think regarding this particular PR?

I believe this PR is ready for review and merge. The PR it addresses is different from the one that @sxu encountered.