Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix promotion bug in MultiHeadDotProductAttention: #3571

Merged
merged 1 commit into from Dec 21, 2023

Conversation

giovannic
Copy link
Contributor

If x64 is enabled, autoregressive decoding would lead to dynamic slicing errors

Changes:

  • Add explicit types for dynamic_slice arguments
  • Add test for autoregressive decoding with x64 enabled

Please refer to the test case to recreate.

What does this PR do?

Fixes # (issue)

Checklist

  • This PR fixes a minor issue (e.g.: typo or small bug) or improves the docs (you can dismiss the other
    checks if that's the case).
  • This change is discussed in a Github issue/
    discussion (please add a
    link).
  • The documentation and docstrings adhere to the
    documentation guidelines.
  • This change includes necessary high-coverage tests.
    (No quality testing = no merge!)

Copy link

google-cla bot commented Dec 18, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@giovannic giovannic marked this pull request as ready for review December 18, 2023 15:17
@cgarciae
Copy link
Collaborator

Thanks @giovannic for doing this!

(@chiamp we should port this to NNX as well)

 * if x64 is enabled, autoregressive decoding would lead to dynamic
   slicing errors
 * Add explicit types for dynamic_slice arguments
 * Add test for x64 autoregressive decoding

Amended for more idiomatic/correct test case
@giovannic
Copy link
Contributor Author

Thanks for the review! Should be resolved.

@chiamp
Copy link
Collaborator

chiamp commented Dec 20, 2023

Thanks @giovannic for doing this!

(@chiamp we should port this to NNX as well)

Sounds good, I added it in #3576

@copybara-service copybara-service bot merged commit 85dfad2 into google:main Dec 21, 2023
19 checks passed
@giovannic giovannic deleted the bug/attention_promotion branch December 21, 2023 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants