Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade torch to 2.1.2 #671

Merged
merged 10 commits into from
Mar 26, 2024
Merged

Upgrade torch to 2.1.2 #671

merged 10 commits into from
Mar 26, 2024

Conversation

davidkyle
Copy link
Member

@davidkyle davidkyle commented Feb 22, 2024

Elasticsearch is upgraded to PyTorch v2.1.2 for v8.13.0, this change makes Eland use the same version

Due to potential backwards compatibility issues with models traced in PyTorch 2.1.2 running on earlier versions the eland_import_hub_model script will not upload models to a cluster that is earlier than Elasticsearch v8.13.0

The tests caught a problem tracing the valhalla/distilbart-mnli-12-6 model:

ValueError: Attention using SDPA can not be traced with torch.jit.trace when no attention_mask is provided. To solve this issue, please either load your model with the argument `attn_implementation="eager"` or pass an attention_mask input when tracing the model. 

The error comes from the decoder layer, in the transformers code the decoder attention mask is not initialised and appears to be added in this PR.

or pass an attention_mask input when tracing the model.

The error message is misleading as the input included attention_mask.

The change was added in transformers v4.36.0 released in December 2023. Rather than work around the problem by injecting the missing parameters the simplest solution is to pin the transformers version to 4.35.2.

Closes #669

@davidkyle davidkyle added the enhancement New feature or request label Feb 22, 2024
@davidkyle davidkyle marked this pull request as ready for review March 4, 2024 09:05
@davidkyle davidkyle force-pushed the upgrade branch 2 times, most recently from 5e4363f to 2e01a2d Compare March 21, 2024 10:14
Copy link
Contributor

@valeriy42 valeriy42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@davidkyle davidkyle merged commit ae0bba3 into elastic:main Mar 26, 2024
4 checks passed
@davidkyle davidkyle deleted the upgrade branch March 26, 2024 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade PyTorch to version 2.1.2 and
2 participants