Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix/attention mask and implementation #49

Merged

Conversation

Alvant
Copy link
Contributor

@Alvant Alvant commented Dec 27, 2023

Issue: #46

I had to change attn implementation initialization. Unexpectedly (at least for me 😅), it turned out that it is not possible to specify attention in the model's config.json file. One can only set it as an argument when creating an object (attn_implementation is taken from "kwargs", not "config_dict", see: https://github.com/huggingface/transformers/blob/v4.36.1/src/transformers/configuration_utils.py#L772). So, I add attention to args and give it to config object when it is created.

I hope that this change is OK (new argument to parser + modified AutoConfig.from_pretrained call).

@ChenMnZ
Copy link
Collaborator

ChenMnZ commented Dec 27, 2023

Good job! Thanks for your time again.

@ChenMnZ ChenMnZ merged commit 9790164 into OpenGVLab:main Dec 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants