Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: default use_qk_norm false in cohere #1758

Closed
wants to merge 1 commit into from

Conversation

drbh
Copy link
Collaborator

@drbh drbh commented Apr 17, 2024

This PR simply defaults use_qk_norm to false if its not included in the config

Comment on lines +219 to +221
self.use_qk_norm = (
config.use_qk_norm if hasattr(config, "use_qk_norm") else False
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should update transformers instead and that should solve it.

@drbh
Copy link
Collaborator Author

drbh commented Apr 22, 2024

closing due to upstream change huggingface/transformers#30027

@drbh drbh closed this Apr 22, 2024
@drbh drbh deleted the cohere-default-use-qk-norm branch April 22, 2024 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants