Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix KV cache shapes error #75

Merged
merged 7 commits into from
Sep 27, 2023
Merged

Fix KV cache shapes error #75

merged 7 commits into from
Sep 27, 2023

Conversation

casper-hansen
Copy link
Owner

@casper-hansen casper-hansen commented Sep 26, 2023 •
edited
Loading

Resolves #70

  • roll KV cache

casperhansen/vicuna-7b-v1.5-awq

Without fused modules:

Task Version Metric Value Stderr
wikitext 1 word_perplexity 11.7194
byte_perplexity 1.5845
bits_per_byte 0.6640

With fused modules:

Task Version Metric Value Stderr
wikitext 1 word_perplexity 11.7192
byte_perplexity 1.5845
bits_per_byte 0.6640

@casper-hansen casper-hansen mentioned this pull request Sep 26, 2023
Closed
@casper-hansen casper-hansen merged commit c57da6b into main Sep 27, 2023
@casper-hansen casper-hansen deleted the fix_runtime branch September 27, 2023 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CodeLlama 34B errors out after 3+ completions
1 participant
@casper-hansen