Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix gptq fp16 inference #104

Merged
merged 2 commits into from
Dec 5, 2023
Merged

fix gptq fp16 inference #104

merged 2 commits into from
Dec 5, 2023

Conversation

flozi00
Copy link
Collaborator

@flozi00 flozi00 commented Dec 5, 2023 •
edited

Now its working with fp16
ready to merge from my side
inference output checked, all adapters working now as expected

@flozi00 flozi00 requested a review from tgaddair December 5, 2023 20:30
@flozi00 flozi00 changed the title fix inference fix gptq fp16 inference Dec 5, 2023
@flozi00 flozi00 merged commit d481e7e into main Dec 5, 2023
1 check passed
@flozi00 flozi00 deleted the fix_exllama_fp16 branch December 5, 2023 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tgaddair tgaddair tgaddair approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@flozi00 @tgaddair