Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

What do you think of omniquant? #1

Closed
Ph0rk0z opened this issue Sep 12, 2023 · 1 comment
Closed

What do you think of omniquant? #1

Ph0rk0z opened this issue Sep 12, 2023 · 1 comment

Comments

@Ph0rk0z
Copy link

Ph0rk0z commented Sep 12, 2023

https://arxiv.org/abs/2308.13137

Is it comparable to the new version? This guy jammed falcon into 65G with decent PPL.

Uses the AutoGPTQ kernel so it must be similar. Maybe something can be plifered from it.

https://github.com/OpenGVLab/OmniQuant
@turboderp
Copy link
Owner

I've been way too preoccupied to look at it. But it does look interesting. I'm going to be checking it out eventually, for sure.

@turboderp turboderp closed this as not planned Won't fix, can't repro, duplicate, stale Sep 17, 2023
turboderp pushed a commit that referenced this issue Apr 27, 2024
@Orion-zhen
Merge pull request #1 from Orion-zhen/rocm-aio
e5cdc54 
feat: rocm aio build
anchortense pushed a commit to anchortense/exllamav2-logit-threshold-samplers that referenced this issue Oct 21, 2024
@Orion-zhen
Merge pull request turboderp#1 from Orion-zhen/rocm-aio
0ffd5a8 
feat: rocm aio build
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@turboderp @Ph0rk0z