Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for AWQ models #645

Open
ptanov opened this issue Nov 14, 2023 · 0 comments
Open

Add support for AWQ models #645

ptanov opened this issue Nov 14, 2023 · 0 comments

Comments

@ptanov
Copy link
Contributor

ptanov commented Nov 14, 2023

There is already available implementation here: https://github.com/casper-hansen/AutoAWQ

Here are some other links:
https://arxiv.org/abs/2306.00978
https://friendli.ai/blog/activation-aware-weight-quantization-llm/
https://github.com/mit-han-lab/llm-awq
ptanov added a commit to ptanov/localGPT that referenced this issue Nov 14, 2023
@ptanov
PromtEngineer#645 Add support for AWQ models
17e8cfb
PromtEngineer added a commit that referenced this issue Nov 22, 2023
@PromtEngineer
Merge pull request #646 from ptanov/645-support-awq-models
d30aaef 
#645 Add support for AWQ models
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ptanov