Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

什么时候有量化后的模型 #48

Closed
15899885850 opened this issue Jul 13, 2023 · 5 comments
Closed

什么时候有量化后的模型 #48

15899885850 opened this issue Jul 13, 2023 · 5 comments

Comments

@15899885850
Copy link

RT
现在每次启动都需要使用CPU量化，速度太慢了
@jameswu2014
Copy link
Collaborator

这两天会发布

@ShadowPower
Copy link

可以试试 bitsandbytes 量化，我写了一篇教程：
https://zhuanlan.zhihu.com/p/643307410

@15899885850 15899885850 reopened this Jul 13, 2023
@GradientGuru
Copy link
Contributor

已经有Chat的量化版本。见更新后的README

@Gzj369
Copy link

Gzj369 commented Aug 9, 2023

@GradientGuru 大佬，Chat的量化版本链接好像失效了，无法下载，麻烦帮忙看看

@Gzj369
Copy link

Gzj369 commented Aug 12, 2023

找到了，可以用这个链接访问下载 8bit量化后的模型，https://huggingface.co/trillionmonster/Baichuan-13B-Chat-8bit/tree/main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ShadowPower @jameswu2014 @Gzj369 @15899885850 @GradientGuru