制作了gptq 4bit版本 #107
Replies: 3 comments 3 replies
-
咋用的啊 |
Beta Was this translation helpful? Give feedback.
-
感谢热心分享,由于不属于 |
Beta Was this translation helpful? Give feedback.
-
感谢分享,亲测可用,大约占用 8G 显存。附上 GPTQ-for-LLaMa CUDA 命令
|
Beta Was this translation helpful? Give feedback.
-
https://huggingface.co/mrtoy/chinese-llama-13b-4bit-128g
效果还行,和fp16输出的内容差不多
Beta Was this translation helpful? Give feedback.
All reactions