Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add a new model #2210

Closed
cacaview opened this issue Apr 11, 2024 · 1 comment
Closed

[Feature] Add a new model #2210

cacaview opened this issue Apr 11, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@cacaview
Copy link

I found an AI model on GitHub:

https://github.com/baichuan-inc/Baichuan2

I using llama.cpp convert the model to gguf, but the results aren't ideal and feedback is all wrong.
Used the following parameters: --ctx 4096 --outtype q8_0.
CommentPicture_a236fed4-0df4-4f31-9b2e-c7624db1958a
I think this might be my problem, and I want to make it usable.
I want it to be available on gpt4all and add it to the download list.

@cacaview cacaview added the enhancement New feature or request label Apr 11, 2024
@cebtenzzre
Copy link
Member

cebtenzzre commented Apr 12, 2024

This model should already work, including GPU support. Try downloading a Q4_0 GGUF, e.g. the one from here (this is a finetune but it's the same model architecture). Make sure to set the prompt template (including the blank line after):

|Human|: %1
|Bot|: %2

@cebtenzzre cebtenzzre closed this as not planned Won't fix, can't repro, duplicate, stale Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants