Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bump llama.cpp, add gguf support #943

Merged
merged 7 commits into from Aug 23, 2023
Merged

feat: bump llama.cpp, add gguf support #943

merged 7 commits into from Aug 23, 2023

Conversation

mudler
Copy link
Owner

@mudler mudler commented Aug 22, 2023

Description

This PR syncs up the llama backend to use gguf (go-skynet/go-llama.cpp#180). It also adds llama-stable to the targets so we can still load ggml. It adapts the current tests to use the llama-backend for ggml and uses a gguf model to run tests on the new backend.

In order to consume the new version of go-llama.cpp, it also bump go to 1.21 (images, pipelines, etc)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler merged commit 1120847 into master Aug 23, 2023
14 checks passed
@mudler mudler deleted the gguf branch August 23, 2023 23:18
@mudler
Copy link
Owner Author

mudler commented Aug 23, 2023

let's see how it goes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant