-
Notifications
You must be signed in to change notification settings - Fork 8.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
adding support for linux binaries #5106
Comments
What issues? |
It compiles fine for me, what is your build environment. The CI tests if it can compile on Linux and it does. It's more likely to be specific to your build environment. Edit: try using zig! |
the error is random at times, my environment is conda with cuda and gcc=11 installed, but that isn't the point, also @Titaniumtown, i am not a programmer to use/know zig, i wanna proper binary support ;-; |
I package RPMs using Fedora COPR for a few architectures and platforms (Fedora, CentOS, Amazon Linux, etc) on x86_64, aarch64, etc. These are basic builds without AVX512 or some optimizations. I use build date as version because code tags/hashes aren't ideal and don't sort well. https://copr.fedorainfracloud.org/coprs/boeroboy/brynzai/monitor/ The OSS nature of COPR means I can't really pre-build the CUBLAS releases with CUDA being a proprietary license but they can still be built locally. I've been pushing for these to be mainlined in Fedora. What is your OS? I also notice that OpenBLAS builds in Fedora aren't straightforward because Then again I experience much slower inference with OpenBLAS than standard CPU compiled with What Linux distros are you looking for? I don't have .deb packaged. What I would recommend is a generic statically linked binary in releases. That way it should work with most Linux distros regardless of packaging. Again CUDA still can't be statically linked so this option would only support CPU, CLBLAST, or OpenBLAS. |
It looks like you're enabling both LLAMA_OPENBLAS and LLAMA_CUBLAS which is probably your problem. Remove one of them and try again? (Env variables possibly?) |
FYI, if you're not familiar with compiling, just use docker. Install docker using one command: https://gist.github.com/zulhfreelancer/254c4a157c586dd232c1a51db0f6eac3 Then heading to the "Docker" section in README: https://github.com/ggerganov/llama.cpp#docker |
came here to +1 this |
Came here to +1, too - especially for Mac! |
For some cloud servers that charge by the second, using precompiled llamacpp saves compilation time, which means saving money. |
Prerequisites
Please answer the following questions for yourself before submitting an issue.
(yup, i searched 'linux binary' and 'linux binaries' and didn't found a single issue)
Feature Description
as i said in the title, it would be great if llama.cpp officially adds support for automated compile in
https://github.com/ggerganov/llama.cpp/releases
because i get compile issues and i'm not programmer, so it would be great if you would officially include such binary, even if it's 343.2 MIB (359.8712832 MB) of size like what kobold.cpp is
Motivation
alot of compile errors and discord calls, and the fact that kobold.cpp has it already, as for why it's 'necessary', because newbies like me could also use it, and the fact that kobold.cpp is very slow in updates and can't use the mainline features
Possible Implementation
maybe this would be useful?
https://github.com/Nexesenex/kobold.cpp/blob/concedo_experimental/.github/workflows/kcpp-build-release-linux.yaml
but overall, feel free to close and ignore this issue if it isn't a priority, but it hugely sucks
The text was updated successfully, but these errors were encountered: