Repository navigation
old NVIDIA GPU arch run(Fermi,Kepler) support, and LLVM-IR NVPTX(Clang) compiler support #29961
6uO
started this conversation in
Show and tell
Replies: 3 comments
|
@Kontomoka5932 If you are using an NVIDIA GPU, please try compiling the x32 version of ggml-cuda (I untest compile):
|
0 replies
Changelog (not-release, testing...)
|
0 replies
|
GT 430 for ggml-0.26.0 |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
TL;DR
DOWNLOAD: ggml-0.25.3_patch.zip, compile target CUDA Toolkit 8.0+sm_20.
⚠️ running requirements installed CUDA Toolkit 8.0 and Visual C++ v14 Redistributable⚠️
old nvcc unsupport C++ 17 host compiler, so use LLVM-IR NVPTX(-DCMAKE_CUDA_COMPILER=clang) compiler support
Welcome to discuss how to improve the patch.
Hope to Merge the patch to the repository, can also filter a portion of the diff to apply the patch. (I am sorry for not complying CONTRIBUTING.md, because there is an issue with the GitHub account.)
Aboot
Purpose:
To adapt to the increasingly expensive GPU era, adapt to cheap NVIDIA GPUs, and lower the minimum running requirements for ggml-cuda
Anyone can run inference at a very low cost.
Expand the user base and attract more contributors.
Content:
Quality Optimization/Bug Fixes.
Extensively used the
#if CUDART_VERSION > 8000macro to handle legacy syntax, in order to maintain compatibility with Fermi architecture.The code comment containing
unconfirmed patch:means I don't know what to do.Specially handle code that does not support LLVM-IR NVPTX using the
__NVCC__macro.The code comment containing a small part of
TODOand AI-generated codeAI code.Build
In order to build ggml-cuda, you need the following support libraries and
tools:
Download ggml
extract ggml-0.25.3
Notes
The final CUDA Driver API supported by the Kepler architecture is 11.4, but the final CUDA Toolkit supported by the Kepler architecture is 11.8, so therefore running (ggml-cuda for sm_35) with CUDA Toolkit 11.8 may have bug.
The Fermi architecture lacks the
__shflinstruction, so logical errors may occur when using it.For Multi-GPU setups, it is not recommended to rely on NVIDIA SLI for acceleration.
If
USE_GRAPHS = 1,CUDA : ARCHSmust include the sm of the current GPU? andGGML_CUDA_DISABLE_GRAPHSenv var is broken?sm_75 device(20 Series) use CUDA Toolkit 11.8 build and run:
CUDA : ARCHShave750run errCUDA : ARCHSdoes not have750is normal run.Support Matrix legend:
🔷 : Fully usable no obvious bugs (benchmark, work environment)
🟢 : Tested in GGML_BUILD_TESTS
🟡 : Should work, but is untested
🔶 : late-stage support
🔴 : Unsupported
Note: CUDA with LLVM 21 is broken due to a bug in Clang, plz using LLVM 21 developer upgrade LLVM ver.
Binary packages
Because the compilation time is too long...
So only Windows(ggml-cuda.dll), other OS stay tuned...
TODO
feedback to LLVM
fix cudaOccupancyMaxActiveBlocksPerMultiprocessor() (It should be that my run env is too complex)
fix GGML_CUDA_DISABLE_GRAPHS
add replicate
__shfl, maintain consistent data behavior for Fermilook solution for support Tesla archAll reactions