You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sometimes the cuda version detection in linux is not correct.
Reproduction Steps
pull and run the docker nvidia/cuda:11.7.1-devel-ubuntu22.04.
Run the LLamaSharp examples
Environment & Configuration
Operating system: Ubuntu
.NET runtime version: 8.0
LLamaSharp version: master
CUDA version (if you are using cuda backend): 11.7
CPU & GPU device: 2080Ti
Known Workarounds
SkipCheck or WithLibrary is a quick fix but not good.
The main reason is that the version.json or version.txt does not exist. A possible solution is to create a repo about calling nvml APIs but that will cost a lot of time.
The text was updated successfully, but these errors were encountered:
else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
var manualVersionSet = Environment.GetEnvironmentVariable("CUDA_MAJOR_VERSION");
if(!string.IsNullOrEmpty(manualVersionSet)){
return int.Parse(manualVersionSet);
}
// Try the default first
cudaPath = "/usr/local/bin/cuda";
version = GetCudaVersionFromPath(cudaPath);
if (string.IsNullOrEmpty(version))
{
cudaPath = Environment.GetEnvironmentVariable("LD_LIBRARY_PATH");
if (cudaPath is null)
{
return -1;
}
foreach (var path in cudaPath.Split(':'))
{
version = GetCudaVersionFromPath(Path.Combine(path, ".."));
if (string.IsNullOrEmpty(version))
{
break;
}
}
}
}
I just set the same environment variable in the dockerfile, and the check always works. One can also set it via the docker run -e command, or export it in the shell.
I don't think you need to implement something super complex just to parse some nvidia tool result. You can expect an environment variable or maybe even a config set within the project, that simply tells you what cuda version is installed. Mention it in the docu and you're done.
Description
Sometimes the cuda version detection in linux is not correct.
Reproduction Steps
nvidia/cuda:11.7.1-devel-ubuntu22.04
.Environment & Configuration
Known Workarounds
SkipCheck
orWithLibrary
is a quick fix but not good.The main reason is that the
version.json
orversion.txt
does not exist. A possible solution is to create a repo about calling nvml APIs but that will cost a lot of time.The text was updated successfully, but these errors were encountered: