"native" is not a defined option for 'gpu-architecture'

[wesgould]

I don't know if this is specific to me. I am running a headless GTX 1080. I have the nvidia-530 drivers and the nvidia-cuda-toolkit installed (running pop_os so these are likely whats in the ubuntu repos as well).

When I attempted to perform:

make clean
WHISPER_CUBLAS=1 make -j

I received this error

nvcc fatal   : Value 'native' is not defined for option 'gpu-architecture'
make: *** [Makefile:171: ggml-cuda.o] Error 1
make: *** Waiting for unfinished jobs....

nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2021 NVIDIA Corporation Built on Thu_Nov_18_09:45:30_PST_2021 Cuda compilation tools, release 11.5, V11.5.119 Build cuda_11.5.r11.5/compiler.30672275_0

I fixed this by editing the make file with:

NVCCFLAGS = --forward-unknown-to-host-compiler -arch=compute_75

The compute_75 was one of the listed available options when I did nvcc --help

--gpu-architecture <arch> (-arch) Specify the name of the class of NVIDIA 'virtual' GPU architecture for which the CUDA input files must be compiled. With the exception as described for the shorthand below, the architecture specified with this option must be a 'virtual' architecture (such as compute_50). Normally, this option alone does not trigger assembly of the generated PTX for a 'real' architecture (that is the role of nvcc option '--gpu-code', see below); rather, its purpose is to control preprocessing and compilation of the input to PTX. For convenience, in case of simple nvcc compilations, the following shorthand is supported. If no value for option '--gpu-code' is specified, then the value of this option defaults to the value of '--gpu-architecture'. In this situation, as only exception to the description above, the value specified for '--gpu-architecture' may be a 'real' architecture (such as a sm_50), in which case nvcc uses the specified 'real' architecture and its closest 'virtual' architecture as effective architecture values. For example, 'nvcc --gpu-architecture=sm_50' is equivalent to 'nvcc --gpu-architecture=compute_50 --gpu-code=sm_50,compute_50'. -arch=all build for all supported architectures (sm_*), and add PTX for the highest major architecture to the generated code. -arch=all-major build for just supported major versions (sm_*0), plus the earliest supported, and add PTX for the highest major architecture to the generated code. Note: -arch=all, -arch=all-major cannot be used with the -code option, but can be used with -gencode options Note: the values compute_30, compute_32, compute_35, compute_37, compute_50, sm_30, sm_32, sm_35, sm_37 and sm_50 are deprecated and may be removed in a future release. Allowed values for this option: 'all','all-major','compute_35','compute_37', 'compute_50','compute_52','compute_53','compute_60','compute_61','compute_62', 'compute_70','compute_72','compute_75','compute_80','compute_86','compute_87', 'lto_35','lto_37','lto_50','lto_52','lto_53','lto_60','lto_61','lto_62', 'lto_70','lto_72','lto_75','lto_80','lto_86','lto_87','sm_35','sm_37','sm_50', 'sm_52','sm_53','sm_60','sm_61','sm_62','sm_70','sm_72','sm_75','sm_80', 'sm_86','sm_87'.

People may have to see what is available for their specific gfx card and edit the Makefile accordingly. Although "all" may be an option that works instead of native.

[wesgould]

I just confirmed that that the "all" options works, so that might be a better default than "native" for nvidia GPUs at least.

[ururk]

Neither options (all or native) worked for me - I had to use sm_80.

[gardner]

The compute capability of a given card is available here: https://developer.nvidia.com/cuda-gpus

RTX 3060 is 8.6 so I passed -arch=sm_86

[kiang]

In my case, I have to choose one from the list provided by nvcc --list-gpu-arch

[gardner]

Please note: The cmake build works without modification.

[JunyaoC]

I just confirmed that that the "all" options works, so that might be a better default than "native" for nvidia GPUs at least.

Yes, this helped me out. Just in case someone fresh to the scene like me also run into this, i got this working by editing the Makefile, look for a line with NVCCFLAGS, change the -arch=native to all

[vadi2]

Just an fyi: if your nvcc doesn't show the compute availability that your video card supports, get the latest one from the nvidia website.

[fhirfly]

Same here, fixed with edit to Makefile line 228. NVCCFLAGS += -arch=compute_75, where compute_75 was one of the archs on my 4090. List of archs came from NVCCFLAGS += -arch=compute_75

[AlessandroBorges]

Just another workaround:
Do a make clean

make clean

and then run

make LLAMA_CUBLAS=1 CUDA_DOCKER_ARCH=all

The CUDA_DOCKER_ARCH can be:

• all -> for all supported CUDA GPU
• compute_61 -> for GTX 10XX series and TITAN X
• compute_75 -> for RTX 20xx series
• compute_86 -> for RTX 30xx series
• compute_89 -> for RTX 40xx series
• sm_XX -> same as compute_xx above

Check CUDA GPU capability here: https://developer.nvidia.com/cuda-gpus

[toncho11]

I made some progress compiling cuBLAS on Windows: ggerganov/llama.cpp#1431 (comment)