CUDA compilation fails on Blackwell (sm_120) with MXFP4: "Instruction 'mma with block scale' not supported"

[karolysovagoozeki]

Git commit

git rev-parse HEAD
d5dfc33

Operating systems

Linux

GGML backends

CUDA

Problem description & steps to reproduce

Environment
GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (compute capability 12.0)

CUDA Toolkit: 13.1.115 (installed to /usr/local/cuda-13.1)

OS: Ubuntu 24.04

llama.cpp commit: Latest master (as of 2026-02-16)

CMake version: 3.28.3

Compiler: GCC 13.3.0

Problem
Building llama.cpp with CUDA support fails during PTX assembly when compiling MXFP4 template instances for sm_120 (Blackwell architecture). The build process encounters hundreds of errors from ptxas indicating that MXFP4-related instructions are not supported on the sm_120 target.

Build Configuration

cmake llama.cpp -B llama.cpp/build
-DBUILD_SHARED_LIBS=OFF
-DGGML_CUDA=ON
-DLLAMA_CURL=ON
-DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
-DCMAKE_CUDA_ARCHITECTURES=120

Error Output

ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error : Instruction 'mma with block scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error : Feature '.kind::mxf4' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error : Feature '.block_scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error : Feature '.scale_vec::2X' not supported on .target 'sm_120'
...
ptxas fatal : Ptx assembly aborted due to errors
gmake[3]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:1547: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/mmq-instance-mxfp4.cu.o] Error 255
Additional Details
The error occurs specifically in mmq-instance-mxfp4.cu compilation

Using -DGGML_CUDA_MXFP4=OFF does not prevent the compilation of MXFP4 template instances

The MXFP4 code appears to be compiled unconditionally for sm_120 targets

ptxas --version confirms CUDA 13.1.115 is being used

nvidia-smi confirms the GPU is properly detected with compute capability 12.0

Workaround
Rolling back to commit caf5681 (pre-MXFP4) allows successful compilation on sm_120, but this version lacks support for newer model architectures like mistral3 (Devstral models).

Expected Behavior
Either:

MXFP4 code should compile successfully for sm_120 (Blackwell) when using CUDA 13.1+, or

There should be a working CMake flag to disable MXFP4 compilation entirely for architectures where it's not yet supported, or

MXFP4 template instantiation should be automatically skipped for sm_120 until proper support is available

Use Case
We're trying to run Devstral-Small-2-24B-Instruct models on Blackwell hardware with CUDA acceleration. The current state makes it impossible to build a version that supports both the new architecture and the new GPU.

Question
Is MXFP4 + Blackwell (sm_120) support planned? If so, what's the timeline? Should we expect this to work with CUDA 13.1, or is a newer toolkit version required?

First Bad Commit

No response

Compile command

cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-gguf-split

Relevant log output

ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error   : Instruction 'mma with block scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error   : Feature '.kind::mxf4' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error   : Feature '.block_scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_00021d12_00000000-7_mmq-instance-mxfp4.ptx, line 110582; error   : Feature '.scale_vec::2X' not supported on .target 'sm_120'
...
ptxas fatal   : Ptx assembly aborted due to errors
gmake[3]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:1547: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/template-instances/mmq-instance-mxfp4.cu.o] Error 255

[am17an]

I'm not able to replicate this, arch 120 is replaced by 120a-real automatically in CMakeLists.txt, can you share the output of the CMake command?

[ORippler]

We're trying to run Devstral-Small-2-24B-Instruct models on Blackwell hardware with CUDA acceleration. The current state makes it impossible to build a version that supports both the new architecture and the new GPU.

Yeah afaik this is not possible atm due to aforementioned conversion from 120 to 120a-real (the ptx instruction for mxfp4-hw-acceleration are available only for 120f, not for 120. So sm 120 is expected to fail)

[am17an]

Yeah afaik this is not possible atm due to aforementioned conversion from 120 to 120a-real (the ptx instruction for mxfp4-hw-acceleration are available only for 120f, not for 120. So sm 120 is expected to fail)

It is not possible atm to compile with 120, CMakeLists.txt will make the adjustment unconditionally

[ORippler]

Okay to reword: It is impossible to compile forward-compatible 120-virtual builds of the CUDA-backend at the moment, meaning that everything build post-mxfp4-fixes with "native" will not run on vera rubin+GPUs. It is possible to build for BW + mxfp4 acceleration, and it should be possible to rely on forward-compatibility of 89-virtual for vera rubin+GPUs in current builds

[mtcl]

i am facing the same issue...

cmake -B build -DGGML_CUDA=ON -DGGML_BLAS=OFF
cmake --build build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split

and after a little, i get this error, like 100s of lines of these... is there any solution to this?

ptxas /tmp/tmpxft_003aed92_00000000-7_mmq-instance-mxfp4.compute_120.ptx, line 38007; error   : Feature '.kind::mxf4' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_003aed92_00000000-7_mmq-instance-mxfp4.compute_120.ptx, line 38007; error   : Feature '.block_scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_003aed92_00000000-7_mmq-instance-mxfp4.compute_120.ptx, line 38007; error   : Feature '.scale_vec::2X' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_003aed92_00000000-7_mmq-instance-mxfp4.compute_120.ptx, line 38021; error   : Instruction 'mma with block scale' not supported on .target 'sm_120'
ptxas /tmp/tmpxft_003aed92_00000000-7_mmq-instance-mxfp4.compute_120.ptx, line 38021; error   : Feature '.kind::mxf4' not supported on .target 'sm_120'

[ORippler]

Please post the output of your cmake configure step, especially the line using CMAKE_CUDA_ARCHITECTURES=...

-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Replacing 120-real in CMAKE_CUDA_ARCHITECTURES_NATIVE with 120a-real
-- Using CMAKE_CUDA_ARCHITECTURES=120a-real CMAKE_CUDA_ARCHITECTURES_NATIVE=120a-real
-- CUDA host compiler is GNU 13.3.0
-- Including CUDA backend

On SM-120 systems, it's supposed to look like this (i.e. using 120a and not 120)

[loscrossos]

i am also getting lots of this on windows:

tmpxft_00003b18_00000000-8_mmq-instance-mxfp4.ptx, line 107648; error : Featur e '.kind::mxf4' not supported on .target 'sm_120' [D:\temp\winllamacpp_source\build\ggml\src\ggml-cu da\ggml-cuda.vcxproj]

only these get built

17/02/2026  17:54           542.720 ggml-base.dll
17/02/2026  17:54           832.000 ggml-cpu.dll
17/02/2026  18:14            26.624 llama-gemma3-cli.exe
17/02/2026  18:14            26.624 llama-llava-cli.exe
17/02/2026  18:14            26.624 llama-minicpmv-cli.exe
17/02/2026  18:14            26.624 llama-qwen2vl-cli.exe

cmake -B build -DGGML_CUDA=ON 
c:\winllamacpp_sm120>cmake -B build -DGGML_CUDA=ON
-- Building for: Visual Studio 17 2022
-- The C compiler identification is MSVC 19.44.35222.0
-- The CXX compiler identification is MSVC 19.44.35222.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE=
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.53.0.windows.1")
-- The ASM compiler identification is MSVC
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- CMAKE_GENERATOR_PLATFORM:
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- x86 detected
-- Performing Test HAS_AVX_1
-- Performing Test HAS_AVX_1 - Success
-- Performing Test HAS_AVX2_1
-- Performing Test HAS_AVX2_1 - Success
-- Performing Test HAS_FMA_1
-- Performing Test HAS_FMA_1 - Success
-- Performing Test HAS_AVX512_1
-- Performing Test HAS_AVX512_1 - Failed
-- Performing Test HAS_AVX512_2
-- Performing Test HAS_AVX512_2 - Failed
-- Adding CPU backend variant ggml-cpu: /arch:AVX2 GGML_AVX2;GGML_FMA;GGML_F16C
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.1/include;C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.1/include/cccl (found version "13.1.115")
-- CUDA Toolkit found
-- The CUDA compiler identification is NVIDIA 13.1.115 with host compiler MSVC 19.44.35222.0
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v13.1/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Using CMAKE_CUDA_ARCHITECTURES=native CMAKE_CUDA_ARCHITECTURES_NATIVE=No CUDA devices found.-real
-- Including CUDA backend
-- ggml version: 0.9.7
-- ggml commit:  afa6bfe4f
-- Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY OPENSSL_INCLUDE_DIR)
CMake Warning at vendor/cpp-httplib/CMakeLists.txt:150 (message):
  OpenSSL not found, HTTPS support disabled


-- Generating embedded license file for target: common
-- Configuring done (16.5s)
-- Generating done (0.5s)
-- Build files have been written to: c:/winllamacpp_sm120/build

it seems the card is not properly recognized no matter what is set.. even cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120a -DCMAKE_CUDA_ARCHITECTURES_NATIVE=120a-real does has the same "not found" result.

at most i get this setting things like above:

-- Using CMAKE_CUDA_ARCHITECTURES=120a CMAKE_CUDA_ARCHITECTURES_NATIVE=No CUDA devices found.-real

[loscrossos]

@am17an it seems the PR #18457 didnt work?

my 5060ti is not being revcognized. Build does not work for me

[am17an]

From your logs it looks like Cmake isn't able to find any device?

[loscrossos]

seems like it yes...
yet my card is there and working.

>nvidia-smi
Fri Feb 20 12:42:05 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 576.88                 Driver Version: 576.88         CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5060 Ti   WDDM  |   00000000:01:00.0  On |                  N/A |
|  0%   34C    P8              3W /  180W |     380MiB /  16311MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

hm.. the driver is a bit oldand says it supports CUDA 12.9. meanwhile i just upgraded to CUDA 13.1
ill update drivers and report back. (i know im sounding like an LLM here lol)

on linux (same machine) it works as expected. so not sure what mechanism is used on windows.

`$ cmake -B build -DGGML_CUDA=ON 
-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE=Release
-- Found Git: /usr/bin/git (found version "2.43.0") 
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native 
-- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include (found version "13.1.115") 
-- CUDA Toolkit found
-- The CUDA compiler identification is NVIDIA 13.1.115
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Replacing 120-real in CMAKE_CUDA_ARCHITECTURES_NATIVE with 120a-real
-- Using CMAKE_CUDA_ARCHITECTURES=120a-real CMAKE_CUDA_ARCHITECTURES_NATIVE=120a-real
-- CUDA host compiler is GNU 13.3.0
-- Including CUDA backend
-- ggml version: 0.9.7
-- ggml commit:  b908baf18
-- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "3.0.13")  
-- Performing Test OPENSSL_VERSION_SUPPORTED
-- Performing Test OPENSSL_VERSION_SUPPORTED - Success
-- OpenSSL found: 3.0.13
-- Generating embedded license file for target: common
-- Configuring done (4.8s)
-- Generating done (0.2s)