Can not get USE_MIMALLOC activated in Windows

[AndreyOrb]

Describe the issue

The CPUAllocator does not use the mimalloc, although it was build:

I even tried to add USE_MIMALLOC into project's Preprocessor Definitions

and also into the code itself as
#define USE_MIMALLOC 1

To reproduce

Script I used to compile the ORT:
.\build.bat --config Debug --build_shared_lib --parallel --use_cuda --cuda_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8" --cudnn_home "C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\cudnn-windows-x86_64-8.9.0.131_cuda11-archive" --cuda_version 11.8 --use_tensorrt --tensorrt_home "C:\Program Files\NVIDIA GPU Computing Toolkit\TensorRT\TensorRT-10.9.0.34.Windows.win10.cuda-11.8\TensorRT-10.9.0.34" --use_tensorrt_oss_parser --cmake_generator "Visual Studio 16 2019" --compile_no_warning_as_error --cmake_path E:\3rdParties\cmake-4.0.3\build\bin\Release\cmake.exe --skip_tests --enable_cuda_line_info --use_mimalloc --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF --skip_tests --cmake_extra_defines onnxruntime_ENABLE_MEMLEAK_CHECKER=OFF

Urgency

Not urgent

Platform

Windows

OS Version

11

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.22.0

ONNX Runtime API

C++

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[skottmckay]

Works for me. CPU build with --use_mimalloc.

The framework project has USE_MIMALLOC set as expected.

I would suggest you try a clean build. If the preprocessor definition is set, as it appears to be in your screen shot, you should see the same behavior.

[AndreyOrb]

I checked the onnxruntime_project Preprocessor Definitions, and the USE_MIMALLOC is not there.
I deleted the build\Windows\Debug\ folder and ran the build.bat again.
Unfortunately, the newly generated onnxruntime_project still does not have the USE_MIMALLOC definition.

No warning messages during the compile time pointing why the USE_MIMALLOC was not used.

I will probably have to debug the Cmake to understand why USE_MIMALLOC is not propagated into the project,

USE_MIMALLOC is activated in build\Windows\Debug\CMakeCache.txt:
//Override new/delete and arena allocator with mimalloc
onnxruntime_USE_MIMALLOC:INTERNAL=ON

Logs that have mimalloc:
2025-06-30 08:43:42,281 build [DEBUG] - Command line arguments:
--build_dir 'E:\3rdParties\onnxruntime_v1.22.0\build\Windows' --config Debug --build_shared_lib --parallel --use_cuda --cuda_home 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8' --cudnn_home 'C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\cudnn-windows-x86_64-8.9.0.131_cuda11-archive' --cuda_version 11.8 --use_tensorrt --tensorrt_home 'C:\Program Files\NVIDIA GPU Computing Toolkit\TensorRT\TensorRT-10.9.0.34.Windows.win10.cuda-11.8\TensorRT-10.9.0.34' --use_tensorrt_oss_parser --cmake_generator 'Visual Studio 16 2019' --compile_no_warning_as_error --cmake_path 'E:\3rdParties\cmake-4.0.3\build\bin\Release\cmake.exe' --skip_tests --enable_cuda_line_info --use_mimalloc --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF --skip_tests --cmake_extra_defines onnxruntime_ENABLE_MEMLEAK_CHECKER=OFF

2025-06-30 08:43:47,162 build [INFO] - E:\3rdParties\cmake-4.0.3\build\bin\Release\cmake.exe E:\3rdParties\onnxruntime_v1.22.0\cmake --compile-no-warning-as-error -Donnxruntime_USE_DML=OFF -Donnxruntime_USE_WINML=OFF -Donnxruntime_USE_TELEMETRY=OFF -Donnxruntime_ENABLE_PIX_FOR_WEBGPU_EP=OFF -Donnxruntime_ENABLE_MEMLEAK_CHECKER=ON -Donnxruntime_RUN_ONNX_TESTS=OFF -Donnxruntime_GENERATE_TEST_REPORTS=ON -DPython_EXECUTABLE=C:\Program Files\Python310\python.exe -Donnxruntime_USE_VCPKG=OFF -Donnxruntime_USE_MIMALLOC=ON -Donnxruntime_ENABLE_PYTHON=OFF -Donnxruntime_BUILD_CSHARP=OFF -Donnxruntime_BUILD_JAVA=OFF -Donnxruntime_BUILD_NODEJS=OFF -Donnxruntime_BUILD_OBJC=OFF -Donnxruntime_BUILD_SHARED_LIB=ON -Donnxruntime_BUILD_APPLE_FRAMEWORK=OFF -Donnxruntime_USE_DNNL=OFF -Donnxruntime_USE_NNAPI_BUILTIN=OFF -Donnxruntime_USE_VSINPU=OFF -Donnxruntime_USE_RKNPU=OFF -Donnxruntime_ENABLE_MICROSOFT_INTERNAL=OFF -Donnxruntime_USE_VITISAI=OFF -Donnxruntime_USE_TENSORRT=ON -Donnxruntime_USE_NV=OFF -Donnxruntime_USE_TENSORRT_BUILTIN_PARSER=OFF -Donnxruntime_USE_TENSORRT_INTERFACE=OFF -Donnxruntime_USE_CUDA_INTERFACE=OFF -Donnxruntime_USE_NV_INTERFACE=OFF -Donnxruntime_USE_OPENVINO_INTERFACE=OFF -Donnxruntime_USE_VITISAI_INTERFACE=OFF -Donnxruntime_USE_QNN_INTERFACE=OFF -Donnxruntime_USE_MIGRAPHX=OFF -Donnxruntime_DISABLE_CONTRIB_OPS=OFF -Donnxruntime_DISABLE_ML_OPS=OFF -Donnxruntime_DISABLE_RTTI=OFF -Donnxruntime_DISABLE_EXCEPTIONS=OFF -Donnxruntime_MINIMAL_BUILD=OFF -Donnxruntime_EXTENDED_MINIMAL_BUILD=OFF -Donnxruntime_MINIMAL_BUILD_CUSTOM_OPS=OFF -Donnxruntime_REDUCED_OPS_BUILD=OFF -Donnxruntime_BUILD_MS_EXPERIMENTAL_OPS=OFF -Donnxruntime_ENABLE_LTO=OFF -Donnxruntime_USE_ACL=OFF -Donnxruntime_USE_ARMNN=OFF -Donnxruntime_ARMNN_RELU_USE_CPU=ON -Donnxruntime_ARMNN_BN_USE_CPU=ON -Donnxruntime_USE_JSEP=OFF -Donnxruntime_USE_WEBGPU=OFF -Donnxruntime_USE_EXTERNAL_DAWN=OFF -Donnxruntime_ENABLE_NVTX_PROFILE=OFF -Donnxruntime_ENABLE_TRAINING=OFF -Donnxruntime_ENABLE_TRAINING_OPS=OFF -Donnxruntime_ENABLE_TRAINING_APIS=OFF -Donnxruntime_ENABLE_CPU_FP16_OPS=OFF -Donnxruntime_USE_NCCL=OFF -Donnxruntime_BUILD_BENCHMARKS=OFF -Donnxruntime_GCOV_COVERAGE=OFF -Donnxruntime_ENABLE_MEMORY_PROFILE=OFF -Donnxruntime_ENABLE_CUDA_LINE_NUMBER_INFO=ON -Donnxruntime_USE_CUDA_NHWC_OPS=ON -Donnxruntime_BUILD_WEBASSEMBLY_STATIC_LIB=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_EXCEPTION_CATCHING=ON -Donnxruntime_ENABLE_WEBASSEMBLY_API_EXCEPTION_CATCHING=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_EXCEPTION_THROWING=ON -Donnxruntime_WEBASSEMBLY_RUN_TESTS_IN_BROWSER=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_THREADS=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_DEBUG_INFO=OFF -Donnxruntime_ENABLE_WEBASSEMBLY_PROFILING=OFF -Donnxruntime_ENABLE_LAZY_TENSOR=OFF -Donnxruntime_ENABLE_CUDA_PROFILING=OFF -Donnxruntime_USE_XNNPACK=OFF -Donnxruntime_USE_WEBNN=OFF -Donnxruntime_USE_CANN=OFF -Donnxruntime_USE_TRITON_KERNEL=OFF -Donnxruntime_DISABLE_FLOAT8_TYPES=OFF -Donnxruntime_DISABLE_SPARSE_TENSORS=OFF -Donnxruntime_DISABLE_OPTIONAL_TYPE=OFF -Donnxruntime_CUDA_MINIMAL=OFF -Donnxruntime_TENSORRT_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\TensorRT\TensorRT-10.9.0.34.Windows.win10.cuda-11.8\TensorRT-10.9.0.34 -Donnxruntime_NVCC_THREADS=2 -DCMAKE_CUDA_COMPILER=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8/bin/nvcc -Donnxruntime_USE_KLEIDIAI=ON -DCMAKE_VS_GLOBALS=UseMultiToolTask=true;EnforceProcessCountAcrossBuilds=true -Donnxruntime_BUILD_UNIT_TESTS=OFF -Donnxruntime_ENABLE_MEMLEAK_CHECKER=OFF -Donnxruntime_USE_CUDA=ON -Donnxruntime_CUDA_VERSION=11.8 -Donnxruntime_CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 -Donnxruntime_CUDNN_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\cudnn-windows-x86_64-8.9.0.131_cuda11-archive -DONNX_USE_MSVC_STATIC_RUNTIME=OFF -Dprotobuf_MSVC_STATIC_RUNTIME=OFF -DABSL_MSVC_STATIC_RUNTIME=OFF -Dgtest_force_shared_crt=ON -DCMAKE_MSVC_DEBUG_INFORMATION_FORMAT=ProgramDatabase -A x64 -T host=x64,cuda=11.8 -G Visual Studio 16 2019 -DCMAKE_C_FLAGS=/MP -DCMAKE_CXX_FLAGS=/MP -DCMAKE_CUDA_FLAGS_INIT=-allow-unsupported-compiler -DCMAKE_BUILD_TYPE=Debug

2025-06-30 09:22:12,292 build [INFO] - Build complete
Namespace(build_dir='E:\3rdParties\onnxruntime_v1.22.0\\build\Windows', config=['Debug'], update=True, build=True, clean=False, parallel=0, target=None, compile_no_warning_as_error=True, build_shared_lib=True, build_apple_framework=False, enable_lto=False, use_cache=False, use_binskim_compliant_compile_flags=False, cmake_extra_defines=[['onnxruntime_BUILD_UNIT_TESTS=OFF'], ['onnxruntime_ENABLE_MEMLEAK_CHECKER=OFF']], cmake_path='E:\3rdParties\cmake-4.0.3\build\bin\Release\cmake.exe', cmake_generator='Visual Studio 16 2019', use_vcpkg=False, use_vcpkg_ms_internal_asset_cache=False, skip_submodule_sync=False, test=False, skip_tests=True, ctest_path='ctest', enable_onnx_tests=False, path_to_protoc_exe=None, fuzz_testing=False, enable_symbolic_shape_infer_tests=False, skip_onnx_tests=False, skip_winml_tests=False, skip_nodejs_tests=False, test_all_timeout='10800', enable_transformers_tool_test=False, build_micro_benchmarks=False, code_coverage=False, enable_training=False, enable_training_apis=False, enable_training_ops=False, enable_nccl=False, nccl_home=None, enable_memory_profile=False, enable_address_sanitizer=False, gen_doc=None, rv64=False, riscv_toolchain_root='', riscv_qemu_path='', android=False, android_abi='arm64-v8a', android_api=27, android_sdk_path='', android_ndk_path='', android_cpp_shared=False, android_run_emulator=False, build_wasm=False, build_wasm_static_lib=False, emsdk_version='4.0.8', enable_wasm_simd=False, enable_wasm_relaxed_simd=False, enable_wasm_threads=False, disable_wasm_exception_catching=False, enable_wasm_api_exception_catching=False, enable_wasm_exception_throwing_override=True, wasm_run_tests_in_browser=False, enable_wasm_profiling=False, enable_wasm_debug_info=False, wasm_malloc=None, emscripten_settings=None, use_full_protobuf=False, use_mimalloc=True, external_graph_transformer_path=None, use_extensions=False, extensions_overridden_path=None, minimal_build=None, include_ops_by_config=None, enable_reduced_operator_type_support=False, disable_contrib_ops=False, disable_ml_ops=False, disable_rtti=False, disable_types=[], disable_exceptions=False, enable_pybind=False, build_wheel=False, wheel_name_suffix=None, skip_keras_test=False, build_csharp=False, build_nuget=False, msbuild_extra_options=None, build_java=False, build_nodejs=False, build_objc=False, use_cuda=True, cuda_version='11.8', cuda_home='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8', cudnn_home='C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\cudnn-windows-x86_64-8.9.0.131_cuda11-archive', enable_cuda_line_info=True, enable_cuda_nhwc_ops=False, disable_cuda_nhwc_ops=False, enable_cuda_minimal_build=False, nvcc_threads=-1, enable_nvtx_profile=False, enable_cuda_profiling=False, use_dnnl=False, dnnl_gpu_runtime='', dnnl_opencl_root='', dnnl_aarch64_runtime='', dnnl_acl_root='', use_openvino=None, use_tensorrt=True, use_tensorrt_builtin_parser=True, use_tensorrt_oss_parser=True, tensorrt_home='C:\Program Files\NVIDIA GPU Computing Toolkit\TensorRT\TensorRT-10.9.0.34.Windows.win10.cuda-11.8\TensorRT-10.9.0.34', use_nv_tensorrt_rtx=False, use_dml=False, dml_path='', dml_external_project=False, use_nnapi=False, nnapi_min_api=None, use_coreml=False, use_qnn=None, qnn_home=None, use_snpe=False, snpe_root=None, use_vitisai=False, use_armnn=False, armnn_relu=False, armnn_bn=False, armnn_home=None, armnn_libs=None, use_acl=False, acl_home=None, acl_libs=None, no_kleidiai=False, use_rknpu=False, use_cann=False, cann_home=None, use_migraphx=False, migraphx_home=None, use_rocm=False, rocm_version=None, rocm_home=None, use_webnn=False, use_jsep=False, use_webgpu=False, use_external_dawn=False, use_xnnpack=False, use_vsinpu=False, use_azure=False, enable_lazy_tensor=False, ms_experimental=False, enable_msinternal=False, use_triton_kernel=False, use_lock_free_queue=False, enable_generic_interface=False, msvc_toolset=None, windows_sdk_version=None, enable_msvc_static_runtime=False, use_telemetry=False, caller_framework=None, x86=False, arm=False, arm64=False, arm64ec=False, buildasx=False, disable_memleak_checker=False, enable_pix_capture=False, enable_wcos=False, use_gdk=False, gdk_edition='', gdk_platform='Scarlett', use_winml=False, winml_root_namespace_override=None)

[AndreyOrb]

This is what I have in CMakeLists.txt:

cmake_dependent_option(onnxruntime_USE_MIMALLOC "Override new/delete and arena allocator with mimalloc" OFF "WIN32;NOT onnxruntime_USE_CUDA;NOT onnxruntime_USE_OPENVINO" OFF)

Does it mean that I can not use mimalloc with CUDA?

@snnn I see that you made this commit:
04900f9#diff-6857e85a31ef258bd57de3cd5fb1e0c31239cd93e0a16fba07dba83864bec5b6R104

What is the motivation behind the "WIN32;NOT onnxruntime_USE_CUDA;NOT onnxruntime_USE_OPENVINO" ?
This conversation?
https://github.com/microsoft/onnxruntime/pull/2071/files#r335210443

[AndreyOrb]

If indeed the mimalloc is not compatible with CUDA EP, there are 2 problems:

1. I pass the --use_mimalloc as a flag, there is no indication that it was voided because of --use_cuda
2. The documentation does not mention this limitation: https://onnxruntime.ai/docs/performance/tune-performance/memory.html

[AndreyOrb]

I edited the CMakeLists.txt like this:

if (WIN32)
  message(STATUS "Entered WIN32 block in CMakeLists.txt")

  if (onnxruntime_USE_MIMALLOC)
    list(APPEND onnxruntime_EXTERNAL_LIBRARIES mimalloc-static)
    message(STATUS "mimalloc-static appended to onnxruntime_EXTERNAL_LIBRARIES")
  else()
    message(STATUS "onnxruntime_USE_MIMALLOC is OFF")
  endif()

The build log:
-- Entered WIN32 block in CMakeLists.txt
-- onnxruntime_USE_MIMALLOC is OFF

[snnn]

I think I disabled it because I couldn't get the code compiled, and we do not have any pipeline to test the build config combination.

If you remove the restrictions:

cmake_dependent_option(onnxruntime_USE_MIMALLOC "Override new/delete and arena allocator with mimalloc" OFF "WIN32;NOT onnxruntime_USE_CUDA;NOT onnxruntime_USE_OPENVINO" OFF)

Will it build and pass all tests?