[Build] unable to build against `rel-1.20.0` due to code assertion

[mc-nv]

Describe the issue

I see following assertion in the code:

onnxruntime/onnxruntime/core/flatbuffers/schema/ort_training_checkpoint.fbs.h

Lines 11 to 13 in 03ea5dc

	static_assert(FLATBUFFERS_VERSION_MAJOR == 23 &&
	FLATBUFFERS_VERSION_MINOR == 5 &&
	FLATBUFFERS_VERSION_REVISION == 26,

This assertion has specific expectation in value, is it a straight dependency or can be omitted?

Build environment:

BUILDTOOLS_VERSION:17.10.35201.131 
CMAKE_VERSION:3.30.0 
CUDA_VERSION:12.6.2 
CUDNN_VERSION:9.5.1.17 
PYTHON_VERSION:3.12.3 
TENSORRT_VERSION:10.6.0.26 
VCPGK_VERSION:2024.03.19

Urgency

before release

Target platform

windows

Build script

onnxruntime/tools/ci_build/build.py `
      --cmake_generator "Visual Studio 17 2022" `
      --config Release `
      --cmake_extra_defines "CMAKE_CUDA_ARCHITECTURES=75;80;86;90" `
      --skip_submodule_sync `
      --parallel `
      --build_shared_lib `
      --compile_no_warning_as_error `
      --skip_tests `
      --update `
      --build `
      --build_dir /workspace/build `
      --use_cuda `
      --cuda_home ${env:CUDA_PATH} `
      --cudnn_home ${env:CUDA_PATH} `
      --use_tensorrt --tensorrt_home "/tensorrt" 

Error / output

C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\workspace\build\Release\onnxruntime_flatbuffers.vcxproj]

Visual Studio Version

No response

GCC / Compiler Version

No response

[mc-nv]

@snnn

[snnn]

If you have flatc in your PATH, you need to run:

flatc --version
python onnxruntime/core/flatbuffers/schema/compile_schema.py --flatc "$(which flatc)"
python onnxruntime/lora/adapter_format/compile_schema.py --flatc "$(which flatc)"

[mc-nv]

I do have it installed with VCPKG:

PS C:\workspace\build\Release> C:\vcpkg\vcpkg.exe list
b64:x64-windows                                   2.0.0.1#2           libb64 is a library of ANSI C routines for fast ...
boost-assert:x64-windows                          1.84.0              Boost assert module
boost-build:x64-windows                           1.84.0              Boost.Build
boost-config:x64-windows                          1.84.0              Boost config module
boost-container-hash:x64-windows                  1.84.0              Boost container_hash module
boost-container:x64-windows                       1.84.0              Boost container module
boost-core:x64-windows                            1.84.0              Boost core module
boost-describe:x64-windows                        1.84.0              Boost describe module
boost-interprocess:x64-windows                    1.84.0              Boost interprocess module
boost-intrusive:x64-windows                       1.84.0              Boost intrusive module
boost-modular-build-helper:x64-windows            1.84.0#3            Internal vcpkg port used to build Boost libraries
boost-move:x64-windows                            1.84.0              Boost move module
boost-mp11:x64-windows                            1.84.0              Boost mp11 module
boost-predef:x64-windows                          1.84.0              Boost predef module
boost-stacktrace:x64-windows                      1.84.0              Boost stacktrace module
boost-static-assert:x64-windows                   1.84.0              Boost static_assert module
boost-throw-exception:x64-windows                 1.84.0              Boost throw_exception module
boost-type-traits:x64-windows                     1.84.0              Boost type_traits module
boost-uninstall:x64-windows                       1.84.0              Internal vcpkg port used to uninstall Boost
boost-unordered:x64-windows                       1.84.0              Boost unordered module
boost-vcpkg-helpers:x64-windows                   1.84.0              Internal vcpkg port used to modularize Boost
boost-winapi:x64-windows                          1.84.0              Boost winapi module
flatbuffers:x64-windows                           23.5.26#1           Memory Efficient Serialization Library
openssl-windows:x64-windows                       deprecated          Deprecated OpenSSL port. Use port openssl instead.
openssl:x64-windows                               3.2.1               OpenSSL is an open source project that provides ...
pthread:x64-windows                               3.0.0#2             empty package, linking to other port
pthreads:x64-windows                              3.0.0#14            Meta-package that provides PThreads4W on Windows...
rapidjson:x64-windows                             2023-07-17#1        A fast JSON parser/generator for C++ with both S...
vcpkg-cmake-config:x64-windows                    2022-02-06#1
vcpkg-cmake-get-vars:x64-windows                  2023-12-31
vcpkg-cmake:x64-windows                           2023-05-04
zlib:x64-windows                                  1.3.1               A compression library

In the same time I'm getting error:

C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\workspace\build\Release\onnx_test_runner_common.vcxproj]
  flatbuffers_utils.cc
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\workspace\build\Release\onnxruntime_flatbuffers.vcxproj]
  (compiling source file '../../onnxruntime/onnxruntime/core/flatbuffers/flatbuffers_utils.cc')
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\workspace\build\Release\onnxruntime_framework.vcxproj]
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\workspace\build\Release\onnxruntime_framework.vcxproj]
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\workspace\build\Release\onnxruntime_graph.vcxproj]

[mc-nv]

Wouldn't be better to validate the presence of the package with specific version during the generation stage?
That assertion within code looks like "hardcode": https://github.com/microsoft/onnxruntime/blob/rel-1.20.0/onnxruntime/core/flatbuffers/schema/ort_training_checkpoint.fbs.h#L11-L13
While CMake configuration allows installation of the versions higher then used with assertion:
https://github.com/microsoft/onnxruntime/blob/rel-1.20.0/cmake/vcpkg.json#L23-L29

[mc-nv]

I'm keep getting issue with assertion in code now with version rel-1.20.1:

C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\wor
kspace\build\Release\onnxruntime_graph.vcxproj]
  (compiling source file '../../onnxruntime/onnxruntime/core/graph/graph.cc')

  onnxruntime_mlas.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mlas.lib
  onnxruntime_optimizer.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_optimizer.lib
  onnxruntime_providers.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_providers.lib
  onnxruntime_test_utils.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_test_utils.lib
  onnxruntime_util.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_util.lib
  win_getopt_wide.vcxproj -> C:\workspace\build\Release\Debug\win_getopt_wide.lib
  onnxruntime_mocked_allocator.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mocked_allocator.lib
  onnxruntime_mlas_q4dq.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mlas_q4dq.exe
  onnxruntime_mlas_test.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mlas_test.exe
PS C:\workspace\build\Release> C:\vcpkg\vcpkg.exe list
b64:x64-windows                                   2.0.0.1#2           libb64 is a library of ANSI C routines for fast ...
boost-assert:x64-windows                          1.84.0              Boost assert module
boost-build:x64-windows                           1.84.0              Boost.Build
boost-config:x64-windows                          1.84.0              Boost config module
boost-container-hash:x64-windows                  1.84.0              Boost container_hash module
boost-container:x64-windows                       1.84.0              Boost container module
boost-core:x64-windows                            1.84.0              Boost core module
boost-describe:x64-windows                        1.84.0              Boost describe module
boost-interprocess:x64-windows                    1.84.0              Boost interprocess module
boost-intrusive:x64-windows                       1.84.0              Boost intrusive module
boost-modular-build-helper:x64-windows            1.84.0#3            Internal vcpkg port used to build Boost libraries
boost-move:x64-windows                            1.84.0              Boost move module
boost-mp11:x64-windows                            1.84.0              Boost mp11 module
boost-predef:x64-windows                          1.84.0              Boost predef module
boost-stacktrace:x64-windows                      1.84.0              Boost stacktrace module
boost-static-assert:x64-windows                   1.84.0              Boost static_assert module
boost-throw-exception:x64-windows                 1.84.0              Boost throw_exception module
boost-type-traits:x64-windows                     1.84.0              Boost type_traits module
boost-uninstall:x64-windows                       1.84.0              Internal vcpkg port used to uninstall Boost
boost-unordered:x64-windows                       1.84.0              Boost unordered module
boost-vcpkg-helpers:x64-windows                   1.84.0              Internal vcpkg port used to modularize Boost
boost-winapi:x64-windows                          1.84.0              Boost winapi module
flatbuffers:x64-windows                           23.5.26#1           Memory Efficient Serialization Library

[mc-nv]

@snnn adding more context as per your request:

PS C:\workspace\build\Release> python.exe C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\compile_schema.py --flatc (get-command flatc).path
PS C:\workspace\build\Release> python.exe C:\workspace\onnxruntime\onnxruntime\lora\adapter_format\compile_schema.py --flatc (get-command flatc).path
PS C:\workspace\build\Release> flatc --version
flatc version 23.5.26

Problem remain the same.

PS C:\workspace\build\Release> cmake --build .
MSBuild version 17.12.6+db5f6012c for .NET Framework

  Checking File Globs
  custom_op_get_const_input_test_library.vcxproj -> C:\workspace\build\Release\Debug\custom_op_get_const_input_test_library.dll
  custom_op_invalid_library.vcxproj -> C:\workspace\build\Release\Debug\custom_op_invalid_library.dll
  custom_op_local_function.vcxproj -> C:\workspace\build\Release\Debug\custom_op_local_function.dll
  onnx_proto.vcxproj -> C:\workspace\build\Release\_deps\onnx-build\Debug\onnx_proto.lib
  onnxruntime_providers_shared.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_providers_shared.dll
  onnxruntime_providers_cuda.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_providers_cuda.dll
  onnxruntime_providers_tensorrt.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_providers_tensorrt.dll
  onnx.vcxproj -> C:\workspace\build\Release\_deps\onnx-build\Debug\onnx.lib
  test_execution_provider.vcxproj -> C:\workspace\build\Release\Debug\test_execution_provider.dll
  custom_op_library.vcxproj -> C:\workspace\build\Release\Debug\custom_op_library.dll
  gmock.vcxproj -> C:\workspace\build\Release\lib\Debug\gmock.lib
  gmock_main.vcxproj -> C:\workspace\build\Release\lib\Debug\gmock_main.lib
  gtest.vcxproj -> C:\workspace\build\Release\lib\Debug\gtest.lib
  gtest_main.vcxproj -> C:\workspace\build\Release\lib\Debug\gtest_main.lib
  onnx_test_data_proto.vcxproj -> C:\workspace\build\Release\Debug\onnx_test_data_proto.lib
  onnx_model_info.cc
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\wor
kspace\build\Release\onnx_test_runner_common.vcxproj]
  (compiling source file '../../onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc')

  onnxruntime_common.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_common.lib
  flatbuffers_utils.cc
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\wor
kspace\build\Release\onnxruntime_flatbuffers.vcxproj]
  (compiling source file '../../onnxruntime/onnxruntime/core/flatbuffers/flatbuffers_utils.cc')

  kernel_type_str_resolver.cc
  kernel_type_str_resolver_utils.cc
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\wor
kspace\build\Release\onnxruntime_framework.vcxproj]
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\wor
kspace\build\Release\onnxruntime_framework.vcxproj]
  (compiling source file '../../onnxruntime/onnxruntime/core/framework/kernel_type_str_resolver_utils.cc')

  (compiling source file '../../onnxruntime/onnxruntime/core/framework/kernel_type_str_resolver.cc')

  graph.cc
  graph_flatbuffers_utils.cc
C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\wor
kspace\build\Release\onnxruntime_graph.vcxproj]
  (compiling source file '../../onnxruntime/onnxruntime/core/graph/graph_flatbuffers_utils.cc')

C:\workspace\onnxruntime\onnxruntime\core\flatbuffers\schema\ort.fbs.h(12,46): error C2338: static_assert failed: 'Non-compatible flatbuffers version included' [C:\wor
kspace\build\Release\onnxruntime_graph.vcxproj]
  (compiling source file '../../onnxruntime/onnxruntime/core/graph/graph.cc')

  onnxruntime_mlas.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mlas.lib
  onnxruntime_optimizer.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_optimizer.lib
  onnxruntime_providers.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_providers.lib
  onnxruntime_test_utils.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_test_utils.lib
  onnxruntime_util.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_util.lib
  win_getopt_wide.vcxproj -> C:\workspace\build\Release\Debug\win_getopt_wide.lib
  onnxruntime_mocked_allocator.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mocked_allocator.lib
  onnxruntime_mlas_q4dq.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mlas_q4dq.exe
  onnxruntime_mlas_test.vcxproj -> C:\workspace\build\Release\Debug\onnxruntime_mlas_test.exe
PS C:\workspace\build\Release>

Can you give me a clue how flatc was distributed in your build environment?

[mc-nv]

cc: @pranavsharma

[mc-nv]

cc: @metang

[mc-nv]

Isn't this value must be "version=": "23.5.26"
https://github.com/microsoft/onnxruntime/blob/rel-1.20.1/cmake/vcpkg.json#L25

for scenario:

onnxruntime/onnxruntime/core/flatbuffers/schema/ort_training_checkpoint.fbs.h

Lines 11 to 13 in 03ea5dc

	static_assert(FLATBUFFERS_VERSION_MAJOR == 23 &&
	FLATBUFFERS_VERSION_MINOR == 5 &&
	FLATBUFFERS_VERSION_REVISION == 26,