CMake __BUILD_FOR_R fails to find R source files

[david-cortes]

The CMake build has an option to build for R which was working some versions ago, at least under windows:

LightGBM/CMakeLists.txt

Line 23 in 6e78e69

option(__BUILD_FOR_R "Set to ON if building lib_lightgbm for use with the R package" OFF)

Building with this option by calling cmake directly with -D__BUILD_FOR_R=1 -DCMAKE_R_VERSION=<version> without using the build_r.R script is not working anymore, since it assumes that the R package files are under the same directory as the rest of the source files:

CMake Error at CMakeLists.txt:358 (add_library):
  Cannot find source file:

    src/lightgbm_R.cpp

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .mpp .m .M .mm .ixx .cppm
  .ccm .cxxm .c++m .h .hh .h++ .hm .hpp .hxx .in .txx .f .F .for .f77 .f90
  .f95 .f03 .hip .ispc


CMake Error at CMakeLists.txt:358 (add_library):
  No SOURCES given to target: _lightgbm


CMake Generate step failed.  Build files cannot be regenerated correctly.

Building with the latest MSVC using the CI script build_r.R under the VS developer console is not working either, since it doesn't recognize all the installed versions - for example, if I use vs2022 in windows 10, I get this error as it tries to look for a windows 8.1 version (note that this doesn't happen when calling cmake directly):

installing via 'install.libs.R' to C:/Users/david/AppData/Local/R/win-library/4.3/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 16 2019'
-- Configuring incomplete, errors occurred!
CMake Error at CMakeLists.txt:33 (PROJECT):
See also "C:/Users/david/AppData/Local/Temp/Rtmpk3gflg/R.INSTALLafc1e183436/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
  Generator

    Visual Studio 16 2019

  could not find any instance of Visual Studio.



Trying 'Visual Studio 15 2017'
-- Configuring incomplete, errors occurred!
CMake Error at CMakeLists.txt:33 (PROJECT):
See also "C:/Users/david/AppData/Local/Temp/Rtmpk3gflg/R.INSTALLafc1e183436/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
  Generator

    Visual Studio 15 2017

  could not find any instance of Visual Studio.



Trying 'Visual Studio 14 2015'
-- Configuring incomplete, errors occurred!
CMake Error at CMakeLists.txt:33 (PROJECT):
See also "C:/Users/david/AppData/Local/Temp/Rtmpk3gflg/R.INSTALLafc1e183436/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
  Failed to run MSBuild command:

    MSBuild.exe

  to get the value of VCTargetsPath:

    MSBuild version 17.9.8+b34f75857 for .NET Framework
    Build started 5/6/2024 7:21:50 PM.

    Project "C:\Users\david\AppData\Local\Temp\Rtmpk3gflg\R.INSTALLafc1e183436\lightgbm\src\build\CMakeFiles\3.24.3\VCTargetsPath.vcxproj" on node 1 (default targets).
    C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Platforms\x64\PlatformToolsets\v140\Toolset.targets(36,5): error MSB8036: The Windows SDK version 8.1 was not found. Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution". [C:\Users\david\AppData\Local\Temp\Rtmpk3gflg\R.INSTALLafc1e183436\lightgbm\src\build\CMakeFiles\3.24.3\VCTargetsPath.vcxproj]
    Done Building Project "C:\Users\david\AppData\Local\Temp\Rtmpk3gflg\R.INSTALLafc1e183436\lightgbm\src\build\CMakeFiles\3.24.3\VCTargetsPath.vcxproj" (default targets) -- FAILED.

    Build FAILED.

    "C:\Users\david\AppData\Local\Temp\Rtmpk3gflg\R.INSTALLafc1e183436\lightgbm\src\build\CMakeFiles\3.24.3\VCTargetsPath.vcxproj" (default target) (1) ->
    (Desktop_PlatformPrepareForBuild target) ->
      C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Platforms\x64\PlatformToolsets\v140\Toolset.targets(36,5): error MSB8036: The Windows SDK version 8.1 was not found. Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution". [C:\Users\david\AppData\Local\Temp\Rtmpk3gflg\R.INSTALLafc1e183436\lightgbm\src\build\CMakeFiles\3.24.3\VCTargetsPath.vcxproj]

        0 Warning(s)
        1 Error(s)

    Time Elapsed 00:00:00Warning in eval(ei, envir) :.15


  Exit code: 1




  Building with Visual Studio failed. Attempting with MSYS2
CMake Error: Error: generator : MSYS Makefiles
Does not match the generator used previously: Visual Studio 14 2015
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
CMake Error: Error: generator : MSYS Makefiles
Does not match the generator used previously: Visual Studio 14 2015
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
Error in .run_shell_command("cmake", c(cmake_args, "..")) :
  Command failed with exit code: 1
* removing 'C:/Users/david/AppData/Local/R/win-library/4.3/lightgbm'
Error in .run_shell_command(install_cmd, install_args) :
  Command failed with exit code: 1
Execution halted

[jameslamb]

Thanks for reporting this.

Building with this option by calling cmake directly with -D__BUILD_FOR_R=1 -DCMAKE_R_VERSION= without using the build_r.R script is not working anymore

If that ever worked before, it was by accident.

__BUILD_FOR_R is not intended to be part of the public API of this project's CMake build. It's intended only for use by the build_r.R script.

Use build_r.R to drive CMake-based builds of this project's R package.

Building with the latest MSVC using the CI script build_r.R under the VS developer console is not working either

Could you clarify what "using ... build_r.R under the VS developer console" means specifically? Does that just mean running Rscript build_r.R from a terminal inside Visual Studio, or something else? I'm sorry if that's a silly question, I personally am very unfamiliar with the Visual Studio IDE on Windows.

I get this error as it tries to look for a windows 8.1 version

I wonder if this is a difference between the cmake that's bundled with RTools and the one that you are using outside of R stuff? I'd observed some similar issue when working on #6213 and the related things a few days ago. Don't have logs available right now, will try to share them when I can.

[david-cortes]

By "VS developer console", I mean the start menu entry named "Developer Command Prompt" which is added after installing some msvc tool such as "visual studio build tools", which consists of the regular windows command prompt with all msvc-related environment variables pre-loaded.

__BUILD_FOR_R is not intended to be part of the public API of this project's CMake build. It's intended only for use by the build_r.R script.

From what I can tell the build_r.R script doesn't use this option.

[david-cortes]

I wonder if this is a difference between the cmake that's bundled with RTools and the one that you are using outside of R stuff? I'd observed some similar issue when working on #6213 and the related things a few days ago. Don't have logs available right now, will try to share them when I can.

I get the same error when using the cmake that comes bundled in RTools 4.3.

[jameslamb]

Thanks for that explanation of Visual Studio and for checking the cmake bundled in RTools, much appreciated!

From what I can tell the build_r.R script doesn't use this option.

This one is very subtle/complex, I can explain!

build_r.R creates a lightgbm_{version}.tar.gz source distribution of the R package which includes src/install.libs.R.

LightGBM/build_r.R

Lines 193 to 197 in f539536

	# overwrite src/install.libs.R with new content based on command-line flags
	writeLines(
	text = install_libs_content
	, con = file.path(TEMP_SOURCE_DIR, "install.libs.R")
	)

That src/install.libs.R then formats a CMake configure command and includes -D__BUILD_FOR_R.

LightGBM/R-package/src/install.libs.R

Line 177 in f539536

cmake_args <- c(cmake_args, "-D__BUILD_FOR_R=ON")

That install.libs.R is a rarely-used but supported alternative to using CMake or autotools directly in an R package. See https://cran.r-project.org/doc/manuals/R-exts.html#Package-subdirectories.