Compile R package by custom tool chain. #584
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@guolinke Installation seems correct, ran all examples successfully and without any apparent issue. Compiled with Visual Studio 2017. I used Do you think there is a way to fallback to the old compilation way (directly from R toolset) if the dll/lib file is not found? Also, you can update/delete https://github.com/Microsoft/LightGBM/blob/master/docs/GPU-Windows.md doc, as it is now obsolete. For Visual Studio version in wiki, you can recommend VS 2017 with C++ tools because it is a lightweight installation (around 4GB) compared to 2015 or older (8GB or more). BTW, do you think we can put a "default" DLL in this repository or is there any licensing issue? It would allow R users to install from repository. I could host it on my fork if it causes issues. Install log: > setwd("C:/xgboost/LightGBM/R-package")
> devtools::install()
Installing lightgbm
"C:/PROGRA~1/R/R-34~1.0/bin/x64/R" --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL "C:/xgboost/LightGBM/R-package" \
--library="C:/Program Files/R/R-3.4.0/library" --install-tests
* installing *source* package 'lightgbm' ...
** libs
installing via 'install.libs.R' to C:/Program Files/R/R-3.4.0/library/lightgbm
[1] "find library file: C:/xgboost/LightGBM/R-package/../windows/x64/DLL//lib_lightgbm.dll"
** R
** data
** demo
** tests
** preparing package for lazy loading
** help
*** installing help indices
converting help for package 'lightgbm'
finding HTML links ... done
agaricus.test html
agaricus.train html
dim html
dimnames.lgb.Dataset html
getinfo html
lgb.Dataset html
lgb.Dataset.construct html
lgb.Dataset.create.valid html
lgb.Dataset.save html
lgb.Dataset.set.categorical html
lgb.Dataset.set.reference html
lgb.dump html
lgb.get.eval.result html
lgb.importance html
lgb.interprete html
lgb.load html
lgb.model.dt.tree html
lgb.plot.importance html
lgb.plot.interpretation html
lgb.prepare html
lgb.prepare2 html
lgb.prepare_rules html
lgb.prepare_rules2 html
lgb.save html
lgb.train html
lgb.unloader html
predict.lgb.Booster html
readRDS.lgb.Booster html
saveRDS.lgb.Booster html
setinfo html
slice html
** building package indices
** testing if installed package can be loaded
* DONE (lightgbm) |
|
Thanks @Laurae2 .
I think we can try to add some make command in The problem of putting "default" DLL is the cross-platform. We need to build many "DLLs" ([windows, linux, OSX] x [cpu, gpu] ). |
Currently we can get rid of GPU for automated builds, it is too much GPU vendor dependent.
Objectives would be:
I have 5 ideas, but I have no idea if they are feasible (solution 5 is feasible 100% guaranteed): Solution 1: how is a good question... I think with a
Solution 2: another way of installing it: using R Solution 3: create a secondary R package (easiest) with Solution 4: use Makevars Solution 5: identical to solution 2, except that script would handle both the fetching of the repository ( |
|
@guolinke Do you know how to create DLL from Visual Studio compiler from command line? I am trying to allow compilation using VS make. I can create a helper package to install LightGBM (we can put it in Microsoft/LightGBM or at Laurae2, up to you). I managed to get compilation done using R script: > lgb_git_dir <- tempdir()
> lgb_git_file <- file.path(lgb_git_dir, "temp.bat", fsep = "\\")
> unlink(paste0(lgb_git_dir, "\\LightGBM"), recursive = TRUE, force = TRUE)
> cat(paste0("cd ", lgb_git_dir, "\n"), file = lgb_git_file)
> cat(paste0("git clone --recursive https://github.com/Microsoft/LightGBM", "\n"), file = lgb_git_file, append = TRUE)
> cat(paste0("cd LightGBM", "\n"), file = lgb_git_file, append = TRUE)
> cat(paste0("git checkout r-withvs", "\n"), file = lgb_git_file, append = TRUE)
> cat(paste0("mkdir build && cd build", "\n"), file = lgb_git_file, append = TRUE)
> cat(paste0("cmake -G \"MinGW Makefiles\" ..", "\n"), file = lgb_git_file, append = TRUE)
> cat(paste0("cmake -G \"MinGW Makefiles\" ..", "\n"), file = lgb_git_file, append = TRUE)
> cat(paste0("mingw32-make.exe"), file = lgb_git_file, append = TRUE)
> system(lgb_git_file)
C:\Users\Laurae\Documents>cd C:\tmp\RtmpSa5Bdv
C:\tmp\RtmpSa5Bdv>git clone --recursive https://github.com/Microsoft/LightGBM
Cloning into 'LightGBM'...
Submodule 'include/boost/compute' (https://github.com/boostorg/compute) registered for path 'compute'
Cloning into 'compute'...
Submodule path 'compute': checked out '6de7f6448796f67958dde8de4569fb1ae649ee91'
C:\tmp\RtmpSa5Bdv>cd LightGBM
C:\tmp\RtmpSa5Bdv\LightGBM>git checkout r-withvs
Branch r-withvs set up to track remote branch r-withvs from origin.
Switched to a new branch 'r-withvs'
C:\tmp\RtmpSa5Bdv\LightGBM>mkdir build && cd build
C:\tmp\RtmpSa5Bdv\LightGBM\build>cmake -G "MinGW Makefiles" ..
CMake Error at C:/Program Files/CMake/share/cmake-3.8/Modules/CMakeMinGWFindMake.cmake:12 (message):
sh.exe was found in your PATH, here:
C:/Rtools/bin/sh.exe
For MinGW make to work correctly sh.exe must NOT be in your path.
Run cmake from a shell that does not have sh.exe in your PATH.
If you want to use a UNIX shell, then use MSYS Makefiles.
Call Stack (most recent call first):
CMakeLists.txt:8 (PROJECT)
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
C:\tmp\RtmpSa5Bdv\LightGBM\build>cmake -G "MinGW Makefiles" ..
-- The C compiler identification is GNU 7.1.0
-- The CXX compiler identification is GNU 7.1.0
-- Check for working C compiler: C:/Rtools/mingw_64/bin/gcc.exe
-- Check for working C compiler: C:/Rtools/mingw_64/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Rtools/mingw_64/bin/g++.exe
-- Check for working CXX compiler: C:/Rtools/mingw_64/bin/g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -fopenmp
-- Configuring done
-- Generating done
-- Build files have been written to: C:/tmp/RtmpSa5Bdv/LightGBM/build
C:\tmp\RtmpSa5Bdv\LightGBM\build>mingw32-make.exe
Scanning dependencies of target lightgbm
[ 1%] Building CXX object CMakeFiles/lightgbm.dir/src/main.cpp.obj
[ 3%] Building CXX object CMakeFiles/lightgbm.dir/src/application/application.cpp.obj
[ 5%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/boosting.cpp.obj
[ 7%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/gbdt.cpp.obj
[ 9%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/gbdt_prediction.cpp.obj
[ 10%] Building CXX object CMakeFiles/lightgbm.dir/src/boosting/prediction_early_stop.cpp.obj
[ 12%] Building CXX object CMakeFiles/lightgbm.dir/src/io/bin.cpp.obj
[ 14%] Building CXX object CMakeFiles/lightgbm.dir/src/io/config.cpp.obj
[ 16%] Building CXX object CMakeFiles/lightgbm.dir/src/io/dataset.cpp.obj
[ 18%] Building CXX object CMakeFiles/lightgbm.dir/src/io/dataset_loader.cpp.obj
[ 20%] Building CXX object CMakeFiles/lightgbm.dir/src/io/metadata.cpp.obj
[ 21%] Building CXX object CMakeFiles/lightgbm.dir/src/io/parser.cpp.obj
[ 23%] Building CXX object CMakeFiles/lightgbm.dir/src/io/tree.cpp.obj
[ 25%] Building CXX object CMakeFiles/lightgbm.dir/src/metric/dcg_calculator.cpp.obj
[ 27%] Building CXX object CMakeFiles/lightgbm.dir/src/metric/metric.cpp.obj
[ 29%] Building CXX object CMakeFiles/lightgbm.dir/src/objective/objective_function.cpp.obj
[ 30%] Building CXX object CMakeFiles/lightgbm.dir/src/network/linker_topo.cpp.obj
[ 32%] Building CXX object CMakeFiles/lightgbm.dir/src/network/linkers_mpi.cpp.obj
[ 34%] Building CXX object CMakeFiles/lightgbm.dir/src/network/linkers_socket.cpp.obj
[ 36%] Building CXX object CMakeFiles/lightgbm.dir/src/network/network.cpp.obj
[ 38%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.obj
[ 40%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/feature_parallel_tree_learner.cpp.obj
[ 41%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/gpu_tree_learner.cpp.obj
[ 43%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/serial_tree_learner.cpp.obj
[ 45%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/tree_learner.cpp.obj
[ 47%] Building CXX object CMakeFiles/lightgbm.dir/src/treelearner/voting_parallel_tree_learner.cpp.obj
[ 49%] Linking CXX executable ..\lightgbm.exe
[ 49%] Built target lightgbm
Scanning dependencies of target _lightgbm
[ 50%] Building CXX object CMakeFiles/_lightgbm.dir/src/c_api.cpp.obj
[ 52%] Building CXX object CMakeFiles/_lightgbm.dir/src/lightgbm_R.cpp.obj
[ 54%] Building CXX object CMakeFiles/_lightgbm.dir/src/application/application.cpp.obj
[ 56%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/boosting.cpp.obj
[ 58%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt.cpp.obj
[ 60%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt_prediction.cpp.obj
[ 61%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/prediction_early_stop.cpp.obj
[ 63%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/bin.cpp.obj
[ 65%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/config.cpp.obj
[ 67%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset.cpp.obj
[ 69%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset_loader.cpp.obj
[ 70%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/metadata.cpp.obj
[ 72%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/parser.cpp.obj
[ 74%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/tree.cpp.obj
[ 76%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/dcg_calculator.cpp.obj
[ 78%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/metric.cpp.obj
[ 80%] Building CXX object CMakeFiles/_lightgbm.dir/src/objective/objective_function.cpp.obj
[ 81%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linker_topo.cpp.obj
[ 83%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_mpi.cpp.obj
[ 85%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_socket.cpp.obj
[ 87%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/network.cpp.obj
[ 89%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.obj
[ 90%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/feature_parallel_tree_learner.cpp.obj
[ 92%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/gpu_tree_learner.cpp.obj
[ 94%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/serial_tree_learner.cpp.obj
[ 96%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/tree_learner.cpp.obj
[ 98%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/voting_parallel_tree_learner.cpp.obj
[100%] Linking CXX shared library ..\lib_lightgbm.dll
[100%] Built target _lightgbm
> devtools::install(file.path(lgb_git_dir, "LightGBM", "R-package", fsep = "\\"))
Installing lightgbm
"C:/PROGRA~1/R/R-34~1.0/bin/x64/R" --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL "C:/tmp/RtmpSa5Bdv/LightGBM/R-package" \
--library="C:/Program Files/R/R-3.4.0/library" --install-tests
* installing *source* package 'lightgbm' ...
** libs
installing via 'install.libs.R' to C:/Program Files/R/R-3.4.0/library/lightgbm
[1] "find library file: C:/tmp/RtmpSa5Bdv/LightGBM/R-package/../lib_lightgbm.dll"
** R
** data
** demo
** tests
** preparing package for lazy loading
** help
*** installing help indices
converting help for package 'lightgbm'
finding HTML links ... done
agaricus.test html
agaricus.train html
dim html
dimnames.lgb.Dataset html
getinfo html
lgb.Dataset html
lgb.Dataset.construct html
lgb.Dataset.create.valid html
lgb.Dataset.save html
lgb.Dataset.set.categorical html
lgb.Dataset.set.reference html
lgb.dump html
lgb.get.eval.result html
lgb.importance html
lgb.interprete html
lgb.load html
lgb.model.dt.tree html
lgb.plot.importance html
lgb.plot.interpretation html
lgb.prepare html
lgb.prepare2 html
lgb.prepare_rules html
lgb.prepare_rules2 html
lgb.save html
lgb.train html
lgb.unloader html
predict.lgb.Booster html
readRDS.lgb.Booster html
saveRDS.lgb.Booster html
setinfo html
slice html
** building package indices
** testing if installed package can be loaded
* DONE (lightgbm)
> unlink(paste0(lgb_git_dir, "\\LightGBM"), recursive = TRUE, force = TRUE) |
|
This should do it, but Visual Studio compilation is not working properly: # Known elements
lgb_git_commit <- "r-withvs" # Put "" if master branch
lgb_git_compiler <- "vs" # Windows only: ["gcc", "vs"]
lgb_git_lib <- "C:\\LightGBM\\windows\\x64\\DLL\\lib_lightgbm.dll" # Put "" if using no precompiled dll/lib
lgb_git_repo <- "https://github.com/Microsoft/LightGBM" # Set it to something else from GitHub for other repositories
lgb_git_cores <- 4 # Specify number of cores for compilation here, ignored for Visual Studio build
# Generates temporary dir
lgb_git_dir <- tempdir()
# Check if it is Windows, because it create most issues
if (.Platform$OS.type == "windows") {
# Create temp file
lgb_git_file <- file.path(lgb_git_dir, "temp.bat", fsep = "\\")
# Delete (old) temp LightGBM folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM", fsep = "\\")), recursive = TRUE, force = TRUE)
# Use git to fetch data from repository
cat(paste0("cd ", lgb_git_dir, "\n"), file = lgb_git_file)
cat(paste0("git clone --recursive ", lgb_git_repo, "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cd LightGBM", "\n"), file = lgb_git_file, append = TRUE)
# Checkout specific commit if needed
if (lgb_git_commit != "") {
cat(paste0("git checkout ", lgb_git_commit, "\n"), file = lgb_git_file, append = TRUE)
}
# If no lib is specified, force compilation
if (lgb_git_lib == "") {
cat(paste0("mkdir build && cd build", "\n"), file = lgb_git_file, append = TRUE)
# Check if compilation must be done using MinGW/gcc (default) or Visual Studio
if (lgb_git_compiler == "gcc") {
cat(paste0("cmake -G \"MinGW Makefiles\" ..", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake -G \"MinGW Makefiles\" ..", "\n"), file = lgb_git_file, append = TRUE) # Failsafe
cat(paste0("mingw32-make.exe -j", lgb_git_cores), file = lgb_git_file, append = TRUE)
} else {
cat(paste0("cmake -DCMAKE_GENERATOR_PLATFORM=x64 ..", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake -DCMAKE_GENERATOR_PLATFORM=x64 ..", "\n"), file = lgb_git_file, append = TRUE) # Failsafe
cat(paste0("cmake --build . --target ALL_BUILD --config DLL"), file = lgb_git_file, append = TRUE)
}
} else {
cat(paste0("cp ", lgb_git_lib, " ", file.path(lgb_git_dir, "LightGBM", fsep = "\\"), "\n"), file = lgb_git_file, append = TRUE) # Move dll/lib
}
# Do actions
system(lgb_git_file)
# Install package
devtools::install(file.path(lgb_git_dir, "LightGBM", "R-package", fsep = "\\"))
# Get rid of the created temporary folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM", fsep = "\\")), recursive = TRUE, force = TRUE)
} else {
# Create temp file
lgb_git_file <- file.path(lgb_git_dir, "temp.sh")
# Delete (old) temp LightGBM folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM")), recursive = TRUE, force = TRUE)
# Use git to fetch data from repository
cat(paste0("cd ", lgb_git_dir, "\n"), file = lgb_git_file)
cat(paste0("git clone --recursive ", lgb_git_repo, "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cd LightGBM", "\n"), file = lgb_git_file, append = TRUE)
# Checkout specific commit if needed
if (lgb_git_commit != "") {
cat(paste0("git checkout ", lgb_git_commit, "\n"), file = lgb_git_file, append = TRUE)
}
# If no lib is specified, force compilation
if (lgb_git_lib == "") {
cat(paste0("mkdir build && cd build", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake ..", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake ..", "\n"), file = lgb_git_file, append = TRUE) # Failsafe
cat(paste0("make -j", lgb_git_cores), file = lgb_git_file, append = TRUE)
} else {
cat(paste0("cp ", lgb_git_lib, " ", file.path(lgb_git_dir, "LightGBM", fsep = "\\"), "\n"), file = lgb_git_file, append = TRUE) # Move dll/lib
}
# Do actions
system(lgb_git_file)
# Install package
devtools::install(file.path(lgb_git_dir, "LightGBM", "R-package"))
# Get rid of the created temporary folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM")), recursive = TRUE, force = TRUE)
}Gives weird VS error about 2010 when I don't use 2010: C:\Users\Laurae\Documents>cd C:\tmp\RtmpSa5Bdv
C:\tmp\RtmpSa5Bdv>git clone --recursive https://github.com/Microsoft/LightGBM
Cloning into 'LightGBM'...
Submodule 'include/boost/compute' (https://github.com/boostorg/compute) registered for path 'compute'
Cloning into 'compute'...
Submodule path 'compute': checked out '6de7f6448796f67958dde8de4569fb1ae649ee91'
C:\tmp\RtmpSa5Bdv>cd LightGBM
C:\tmp\RtmpSa5Bdv\LightGBM>git checkout r-withvs
Branch r-withvs set up to track remote branch r-withvs from origin.
Switched to a new branch 'r-withvs'
C:\tmp\RtmpSa5Bdv\LightGBM>mkdir build && cd build
C:\tmp\RtmpSa5Bdv\LightGBM\build>cmake -DCMAKE_GENERATOR_PLATFORM=x64 ..
-- Building for: Visual Studio 15 2017
-- The C compiler identification is MSVC 19.11.25303.0
-- The CXX compiler identification is MSVC 19.11.25303.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.11.25301/bin/HostX86/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.11.25301/bin/HostX86/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.11.25301/bin/HostX86/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.11.25301/bin/HostX86/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Try OpenMP C flag = [/openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [/openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: /openmp
-- Configuring done
-- Generating done
-- Build files have been written to: C:/tmp/RtmpSa5Bdv/LightGBM/build
C:\tmp\RtmpSa5Bdv\LightGBM\build>cmake -DCMAKE_GENERATOR_PLATFORM=x64 ..
-- Configuring done
-- Generating done
-- Build files have been written to: C:/tmp/RtmpSa5Bdv/LightGBM/build
C:\tmp\RtmpSa5Bdv\LightGBM\build>cmake --build . --target ALL_BUILD --config DLL
Microsoft (R) Build Engine version 15.3.118.39484
Copyright (C) Microsoft Corporation. All rights reserved.
Build started 04/06/2017 08:10:13 PM.
Project "C:\tmp\RtmpSa5Bdv\LightGBM\build\ALL_BUILD.vcxproj" on node 1 (default targets).
Project "C:\tmp\RtmpSa5Bdv\LightGBM\build\ALL_BUILD.vcxproj" (1) is building "C:\tmp\RtmpSa5Bdv\LightGBM\build\ZERO_CHECK.vcxproj" (2) on node 1 (default targets).
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Community\Common7\IDE\VC\VCTargets\Microsoft.Cpp.Platform.targets(55,5): error MSB8020: The build tools for Visual Studio 2010 (Platform Toolset = 'v100') cannot be found. To build using the v100 build tools, please install Visual Studio 2010 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [C:\tmp\RtmpSa5Bdv\LightGBM\build\ZERO_CHECK.vcxproj]
Done Building Project "C:\tmp\RtmpSa5Bdv\LightGBM\build\ZERO_CHECK.vcxproj" (default targets) -- FAILED.
Done Building Project "C:\tmp\RtmpSa5Bdv\LightGBM\build\ALL_BUILD.vcxproj" (default targets) -- FAILED.
Build FAILED.
"C:\tmp\RtmpSa5Bdv\LightGBM\build\ALL_BUILD.vcxproj" (default target) (1) ->
"C:\tmp\RtmpSa5Bdv\LightGBM\build\ZERO_CHECK.vcxproj" (default target) (2) ->
(PlatformPrepareForBuild target) ->
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Community\Common7\IDE\VC\VCTargets\Microsoft.Cpp.Platform.targets(55,5): error MSB8020: The build tools for Visual Studio 2010 (Platform Toolset = 'v100') cannot be found. To build using the v100 build tools, please install Visual Studio 2010 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [C:\tmp\RtmpSa5Bdv\LightGBM\build\ZERO_CHECK.vcxproj]
0 Warning(s)
1 Error(s)
Time Elapsed 00:00:00.25 |
|
After many attempts, it eventually works with Visual Studio although user has to put path to # Known elements
lgb_git_commit <- "r-withvs" # Put "" if master branch
lgb_git_compiler <- "vs" # Windows only: ["gcc", "vs"] (MinGW or Visual Studio)
lgb_git_upgrade <- "C:\\Program Files (x86)\\Microsoft Visual Studio\\Preview\\Community\\Common7\\IDE" # For VS compilation: path to devenv of the appropriate Visual Studio compiler
lgb_git_msbuild <- "C:\\Program Files (x86)\\Microsoft Visual Studio\\Preview\\Community\\MSBuild\\15.0\\Bin" # For VS compilation: path to msbuild
lgb_git_lib <- "C:\\LightGBM\\windows\\x64\\DLL\\lib_lightgbm.dll" # Put "" if using no precompiled dll/lib
lgb_git_repo <- "https://github.com/Microsoft/LightGBM" # Set it to something else from GitHub for other repositories
lgb_git_cores <- 4 # Specify number of cores for compilation here, ignored for Visual Studio build
# Generates temporary dir
lgb_git_dir <- tempdir()
# Check if it is Windows, because it create most issues
if (.Platform$OS.type == "windows") {
# Create temp file
lgb_git_file <- file.path(lgb_git_dir, "temp.bat", fsep = "\\")
# Delete (old) temp LightGBM folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM", fsep = "\\")), recursive = TRUE, force = TRUE)
# Use git to fetch data from repository
cat(paste0("cd ", lgb_git_dir, "\n"), file = lgb_git_file)
cat(paste0("git clone --recursive ", lgb_git_repo, "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cd LightGBM", "\n"), file = lgb_git_file, append = TRUE)
# Checkout specific commit if needed
if (lgb_git_commit != "") {
cat(paste0("git checkout ", lgb_git_commit, "\n"), file = lgb_git_file, append = TRUE)
}
# If no lib is specified, force compilation
if (lgb_git_lib == "") {
# Check if compilation must be done using MinGW/gcc (default) or Visual Studio
if (lgb_git_compiler == "gcc") {
cat(paste0("mkdir build && cd build", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake -G \"MinGW Makefiles\" ..", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake -G \"MinGW Makefiles\" ..", "\n"), file = lgb_git_file, append = TRUE) # Failsafe
cat(paste0("mingw32-make.exe -j", lgb_git_cores), file = lgb_git_file, append = TRUE)
} else {
cat(paste0("\"", file.path(lgb_git_upgrade, "devenv", fsep = "\\"), "\" ./windows/LightGBM.sln /Upgrade", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("\"", file.path(lgb_git_msbuild, "msbuild", fsep = "\\"), "\" ./windows/LightGBM.vcxproj /p:configuration=DLL /p:platform=x64", "\n"), file = lgb_git_file, append = TRUE)
#cat(paste0("cmake -DCMAKE_GENERATOR_PLATFORM=x64 ..", "\n"), file = lgb_git_file, append = TRUE)
#cat(paste0("cmake -DCMAKE_GENERATOR_PLATFORM=x64 ..", "\n"), file = lgb_git_file, append = TRUE) # Failsafe
#cat(paste0("cmake --build . --target ALL_BUILD --config DLL"), file = lgb_git_file, append = TRUE)
}
} else {
cat(paste0("cp ", lgb_git_lib, " ", file.path(lgb_git_dir, "LightGBM", fsep = "\\"), "\n"), file = lgb_git_file, append = TRUE) # Move dll/lib
}
# Do actions
system(lgb_git_file)
# Install package
devtools::install(file.path(lgb_git_dir, "LightGBM", "R-package", fsep = "\\"))
# Get rid of the created temporary folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM", fsep = "\\")), recursive = TRUE, force = TRUE)
} else {
# Create temp file
lgb_git_file <- file.path(lgb_git_dir, "temp.sh")
# Delete (old) temp LightGBM folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM")), recursive = TRUE, force = TRUE)
# Use git to fetch data from repository
cat(paste0("cd ", lgb_git_dir, "\n"), file = lgb_git_file)
cat(paste0("git clone --recursive ", lgb_git_repo, "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cd LightGBM", "\n"), file = lgb_git_file, append = TRUE)
# Checkout specific commit if needed
if (lgb_git_commit != "") {
cat(paste0("git checkout ", lgb_git_commit, "\n"), file = lgb_git_file, append = TRUE)
}
# If no lib is specified, force compilation
if (lgb_git_lib == "") {
cat(paste0("mkdir build && cd build", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake ..", "\n"), file = lgb_git_file, append = TRUE)
cat(paste0("cmake ..", "\n"), file = lgb_git_file, append = TRUE) # Failsafe
cat(paste0("make -j", lgb_git_cores), file = lgb_git_file, append = TRUE)
} else {
cat(paste0("cp ", lgb_git_lib, " ", file.path(lgb_git_dir, "LightGBM"), "\n"), file = lgb_git_file, append = TRUE) # Move dll/lib
}
# Do actions
system(lgb_git_file)
# Install package
devtools::install(file.path(lgb_git_dir, "LightGBM", "R-package"))
# Get rid of the created temporary folder
unlink(paste0(file.path(lgb_git_dir, "LightGBM")), recursive = TRUE, force = TRUE)
} |
|
@Laurae2 Thanks, I move these build command to |
guolinke
changed the title
|
@wxchan |
|
@guolinke Can you add a check for dll/lib existence to avoid running compilation? It seems we cannot install R package from local without recompiling from scratch now. Also, I tested |
|
@Laurae2 |
|
@guolinke It does indeed skip the compiled files. Which way should we use to use precompiled Visual Studio DLL so the user do not have to download and install Visual Studio? |
|
@Laurae2 BTW, I try this on ubuntu 14.04. However, the |
|
@guolinke The compilation setup is missing instructions for non Windows OS: if (WINDOWS) {
if(use_mingw){
cmake_cmd <- paste0(cmake_cmd, " -G \"MinGW Makefiles\" ")
build_cmd <- "mingw32-make.exe -j"
} else{
cmake_cmd <- paste0(cmake_cmd, " -DCMAKE_GENERATOR_PLATFORM=x64 ")
build_cmd <- "cmake --build . --target _lightgbm --config Release"
lib_folder <- paste0(R_PACKAGE_SOURCE, '/src/Release')
}
}This is weird also if it is not called at all, because there are CRAN packages using |
|
@guolinke I think you can revert your edits to commit c379a2f. I created a very simple package to install LightGBM from repository: lgb.dl(commit = "c379a2f",
#libdll = "C:\\xgboost\\LightGBM\\windows\\x64\\DLL\\lib_lightgbm.dll",
compiler = "vs")It will install LightGBM correctly, while using the DLL you provide if you provide one (otherwise, it will build LightGBM from scratch). Use this to install my package installer: https://github.com/Laurae2/lgbdl (if users do not want to install my package, they can copy the function to run it). |
|
@Laurae2 The command of linux is set here: After add the empty BTW, why need to revert that commit ? |
3 tasks
|
@Laurae2 |
|
Successfully built with this instructions but crashed at the last line (even though as I said there are no problem with standard build instructions and Python interface). Any suggestions are greatly appreciated! |
|
@bushmanov |
|
@guolinke Seems OK for me. I can work later on a script which will allow to customize installation so we can install more easily from source while using custom flags. You can merge, I tested on all boxes (Win/Mac/Linux), compiles and installs without error. |
|
|
@bushmanov From the log, the build with GPU is fine. I am not sure why it freezes. |
|
@bushmanov Can you also provide the log of your successful python build (include the build log and python output log) . |
|
The issue has been resolved after restarting RStudio |
guolinke
added a commit
that referenced
this pull request
Oct 9, 2017
* add R's library file to vs project and cmake. * support using dll built by vs. * better search for the library file. * remove mingw related doc . * update document. * Let R handle the library compile. * try fix build from github. * Update README.md * cleaner build. * fix the install problem in linux. * Update README.md
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
refer to #542 .
with this, we can get rid of MinGW in windows. And it is easier to build R-package in OSX as well.
Also, it is easier to build GPU version for R-package in windows.
@Laurae2 Any comment? Can you also help to test the new build?