Build w/ GPU but got CPU-only ver install in python (Ubuntu 16.04)

[cttsai1985]

After managed to built a LightGBM w/ GPU on Ubuntu 16.04, it asked to "Recompile with CMake option -DUSE_GPU=1'" while calling LightGBM in python 3.5. However, the binay ./LightGBM/lightgbm is able to enable GPU for computation. GPU is availabe after replacing lib_lightgbm.so in /usr/local/lib/python3.5/dist-packages/lightgbm by one ./LightGBM/ . The lib_lightgbm.so installed within python seems a CPU-only built.

The compiling LightGBM w/ GPU is following by the issue 458 by "sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ .." from #458 . Also, pip3 installation did not work for my pc.

Meanwhile, I've notice there are two different sizes of lib_lightgbm.so
2.2MB ./LightGBM/lib_lightgbm.so
1.5MB ./LightGBM/python-package/lightgbm/lib_lightgbm.so
1.5MB ./LightGBM/python-package/build/lib/lightgbm/lib_lightgbm.so
1.5MB .//usr/local/lib/python3.5/dist-packages/lightgbm/lib_lightgbm.so

It seems to me that another CPU-only lib_lightgbm.so is compiled while run "python3 setup.py install". Therefore, I decided to copy the lib_lightgbm.so and so far it works fine.

Environment info

Operating System: Ubuntu 16.04
CPU: i3-7350K / GTX 1080 (375.66)
C++/Python/R version: Python

Error Message:

[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1
Traceback (most recent call last):
File "hybrid.py", line 211, in
clf = lgb.train(params, d_train, 430)
File "/usr/local/lib/python3.5/dist-packages/lightgbm/engine.py", line 168, in train
booster = Booster(params=params, train_set=train_set)
File "/usr/local/lib/python3.5/dist-packages/lightgbm/basic.py", line 1215, in init
ctypes.byref(self.handle)))
File "/usr/local/lib/python3.5/dist-packages/lightgbm/basic.py", line 47, in _safe_call
raise LightGBMError(_LIB.LGBM_GetLastError())
lightgbm.basic.LightGBMError: b'GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1'

Steps to reproduce

1. sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ ..
   -- The C compiler identification is GNU 5.4.0
   -- The CXX compiler identification is GNU 5.4.0
   -- Check for working C compiler: /usr/bin/cc
   -- Check for working C compiler: /usr/bin/cc -- works
   -- Detecting C compiler ABI info
   -- Detecting C compiler ABI info - done
   -- Detecting C compile features
   -- Detecting C compile features - done
   -- Check for working CXX compiler: /usr/bin/c++
   -- Check for working CXX compiler: /usr/bin/c++ -- works
   -- Detecting CXX compiler ABI info
   -- Detecting CXX compiler ABI info - done
   -- Detecting CXX compile features
   -- Detecting CXX compile features - done
   -- Try OpenMP C flag = [-fopenmp]
   -- Performing Test OpenMP_FLAG_DETECTED
   -- Performing Test OpenMP_FLAG_DETECTED - Success
   -- Try OpenMP CXX flag = [-fopenmp]
   -- Performing Test OpenMP_FLAG_DETECTED
   -- Performing Test OpenMP_FLAG_DETECTED - Success
   -- Found OpenMP: -fopenmp
   -- Looking for CL_VERSION_2_0
   -- Looking for CL_VERSION_2_0 - not found
   -- Looking for CL_VERSION_1_2
   -- Looking for CL_VERSION_1_2 - found
   -- Found OpenCL: /usr/local/cuda-8.0/lib64/libOpenCL.so (found version "1.2")
   -- OpenCL include directory:/usr/local/cuda-8.0/include
   -- Boost version: 1.58.0
   -- Found the following Boost libraries:
   -- filesystem
   -- system
   -- Configuring done
   -- Generating done
   -- Build files have been written to: /home/cttsai/DataSci/mlc/LightGBM/build

2. sudo make -j$(nproc)

3. cd ../python-package; sudo python3 setup.py install
   running install
   creating lightgbm/include
   creating lightgbm/include/LightGBM
   copying ../include/LightGBM/prediction_early_stop.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/dataset_loader.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/dataset.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/application.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/bin.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/c_api.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/tree.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/export.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/lightgbm_R.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/meta.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/metric.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/network.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/objective_function.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/R_object_helper.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/boosting.h -> ./lightgbm/include/LightGBM
   creating lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/pipeline_reader.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/random.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/array_args.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/common.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/log.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/threading.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/text_reader.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/utils/openmp_wrapper.h -> ./lightgbm/include/LightGBM/utils
   copying ../include/LightGBM/feature_group.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/tree_learner.h -> ./lightgbm/include/LightGBM
   copying ../include/LightGBM/config.h -> ./lightgbm/include/LightGBM
   creating lightgbm/src
   creating lightgbm/src/application
   copying ../src/application/application.cpp -> ./lightgbm/src/application
   copying ../src/application/predictor.hpp -> ./lightgbm/src/application
   copying ../src/main.cpp -> ./lightgbm/src
   creating lightgbm/src/objective
   copying ../src/objective/regression_objective.hpp -> ./lightgbm/src/objective
   copying ../src/objective/multiclass_objective.hpp -> ./lightgbm/src/objective
   copying ../src/objective/rank_objective.hpp -> ./lightgbm/src/objective
   copying ../src/objective/binary_objective.hpp -> ./lightgbm/src/objective
   copying ../src/objective/objective_function.cpp -> ./lightgbm/src/objective
   copying ../src/objective/xentropy_objective.hpp -> ./lightgbm/src/objective
   creating lightgbm/src/boosting
   copying ../src/boosting/dart.hpp -> ./lightgbm/src/boosting
   copying ../src/boosting/goss.hpp -> ./lightgbm/src/boosting
   copying ../src/boosting/gbdt_prediction.cpp -> ./lightgbm/src/boosting
   copying ../src/boosting/gbdt.h -> ./lightgbm/src/boosting
   copying ../src/boosting/boosting.cpp -> ./lightgbm/src/boosting
   copying ../src/boosting/gbdt.cpp -> ./lightgbm/src/boosting
   copying ../src/boosting/rf.hpp -> ./lightgbm/src/boosting
   copying ../src/boosting/score_updater.hpp -> ./lightgbm/src/boosting
   copying ../src/boosting/prediction_early_stop.cpp -> ./lightgbm/src/boosting
   creating lightgbm/src/metric
   copying ../src/metric/xentropy_metric.hpp -> ./lightgbm/src/metric
   copying ../src/metric/rank_metric.hpp -> ./lightgbm/src/metric
   copying ../src/metric/dcg_calculator.cpp -> ./lightgbm/src/metric
   copying ../src/metric/metric.cpp -> ./lightgbm/src/metric
   copying ../src/metric/regression_metric.hpp -> ./lightgbm/src/metric
   copying ../src/metric/multiclass_metric.hpp -> ./lightgbm/src/metric
   copying ../src/metric/binary_metric.hpp -> ./lightgbm/src/metric
   copying ../src/metric/map_metric.hpp -> ./lightgbm/src/metric
   creating lightgbm/src/treelearner
   copying ../src/treelearner/gpu_tree_learner.cpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/split_info.hpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/feature_parallel_tree_learner.cpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/leaf_splits.hpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/data_parallel_tree_learner.cpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/serial_tree_learner.cpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/data_partition.hpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/feature_histogram.hpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/tree_learner.cpp -> ./lightgbm/src/treelearner
   copying ../src/treelearner/parallel_tree_learner.h -> ./lightgbm/src/treelearner
   copying ../src/treelearner/voting_parallel_tree_learner.cpp -> ./lightgbm/src/treelearner
   creating lightgbm/src/treelearner/ocl
   copying ../src/treelearner/ocl/histogram256.cl -> ./lightgbm/src/treelearner/ocl
   copying ../src/treelearner/ocl/histogram64.cl -> ./lightgbm/src/treelearner/ocl
   copying ../src/treelearner/ocl/histogram16.cl -> ./lightgbm/src/treelearner/ocl
   copying ../src/treelearner/gpu_tree_learner.h -> ./lightgbm/src/treelearner
   copying ../src/treelearner/serial_tree_learner.h -> ./lightgbm/src/treelearner
   creating lightgbm/src/io
   copying ../src/io/parser.hpp -> ./lightgbm/src/io
   copying ../src/io/metadata.cpp -> ./lightgbm/src/io
   copying ../src/io/dense_bin.hpp -> ./lightgbm/src/io
   copying ../src/io/sparse_bin.hpp -> ./lightgbm/src/io
   copying ../src/io/ordered_sparse_bin.hpp -> ./lightgbm/src/io
   copying ../src/io/bin.cpp -> ./lightgbm/src/io
   copying ../src/io/dense_nbits_bin.hpp -> ./lightgbm/src/io
   copying ../src/io/tree.cpp -> ./lightgbm/src/io
   copying ../src/io/parser.cpp -> ./lightgbm/src/io
   copying ../src/io/dataset_loader.cpp -> ./lightgbm/src/io
   copying ../src/io/config.cpp -> ./lightgbm/src/io
   copying ../src/io/dataset.cpp -> ./lightgbm/src/io
   copying ../src/lightgbm_R.cpp -> ./lightgbm/src
   copying ../src/c_api.cpp -> ./lightgbm/src
   creating lightgbm/src/network
   copying ../src/network/socket_wrapper.hpp -> ./lightgbm/src/network
   copying ../src/network/linker_topo.cpp -> ./lightgbm/src/network
   copying ../src/network/network.cpp -> ./lightgbm/src/network
   copying ../src/network/linkers_socket.cpp -> ./lightgbm/src/network
   copying ../src/network/linkers.h -> ./lightgbm/src/network
   copying ../src/network/linkers_mpi.cpp -> ./lightgbm/src/network
   copying ../CMakeLists.txt -> ./lightgbm/
   copying ../LICENSE -> ./
   Start to compile libarary.
   -- The C compiler identification is GNU 5.4.0
   -- The CXX compiler identification is GNU 5.4.0
   -- Check for working C compiler: /usr/bin/cc
   -- Check for working C compiler: /usr/bin/cc -- works
   -- Detecting C compiler ABI info
   -- Detecting C compiler ABI info - done
   -- Detecting C compile features
   -- Detecting C compile features - done
   -- Check for working CXX compiler: /usr/bin/c++
   -- Check for working CXX compiler: /usr/bin/c++ -- works
   -- Detecting CXX compiler ABI info
   -- Detecting CXX compiler ABI info - done
   -- Detecting CXX compile features
   -- Detecting CXX compile features - done
   -- Try OpenMP C flag = [-fopenmp]
   -- Performing Test OpenMP_FLAG_DETECTED
   -- Performing Test OpenMP_FLAG_DETECTED - Success
   -- Try OpenMP CXX flag = [-fopenmp]
   -- Performing Test OpenMP_FLAG_DETECTED
   -- Performing Test OpenMP_FLAG_DETECTED - Success
   -- Found OpenMP: -fopenmp
   -- Configuring done
   -- Generating done
   -- Build files have been written to: /home/cttsai/DataSci/mlc/LightGBM/python-package/build_cpp
   Scanning dependencies of target _lightgbm
   [ 3%] Building CXX object CMakeFiles/_lightgbm.dir/src/c_api.cpp.o
   [ 7%] Building CXX object CMakeFiles/./LightGBM/_lightgbm.dir/src/lightgbm_R.cpp.o
   [ 10%] Building CXX object CMakeFiles/_lightgbm.dir/src/application/application.cpp.o
   [ 14%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt_prediction.cpp.o
   [ 17%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/boosting.cpp.o
   [ 21%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt.cpp.o
   [ 25%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/prediction_early_stop.cpp.o
   [ 28%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/metadata.cpp.o
   [ 32%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/bin.cpp.o
   [ 35%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/tree.cpp.o
   [ 39%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/parser.cpp.o
   [ 42%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset_loader.cpp.o
   [ 46%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/config.cpp.o
   [ 50%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset.cpp.o
   [ 53%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/dcg_calculator.cpp.o
   [ 57%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/metric.cpp.o
   [ 60%] Building CXX object CMakeFiles/_lightgbm.dir/src/objective/objective_function.cpp.o
   [ 64%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linker_topo.cpp.o
   [ 67%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/network.cpp.o
   [ 71%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_socket.cpp.o
   [ 75%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_mpi.cpp.o
   [ 78%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/gpu_tree_learner.cpp.o
   [ 82%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/feature_parallel_tree_learner.cpp.o
   [ 85%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.o
   [ 89%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/serial_tree_learner.cpp.o
   [ 92%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/tree_learner.cpp.o
   [ 96%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/voting_parallel_tree_learner.cpp.o
   [100%] Linking CXX shared library /home/cttsai/DataSci/mlc/LightGBM/python-package/lightgbm/lib_lightgbm.so
   [100%] Built target _lightgbm

[wxchan]

I think you should run python3 setup.py install --precompile after install c++ version, or python3 setup.py install --gpu to direct install gpu version for python. @guolinke can confirm this.

[guolinke]

@cttsai1985
We had simplified the python installation. You don't need the build the cli version first.
Refer to: https://github.com/Microsoft/LightGBM/tree/master/python-package

[wxchan]

@guolinke Can we check precompile first when run python setup.py install?

[guolinke]

@wxchan I don't think so. I think make it independently is better.

[wxchan]

It's kind of counter-intuitive. I think most users don't know those --gpu, --precompile options.

[cttsai1985]

@guolinke @wxchan
Thanks for your reply. I followed the instruction below but got more error messages:
https://github.com/Microsoft/LightGBM/tree/master/python-package
Now the installation works for me to enable GPU for python still is the old way. Install a CPU-only version then replace the lib_lightgbm.so . Does it have anything to do with libOpenCL.so?

1. sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ ..
2. make -j$(nproc)
3. sudo python3 setup.py install
4. cd /usr/local/lib/python3.5/dist-packages/lightgbm
5. sudo cp ~/LightGBM/lib_lightgbm.so ./lib_lightgbm.so

Here is some log I collected:

Using pip installation:

1.1 sudo pip3 install --no-binary :all: lightgbm

It installed CPU-only version, so I get
lightgbm.basic.LightGBMError: b'GPU Tree Learner was not enabled in this build. Recompile with CMake option -DUSE_GPU=1'

1.2 sudo pip3 install lightgbm --install-option=--gpu

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-build-mnysq7z9/lightgbm/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-42d4zwxk-record/install-record.txt --single-version-externally-managed --compile --gpu" failed with error code 1 in /tmp/pip-build-mnysq7z9/lightgbm/

Using python3 setup.py install --precompile or --gpu

Both got the same error message:

Traceback (most recent call last):
File "hybrid.py", line 133, in
import lightgbm as lgb
File "/usr/local/lib/python3.5/dist-packages/lightgbm/init.py", line 8, in
from .basic import Booster, Dataset
File "/usr/local/lib/python3.5/dist-packages/lightgbm/basic.py", line 31, in
_LIB = _load_lib()
File "/usr/local/lib/python3.5/dist-packages/lightgbm/basic.py", line 26, in _load_lib
lib = ctypes.cdll.LoadLibrary(lib_path[0])
File "/usr/lib/python3.5/ctypes/init.py", line 425, in LoadLibrary
return self._dlltype(name)
File "/usr/lib/python3.5/ctypes/init.py", line 347, in init
self._handle = _dlopen(self._name, mode)

OSError: /usr/local/lib/python3.5/dist-packages/lightgbm/lib_lightgbm.so: symbol clCreateCommandQueueWithProperties, version OPENCL_2.0 not defined in file libOpenCL.so.1 with link time reference

2.1 sudo python3 setup.py install --precompile

running install
Install lib_lightgbm from: ['../lib_lightgbm.so']
running build
running build_py
creating build
creating build/lib
creating build/lib/lightgbm
copying lightgbm/plotting.py -> build/lib/lightgbm
copying lightgbm/init.py -> build/lib/lightgbm
copying lightgbm/libpath.py -> build/lib/lightgbm
copying lightgbm/callback.py -> build/lib/lightgbm
copying lightgbm/engine.py -> build/lib/lightgbm
copying lightgbm/sklearn.py -> build/lib/lightgbm
copying lightgbm/compat.py -> build/lib/lightgbm
copying lightgbm/basic.py -> build/lib/lightgbm
running egg_info
creating lightgbm.egg-info
writing lightgbm.egg-info/PKG-INFO
writing top-level names to lightgbm.egg-info/top_level.txt
writing dependency_links to lightgbm.egg-info/dependency_links.txt
writing requirements to lightgbm.egg-info/requires.txt
writing manifest file 'lightgbm.egg-info/SOURCES.txt'
reading manifest file 'lightgbm.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'build'
warning: no files found matching 'LICENSE'
warning: no files found matching '.txt'
warning: no files found matching '.so' under directory 'lightgbm'
warning: no files found matching '.dll' under directory 'lightgbm/Release'
warning: no files found matching '' under directory 'lightgbm/include'
warning: no files found matching '' under directory 'lightgbm/src'
warning: no previously-included files matching '.pyo' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
writing manifest file 'lightgbm.egg-info/SOURCES.txt'
copying lightgbm/VERSION.txt -> build/lib/lightgbm
running install_lib
copying build/lib/lightgbm/plotting.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/init.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/libpath.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/callback.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/engine.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/VERSION.txt -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/sklearn.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/compat.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
copying build/lib/lightgbm/basic.py -> /usr/local/lib/python3.5/dist-packages/lightgbm
Install lib_lightgbm from: ['../lib_lightgbm.so']
copying ../lib_lightgbm.so -> /usr/local/lib/python3.5/dist-packages/lightgbm
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/plotting.py to plotting.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/init.py to init.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/libpath.py to libpath.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/callback.py to callback.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/engine.py to engine.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/sklearn.py to sklearn.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/compat.py to compat.cpython-35.pyc
byte-compiling /usr/local/lib/python3.5/dist-packages/lightgbm/basic.py to basic.cpython-35.pyc
running install_data
copying ../lib_lightgbm.so -> /usr/local/lightgbm
running install_egg_info
removing '/usr/local/lib/python3.5/dist-packages/lightgbm-2.0.4-py3.5.egg-info' (and everything under it)
Copying lightgbm.egg-info to /usr/local/lib/python3.5/dist-packages/lightgbm-2.0.4-py3.5.egg-info
running install_scripts

2.2 sudo python3 setup.py install --gpu
The installation log is very long and doing a lot of copy. It may recomple a new binary according to the following message.

Start to compile libarary.
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -fopenmp
-- Looking for CL_VERSION_2_0
-- Looking for CL_VERSION_2_0 - found
-- Found OpenCL: /usr/lib/x86_64-linux-gnu/libOpenCL.so (found version "2.0")
-- OpenCL include directory:/usr/include
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- filesystem
-- system
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cttsai/DataSci/mlc/LightGBM/python-package/build_cpp
Scanning dependencies of target _lightgbm
[ 3%] Building CXX object CMakeFiles/_lightgbm.dir/src/c_api.cpp.o
[ 7%] Building CXX object CMakeFiles/_lightgbm.dir/src/lightgbm_R.cpp.o
[ 10%] Building CXX object CMakeFiles/_lightgbm.dir/src/application/application.cpp.o
[ 14%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt_prediction.cpp.o
[ 17%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/boosting.cpp.o
[ 21%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/gbdt.cpp.o
[ 25%] Building CXX object CMakeFiles/_lightgbm.dir/src/boosting/prediction_early_stop.cpp.o
[ 28%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/metadata.cpp.o
[ 32%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/bin.cpp.o
[ 35%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/tree.cpp.o
[ 39%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/parser.cpp.o
[ 42%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset_loader.cpp.o
[ 46%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/config.cpp.o
[ 50%] Building CXX object CMakeFiles/_lightgbm.dir/src/io/dataset.cpp.o
[ 53%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/dcg_calculator.cpp.o
[ 57%] Building CXX object CMakeFiles/_lightgbm.dir/src/metric/metric.cpp.o
[ 60%] Building CXX object CMakeFiles/_lightgbm.dir/src/objective/objective_function.cpp.o
[ 64%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linker_topo.cpp.o
[ 67%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/network.cpp.o
[ 71%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_socket.cpp.o
[ 75%] Building CXX object CMakeFiles/_lightgbm.dir/src/network/linkers_mpi.cpp.o
[ 78%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/gpu_tree_learner.cpp.o
[ 82%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/feature_parallel_tree_learner.cpp.o
[ 85%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/data_parallel_tree_learner.cpp.o
[ 89%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/serial_tree_learner.cpp.o
[ 92%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/tree_learner.cpp.o
[ 96%] Building CXX object CMakeFiles/_lightgbm.dir/src/treelearner/voting_parallel_tree_learner.cpp.o
[100%] Linking CXX shared library /home/cttsai/DataSci/mlc/LightGBM/python-package/lightgbm/lib_lightgbm.so
[100%] Built target _lightgbm
Install lib_lightgbm from: ['lightgbm/lib_lightgbm.so', '../lib_lightgbm.so']
running build

[guolinke]

@cttsai1985
it seems there are some problem of your local opencl env.
Can you run the example by the gpu-version cli LightGBM ?

[guolinke]

@huanzhang12
any idea about this error ?

OSError: /usr/local/lib/python3.5/dist-packages/lightgbm/lib_lightgbm.so: symbol clCreateCommandQueueWithProperties, version OPENCL_2.0 not defined in file libOpenCL.so.1 with link time reference

[cttsai1985]

@guolinke as you wish.
../../lightgbm config=train.conf data=binary.train valid=binary.test device=gpu
[LightGBM] [Info] Finished loading parameters
[LightGBM] [Info] Loading weights...
[LightGBM] [Info] Loading weights...
[LightGBM] [Info] Finished loading data in 0.189428 seconds
[LightGBM] [Info] Number of positive: 3716, number of negative: 3284
[LightGBM] [Info] This is the GPU trainer!!
[LightGBM] [Info] Total Bins 6143
[LightGBM] [Info] Number of data: 7000, number of used features: 28
[LightGBM] [Info] Using GPU Device: GeForce GTX 1080, Vendor: NVIDIA Corporation
[LightGBM] [Info] Compiling OpenCL Kernel with 256 bins...
[LightGBM] [Info] GPU programs have been built
[LightGBM] [Info] Size of histogram bin entry: 12
[LightGBM] [Info] 28 dense feature groups (0.19 MB) transfered to GPU in 0.012709 secs. 0 sparse feature groups.
[LightGBM] [Info] Finished initializing training
[LightGBM] [Info] Started training...
[LightGBM] [Info] Size of histogram bin entry: 12
[LightGBM] [Info] 28 dense feature groups (0.15 MB) transfered to GPU in 0.034523 secs. 0 sparse feature groups.
[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=10
[LightGBM] [Info] Iteration:1, training auc : 0.778676
[LightGBM] [Info] Iteration:1, training binary_logloss : 0.669124
[LightGBM] [Info] Iteration:1, valid_1 auc : 0.716702
[LightGBM] [Info] Iteration:1, valid_1 binary_logloss : 0.674992
[LightGBM] [Info] 10.432616 seconds elapsed, finished iteration 1
[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11
[LightGBM] [Info] Iteration:2, training auc : 0.80241
[LightGBM] [Info] Iteration:2, training binary_logloss : 0.651141
[LightGBM] [Info] Iteration:2, valid_1 auc : 0.740132
[LightGBM] [Info] Iteration:2, valid_1 binary_logloss : 0.659703
[LightGBM] [Info] 19.234123 seconds elapsed, finished iteration 2
[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=9
[LightGBM] [Info] Iteration:3, training auc : 0.824324
[LightGBM] [Info] Iteration:3, training binary_logloss : 0.635504
[LightGBM] [Info] Iteration:3, valid_1 auc : 0.765649
[LightGBM] [Info] Iteration:3, valid_1 binary_logloss : 0.647252
[LightGBM] [Info] 26.543888 seconds elapsed, finished iteration 3

[huanzhang12]

Two OpenCL libraries are installed on this system. One is provided by Nvidia (version 1.2), and another one is a generic installable client driver (ICD) OpenCL provided by OS (version 2.0). I guess the reason is that LightGBM seems to be linked to the system OpenCL (2.0), but a LD_LIBRARY_PATH setting causes the Nvidia OpenCL to be loaded first, and is incompatible with library that LightGBM was linked with.

The problem with the --gpu flag is that we can't specify OpenCL library path. So first clean up everything, then run sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ .. to build LightGBM manually, and then run sudo python3 setup.py install --precompile. Don't run sudo python3 setup.py install --gpu, as this will link to the default (incorrect) OpenCL library.

@cttsai1985 For further debugging this problem, you can try to run ldd with lib_lightgbm.so and see which libraries it links to. You can also set LD_DEBUG=libs and run lightgbm CLI and see the linker debugging information. Make sure lightgbm is using the same OpenCL library as the one cmake finds during compilation time.

[guolinke]

Thanks @huanzhang12

@cttsai1985 before you using huanzhuang12's solution, you should delete .so file in LightGBM/python-package (or LightGBM/python-package/lightgbm) folder.
and you may need to uninstall lightgbm first by pip uninstall lightgbm

[guolinke]

@huanzhang12
To avoid using -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/, can we set some environment variables for OpenCL_LIBRARY and OpenCL_INCLUDE_DIR ?

[cttsai1985]

Thanks @huanzhang12 @guolinke

In fact, the installation instruction mentioned by @huanzhang12 is how I built LightGBM w/ GPU currently. Without addling these paths I can not pass the cmake. That is why I had already followed the issue #458 by "sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ .." to get it worked. And you've all see the results using "sudo python3 setup.py install --precompile" after successful compiled. It is there #715 (comment)

Regarding to the ldd part:
ldd lib_lightgbm.so

linux-vdso.so.1 =>  (0x00007ffff41fc000)
libOpenCL.so.1 => /usr/local/cuda-8.0/lib64/libOpenCL.so.1 (0x00007f17e64b8000)
libboost_filesystem.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.58.0 (0x00007f17e627f000)
libboost_system.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.58.0 (0x00007f17e607a000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f17e5cf8000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f17e59ef000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f17e57cc000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f17e55b6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f17e5399000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f17e4fce000)
/lib64/ld-linux-x86-64.so.2 (0x0000558975e03000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f17e4dca000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f17e4bc2000)

[guolinke]

@cttsai1985
It can work if you manually replace the lib_lightgbm.so file ?
if yes, using --precomplie should work as well. Maybe you can try to delete the LightGBM folder and re-clone then re-build again.

[cttsai1985]

@guolinke
Yeah, it was my work around solution. I don’t mind to delete LightGBM one more time today. :)
But this time it works! Thanks you guys @huanzhang12 @guolinke @wxchan working so hard to get this issue solved. The rest part is what did I do this time to remove previous installation and have a new one.

Before this time I remove LightGBM packages in python by a solution found in stacked-overflow:

sudo python3 setup.py install --precompile --record files.txt
cat files.txt | sudo xargs rm -rf

Since the lightgbm lightgbm-2.0.4-py3.5.egg-info folder I would going to replace it later so I deleted both of them anyway this time. Maybe there was something mess around.

sudo python3 setup.py install --precompile --record files.txt
cat files.txt | sudo xargs rm -rf
cd /usr/local/lib/python3.5/dist-packages/
sudo rm -r -f lightgbm lightgbm-2.0.4-py3.5.egg-info/

I re-installed by the following steps:

git clone --recursive https://github.com/Microsoft/LightGBM
cd ./LightGBM
mkdir build; cd build
sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ ..
sudo make -j$(nproc)
cd ../python-package; sudo python3 setup.py install --precompile

[chjq201410695]

I successly install lightgbm gpu version like this link. but when I use lightgbm in jupyter lab, there is a

ModuleNotFoundError: No module named 'lightgbm'

How could I solve this problem? do you have any advise? @guolinke

[guolinke]

i think the reason is missing some dependencies in your jupyter, or it is not 64 bit version.