LightGBM GPU Python 3.7 CUDA 10.1 Windows boost 1.69 compilation error

[dishkakrauch]

Environment info

Operating System: Windows 10

CPU/GPU model: Interl Core i7-8700 3.2 Ghz/NVIDIA GeForce GTX 1060 6GB

C++/Python/R version: Python 3.71

LightGBM version or commit hash: 2.2.1

Error message

Reproducible examples

System variable PATH:

Steps to reproduce

1. Install Visual Studio 2017 with C++ packages
2. Install CUDA 10.1
3. Install MinGW 8.1
4. Unpack boost 1.69 in C:\boost and follow steps:
   4.1 cd C:\boost\boost_1_69_0\tools\build
   4.2 bootstrap.bat gcc
   4.3 b2 install --prefix="C:\boost\boost-build" toolset=gcc
   4.4 cd C:\boost\boost_1_69_0
   4.5 b2 install --build_dir="C:\boost\boost-build" --prefix="C:\boost\boost-build" toolset=gcc --with=filesystem,system threading=multi --layout=system release -j 12 (I have 12 cores thats why last parameter is equal 12)
   Then I get error above.
   Can you please give me any suggestions?
   I have read all guides related to lightgbm gpu version compilation include this:
   https://lightgbm.readthedocs.io/en/latest/GPU-Windows.html
   Tried to reinstall everything from start to end and still can't solve this problem...

[dishkakrauch]

@StrikerRUS hope it's enough information, waiting for you help, thanks.

[StrikerRUS]

@dishkakrauch Sure, let's try!

1. Install Visual Studio 2017 with C++ packages

3. Install MinGW 8.1

How would you like to use GPU version of LightGBM: with VS or MinGW?

Note: the first variant (Visual Studio) is recommended UPD: ... and easier.

[dishkakrauch]

@StrikerRUS of course, I prefer VS.

p.s. sorry, I accidentally closed issue, but reopened it in a second.

[StrikerRUS]

OK. Then please download https://bintray.com/boostorg/release/boost-binaries/1.69.0#files (if not already) and unzip them into some folder (C:\boost, for example).

[StrikerRUS]

Please post screenshot here of resulted folder structure of unzipped Boost - I'll adjust paths in next commands.

[dishkakrauch]

@StrikerRUS Done! I downloaded all files according to the link you sent before and unzipped file boost_1_69_0-bin-msvc-all-32-64.7z to folder boost_1_69_0.
Here is screenshot of all files in folder C:\boost

Waiting for your reply!

[StrikerRUS]

@dishkakrauch
Please try to run the following commands:

git clone --recursive https://github.com/Microsoft/LightGBM
cd LightGBM
mkdir build
cd build
cmake -DCMAKE_GENERATOR_PLATFORM=x64 -DUSE_GPU=1 -DBOOST_ROOT=C:/boost/boost_1_69_0 -DBOOST_LIBRARYDIR=C:/boost/boost_1_69_0/lib64-msvc-14.1 ..
cmake --build . --target ALL_BUILD --config Release

[dishkakrauch]

@StrikerRUS Done!
It looks like everying went well!


What's next? Should I install LightGBM-gpu via conda or pip?

[StrikerRUS]

@dishkakrauch Good!

Now install Python package with this just precompiled dll.

cd ../python-package  # assuming you're in LightGBM/build folder now
python setup.py install --precompile

UPD: Don't forget to remove previously installed LightGBM Python package (if any) before running these commands.

[dishkakrauch]

@StrikerRUS Done without errors!

[StrikerRUS]

@dishkakrauch Great!
Now check that you can use LightGBM GPU:

import lightgbm as lgb
from sklearn.datasets import load_boston

data = lgb.Dataset(*load_boston(True))
lgb.train({'device': 'gpu'}, data)

[dishkakrauch]

@StrikerRUS Thanks!

Holy Moly! It's working! Can you tell why didn't we use conda or pip install? How should I update LightGBM to the newest version in future?

[StrikerRUS]

@dishkakrauch No problem!
Рад, что смог помочь! :-)

Unfortunately, we do not maintain conda receipts. So you cannot install LightGBM via conda (at least official one, I'm not sure, but you can search conda-forge for unofficial one).

By the commands above you've installed the latest version of LightGBM from GitHub. You can repeat them in the future to update LightGBM. But again, don't forget to remove previous installation. I suppose you're using conda, so it's possible to do via pip uninstall lightgbm.

Speaking about pip, there is a delay between GitHub and latest PyPI files. Usually, it's about 2 months or more. Also, by default, pip install lightgbm will install ordinary CPU version. To install GPU version, in your case you should run the following command:

pip install lightgbm --install-option=--gpu --install-option="--boost-root=C:/boost/boost_1_69_0" --install-option="--boost-include-dir=C:/boost/boost_1_69_0/lib64-msvc-14.1"

https://github.com/Microsoft/LightGBM/tree/master/python-package#build-gpu-version

But please don't forget that it will not be the latest version!

In addition, you may want to know about that after we've successfully adjusted all paths, it's highly likely that the following command will do all job:

git clone --recursive https://github.com/Microsoft/LightGBM
cd LightGBM/python-package
python setup.py install --gpu --boost-root=C:/boost/boost_1_69_0 --boost-include-dir=C:/boost/boost_1_69_0/lib64-msvc-14.1

It will install latest GPU version from GitHub, just like you did today but with fewer commands.

[dishkakrauch]

@StrikerRUS Thanks for the details!
I understand now a bit more about GPU python libraries compilation.
Let me ask a few questions - why lightgbm lib is not in pip list?

And one more - is pip installation of lightgbm gpu version slower than building it from source like we did?
BTW, I installed xgboost like that too.

p.s. особенно приятно, что можно и по-русски))

[StrikerRUS]

@dishkakrauch

why lightgbm lib is not in pip list?

Hmmm... I have no idea. Can you try conda list? LightGBM should be there with pip channel.

And one more - is pip installation of lightgbm gpu version slower than building it from source like we did?

Speaking about GPU version, there is no difference. All actions will be the same under the hood - compilation from sources will be performed anyway. The difference will be only in code freshness: the latest from GitHub and 2 months+ from PyPI.
Fast installation via pip is possible only from wheel file in which precompiled dll is already placed, then (roughly) just copy-paste operation is performed. But we do not provide GPU wheels due to compatibility reasons. GPU version can be installed only from sources.
Why we can then install GPU version via pip? The thing is that any option after pip install package_name, like --gpu or --boost-root, forces pip to download not wheel file, but tar.gz archive with sources (without precompiled dll) and perform compilation from sources.

$> pip install lightgbm --install-option=--gpu
UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.

https://pypi.org/project/lightgbm/#files

[dishkakrauch]

@StrikerRUS thanks again for full explanation!
It's really strange cause I can't see lightgbm packge via conda list or pip list command in cmd, but it's available inside jupyter notebook cmd command...

[StrikerRUS]

@dishkakrauch Да не за что!

For LightGBM-not-listed error I got an idea: please provide the output of conda info --envs command. Maybe those lists are from different conda environments...

[dishkakrauch]

@StrikerRUS I have just base env...

[StrikerRUS]

@dishkakrauch Strange... Something is wrong with conda, definitely.
What does output conda info --envs inside notebook's cell?

[dishkakrauch]

@StrikerRUS same...

[dishkakrauch]

@StrikerRUS solved aftrer "conda install annaconda" command!

But after opening and closing juptyter notebook I got this error:

Is always suggest me to downgrade python to 3.6...

[dishkakrauch]

Hm... Actually my python version had been downgraded from 3.7.1 to 3.6.5.final.0 but I don't understad which package did that.
I built lightgbm from source, then xgboost form source and then installed everying related to tensorflow, keras e.t.c. but there weren't suggestions for python downgrading. Interesting...

[dishkakrauch]

@StrikerRUS finally I reinstalled Anaconda and all packages from scratch.
I can see LightGBM in conda and pip list right now and it's working on gpu.
Thanks again, now we can close the issue.

p.s. спасибо, не ожидал, что по open-source пакету может быть такой фидбэк; в итоге со всем разобрался!

[StrikerRUS]

@dishkakrauch I'm very glad to see that all your problems have been solved!

finally I reinstalled Anaconda and all packages from scratch.

Yeah, sometimes only reinstalling from scratch can help 😃 .

Actually my python version had been downgraded from 3.7.1 to 3.6.5.final.0 but I don't understad which package did that.

It wasn't LightGBM. That's for sure. We support Python 3.7 and 3.6. Moreover, we do not restrict Python version by python_requires option, allowing users to try to build under any version they want.

Не за что! Здорово, что всё в итоге получилось) А фидбэк везде по-разному: где-то скинут ссыль на мануал и иди кури его (хорошо, если ещё главу подскажут); а вот, например, в соседнем проекте чувак предлагает даже созвониться по скайпу, чтобы помочь устранить проблему. И такое бывает)

Удачи на Каггле (вижу, что ноутбук из той папки)!