Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip install failure #463

Closed
cliveseldon opened this issue Sep 2, 2015 · 72 comments
Closed

pip install failure #463

cliveseldon opened this issue Sep 2, 2015 · 72 comments

Comments

@cliveseldon
Copy link

@cliveseldon cliveseldon commented Sep 2, 2015

root@0c6c17725a7b:/# pip install xgboost
Downloading/unpacking xgboost
Could not find a version that satisfies the requirement xgboost (from versions: 0.4a12, 0.4a13)
Cleaning up...
No distributions matching the version for xgboost
Storing debug log for failure in /root/.pip/pip.log

You can repeat in docker with:
docker run -it --rm ubuntu:trusty

apt-get update
apt-get install python-pip
pip install xgboost

see this also:

http://stackoverflow.com/questions/32258463/install-xgboost-under-python-failing

@grfiv
Copy link

@grfiv grfiv commented Sep 5, 2015

Does not seem to work for me

ubuntu@ip-172-31-11-169:$ sudo apt-get update
... yadda yadda
ubuntu@ip-172-31-11-169:
$ sudo apt-get install python-pip
Reading package lists... Done
Building dependency tree
Reading state information... Done
python-pip is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 74 not upgraded.
ubuntu@ip-172-31-11-169:$ sudo pip install xgboost
Downloading/unpacking xgboost
Could not find a version that satisfies the requirement xgboost (from versions: 0.4a12, 0.4a13)
Cleaning up...
No distributions matching the version for xgboost
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ip-172-31-11-169:
$

@grfiv
Copy link

@grfiv grfiv commented Sep 5, 2015

The stackoverflow suggestion of --pre produced this:

ubuntu@ip-172-31-11-169:~$ sudo pip install --pre xgboost
Downloading/unpacking xgboost
Downloading xgboost-0.4a13.tar.gz (690kB): 690kB downloaded
Running setup.py (path:/tmp/pip_build_root/xgboost/setup.py) egg_info for package xgboost
xgboost/build-python.sh: 13: xgboost/build-python.sh: pushd: not found
make: *** No rule to make target python'. Stop. ----------------------------- Building multi-thread xgboost failed Start to build single-thread xgboost make: *** No rule to make targetclean'. Stop.
make: *** No rule to make target `python'. Stop.
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md
xgboost/build-python.sh: 26: xgboost/build-python.sh: popd: not found
(None, None)
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/xgboost/setup.py", line 20, in
import xgboost
File "./xgboost/init.py", line 8, in
from .core import DMatrix, Booster
File "./xgboost/core.py", line 78, in
_LIB = _load_lib()
File "./xgboost/core.py", line 72, in _load_lib
lib = ctypes.cdll.LoadLibrary(lib_path[0])
File "/usr/lib/python2.7/ctypes/init.py", line 443, in LoadLibrary
return self._dlltype(name)
File "/usr/lib/python2.7/ctypes/init.py", line 365, in init
self._handle = _dlopen(self._name, mode)
OSError: /tmp/pip_build_root/xgboost/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header
Complete output from command python setup.py egg_info:
xgboost/build-python.sh: 13: xgboost/build-python.sh: pushd: not found

make: *** No rule to make target `python'. Stop.


Building multi-thread xgboost failed

Start to build single-thread xgboost

make: *** No rule to make target `clean'. Stop.

make: *** No rule to make target `python'. Stop.

Successfully build single-thread xgboost

If you want multi-threaded version

See additional instructions in doc/build.md

xgboost/build-python.sh: 26: xgboost/build-python.sh: popd: not found

(None, None)

Traceback (most recent call last):

File "", line 17, in

File "/tmp/pip_build_root/xgboost/setup.py", line 20, in

import xgboost

File "./xgboost/init.py", line 8, in

from .core import DMatrix, Booster

File "./xgboost/core.py", line 78, in

_LIB = _load_lib()

File "./xgboost/core.py", line 72, in _load_lib

lib = ctypes.cdll.LoadLibrary(lib_path[0])

File "/usr/lib/python2.7/ctypes/init.py", line 443, in LoadLibrary

return self._dlltype(name)

File "/usr/lib/python2.7/ctypes/init.py", line 365, in init

self._handle = _dlopen(self._name, mode)

OSError: /tmp/pip_build_root/xgboost/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header


Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/xgboost
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ip-172-31-11-169:~$

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Sep 18, 2015

@cliveseldon @grfiv the versioning of pip installation has some confusions, since the official version 0.4 has been there for a long time, and the pip installation contains some recent update from the source code, so I have named the version with 0.4a12 0.4a13 something. It will be corrected later from the next official release.

A good solution for updating pip installation can be, deleting the xgboost directory and the xgboost-0.4a13.dist-info directory both from the site package directory, e.g. /User/phunterlau/python2.7/site-packages/ (the installation location can be found from print xgboost.__file__ ), and redo 'pip install xgboost' . Thanks.

@Fkawala
Copy link

@Fkawala Fkawala commented Sep 23, 2015

Hello,
I have the same problem on my ubuntu box:

To run :

sudo -H pip install --pre xgboost
python -c "import xgboost"

Fails as follows:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/xgboost/__init__.py", line 11, in <module>
    from .core import DMatrix, Booster
  File "/usr/local/lib/python2.7/dist-packages/xgboost/core.py", line 92, in <module>
    _LIB = _load_lib()
  File "/usr/local/lib/python2.7/dist-packages/xgboost/core.py", line 86, in _load_lib
    lib = ctypes.cdll.LoadLibrary(lib_path[0])
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python2.7/dist-packages/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header

I've also tried without --pre, it fails identically.

Any clue ?
Thanks,
F.

@AZaitzeff
Copy link

@AZaitzeff AZaitzeff commented Sep 23, 2015

I used pip install xgboost to install xgboost-0.4a14 on a mac OSX. When I tried loading it I got this error.

OSError Traceback (most recent call last)
in ()
----> 1 import xgboost as xgb

/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/init.py in ()
9 import os
10
---> 11 from .core import DMatrix, Booster
12 from .training import train, cv
13 from .sklearn import XGBModel, XGBClassifier, XGBRegressor

/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/core.py in ()
90
91 # load the XGBoost library globally
---> 92 _LIB = _load_lib()
93
94 def _check_call(ret):

/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/core.py in _load_lib()
84 if len(lib_path) == 0:
85 return None
---> 86 lib = ctypes.cdll.LoadLibrary(lib_path[0])
87 lib.XGBGetLastError.restype = ctypes.c_char_p
88 return lib

/Users/azaitzeff/anaconda/lib/python2.7/ctypes/init.pyc in LoadLibrary(self, name)
441
442 def LoadLibrary(self, name):
--> 443 return self._dlltype(name)
444
445 cdll = LibraryLoader(CDLL)

/Users/azaitzeff/anaconda/lib/python2.7/ctypes/init.pyc in init(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle

OSError: dlopen(/Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/./wrapper/libxgboostwrapper.so, 6): Library not loaded: /usr/local/lib/gcc/4.9/libgomp.1.dylib
Referenced from: /Users/azaitzeff/anaconda/lib/python2.7/site-packages/xgboost/./wrapper/libxgboostwrapper.so
Reason: image not found

Any help. Thanks
AZ

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Sep 23, 2015

@AZaitzeff @Fkawala Thanks. Can you confirm if you have gcc installed on your ubuntu or Mac? With your information, I see the problem of precompiled lib: if there is no gcc installed on the system, the build script in pip would fail. However, in the pip installation package, I included a precompiled lib libxgboostwrapper.so using gcc 4.9 from brew in Mac, so if the build script failed, xgboost will search for this precompiled lib. Linux doesn't recognize this Mac lib, so it has ELF code error; For mac with no OpenMP, it will trigger the link error. Now I see it causes some confusions and it might not be a good idea. I will update the pip installation, and display the message if there is no gcc. P.S. We will have precompiled and static linked lib in the next official release, which is a better solution of shipping precompiled code.

@AZaitzeff
Copy link

@AZaitzeff AZaitzeff commented Sep 23, 2015

@phunterlau Thanks. I do not have OpenMP on my Mac. So that is what was causing the error.

@Fkawala
Copy link

@Fkawala Fkawala commented Sep 23, 2015

@phunterlau ok, I'll set up a proper compilation environment and test the pip installation again.

Thanks for the quick reply.

@AZaitzeff
Copy link

@AZaitzeff AZaitzeff commented Sep 23, 2015

So I ran brew install clang-omp and it installed. Deleted the xgboost directory and the xgboost-0.4a13.dist-info and reran pip install xgboost. Closed my terminal windows and reopened them. But I am still getting the same error when trying to import xgboost. Any thoughts?

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Sep 23, 2015

@AZaitzeff actually clang-omp is not recommended, since clang from Xcode doesn't support it so clang-omp may give some strange result. The mac pip installation package was created and tested on gcc 4.9 from brew with OpenMP. Do you want to try with brew install gcc49? Now we have gcc 5 from brew too, but I haven't had a chance for fully test it. I will do it in the next pip upload.

@AZaitzeff
Copy link

@AZaitzeff AZaitzeff commented Sep 24, 2015

Thanks that worked great.

@Fkawala
Copy link

@Fkawala Fkawala commented Sep 24, 2015

@phunterlau the build (using ./xgboost/build-python.sh) works like a charm. However, the pip installation results in the pre-compiled lib. I modified setup.py to show why the compilation is not attempted. With PIP 7.1.2 (python 2.7), to run:

pip install --user -e ./tweaked-xgboost-0.4a13/

gives:

'pip' in __file__: False
not os.name == 'nt': True
@Fkawala
Copy link

@Fkawala Fkawala commented Sep 24, 2015

According to http://stackoverflow.com/questions/9959279/detect-pip-in-setup-py [1] the 'pip' in __file__ test does not work with Py 2.7 2014-10-27 nor with custom installation (as I used in my previous comment). As proposed in [1] 'pip' in os.environ.get('_') is a more standard way to test if pip is runs the setup.py.

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Sep 24, 2015

@Fkawala Thanks. 'pip' in __file__ was an old trick from long time ago, nice to know the new one. I will update the pip installation and later on send a PR (with some other patches).

@antklen
Copy link

@antklen antklen commented Oct 11, 2015

@phunterlau I have similar problem, didn't understand how to fix this ELF error. I'm working on ubuntu and can't import xgboost package after installing it with pip. You wrote about gcc, I installed gcc-4.9, but still don't work, what I'm missing?

Here is my error:

import xgboost

OSError Traceback (most recent call last)
in ()
----> 1 import xgboost
/home/antklen/anaconda/lib/python2.7/site-packages/xgboost/init.py in ()
9 import os
10
---> 11 from .core import DMatrix, Booster
12 from .training import train, cv
13 from .sklearn import XGBModel, XGBClassifier, XGBRegressor
/home/antklen/anaconda/lib/python2.7/site-packages/xgboost/core.py in ()
90
91 # load the XGBoost library globally
---> 92 _LIB = _load_lib()
93
94 def _check_call(ret):
/home/antklen/anaconda/lib/python2.7/site-packages/xgboost/core.py in _load_lib()
84 if len(lib_path) == 0:
85 return None
---> 86 lib = ctypes.cdll.LoadLibrary(lib_path[0])
87 lib.XGBGetLastError.restype = ctypes.c_char_p
88 return lib
/home/antklen/anaconda/lib/python2.7/ctypes/init.pyc in LoadLibrary(self, name)
441
442 def LoadLibrary(self, name):
--> 443 return self._dlltype(name)
444
445 cdll = LibraryLoader(CDLL)
/home/antklen/anaconda/lib/python2.7/ctypes/init.pyc in init(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle
OSError: /home/antklen/anaconda/lib/python2.7/site-packages/xgboost/./wrapper/libxgboostwrapper.so: invalid ELF header

@smargs
Copy link

@smargs smargs commented Oct 11, 2015

@antklen in windows you need to install setuptools first. i'm not sure if that's the case for ubuntu, but maybe worth trying?

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Oct 11, 2015

@antklen it is because the current pip installation is a on-the-fly compiler script and it ships with a Mac version of .so file, in case some mac users didn't install g++ for compilation. It turned out not a good idea and caused some confusions for Linux users.

The solution is that, please install gcc and g++ apt-get install g++, so it will pickup the compiling script.

And, if your pip is newer than 0.7.1.2, it may have another problem of checking if pip is running. I will fix and ship it in the next release. If you have g++ but still the same error, please install from setup.py manually by following the installation document.

@antklen
Copy link

@antklen antklen commented Oct 12, 2015

@smargsongithub @phunterlau Thank you for replies!
apt-get install g++ also didn't work for me, but I succeed with manual installation as described here https://www.kaggle.com/c/springleaf-marketing-response/forums/t/16320/cloud-vs-local-machine-pros-and-cons
I had some issues beacause I'm using Anaconda, so my final recipe is

sudo apt-get install make
sudo apt-get update
sudo apt-get install gcc
sudo apt-get install g++
sudo apt-get install git
sudo git clone https://github.com/dmlc/xgboost
cd xgboost
sudo ./build.sh
cd python-package
sudo /home/username/anaconda/bin/python setup.py install

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Oct 12, 2015

@antklen oh, you need sudo when install g++ :-)

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Oct 18, 2015

@Fkawala I am trying 'pip' in os.environ.get('_') for the updated pip, but os.environ.get('_') always returns None and this if statement gives error. Do we have other method for detecting if pip in the command line? Otherwise, I would create a separate setup.py just for pip and trigger the compiler there. Thanks.

@pcohen89
Copy link

@pcohen89 pcohen89 commented Oct 24, 2015

@phunterlau I am getting the following traceback from pip install xgboost, any thoughts? (running a Mac)

Traceback (most recent call last):
File "", line 20, in
File "/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/setup.py", line 38, in
LIB_PATH = libpath'find_lib_path'
File "/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/libpath.py", line 47, in find_lib_path
'List of candidates:\n' + ('\n'.join(dll_path)))
builtin.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you run build.sh in root path?
List of candidates:
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/libxgboostwrapper.so
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/../../wrapper/libxgboostwrapper.so
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/./wrapper/libxgboostwrapper.so
/private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost/xgboost/../../../xgboost/wrapper/libxgboostwrapper.so

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/7g/thc6mmqd2nb55r21htwx2slr0000gp/T/pip-build-0BlCW9/xgboost

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Oct 24, 2015

@pcohen89 this python package ships with C++ code and compiles on-the-fly, so you might need a complier. Do you have your compiler installed, e.g. gcc-5 or clang LLVM from Xcode? Mac doesn't install xcode command line tools by default and you can follow this instruction http://railsapps.github.io/xcode-command-line-tools.html or, another better choice is installing homebrew http://brew.sh/ and brew install g++-5. The later option (g++-5) gives multi-thread OpenMP support.

After installing compilers, please remove the old xgboost installation, and redo pip install xgboost

tqchen added a commit that referenced this issue Oct 25, 2015
Fix data file shipping confusions on pip install for #463
@jeechu
Copy link

@jeechu jeechu commented Oct 26, 2015

@antklen @phunterlau Tried following the steps mentioned by @antklen . Now, I am able to run import xgboost on terminal. However on anacondas ipython notebook it gives me the below error. Any help please. I am using ubuntu.

XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you run build.sh in root path?
List of candidates:
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/libxgboostwrapper.so
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/../../wrapper/libxgboostwrapper.so
/home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/xgboost/./wrapper/libxgboostwrapper.so

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Oct 26, 2015

@jeechu that means xgboost was not compiled, have you:

  1. removed the old xgboost installation?
  2. installed xcode (mac), or gcc-5 (Mac uses brew, linux use apt-get or yum)?

pip install xgboost -v can tell more about error message.

@israjsingh
Copy link

@israjsingh israjsingh commented Oct 26, 2015

@phunterlau me and @jeechu we have removed the old xgboost installation and installed gcc,g++. XGBoost is running fine via terminal, but using anaconda ipython notebook it is giving that error.

Is there something wrong with the installation path?

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Oct 26, 2015

@rajdeltarobo @jeechu what is the python version of your ipython notebook? and, what is your $PYTHONPATH variables in the .bashrc (or other env files from your shells)?

Seems like your xgboost is search for /home/simpl/anaconda/lib/python2.7/site-packages/xgboost-0.4-py2.7.egg/ which is not a pip installation path (pip installation has its additional version number e.g. xgboost-0.4a24), so the ipython notebook tries searching your xgboost from python setup.py install from github, thus the problem may come from $PYTHONPATH

@israjsingh
Copy link

@israjsingh israjsingh commented Oct 26, 2015

@phunterlau $PYTHONPATH is not defined, its returning a blank. However, below is the value of path

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/simpl/anaconda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

The python path is

➜  ~  which python  
/usr/bin/python

The python version of ipython notebook is 2.7

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Oct 26, 2015

How about adding something like this to your PYTHONPATH:
/Users/simpl/anaconda/lib/python2.7/site-packages/

if you want to use anaconda's python, just stay with it for everything, otherwise use virtualenv instead of having multiple python installations. It is very confusing if there are multiple python installations on the the same machine.

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Apr 3, 2016

please change the default compiler in config.mk to clang-omp. Please follow the build instruction Mac OS X part.

@LiyanJin
Copy link

@LiyanJin LiyanJin commented Apr 4, 2016

Thx for ur instruction. And I changed the default compiler in config.mk to clang-omp like this
` export CC = /usr/local/bin/clang-omp

export CXX = /usr/local/bin/clang-omp++

ADD_CFLAGS = -fopenmp`

Then I compile by
make -j8

However, error appears again:
make: *** [lib/libxgboost.a] Error 1
make: *** Waiting for unfinished jobs....
clang-3.5clang-3.5: : errorerror: : no such file or directory: 'rabit/lib/librabit.a'no such file or directory: 'rabit/lib/librabit.a'

make: *** [lib/libxgboost.so] Error 1
make: *** [xgboost] Error 1

Is there anything wrong for me?

@LiyanJin
Copy link

@LiyanJin LiyanJin commented Apr 4, 2016

And I run
sudo ./build.sh
error appears as below

include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will
be compiled into single-thread code. Use OpenMP-enabled compiler to get
benefit of multi-threading. [-W#pragma-messages]
#pragma message("Warning: OpenMP is not available, "

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Apr 7, 2016

please try reinstalling clang-omp by brew remove clang-omp; brew install clang-omp; only change these two lines (no ADD_FLAG) in config.mk, and make sure to copy it to the same directory as Makefile is.

export CC = clang-omp
export CXX = clang-omp++

and run make

@boral
Copy link

@boral boral commented May 11, 2016

Using Windows 7. Anaconda from yhat Rodeo.

Trying to install xgboost and getting the following error. Please help.

`! pip install xgboost

Collecting xgboost
Using cached xgboost-0.4a30.tar.gz
No files/directories in C:\Users\USER\AppData\Local\Temp\pip-build-dkmqc8n1\xgboost\pip-egg-info (from PKG-INFO)`

@LiyanJin
Copy link

@LiyanJin LiyanJin commented May 15, 2016

@phunterlau Thx for your suggestion. I tried your instruction and getting the following error. Please give some suggestion.
I reinstalled clang-imp and changed two lines (no ADD_FLAG) in config.mk in the same directory as Makefile is

export CC = clang-omp
export CXX = clang-omp++

Then I run sudo make -j4 and got the following error.

ar crv lib/libxgboost.a 
ar: no archive members specified
usage:  ar -d [-TLsv] archive file ...
    ar -m [-TLsv] archive file ...
    ar -m [-abiTLsv] position archive file ...
    ar -p [-TLsv] archive [file ...]
    ar -q [-cTLsv] archive file ...
    ar -r [-cuTLsv] archive file ...
    ar -r [-abciuTLsv] position archive file ...
    ar -t [-TLsv] archive [file ...]
    ar -x [-ouTLsv] archive [file ...]
make: *** [lib/libxgboost.a] Error 1
make: *** Waiting for unfinished jobs....
clang-3.5: warning: argument unused during compilation: '-pthread'
clang-3.5: warning: argument unused during compilation: '-pthread'

Then I run sudo ./build.sh and get the following error

ar crv lib/libxgboost.a 
ar: no archive members specified
usage:  ar -d [-TLsv] archive file ...
    ar -m [-TLsv] archive file ...
    ar -m [-abiTLsv] position archive file ...
    ar -p [-TLsv] archive [file ...]
    ar -q [-cTLsv] archive file ...
    ar -r [-cuTLsv] archive file ...
    ar -r [-abciuTLsv] position archive file ...
    ar -t [-TLsv] archive [file ...]
    ar -x [-ouTLsv] archive [file ...]
make: *** [lib/libxgboost.a] Error 1
-----------------------------
Building multi-thread xgboost failed
@boral
Copy link

@boral boral commented May 15, 2016

Hi everyone,
To install Xgboost on Windows using Python follow the instructions in the following thread.

http://stackoverflow.com/questions/33749735/how-to-install-xgboost-package-in-python-windows-platform

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented May 15, 2016

@boral yep, the current pip install doesn't support window platform. glad that you have figured out

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented May 15, 2016

@LiyanJin you can just do 'make xgboost' for the core part. to have ar, you might need to install ar for mac

@lalmanisha
Copy link

@lalmanisha lalmanisha commented Jun 28, 2016

I am facing similar issue. My error is quite

OSError: dlopen(/Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so, 6): Library not loaded: @rpath/./libgomp.1.dylib
  Referenced from: /Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so
  Reason: image not found

I checked I have gcc installed.
$ brew install gcc49
Warning: homebrew/versions/gcc49-4.9.3 already installed

I am on mac os I installed xgboost through anaconda

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Jun 29, 2016

@lalmanisha I believe you are using Mac. The current best practice on Mac is using clang-omp instead of gcc. Can you try installing clang-omp from brew and change the CC and CXX in the config.mk to clang-omp?

@lalmanisha
Copy link

@lalmanisha lalmanisha commented Jun 29, 2016

Thanks for your reply. I still get the same error message. Here is the
`--------------------------------------------------------------------------
OSError Traceback (most recent call last)
in ()
14 # Any results you write to the current directory are saved as output.
15
---> 16 import xgboost as xgb
17 from sklearn.linear_model import LogisticRegression
18 import pandas as pd

/Users/mlal/anaconda/lib/python2.7/site-packages/xgboost.py in ()
80
81 # load the XGBoost library globally
---> 82 xglib = load_xglib()
83
84

/Users/mlal/anaconda/lib/python2.7/site-packages/xgboost.py in load_xglib()
57 if len(dll_path) == 0:
58 raise XGBoostLibraryNotFound('cannot find find the files in the candicate path ' + str(dll_path))
---> 59 lib = ctypes.cdll.LoadLibrary(lib_path[0])
60
61 # DMatrix functions

/Users/mlal/anaconda/lib/python2.7/ctypes/init.pyc in LoadLibrary(self, name)
441
442 def LoadLibrary(self, name):
--> 443 return self._dlltype(name)
444
445 cdll = LibraryLoader(CDLL)

/Users/mlal/anaconda/lib/python2.7/ctypes/init.pyc in init(self, name, mode, handle, use_errno, use_last_error)
363
364 if handle is None:
--> 365 self._handle = _dlopen(self._name, mode)
366 else:
367 self._handle = handle

OSError: dlopen(/Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so, 6): Library not loaded: @rpath/./libgomp.1.dylib
Referenced from: /Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so
Reason: image not found
`
i did change config.mk file as below

choice of compiler
export CC = clang-omp
export CXX = clang-omp
export MPICXX = mpicxx

@lalmanisha
Copy link

@lalmanisha lalmanisha commented Jun 29, 2016

sudo make -j4
ar crv lib/libxgboost.a
ar: no archive members specified
usage: ar -d [-TLsv] archive file ...
ar -m [-TLsv] archive file ...
ar -m [-abiTLsv] position archive file ...
ar -p [-TLsv] archive [file ...]
ar -q [-cTLsv] archive file ...
ar -r [-cuTLsv] archive file ...
ar -r [-abciuTLsv] position archive file ...
ar -t [-TLsv] archive [file ...]
ar -x [-ouTLsv] archive [file ...]
make: *** [lib/libxgboost.a] Error 1
make: *** Waiting for unfinished jobs....
clang: clang: error: error: unsupported option '-fopenmp'unsupported option '-fopenmp'

make: *** [lib/libxgboost.so] Error 1
make: *** [xgboost] Error 1
When i ran
sudo ./build.sh

maker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit_empty.a -pthread -lm
clang: warning: argument unused during compilation: '-pthread'
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md

I still get same error
dlopen(/Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so, 6): Library not loaded: @rpath/./libgomp.1.dylib
Referenced from: /Users/mlal/anaconda/lib/python2.7/site-packages/libxgboostwrapper.so
Reason: image not found

@lalmanisha
Copy link

@lalmanisha lalmanisha commented Jun 29, 2016

ok, I guess combination of above some help and build.md helped me make xgboost work. Thank you all !

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Jun 30, 2016

@lalmanisha since you only need the python package, you can just make xgboost which removes other unnecessary packages like the parallel and yarn version which needs ar. It should have successful make. And glad that you have figured it out.

@armgilles
Copy link

@armgilles armgilles commented Aug 2, 2016

Try to install Xgboost with git clone on MAC. I follow all instruction from @LiyanJin until this answer, I downloaded clang-omp via homebrew to use multi-threading.

  • config.mk :
export CC = clang-omp
export CXX = clang-omp++
export MPICXX = mpicxx
  • Error :
make
clang-omp++ -std=c++0x -Wall -O3 -msse2  -Wno-unknown-pragmas -funroll-loops -Iinclude -fopenmp  -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
In file included from src/learner.cc:7:
In file included from include/xgboost/logging.h:11:
In file included from dmlc-core/include/dmlc/logging.h:10:
/usr/local/Cellar/clang-omp/2015-04-01/libexec/bin/../include/c++/v1/cstdio:100:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
         ^
1 error generated.
make: *** [build/learner.o] Error 1

pip install works well, but I would like to have the lastest features in python package.

Some tips ?


Edit : Find the solution. Just xcode-select --install and make. Hope it will help other.

@phunterlau
Copy link
Contributor

@phunterlau phunterlau commented Aug 2, 2016

@armgilles that is right: by default mac hasn't install the compilers with headers while clang-omp needs the local installed Xcode, installing it by xcode-select --install everything after upgrade to a new system can solve the problem.

@hnykda
Copy link

@hnykda hnykda commented Oct 21, 2016

I had to build it from scratch:

git clone --recursive https://github.com/dmlc/xgboost.git  
cd xgboost  
./build.sh
pip install -e python-package  
@wuatanabe
Copy link

@wuatanabe wuatanabe commented Jan 29, 2017

hnykda suggestion worked for me on Ubuntu 16.0.4

@ghost
Copy link

@ghost ghost commented Feb 7, 2017

@phunterlau
I got the same error.
ar crv lib/libxgboost.a
ar: no archive members specified
usage: ar -d [-TLsv] archive file ...
ar -m [-TLsv] archive file ...
ar -m [-abiTLsv] position archive file ...
ar -p [-TLsv] archive [file ...]
ar -q [-cTLsv] archive file ...
ar -r [-cuTLsv] archive file ...
ar -r [-abciuTLsv] position archive file ...
ar -t [-TLsv] archive [file ...]
ar -x [-ouTLsv] archive [file ...]
make: *** [lib/libxgboost.a] Error 1

i have installed ar. i am wondering what the "ar crv" do,if i can use other command to replace.
you have mentioned "@LiyanJin you can just do 'make xgboost' for the core part. to have ar, you might need to install ar for mac" , I don't understand what "do 'make xgboost' for the core part" means. hope for your advices

@GnoDGnaF
Copy link

@GnoDGnaF GnoDGnaF commented Mar 11, 2017

I have installed the package on my MAC (macOS 10.12.3) successfully by:

brew install --with-clang llvm
brew install cmake
brew install gcc --without-multilib

git clone --recursive https://github.com/dmlc/xgboost
cd xgboost; cp make/config.mk ./config.mk; make -j4

cd python-package/ ; python setup.py install
@Xu-Tian
Copy link

@Xu-Tian Xu-Tian commented Apr 16, 2017

@hnykda Many times, the working solution is the seemingly simple one. Thanks, man.

@WillKoehrsen
Copy link

@WillKoehrsen WillKoehrsen commented Jun 23, 2017

The solution that I found worked for me was navigating to
/home/username/anaconda3/lib/python3.5/site-packages/xgboost
and running ./build-python.sh

@tcglarry
Copy link

@tcglarry tcglarry commented Jul 10, 2017

It seems there are a lot of issues when installing xgboost. i tried to install onto Mac, and failed too. Could osmeone tell me what is the final solution ?

@LookOnTheBrightSide
Copy link

@LookOnTheBrightSide LookOnTheBrightSide commented Jul 11, 2017

what worked for me was adding the following on linux ... credit (everyone above)

git clone --recursive https://github.com/dmlc/xgboost.git  
cd xgboost  
./build.sh
pip install -e python-package 
cd python-package
python setup.py install
@philipshurpik
Copy link

@philipshurpik philipshurpik commented Jul 22, 2017

For me worked on MacOS Sierra after:
brew install gcc --without-multilib

executing this commands:

export CC=gcc-7
export CXX=g++-7
@apoorva007
Copy link

@apoorva007 apoorva007 commented Jan 5, 2018

@philipshurpik Thanks a ton!

@cchen767
Copy link

@cchen767 cchen767 commented Mar 3, 2018

@philipshurpik That worked pretty great for me, thanks!

@lock lock bot locked as resolved and limited conversation to collaborators Oct 25, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.