macos pip install failure

[yungmsh]

Within a Python 3.6 environment (managed by conda), when I run pip install Kaggler, pip install -U Kaggler, or pip install --no-cache-dir Kaggler I get the following error message (see below).
I also ran pip install -U cython beforehand to update cython, but same error msg occurs.

(python3.6) mike-yung$ pip install -no-cache-dir Kaggler

Usage:
  pip install [options] <requirement specifier> [package-index-options] ...
  pip install [options] -r <requirements file> [package-index-options] ...
  pip install [options] [-e] <vcs project url> ...
  pip install [options] [-e] <local project path> ...
  pip install [options] <archive url/path> ...

no such option: -n
(python3.6) mike-yung-C02WC0F4HTDG:ltvent mike.yung$ pip install --no-cache-dir Kaggler
Looking in indexes: https://yoober7:****@pypi.uberinternal.com/index, https://pypi.python.org/simple
Collecting Kaggler
  Downloading https://pypi.uberinternal.com/packages/af/98/25d2c773369ba56b2e70e584f5ab4ab1ed1708df6ec8dcc153d77f03607e/Kaggler-0.6.9.tar.gz (812kB)
    100% |████████████████████████████████| 819kB 14.3MB/s
Requirement already satisfied: setuptools>=41.0.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (41.0.1)
Requirement already satisfied: cython in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (0.29.7)
Requirement already satisfied: h5py in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (2.9.0)
Requirement already satisfied: ml_metrics in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (0.1.4)
Requirement already satisfied: numpy in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (1.16.2)
Requirement already satisfied: pandas in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (0.24.2)
Requirement already satisfied: matplotlib in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (2.2.4)
Requirement already satisfied: scipy>=0.14.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (1.2.1)
Requirement already satisfied: scikit-learn>=0.15.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (0.20.3)
Requirement already satisfied: statsmodels>=0.5.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (0.9.0)
Requirement already satisfied: kaggle in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (1.5.3)
Requirement already satisfied: tensorflow in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (1.13.1)
Requirement already satisfied: keras in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from Kaggler) (2.2.4)
Requirement already satisfied: six in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from h5py->Kaggler) (1.12.0)
Requirement already satisfied: pytz>=2011k in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from pandas->Kaggler) (2018.9)
Requirement already satisfied: python-dateutil>=2.5.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from pandas->Kaggler) (2.8.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from matplotlib->Kaggler) (2.3.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from matplotlib->Kaggler) (1.0.1)
Requirement already satisfied: cycler>=0.10 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from matplotlib->Kaggler) (0.10.0)
Requirement already satisfied: requests in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from kaggle->Kaggler) (2.21.0)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from kaggle->Kaggler) (1.24.1)
Requirement already satisfied: python-slugify in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from kaggle->Kaggler) (3.0.2)
Requirement already satisfied: certifi in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from kaggle->Kaggler) (2019.3.9)
Requirement already satisfied: tqdm in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from kaggle->Kaggler) (4.32.1)
Requirement already satisfied: wheel>=0.26 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (0.33.1)
Requirement already satisfied: tensorboard<1.14.0,>=1.13.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (1.13.1)
Requirement already satisfied: gast>=0.2.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (0.2.2)
Requirement already satisfied: tensorflow-estimator<1.14.0rc0,>=1.13.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (1.13.0)
Requirement already satisfied: astor>=0.6.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (0.7.1)
Requirement already satisfied: termcolor>=1.1.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (1.1.0)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (1.0.9)
Requirement already satisfied: protobuf>=3.6.1 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (3.7.1)
Requirement already satisfied: grpcio>=1.8.6 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (1.20.1)
Requirement already satisfied: absl-py>=0.1.6 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (0.7.1)
Requirement already satisfied: keras-applications>=1.0.6 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow->Kaggler) (1.0.7)
Requirement already satisfied: pyyaml in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from keras->Kaggler) (5.1)
Requirement already satisfied: idna<2.9,>=2.5 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from requests->kaggle->Kaggler) (2.8)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from requests->kaggle->Kaggler) (3.0.4)
Requirement already satisfied: text-unidecode==1.2 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from python-slugify->kaggle->Kaggler) (1.2)
Requirement already satisfied: werkzeug>=0.11.15 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow->Kaggler) (0.14.1)
Requirement already satisfied: markdown>=2.6.8 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow->Kaggler) (3.1)
Requirement already satisfied: mock>=2.0.0 in /anaconda2/envs/python3.6/lib/python3.6/site-packages (from tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow->Kaggler) (3.0.5)
Installing collected packages: Kaggler
  Running setup.py install for Kaggler ... error
    Complete output from command /anaconda2/envs/python3.6/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/1y/btgkmt992l94_1d37rkvhc380000gn/T/pip-install-ttr3it94/Kaggler/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/1y/btgkmt992l94_1d37rkvhc380000gn/T/pip-record-g7a_hyv1/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.7-x86_64-3.6
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler
    copying kaggler/data_io.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler
    copying kaggler/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler
    copying kaggler/const.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler/feature_selection
    copying kaggler/feature_selection/feature_selection.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/feature_selection
    copying kaggler/feature_selection/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/feature_selection
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler/ensemble
    copying kaggler/ensemble/linear.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/ensemble
    copying kaggler/ensemble/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/ensemble
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler/model
    copying kaggler/model/nn.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/model
    copying kaggler/model/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/model
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler/metrics
    copying kaggler/metrics/regression.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/metrics
    copying kaggler/metrics/classification.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/metrics
    copying kaggler/metrics/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/metrics
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler/online_model
    copying kaggler/online_model/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/online_model
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler/preprocessing
    copying kaggler/preprocessing/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/preprocessing
    copying kaggler/preprocessing/data.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/preprocessing
    creating build/lib.macosx-10.7-x86_64-3.6/kaggler/test
    copying kaggler/test/test_sgd.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/test
    copying kaggler/test/test_ftrl.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/test
    copying kaggler/test/test_lbe.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/test
    copying kaggler/test/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/test
    copying kaggler/test/test_ohe.py -> build/lib.macosx-10.7-x86_64-3.6/kaggler/test
    running build_ext
    skipping 'kaggler/online_model/ftrl.c' Cython extension (up-to-date)
    building 'kaggler.online_model.ftrl' extension
    creating build/temp.macosx-10.7-x86_64-3.6
    creating build/temp.macosx-10.7-x86_64-3.6/kaggler
    creating build/temp.macosx-10.7-x86_64-3.6/kaggler/online_model
    creating build/temp.macosx-10.7-x86_64-3.6/kaggler/online_model/murmurhash
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/anaconda2/envs/python3.6/include -arch x86_64 -I/anaconda2/envs/python3.6/include -arch x86_64 -I. -I/anaconda2/envs/python3.6/include/python3.6m -I/anaconda2/envs/python3.6/lib/python3.6/site-packages/numpy/core/include -c kaggler/online_model/ftrl.c -o build/temp.macosx-10.7-x86_64-3.6/kaggler/online_model/ftrl.o -O3
    In file included from kaggler/online_model/ftrl.c:594:
    In file included from /anaconda2/envs/python3.6/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4:
    In file included from /anaconda2/envs/python3.6/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
    In file included from /anaconda2/envs/python3.6/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1824:
    /anaconda2/envs/python3.6/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with "          "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
    #warning "Using deprecated NumPy API, disable it with " \
     ^
    1 warning generated.
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/anaconda2/envs/python3.6/include -arch x86_64 -I/anaconda2/envs/python3.6/include -arch x86_64 -I. -I/anaconda2/envs/python3.6/include/python3.6m -I/anaconda2/envs/python3.6/lib/python3.6/site-packages/numpy/core/include -c kaggler/online_model/murmurhash/MurmurHash3.cpp -o build/temp.macosx-10.7-x86_64-3.6/kaggler/online_model/murmurhash/MurmurHash3.o -O3
    warning: include path for stdlibc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
    1 warning generated.
    g++ -bundle -undefined dynamic_lookup -L/anaconda2/envs/python3.6/lib -arch x86_64 -L/anaconda2/envs/python3.6/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/kaggler/online_model/ftrl.o build/temp.macosx-10.7-x86_64-3.6/kaggler/online_model/murmurhash/MurmurHash3.o -o build/lib.macosx-10.7-x86_64-3.6/kaggler/online_model/ftrl.cpython-36m-darwin.so
    clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9 [-Wdeprecated]
    ld: library not found for -lstdc++
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command 'g++' failed with exit status 1

    ----------------------------------------
Command "/anaconda2/envs/python3.6/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/1y/btgkmt992l94_1d37rkvhc380000gn/T/pip-install-ttr3it94/Kaggler/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/1y/btgkmt992l94_1d37rkvhc380000gn/T/pip-record-g7a_hyv1/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/1y/btgkmt992l94_1d37rkvhc380000gn/T/pip-install-ttr3it94/Kaggler/

[yungmsh]

Noting that I added export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:. to ~/.bash_profile (suggestion from #32) and still doesn't work

[jeongyoonlee]

Could you try running xcode-select --install? It sounds like the XCode Command Line Tools issue as in https://stackoverflow.com/questions/52441952/os-x-ld-library-not-found-for-lstdc

[yungmsh]

I ran that, and received:
"xcode-select: error: command line tools are already installed, use "Software Update" to install update"

So I did a Software Update through the native System Preferences GUI (where command line tools was one of the updates), and was able to pip install --no-cache-dir Kaggler successfully!

[jeongyoonlee]

@yungmsh:

I noticed that upgrading XCode from 9.x to 10.x causes the issue again. It's because XCode dropped the support for libstdc++ starting from 10.x.

One walk-around is to downgrade the command line tools by removing /Library/Developer/CommandLineTools and installing the command line tools version 9.x.

I will reopen the ticket and investigate how to fix it.

[jeongyoonlee]

Fixed by adding extra_compile_args and extra_link_args for macOS as follows:

# from https://stackoverflow.com/a/52466939/3216742
extra_compile_args = ['-O3']
extra_link_args = []

if platform.system() == "Darwin":
    extra_compile_args += ["-mmacosx-version-min=10.9"]
    extra_link_args += ["-mmacosx-version-min=10.9"]