Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cannot pip install shap or conda install -c conda-forge shap #1386

Closed
lachlanmichalski opened this issue Aug 27, 2020 · 7 comments
Closed

cannot pip install shap or conda install -c conda-forge shap #1386

lachlanmichalski opened this issue Aug 27, 2020 · 7 comments
Labels
stale Indicates that there has been no recent activity on an issue

Comments

@lachlanmichalski
Copy link

lachlanmichalski commented Aug 27, 2020 •
edited
Loading

Hi I am trying to install shap to use in anaconda (spyder) but this error occurs when I pip install shap in the terminal.The output of the error is below.

The error says "Use sudo xcode-select --switch path/to/Xcode.app to specify the Xcode that you wish to use for command line developer tools, or use xcode-select --install to install the standalone command line developer tools."

However, I do not have xcode installed!

Any help to figure out what is going on would be a life saver as I have no idea what is causing this error to occur. Thanks!!

pip install shap

Collecting shap
 Using cached shap-0.35.0.tar.gz (273 kB)
 Requirement already satisfied: numpy in ./opt/anaconda3/lib/python3.7/site-packages (from shap) (1.18.5)
 Requirement already satisfied: scipy in ./opt/anaconda3/lib/python3.7/site-packages (from shap) (1.4.1)
 Requirement already satisfied: scikit-learn in ./opt/anaconda3/lib/python3.7/site-packages (from shap) (0.23.1)
 Requirement already satisfied: pandas in ./opt/anaconda3/lib/python3.7/site-packages (from shap) (1.0.4)
 Requirement already satisfied: tqdm>4.25.0 in ./opt/anaconda3/lib/python3.7/site-packages (from shap) (4.42.1)
 Requirement already satisfied: threadpoolctl>=2.0.0 in ./opt/anaconda3/lib/python3.7/site-packages (from scikit-learn->shap) (2.1.0)
 Requirement already satisfied: joblib>=0.11 in ./opt/anaconda3/lib/python3.7/site-packages (from scikit-learn->shap) (0.14.1)
 Requirement already satisfied: pytz>=2017.2 in ./opt/anaconda3/lib/python3.7/site-packages (from pandas->shap) (2019.3)
 Requirement already satisfied: python-dateutil>=2.6.1 in ./opt/anaconda3/lib/python3.7/site-packages (from pandas->shap) (2.8.1)
 Requirement already satisfied: six>=1.5 in ./opt/anaconda3/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas->shap) (1.14.0)
 Building wheels for collected packages: shap
   Building wheel for shap (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /Users/lockie/opt/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-install-1l6_m1at/shap/setup.py'"'"'; __file__='"'"'/private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-install-1l6_m1at/shap/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-wheel-xs4mjazc
    cwd: /private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-install-1l6_m1at/shap/
Complete output (73 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.9-x86_64-3.7
creating build/lib.macosx-10.9-x86_64-3.7/shap
copying shap/datasets.py -> build/lib.macosx-10.9-x86_64-3.7/shap
copying shap/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap
copying shap/common.py -> build/lib.macosx-10.9-x86_64-3.7/shap
creating build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/tree.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/partition.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/kernel.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/explainer.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/bruteforce.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/gradient.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/linear.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/mimic.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/permutation.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/sampling.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/additive.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/tf_utils.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
copying shap/explainers/pytree.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
creating build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
copying shap/explainers/other/coefficent.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
copying shap/explainers/other/maple.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
copying shap/explainers/other/lime.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
copying shap/explainers/other/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
copying shap/explainers/other/random.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
copying shap/explainers/other/treegain.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
creating build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
copying shap/explainers/deep/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
copying shap/explainers/deep/deep_pytorch.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
copying shap/explainers/deep/deep_tf.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
creating build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/force_matplotlib.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/force.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/embedding.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/monitoring.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/colorconv.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/decision.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/waterfall.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/summary.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/text.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/dependence.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/bar.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/image.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/partial_dependence.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
copying shap/plots/colors.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
creating build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
copying shap/benchmark/metrics.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
copying shap/benchmark/models.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
copying shap/benchmark/methods.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
copying shap/benchmark/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
copying shap/benchmark/measures.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
copying shap/benchmark/plots.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
copying shap/benchmark/experiments.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
creating build/lib.macosx-10.9-x86_64-3.7/shap/plots/resources
copying shap/plots/resources/bundle.js -> build/lib.macosx-10.9-x86_64-3.7/shap/plots/resources
copying shap/plots/resources/logoSmallGray.png -> build/lib.macosx-10.9-x86_64-3.7/shap/plots/resources
copying shap/tree_shap.h -> build/lib.macosx-10.9-x86_64-3.7/shap
running build_ext
numpy.get_include() /Users/lockie/opt/anaconda3/lib/python3.7/site-packages/numpy/core/include
building 'shap._cext' extension
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/shap
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/lockie/opt/anaconda3/include -arch x86_64 -I/Users/lockie/opt/anaconda3/include -arch x86_64 -I/Users/lockie/opt/anaconda3/include/python3.7m -I/Users/lockie/opt/anaconda3/lib/python3.7/site-packages/numpy/core/include -c shap/_cext.cc -o build/temp.macosx-10.9-x86_64-3.7/shap/_cext.o
xcrun: error: active developer path ("/Applications/Xcode.app/Contents/Developer") does not exist
Use `sudo xcode-select --switch path/to/Xcode.app` to specify the Xcode that you wish to use for command line developer tools, or use `xcode-select --install` to install the standalone command line developer tools.
See `man xcode-select` for more details.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for shap
Running setup.py clean for shap
Failed to build shap
DEPRECATION: Could not build wheels for shap which do not use PEP 517. pip will fall back to legacy 'setup.py install' for these. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Installing collected packages: shap
 Running setup.py install for shap ... error
 ERROR: Command errored out with exit status 1:
  command: /Users/lockie/opt/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-install-1l6_m1at/shap/setup.py'"'"'; __file__='"'"'/private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-install-1l6_m1at/shap/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-record-suze3g9s/install-record.txt --single-version-externally-managed --compile --install-headers /Users/lockie/opt/anaconda3/include/python3.7m/shap
      cwd: /private/var/folders/vq/h4jz7fs53y1_mysncrx507s40000gn/T/pip-install-1l6_m1at/shap/
 Complete output (73 lines):
 running install
 running build
 running build_py
 creating build
 creating build/lib.macosx-10.9-x86_64-3.7
 creating build/lib.macosx-10.9-x86_64-3.7/shap
 copying shap/datasets.py -> build/lib.macosx-10.9-x86_64-3.7/shap
 copying shap/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap
 copying shap/common.py -> build/lib.macosx-10.9-x86_64-3.7/shap
 creating build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/tree.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/partition.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/kernel.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/explainer.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/bruteforce.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/gradient.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/linear.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/mimic.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/permutation.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/sampling.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/additive.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/tf_utils.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 copying shap/explainers/pytree.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers
 creating build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
 copying shap/explainers/other/coefficent.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
 copying shap/explainers/other/maple.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
 copying shap/explainers/other/lime.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
 copying shap/explainers/other/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
 copying shap/explainers/other/random.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
 copying shap/explainers/other/treegain.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/other
 creating build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
 copying shap/explainers/deep/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
 copying shap/explainers/deep/deep_pytorch.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
 copying shap/explainers/deep/deep_tf.py -> build/lib.macosx-10.9-x86_64-3.7/shap/explainers/deep
 creating build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/force_matplotlib.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/force.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/embedding.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/monitoring.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/colorconv.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/decision.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/waterfall.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/summary.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/text.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/dependence.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/bar.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/image.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/partial_dependence.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 copying shap/plots/colors.py -> build/lib.macosx-10.9-x86_64-3.7/shap/plots
 creating build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 copying shap/benchmark/metrics.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 copying shap/benchmark/models.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 copying shap/benchmark/methods.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 copying shap/benchmark/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 copying shap/benchmark/measures.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 copying shap/benchmark/plots.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 copying shap/benchmark/experiments.py -> build/lib.macosx-10.9-x86_64-3.7/shap/benchmark
 creating build/lib.macosx-10.9-x86_64-3.7/shap/plots/resources
 copying shap/plots/resources/bundle.js -> build/lib.macosx-10.9-x86_64-3.7/shap/plots/resources
 copying shap/plots/resources/logoSmallGray.png -> build/lib.macosx-10.9-x86_64-3.7/shap/plots/resources
 copying shap/tree_shap.h -> build/lib.macosx-10.9-x86_64-3.7/shap
 running build_ext
 numpy.get_include() /Users/lockie/opt/anaconda3/lib/python3.7/site-packages/numpy/core/include
 building 'shap._cext' extension
 creating build/temp.macosx-10.9-x86_64-3.7
 creating build/temp.macosx-10.9-x86_64-3.7/shap
 gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/lockie/opt/anaconda3/include -arch x86_64 -I/Users/lockie/opt/anaconda3/include -arch x86_64 -I/Users/lockie/opt/anaconda3/include/python3.7m -I/Users/lockie/opt/anaconda3/lib/python3.7/site-packages/numpy/core/include -c shap/_cext.cc -o build/temp.macosx-10.9-x86_64-3.7/shap/_cext.o
 xcrun: error: active developer path ("/Applications/Xcode.app/Contents/Developer") does not exist
 Use `sudo xcode-select --switch path/to/Xcode.app` to specify the Xcode that you wish to use for command line developer tools, or use `xcode-select --install` to install the standalone command line developer tools.
 See `man xcode-select` for more details.
 error: command 'gcc' failed with exit status 1
 ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/lockie/opt/anaconda3/bin/pyt
@slundberg
Copy link
Collaborator

pip will not work without a compiler (XCode), but I would have expected coda forge to work...

@lachlanmichalski
Copy link
Author

Thanks for getting back to us. Issue with Mac. Reset and installed perfectly. Now we wait for explainer.shap_values(X_test) to run 😊😊

Would you have a rough idea how long a test set with 35,000 observations and 150 features would take to run?

@slundberg
Copy link
Collaborator

Totally depends on the explainer and the model. For explainers.Tree is should take very little time (a min?). For model agnostic methods it could take a lot longer, but there should be a progress bar. If you find something running for a long time without a progress bar let us know.

@lachlanmichalski
Copy link
Author

I am using the following RF model and it has been running > 24 hours. y is multi-class (0, 1, 2) and x features are financial ratios, macroeconomic vars etc.

model = RandomForestClassifier(random_state=0)
model.fit(X_train,y_train)
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

As you mentioned it should take very little time, would it be due to using RF for such long run time?

@lachlanmichalski
Copy link
Author

Extra information:

training data is approx 85,000 observations.

I have it running separately as well with a reduced feature set from 150+ to 20 and that has been > 12 hours at the moment. Could it be due to the training dataset observation size?

@github-actions GitHub Actions
Copy link

This issue has been inactive for two years, so it's been automatically marked as 'stale'.

We value your input! If this issue is still relevant, please leave a comment below. This will remove the 'stale' label and keep it open.

If there's no activity in the next 90 days the issue will be closed.

@github-actions github-actions bot added the stale Indicates that there has been no recent activity on an issue label Feb 11, 2024
@github-actions GitHub Actions
Copy link

This issue has been automatically closed due to lack of recent activity.

Your input is important to us! Please feel free to open a new issue if the problem persists or becomes relevant again.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stale Indicates that there has been no recent activity on an issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@slundberg @lachlanmichalski