Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in builds using cmake >= 3.20? #81

Open
jreades opened this issue Apr 22, 2021 · 8 comments
Open

Error in builds using cmake >= 3.20? #81

jreades opened this issue Apr 22, 2021 · 8 comments

Comments

@jreades
Copy link

jreades commented Apr 22, 2021

I'm trying to install MulticoreTSNE into a Docker image built on top of the Jupyter Minimal distribution (Anaconda Python, etc.). Previously I was able to run this without a hitch. I've tested the following combinations:

  1. jupyter/minimal-notebook:584f43f0658 (Late April 2021)
    • cmake 3.18.2 = Build successful (this was the version dumped from my last working image)
    • cmake 3.20.1 = Build unsuccessful (error below)
  2. jupyter/minimal-notebook:016833b15ceb (Late Feb 2021)
    • cmake 3.18.2 = Build successful (this was the version dumped from my last working image)
    • cmake 3.20.1 = Build unsuccessful (error below)

Forcing cmake to downgrade to 3.18.2 seems to force a downgrade of Python from 3.9 to 3.8 so it's possible that is the source of the problem, but the report suggests it's cmake.

The error is:

#13 53.91     running build_ext
#13 53.91     cmake version 3.20.1
#13 53.91
#13 53.91     CMake suite maintained and supported by Kitware (kitware.com/cmake).
#13 53.91     CMake Error: Unknown argument --
#13 53.91     CMake Error: Run 'cmake --help' for all supported options.
#13 53.91
#13 53.91     ERROR: Cannot generate Makefile. See above errors.

Full context is:

> [4/5] RUN conda-env create -n ethos -f ./python.test.yml     && conda clean --all --yes --force-pkgs-dirs     && find /opt/conda/ -follow -type f -name '*.a' -delete     && find /opt/conda/ -follow -type f -name '*.pyc' -delete     && find /opt/conda/ -follow -type f -name '*.js.map' -delete     && pip cache purge     && rm -rf /home/jovyan/.cache/pip     && rm ./python.test.yml:
#12 0.549 Collecting package metadata (repodata.json): ...working... done
#12 25.27 Solving environment: ...working... done
#12 29.94
#12 29.94 Downloading and Extracting Packages
libffi-3.3           | 51 KB     | ########## | 100%
lz4-c-1.9.3          | 179 KB    | ########## | 100%
libgomp-9.3.0        | 376 KB    | ########## | 100%
libedit-3.1.20191231 | 121 KB    | ########## | 100%
xz-5.2.5             | 343 KB    | ########## | 100%
rhash-1.4.1          | 192 KB    | ########## | 100%
krb5-1.17.2          | 1.4 MB    | ########## | 100%
readline-8.1         | 295 KB    | ########## | 100%
bzip2-1.0.8          | 484 KB    | ########## | 100%
_openmp_mutex-4.5    | 22 KB     | ########## | 100%
tzdata-2021a         | 121 KB    | ########## | 100%
libssh2-1.9.0        | 226 KB    | ########## | 100%
certifi-2020.12.5    | 143 KB    | ########## | 100%
sqlite-3.35.4        | 1.4 MB    | ########## | 100%
ca-certificates-2020 | 137 KB    | ########## | 100%
zstd-1.4.9           | 431 KB    | ########## | 100%
libuv-1.41.0         | 1.0 MB    | ########## | 100%
setuptools-49.6.0    | 943 KB    | ########## | 100%
libstdcxx-ng-9.3.0   | 4.0 MB    | ########## | 100%
libcurl-7.76.1       | 328 KB    | ########## | 100%
cmake-3.20.1         | 14.7 MB   | ########## | 100%
libgcc-ng-9.3.0      | 7.8 MB    | ########## | 100%
python-3.9.2         | 27.3 MB   | ########## | 100%
zlib-1.2.11          | 106 KB    | ########## | 100%
pip-21.0.1           | 1.1 MB    | ########## | 100%
ld_impl_linux-64-2.3 | 618 KB    | ########## | 100%
tk-8.6.10            | 3.2 MB    | ########## | 100%
openssl-1.1.1k       | 2.1 MB    | ########## | 100%
libnghttp2-1.43.0    | 808 KB    | ########## | 100%
wheel-0.36.2         | 31 KB     | ########## | 100%
python_abi-3.9       | 4 KB      | ########## | 100%
libev-4.33           | 104 KB    | ########## | 100%
_libgcc_mutex-0.1    | 3 KB      | ########## | 100%
expat-2.3.0          | 168 KB    | ########## | 100%
ncurses-6.2          | 985 KB    | ########## | 100%
c-ares-1.17.1        | 109 KB    | ########## | 100%
#12 44.17 Preparing transaction: ...working... done
#12 44.43 Verifying transaction: ...working... done
#12 45.95 Executing transaction: ...working... done
#12 47.59 Installing pip dependencies: ...working... Ran pip subprocess with arguments:
#12 52.31 ['/opt/conda/envs/ethos/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/jovyan/condaenv.4kzuzpkn.requirements.txt']
#12 52.31 Pip subprocess output:
#12 52.31 Collecting MulticoreTSNE
#12 52.31   Downloading MulticoreTSNE-0.1.tar.gz (20 kB)
#12 52.31 Collecting numpy
#12 52.31   Downloading numpy-1.20.2-cp39-cp39-manylinux2010_x86_64.whl (15.4 MB)
#12 52.31 Collecting cffi
#12 52.31   Downloading cffi-1.14.5-cp39-cp39-manylinux1_x86_64.whl (406 kB)
#12 52.31 Collecting pycparser
#12 52.31   Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
#12 52.31 Building wheels for collected packages: MulticoreTSNE
#12 52.31   Building wheel for MulticoreTSNE (setup.py): started
#12 52.31   Building wheel for MulticoreTSNE (setup.py): finished with status 'error'
#12 52.31   Running setup.py clean for MulticoreTSNE
#12 52.31 Failed to build MulticoreTSNE
#12 52.31 Installing collected packages: pycparser, numpy, cffi, MulticoreTSNE
#12 52.31     Running setup.py install for MulticoreTSNE: started
#12 52.31     Running setup.py install for MulticoreTSNE: finished with status 'error'
#12 52.31
#12 52.31 failed
#12 52.31
#12 52.31
#12 52.31 ==> WARNING: A newer version of conda exists. <==
#12 52.31   current version: 4.10.0
#12 52.31   latest version: 4.10.1
#12 52.31
#12 52.31 Please update conda by running
#12 52.31
#12 52.31     $ conda update -n base conda
#12 52.31
#12 52.31
#12 52.31 Pip subprocess error:
#12 52.31   ERROR: Command errored out with exit status 1:
#12 52.31    command: /opt/conda/envs/ethos/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/setup.py'"'"'; __file__='"'"'/tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-gm7zmxgs
#12 52.31        cwd: /tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/
#12 52.31   Complete output (26 lines):
#12 52.31   running bdist_wheel
#12 52.31   running build
#12 52.31   running build_py
#12 52.31   creating build
#12 52.31   creating build/lib.linux-x86_64-3.9
#12 52.31   creating build/lib.linux-x86_64-3.9/MulticoreTSNE
#12 52.31   copying MulticoreTSNE/__init__.py -> build/lib.linux-x86_64-3.9/MulticoreTSNE
#12 52.31   creating build/lib.linux-x86_64-3.9/MulticoreTSNE/tests
#12 52.31   copying MulticoreTSNE/tests/test_base.py -> build/lib.linux-x86_64-3.9/MulticoreTSNE/tests
#12 52.31   copying MulticoreTSNE/tests/__init__.py -> build/lib.linux-x86_64-3.9/MulticoreTSNE/tests
#12 52.31   running egg_info
#12 52.31   writing MulticoreTSNE.egg-info/PKG-INFO
#12 52.31   writing dependency_links to MulticoreTSNE.egg-info/dependency_links.txt
#12 52.31   writing requirements to MulticoreTSNE.egg-info/requires.txt
#12 52.31   writing top-level names to MulticoreTSNE.egg-info/top_level.txt
#12 52.31   reading manifest file 'MulticoreTSNE.egg-info/SOURCES.txt'
#12 52.31   reading manifest template 'MANIFEST.in'
#12 52.31   writing manifest file 'MulticoreTSNE.egg-info/SOURCES.txt'
#12 52.31   running build_ext
#12 52.31   cmake version 3.20.1
#12 52.31
#12 52.31   CMake suite maintained and supported by Kitware (kitware.com/cmake).
#12 52.31   CMake Error: Unknown argument --
#12 52.31   CMake Error: Run 'cmake --help' for all supported options.
#12 52.31
#12 52.31   ERROR: Cannot generate Makefile. See above errors.
#12 52.31   ----------------------------------------
#12 52.31   ERROR: Failed building wheel for MulticoreTSNE
#12 52.31     ERROR: Command errored out with exit status 1:
#12 52.31      command: /opt/conda/envs/ethos/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/setup.py'"'"'; __file__='"'"'/tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5jx__mkf/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/envs/ethos/include/python3.9/MulticoreTSNE
#12 52.31          cwd: /tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/
#12 52.31     Complete output (26 lines):
#12 52.31     running install
#12 52.31     running build
#12 52.31     running build_py
#12 52.31     creating build
#12 52.31     creating build/lib.linux-x86_64-3.9
#12 52.31     creating build/lib.linux-x86_64-3.9/MulticoreTSNE
#12 52.31     copying MulticoreTSNE/__init__.py -> build/lib.linux-x86_64-3.9/MulticoreTSNE
#12 52.31     creating build/lib.linux-x86_64-3.9/MulticoreTSNE/tests
#12 52.31     copying MulticoreTSNE/tests/test_base.py -> build/lib.linux-x86_64-3.9/MulticoreTSNE/tests
#12 52.31     copying MulticoreTSNE/tests/__init__.py -> build/lib.linux-x86_64-3.9/MulticoreTSNE/tests
#12 52.31     running egg_info
#12 52.31     writing MulticoreTSNE.egg-info/PKG-INFO
#12 52.31     writing dependency_links to MulticoreTSNE.egg-info/dependency_links.txt
#12 52.31     writing requirements to MulticoreTSNE.egg-info/requires.txt
#12 52.31     writing top-level names to MulticoreTSNE.egg-info/top_level.txt
#12 52.31     reading manifest file 'MulticoreTSNE.egg-info/SOURCES.txt'
#12 52.31     reading manifest template 'MANIFEST.in'
#12 52.31     writing manifest file 'MulticoreTSNE.egg-info/SOURCES.txt'
#12 52.31     running build_ext
#12 52.31     cmake version 3.20.1
#12 52.31
#12 52.31     CMake suite maintained and supported by Kitware (kitware.com/cmake).
#12 52.31     CMake Error: Unknown argument --
#12 52.31     CMake Error: Run 'cmake --help' for all supported options.
#12 52.31
#12 52.31     ERROR: Cannot generate Makefile. See above errors.
#12 52.31     ----------------------------------------
#12 52.31 ERROR: Command errored out with exit status 1: /opt/conda/envs/ethos/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/setup.py'"'"'; __file__='"'"'/tmp/pip-install-110qdwqv/multicoretsne_4b5d168de5e04f6c894a77b8595839b9/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5jx__mkf/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/envs/ethos/include/python3.9/MulticoreTSNE Check the logs for full command output.
#12 52.31
#12 52.31
#12 52.31 CondaEnvException: Pip failed
#12 52.31
------
executor failed running [/bin/bash -c conda-env create -n ${env_nm} -f ./${yaml_nm}     && conda clean --all --yes --force-pkgs-dirs     && find /opt/conda/ -follow -type f -name '*.a' -delete     && find /opt/conda/ -follow -type f -name '*.pyc' -delete     && find /opt/conda/ -follow -type f -name '*.js.map' -delete     && pip cache purge     && rm -rf /home/$NB_USER/.cache/pip     && rm ./${yaml_nm}]: exit code: 1
@ndwarshuis
Copy link

I just encountered the same error and can confirm downgrading cmake from 3.20.1 -> 3.19.7 was the issue. It seems the new version of cmake doesn't support the -- in its arguments. After downgrading, pip install multicoretsne works as expected.

This was effect was seen both in a pyenv/virtual environment where the python version was 3.9.2, and in a systemwide python installation where the python version was 3.9.3 (both on arch linux).

@kmoad
Copy link

kmoad commented Jun 14, 2021

Getting the same issue on Fedora 34 system install. Downgrading with dnf install cmake-3.19.7-1.fc34 resolved it, but it would be nice to be able to use latest cmake.

@Pseudomanifold Pseudomanifold mentioned this issue Jun 17, 2021
Closed
@TheAustinator
Copy link

TheAustinator commented Jun 18, 2021 •
edited
Loading

pip install cmake==3.18.4 worked for me.

@qleroy qleroy mentioned this issue Jul 22, 2021
Open
@JennyWeng JennyWeng mentioned this issue Nov 9, 2021
Closed
This was referenced Mar 16, 2022
Closed
Merged
@ARoefer ARoefer mentioned this issue Jun 9, 2022
Closed
@yash-goel
Copy link

Hi, I still get the cmake error with 3.18.4 version.

ERROR: Command errored out with exit status 1:
   command: /home/yash/anaconda3/envs/calvin_try/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-4_pnlmiw/multicoretsne_f755883637e7456cac20fedb190b6da2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-4_pnlmiw/multicoretsne_f755883637e7456cac20fedb190b6da2/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-bgyvtm0q
       cwd: /tmp/pip-install-4_pnlmiw/multicoretsne_f755883637e7456cac20fedb190b6da2/
  Complete output (29 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/MulticoreTSNE
  copying MulticoreTSNE/__init__.py -> build/lib.linux-x86_64-3.8/MulticoreTSNE
  creating build/lib.linux-x86_64-3.8/MulticoreTSNE/tests
  copying MulticoreTSNE/tests/test_base.py -> build/lib.linux-x86_64-3.8/MulticoreTSNE/tests
  copying MulticoreTSNE/tests/__init__.py -> build/lib.linux-x86_64-3.8/MulticoreTSNE/tests
  running egg_info
  writing MulticoreTSNE.egg-info/PKG-INFO
  writing dependency_links to MulticoreTSNE.egg-info/dependency_links.txt
  writing requirements to MulticoreTSNE.egg-info/requires.txt
  writing top-level names to MulticoreTSNE.egg-info/top_level.txt
  reading manifest file 'MulticoreTSNE.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE.txt'
  writing manifest file 'MulticoreTSNE.egg-info/SOURCES.txt'
  running build_ext
  cmake version 3.18.4
  
  CMake suite maintained and supported by Kitware (kitware.com/cmake).
  CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
  CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-4_pnlmiw/multicoretsne_f755883637e7456cac20fedb190b6da2/build/temp.linux-x86_64-3.8/CMakeFiles/CMakeOutput.log".
  
  ERROR: Cannot generate Makefile. See above errors.
  ----------------------------------------
  ERROR: Failed building wheel for MulticoreTSNE

@scottgigante-immunai
Copy link

Solved with pip install git+https://github.com/jorvis/Multicore-TSNE. @DmitryUlyanov please merge #92 !

@divyagupta25
Copy link

sudo apt-get install make worked for me

@iTomxy
Copy link

iTomxy commented Sep 22, 2023

(Sept. 22, 2023) I follow the manual installing method in README, i.e.

git clone https://github.com/DmitryUlyanov/Multicore-TSNE.git
cd Multicore-TSNE/
pip install .    # <- got cmake error

but still got cmake error. My cmake version is 3.20.2. As a workaround, I manually modify the setup.py line 55, i.e. change

if version.parse(self.get_cmake_version()) < version.parse("3.22.0"):
    cmake_passthru_flag = "--"
else:
    cmake_passthru_flag = "-S"

to

# if version.parse(self.get_cmake_version()) < version.parse("3.22.0"):
#     cmake_passthru_flag = "--"
# else:
cmake_passthru_flag = "-S"

Then it install correctly.

@CamiloMartinezM
Copy link

Worked for me on a conda environment and cmake version 3.27.7.

> git clone https://github.com/DmitryUlyanov/Multicore-TSNE.git
Clonando en 'Multicore-TSNE'...
remote: Enumerating objects: 558, done.
remote: Counting objects: 100% (53/53), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 558 (delta 21), reused 35 (delta 13), pack-reused 505
Recibiendo objetos: 100% (558/558), 456.25 KiB | 5.24 MiB/s, listo.
Resolviendo deltas: 100% (314/314), listo.
> cd Multicore-TSNE/
> conda activate multicoretsne
> pip install .
Processing /home/Camilo/Documentos/Repositorios/Multicore-TSNE
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in /home/Camilo/anaconda3/envs/multicoretsne/lib/python3.11/site-packages (from MulticoreTSNE==0.1) (1.26.4)
Requirement already satisfied: cffi in /home/Camilo/anaconda3/envs/multicoretsne/lib/python3.11/site-packages (from MulticoreTSNE==0.1) (1.16.0)
Requirement already satisfied: pycparser in /home/Camilo/anaconda3/envs/multicoretsne/lib/python3.11/site-packages (from cffi->MulticoreTSNE==0.1) (2.21)
Building wheels for collected packages: MulticoreTSNE
  Building wheel for MulticoreTSNE (setup.py) ... done
  Created wheel for MulticoreTSNE: filename=MulticoreTSNE-0.1-cp311-cp311-linux_x86_64.whl size=71819 sha256=a6ba096a2a986753b207a0ddaafb90f1d4e4f5d7d6e0539123b06403369c728c
  Stored in directory: /home/Camilo/.cache/pip/wheels/f2/27/3a/e0b555687057204b7b11ff19c795409d809d348575b1e2eb43
Successfully built MulticoreTSNE
Installing collected packages: MulticoreTSNE
Successfully installed MulticoreTSNE-0.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@jreades @ndwarshuis @yash-goel @kmoad @divyagupta25 @TheAustinator @iTomxy @CamiloMartinezM @scottgigante-immunai