Cannot run pip install (error)

[yerzhik]

Description

succesfuly run make (cmake, make)
then, after running pip install --user . (without -e option)
getting error message (below)

Error Message

WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing /home/user/temp/mxnet/python
Requirement already satisfied: numpy<2.0.0,>1.16.0 in /home/user/.local/lib/python2.7/site-packages (from mxnet==1.6.0) (1.16.6)
Requirement already satisfied: requests<3,>=2.20.0 in /home/user/.local/lib/python2.7/site-packages (from mxnet==1.6.0) (2.23.0)
Requirement already satisfied: graphviz<0.9.0,>=0.8.1 in /home/user/.local/lib/python2.7/site-packages (from mxnet==1.6.0) (0.8.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/user/.local/lib/python2.7/site-packages (from requests<3,>=2.20.0->mxnet==1.6.0) (2019.11.28)
Requirement already satisfied: idna<3,>=2.5 in /home/user/.local/lib/python2.7/site-packages (from requests<3,>=2.20.0->mxnet==1.6.0) (2.9)
Requirement already satisfied: chardet<4,>=3.0.2 in /home/user/.local/lib/python2.7/site-packages (from requests<3,>=2.20.0->mxnet==1.6.0) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/user/.local/lib/python2.7/site-packages (from requests<3,>=2.20.0->mxnet==1.6.0) (1.25.8)
Building wheels for collected packages: mxnet
  Building wheel for mxnet (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-ohO0MR/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-ohO0MR/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-I12gsz
       cwd: /tmp/pip-req-build-ohO0MR/
  Complete output (21 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/operator.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/symbol_doc.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/executor.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/random.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/executor_manager.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/model.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/test_utils.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/lr_scheduler.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/error.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/torch.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/base.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/context.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/tvmop.py -> build/lib.linux-x86_64-2.7/mxnet
  copying mxnet/monitor.py -> build/lib.linux-x86_64-2.7/mxnet
  error: can't copy 'mxnet/space.py': doesn't exist or not a regular file
  ----------------------------------------
  ERROR: Failed building wheel for mxnet
  Running setup.py clean for mxnet
Failed to build mxnet
Installing collected packages: mxnet
  Attempting uninstall: mxnet
    Found existing installation: mxnet 1.6.0
    Uninstalling mxnet-1.6.0:
      Successfully uninstalled mxnet-1.6.0
    Running setup.py install for mxnet ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-ohO0MR/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-ohO0MR/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-gg1hhB/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/user/.local/include/python2.7/mxnet
         cwd: /tmp/pip-req-build-ohO0MR/
    Complete output (21 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/operator.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/symbol_doc.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/executor.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/random.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/executor_manager.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/model.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/test_utils.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/lr_scheduler.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/error.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/torch.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/base.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/context.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/tvmop.py -> build/lib.linux-x86_64-2.7/mxnet
    copying mxnet/monitor.py -> build/lib.linux-x86_64-2.7/mxnet
    error: can't copy 'mxnet/space.py': doesn't exist or not a regular file
    ----------------------------------------
  Rolling back uninstall of mxnet
  Moving to /home/user/.local/lib/python2.7/site-packages/mxnet.egg-link
   from /tmp/pip-uninstall-a7_FbF/mxnet.egg-link
ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-ohO0MR/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-ohO0MR/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-gg1hhB/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/user/.local/include/python2.7/mxnet Check the logs for full command output.

To Reproduce

Just trying to install using pip install --user
happens with pip3 or pip2

Steps to reproduce

(Paste the commands you ran that produced the error.)

1. git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet
2. cd mxnet
3. cp config/linux_gpu.cmake config.cmake
4. rm -rf build
5. mkdir -p build && cd build
6. cmake ..
7. cmake --build .
8. cd ..
9. cd python
10. pip install --user .

What have you tried to solve it?

1. tried with pip install --user -e .
   But when using library (import mxnet)
   got the following error:

line 25, in <module>
    import mxnet
  File "/home/user/temp/mxnet/python/mxnet/__init__.py", line 23, in <module>
    from .context import Context, current_context, cpu, gpu, cpu_pinned
  File "/home/user/temp/mxnet/python/mxnet/context.py", line 23, in <module>
    from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
  File "/home/user/temp/mxnet/python/mxnet/base.py", line 29, in <module>
    import numpy as _np
  File "/home/user/temp/mxnet/python/mxnet/numpy/__init__.py", line 21, in <module>
    from . import random
  File "/home/user/temp/mxnet/python/mxnet/numpy/random.py", line 20, in <module>
    from ..ndarray import numpy as _mx_nd_np
  File "/home/user/temp/mxnet/python/mxnet/ndarray/__init__.py", line 20, in <module>
    from . import _internal, contrib, linalg, op, random, sparse, utils, image, ndarray, numpy
  File "/home/user/temp/mxnet/python/mxnet/ndarray/_internal.py", line 35, in <module>
    from .._ctypes.ndarray import NDArrayBase, CachedOp
  File "/home/user/temp/mxnet/python/mxnet/_ctypes/ndarray.py", line 25, in <module>
    from ..base import _LIB
ImportError: cannot import name _LIB

Environment

We recommend using our script for collecting the diagnositc information. Run the following command and paste the outputs below:

curl --retry 10 -s https://raw.githubusercontent.com/dmlc/gluon-nlp/master/tools/diagnose.py | python

----------Python Info----------
('Version      :', '2.7.15+')
('Compiler     :', 'GCC 7.4.0')
('Build        :', ('default', 'Oct  7 2019 17:39:04'))
('Arch         :', ('64bit', ''))
------------Pip Info-----------
('Version      :', '20.0.2')
('Directory    :', '/usr/local/lib/python2.7/dist-packages/pip')
----------MXNet Info-----------
No MXNet installed.
----------System Info----------
('Platform     :', 'Linux-5.3.0-26-generic-x86_64-with-Ubuntu-18.04-bionic')
('system       :', 'Linux')
('node         :', 'user-desktop')
('release      :', '5.3.0-26-generic')
('version      :', '#28~18.04.1-Ubuntu SMP Wed Dec 18 16:40:14 UTC 2019')
----------Hardware Info----------
('machine      :', 'x86_64')
('processor    :', 'x86_64')
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              12
On-line CPU(s) list: 0-11
Thread(s) per core:  2
Core(s) per socket:  6
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
Stepping:            10
CPU MHz:             3912.041
CPU max MHz:         4700.0000
CPU min MHz:         800.0000
BogoMIPS:            7399.70
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            12288K
NUMA node0 CPU(s):   0-11
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
----------Network Test----------
Setting timeout: 10
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0104 sec, LOAD: 0.5736 sec.
Timing for D2L: http://d2l.ai, DNS: 0.0439 sec, LOAD: 0.1898 sec.
Timing for FashionMNIST: https://repo.mxnet.io/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.1669 sec, LOAD: 0.5573 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0120 sec, LOAD: 0.3780 sec.
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0007 sec, LOAD: 0.6053 sec.
Timing for GluonNLP: http://gluon-nlp.mxnet.io, DNS: 0.1362 sec, LOAD: 0.9050 sec.
Timing for D2L (zh-cn): http://zh.d2l.ai, DNS: 0.0165 sec, LOAD: 0.2485 sec.
Timing for GluonNLP GitHub: https://github.com/dmlc/gluon-nlp, DNS: 0.0007 sec, LOAD: 0.7668 sec.

[leezu]

Try with the -e option. If that works, please update the title of this issue to be more specific. Thanks.

[harish-vnkt]

Facing the same issue with python setup.py install too. It's strange because importing mxnet seems to work with Python3 but not Python2.

[leezu]

Python 2 is EOL and unsupported indeed.

[harish-vnkt]

@leezu Is there a way I can modify CMakeLists.txt to compile mxnet to work with Python2.7? Or perhaps a previous version of mxnet that I could use?

[leezu]

This is unrelated to cmake. But yes, you can use MXNet 1.6, which is the last release of MXNet support Python 2. See https://github.com/apache/incubator-mxnet/releases/tag/1.6.0

However, wanting to use Python 2 seems quite dubious to me. Are you sure you need that?

[harish-vnkt]

I am trying to build mxnet from source because a repository (who's code I am trying to run) has provided custom operators that need to be included in the source. The repository code is in Python2. After I install the python binding, running import mxnet in a Python2 interpreter gives me ImportError: cannot import name _LIB, whereas it runs successfully in a Python3 interpreter.

[leezu]

You probably want to use the same version of MXNet as used by the authors of that repo, as it appears that repo is quite old?

[harish-vnkt]

@leezu That's what I initially did. :(
The authors use https://github.com/apache/incubator-mxnet/tree/62ecb60 for their experiment, but I got compilation errors that I couldn't solve. I have limited computing resources so I couldn't try out that install too many times before deciding to move to the latest version. I would love to get this install working in a Python2 environment. Thanks for your quick replies!

[leezu]

You can try following the old install guide: https://mxnet.apache.org/versions/0.11.0/get_started/install.html