can't install apex #1594

haiqizhang · 2023-02-21T09:08:22Z

 Running command python setup.py egg_info
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/glm/apex/setup.py", line 4, in <module>
      from packaging.version import parse, Version
  ModuleNotFoundError: No module named 'packaging'
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/letrain/miniconda/envs/glm/bin/python -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/home/glm/apex/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /tmp/pip-pip-egg-info-pijoox9c
  cwd: /home/glm/apex/
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

my env:
Pytorch==1.7.0
python==3.8.15
CUDA==11.0

The text was updated successfully, but these errors were encountered:

fazlicodes · 2023-02-23T15:07:08Z

I'm getting the same issue

pkosel · 2023-02-23T17:02:29Z

Same

crcrpar · 2023-02-23T17:27:36Z

File "/home/glm/apex/setup.py", line 4, in
from packaging.version import parse, Version
ModuleNotFoundError: No module named 'packaging'

could you install packaging and retry?

pkosel · 2023-02-23T17:35:09Z

That solved it for me.

fazlicodes · 2023-02-23T17:53:18Z

Doesn't work for me, packaging already installed

honeysuckcle · 2023-02-24T02:15:17Z

File "/home/glm/apex/setup.py", line 4, in
from packaging.version import parse, Version
ModuleNotFoundError: No module named 'packaging'

could you install packaging and retry?

It works. Thank you!

zw-xxx · 2023-02-24T07:52:31Z

Doesn't work for me, packaging already installed

I came to the same situation. I have solved it by installing the packaging by conda install packaging rather than pip install packaging.

And APEX gets installed successfully for me.

DjokerR · 2023-02-27T08:18:09Z

cuda 11.7
python3.7.6
torch==1.13.1
torchvision==0.14.1
torchaudio==0.13.1

unzip apex.zip
cd apex

1、install no cuda
python setup.py install

2、 install with cuda
python setup.py install --cuda_ext
The compilation process is very long, at least ten minutes, and the CPU and memory footprint is 8c 8G

zp2459 · 2023-03-03T05:15:02Z

same problem，
Using pip 22.3.1 from /home/panz/anaconda3/envs/gpt/lib/python3.8/site-packages/pip (python 3.8)
WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option / --install-option. Consider using --config-settings for more flexibility.
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at pypa/pip#11453
Processing /home/panz/project/ColossalAI/examples/language/gpt/gemini/apex
Running command python setup.py egg_info

torch.version = 1.12.0+cu113

running egg_info
creating /tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info
writing /tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info/dependency_links.txt
writing requirements to /tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info/requires.txt
writing top-level names to /tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file '/tmp/pip-pip-egg-info-7w8pc_qc/apex.egg-info/SOURCES.txt'
Preparing metadata (setup.py) ... done
Requirement already satisfied: packaging>20.6 in /home/panz/anaconda3/envs/gpt/lib/python3.8/site-packages (from apex==0.1) (23.0)
Installing collected packages: apex
DEPRECATION: apex is being installed using the legacy 'setup.py install' method, because the '--no-binary' option was enabled for it and this currently disables local wheel building for projects that don't have a 'pyproject.toml' file. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at pypa/pip#11451
Running command Running setup.py install for apex

torch.version = 1.12.0+cu113

Compiling cuda extensions with
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Feb__7_19:32:13_PST_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0
from /home/panz/anaconda3/envs/gpt/bin

Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "/home/panz/project/ColossalAI/examples/language/gpt/gemini/apex/setup.py", line 171, in
check_cuda_torch_binary_vs_bare_metal(CUDA_HOME)
File "/home/panz/project/ColossalAI/examples/language/gpt/gemini/apex/setup.py", line 33, in check_cuda_torch_binary_vs_bare_metal
raise RuntimeError(
RuntimeError: Cuda extensions are being compiled with a version of Cuda that does not match the version used to compile Pytorch binaries. Pytorch binaries were compiled with Cuda 11.3.
In some cases, a minor-version mismatch will not cause later errors: #323 (comment). You can try commenting out this check (at your own risk).
error: subprocess-exited-with-error

× Running setup.py install for apex did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/panz/anaconda3/envs/gpt/bin/python -u -c '
exec(compile('"'"''"'"''"'"'

This is -- a caller that pip uses to run setup.py

- It imports setuptools before invoking setup.py, to enable projects that directly

import from `distutils.core` to work with newer packaging standards.

- It provides a clear error message when setuptools is not installed.

- It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so

setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:

manifest_maker: standard file '"'"'-c'"'"' not found".

- It generates a shim setup.py, for handling setup.cfg-only projects.

import os, sys, tokenize

try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute setup.py since setuptools is not available in "
"the build environment.",
file=sys.stderr,
)
sys.exit(1)

file = %r
sys.argv[0] = file

if os.path.exists(file):
filename = file
with tokenize.open(file) as f:
setup_py_code = f.read()
else:
filename = ""
setup_py_code = "from setuptools import setup; setup()"

exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/home/panz/project/ColossalAI/examples/language/gpt/gemini/apex/setup.py'"'"',), "", "exec"))' --cpp_ext --cuda_ext install --record /tmp/pip-record-ss3s_tnl/install-record.txt --single-version-externally-managed --compile --install-headers /home/panz/anaconda3/envs/gpt/include/python3.8/apex
cwd: /home/panz/project/ColossalAI/examples/language/gpt/gemini/apex/
Running setup.py install for apex ... error
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> apex

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Linghuxc · 2023-03-29T02:56:15Z

thank you very much

SeekPoint · 2023-06-30T15:03:01Z

conda install packaging

not works,
the bug still exist

LambdaGuard · 2023-07-12T14:02:47Z

conda install packaging not works
I tried to 'git checkout' a older version and apex was installed sucessfully

the-yanqi · 2023-07-14T15:41:57Z

conda install packaging not works I tried to 'git checkout' a older version and apex was installed sucessfully

Which older version did you use?

LambdaGuard · 2023-07-14T16:13:31Z

conda install packaging not works I tried to 'git checkout' a older version and apex was installed sucessfully

Which older version did you use?

I checkout the commit "6943fd26e04c59327de32592cf5af68be8f5c44e" and it works (actually chosen randomly an older one)

zarzen · 2023-07-15T21:51:16Z

Using the release 23.05 works for me (https://github.com/NVIDIA/apex/tags)
Hash tag: 0da3ffb

karol-nowakowski · 2023-07-27T03:48:29Z

Using the release 23.05 works for me (https://github.com/NVIDIA/apex/tags) Hash tag: 0da3ffb

This release worked for me too, thanks @zarzen !

My environment:
Pytorch==2.1.0.dev20230725+cu121
Python==3.10.6
CUDA==12.1

shjwudp · 2023-08-23T10:02:32Z

try add --no-build-isolation flag in pip install.

lixin4ever · 2023-08-25T03:28:29Z

try add --no-build-isolation flag in pip install.

After trying all the solutions above (but the issue still kept unresolved), this one eventually worked for me 👍

wangbxj1234 · 2023-08-26T14:23:49Z

try add --no-build-isolation flag in pip install.

thx

aakejiang · 2023-08-29T07:28:12Z

try add --no-build-isolation flag in pip install.

It works for me. The full command is:
pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./

As discussed in NVIDIA/apex#1594, it appears the only solution to make apex build on a recent release is to ignore the build dependencies during packaging... Otherwise, we get the error that the `packaging` package is missing, although we explicitly install it.

ahengg · 2023-11-10T07:16:22Z

try add --no-build-isolation flag in pip install.

Thankyou it's worked

Fire-Star · 2023-11-15T06:42:47Z

6943fd2

I chosed 22.3 in tags，successed

mikenetrino · 2023-11-22T07:05:19Z

pip uninstall setuptools

pip install setuptools==60.2.0

pip install packaging

rm -R apex

git clone https://github.com/NVIDIA/apex

cd apex

pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./

~/It will work :)

VinceChin · 2023-11-28T10:37:22Z

Having tried all of the above, but still not working for me....

JuicyJeong · 2023-12-06T14:46:32Z

pip uninstall setuptools

pip install setuptools==60.2.0

pip install packaging

rm -R apex

git clone https://github.com/NVIDIA/apex

cd apex

pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./

~/It will work :)

it works! thanks mikenetrino :)

rocke2020 · 2023-12-08T14:15:56Z

pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./

pip uninstall setuptools
pip install setuptools==60.2.0
pip install packaging
rm -R apex
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./
~/It will work :)

it works! thanks mikenetrino :)
@JuicyJeong
could you share your env by runing, such as the version of python, cuda, pytorch, etc. thanks!!
python -m torch.utils.collect_env

h2222 · 2023-12-14T12:29:18Z

--no-build-isolation

hero

CrackerHax · 2023-12-25T06:17:00Z

try add --no-build-isolation flag in pip install.

Worked for me

tbbatbb · 2024-01-06T09:11:05Z

Using the release 23.05 works for me (https://github.com/NVIDIA/apex/tags) Hash tag: 0da3ffb

Worked for me. Thank you my hero.

wuusn · 2024-01-08T03:33:22Z

the full script is this:

wget https://github.com/NVIDIA/apex/archive/refs/tags/23.05.zip
unzip 23.05.zip
cd apex-23.05/
pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./

Huanxi2018 · 2024-01-16T03:44:37Z

try add --no-build-isolation flag in pip install.

After trying all the solutions above (but the issue still kept unresolved), this one eventually worked for me 👍

It works. Thanks a lot.

--no-build-isolation flag solves this issue for installation NVIDIA/apex#1594

MikeBrock03 · 2024-03-12T08:34:28Z

pip uninstall setuptools

pip install setuptools==60.2.0

pip install packaging

rm -R apex

git clone https://github.com/NVIDIA/apex

cd apex

pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation --config-settings "--build-option=--cpp_ext" --config-settings "--build-option=--cuda_ext" ./

~/It will work :)

it did! ♥️

kennedyCzar · 2024-03-28T08:34:32Z

Try,
pip install -v --no-build-isolation --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
It solved mine.

FunnyRainn · 2024-04-02T03:04:07Z

try add --no-build-isolation flag in pip install.

cooooooooooooollll!!! it works! thanls!!

FunnyRainn · 2024-04-02T03:04:43Z

try add --no-build-isolation flag in pip install.

cooooooooooooollll!!! it works! thanls!!

i use this :pip install -v --no-cache-dir . --no-build-isolation

kelvin0207 · 2024-05-06T09:12:42Z

Using the release 23.05 works for me (https://github.com/NVIDIA/apex/tags) Hash tag: 0da3ffb

This works for me! Thank you so much

haiqizhang added the bug Something isn't working label Feb 21, 2023

kevingreenman mentioned this issue Jul 25, 2023

ImportError - fast_transformers/causal_product undefined symbol - unable to train or finetune IBM/molformer#6

Open

MaxiBoether mentioned this issue Sep 19, 2023

Fix Apex Build eth-easl/modyn#298

Merged

ming-afk mentioned this issue Nov 14, 2023

Original ImportError was: ModuleNotFoundError("No module named 'amp_C') #573

Closed

rocke2020 mentioned this issue Dec 7, 2023

No module named 'amp_C' error for py3.9 pytorch2.1.0 cuda12.1 #1757

Closed

joshwigginton mentioned this issue Feb 2, 2024

Docker file doesn't work TradeMaster-NTU/TradeMaster#190

Open

baijiuyang added a commit to baijiuyang/ZITS_inpainting that referenced this issue Feb 21, 2024

Update README.md

66ee829

--no-build-isolation flag solves this issue for installation NVIDIA/apex#1594

baijiuyang mentioned this issue Feb 21, 2024

Update README.md DQiaole/ZITS_inpainting#78

Merged

can't install apex #1594

can't install apex #1594

Comments

haiqizhang commented Feb 21, 2023 • edited by crcrpar

fazlicodes commented Feb 23, 2023 • edited

pkosel commented Feb 23, 2023

crcrpar commented Feb 23, 2023

pkosel commented Feb 23, 2023

fazlicodes commented Feb 23, 2023

honeysuckcle commented Feb 24, 2023

zw-xxx commented Feb 24, 2023

DjokerR commented Feb 27, 2023

zp2459 commented Mar 3, 2023

This is -- a caller that pip uses to run setup.py

- It imports setuptools before invoking setup.py, to enable projects that directly

import from distutils.core to work with newer packaging standards.

- It provides a clear error message when setuptools is not installed.

- It sets sys.argv[0] to the underlying setup.py, when invoking setup.py so

setuptools doesn'"'"'t think the script is -c. This avoids the following warning:

manifest_maker: standard file '"'"'-c'"'"' not found".

- It generates a shim setup.py, for handling setup.cfg-only projects.

Linghuxc commented Mar 29, 2023

SeekPoint commented Jun 30, 2023

LambdaGuard commented Jul 12, 2023

the-yanqi commented Jul 14, 2023

LambdaGuard commented Jul 14, 2023

zarzen commented Jul 15, 2023

karol-nowakowski commented Jul 27, 2023 • edited

shjwudp commented Aug 23, 2023

lixin4ever commented Aug 25, 2023

wangbxj1234 commented Aug 26, 2023

aakejiang commented Aug 29, 2023 • edited

ahengg commented Nov 10, 2023

Fire-Star commented Nov 15, 2023

mikenetrino commented Nov 22, 2023

VinceChin commented Nov 28, 2023

JuicyJeong commented Dec 6, 2023

rocke2020 commented Dec 8, 2023

h2222 commented Dec 14, 2023

CrackerHax commented Dec 25, 2023

tbbatbb commented Jan 6, 2024

wuusn commented Jan 8, 2024

Huanxi2018 commented Jan 16, 2024

MikeBrock03 commented Mar 12, 2024

kennedyCzar commented Mar 28, 2024

FunnyRainn commented Apr 2, 2024

FunnyRainn commented Apr 2, 2024

kelvin0207 commented May 6, 2024

haiqizhang commented Feb 21, 2023 •

edited by crcrpar

fazlicodes commented Feb 23, 2023 •

edited

import from `distutils.core` to work with newer packaging standards.

- It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so

setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:

karol-nowakowski commented Jul 27, 2023 •

edited

aakejiang commented Aug 29, 2023 •

edited