Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unpin DeepSpeed to allow 0.13.0 and greater #3905

Closed
arnavgarg1 opened this issue Jan 22, 2024 · 0 comments
Closed

Unpin DeepSpeed to allow 0.13.0 and greater #3905

arnavgarg1 opened this issue Jan 22, 2024 · 0 comments
Assignees

Comments

@arnavgarg1
Copy link
Contributor

arnavgarg1 commented Jan 22, 2024

DeepSpeed 0.13.0 was released on Jan 20th, 2024, and is to be causing our tests to fail with the following error: https://github.com/ludwig-ai/ludwig/actions/runs/7614778487/job/20737857305?pr=3901

Collecting deepspeed!=0.11.0 (from ludwig==0.9.dev0)
  Downloading deepspeed-0.13.0.tar.gz (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 92.8 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [38 lines of output]
Warning: RNING] Unable to import torch, pre-compiling ops will be disabled. Please visit https://pytorch.org/ to see how to properly install torch on your system.
       [WARNING]  unable to import torch, please install it if you want to pre-compile any deepspeed ops.
      Traceback (most recent call last):
        File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/builder.py", line 14, in <module>
          from op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
      ImportError: cannot import name 'TORCH_MAJOR' from 'op_builder.builder' (/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/builder.py)
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 480, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-l1qaau3a/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 37, in <module>
        File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/__init__.py", line 48, in <module>
          module = importlib.import_module(f".{module_name}", package=op_builder_dir)
        File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/importlib/__init__.py", line 127, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
        File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/__init__.py", line 6, in <module>
          from .cpu_adam import CPUAdamBuilder
        File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/cpu_adam.py", line 6, in <module>
          from .builder import SYCLOpBuilder
        File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/op_builder/xpu/builder.py", line 16, in <module>
          from deepspeed.ops.op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
        File "/tmp/pip-install-e8btlrcw/deepspeed_59dc3579cb9240e9b578a8bf59239b19/deepspeed/__init__.py", line 10, in <module>
          import torch
      ModuleNotFoundError: No module named 'torch'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Error: Process completed with exit code 1.

Deepspeed was pinned to <0.13.0 in #3906

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants