import torch: AttributeError: module 'torch.distributed' has no attribute 'BuiltinCommHookType' #47153

kshitij12345 · 2020-10-31T14:59:11Z

After building the latest master ee0033a with,

USE_DISTRIBUTED=0 USE_GLOO=0 BUILD_TEST=0 USE_CUDA=1 USE_MKLDNN=0 DEBUG=0 python setup.py install

On importing torch, I get the following error,

>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.conda/envs/PytorchENV/lib/python3.7/site-packages/torch/__init__.py", line 526, in <module>
    from .functional import *
  File "/home/user/.conda/envs/PytorchENV/lib/python3.7/site-packages/torch/functional.py", line 6, in <module>
    import torch.nn.functional as F
  File "/home/user/.conda/envs/PytorchENV/lib/python3.7/site-packages/torch/nn/__init__.py", line 3, in <module>
    from .parallel import DataParallel
  File "/home/user/.conda/envs/PytorchENV/lib/python3.7/site-packages/torch/nn/parallel/__init__.py", line 5, in <module>
    from .distributed import DistributedDataParallel
  File "/home/user/.conda/envs/PytorchENV/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 93, in <module>
    class DistributedDataParallel(Module):
  File "/home/user/.conda/envs/PytorchENV/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 1064, in DistributedDataParallel
    self, comm_hook_type: dist.BuiltinCommHookType
AttributeError: module 'torch.distributed' has no attribute 'BuiltinCommHookType'
>>>

Previously there was no issue with the same.

Probably related to ee0033a

cc @ezyang @gchanan @zou3519 @bdhirsh @heitorschueroff @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @xush6528 @osalpekar @jiayisuse @agolynski

The text was updated successfully, but these errors were encountered:

jonasteuwen · 2020-11-01T20:03:21Z

Same problem on OS X.

robieta · 2020-11-02T15:24:15Z

@SciPioneer This breaks any USE_DISTRIBUTED=0 build.

… as built-in comm hooks" Revert the diff because of #47153 Differential Revision: [D24691866](https://our.internmc.facebook.com/intern/diff/D24691866/) [ghstack-poisoned]

… as built-in comm hooks" Revert the diff because of #47153 Original PR issue: C++ DDP Communication Hook #46348 Differential Revision: [D24691866](https://our.internmc.facebook.com/intern/diff/D24691866/) [ghstack-poisoned]

… as built-in comm hooks" Revert the diff because of #47153 Original PR issue: C++ DDP Communication Hook #46348 Differential Revision: [D24691866](https://our.internmc.facebook.com/intern/diff/D24691866/) ghstack-source-id: 115720415 Pull Request resolved: #47234

… as built-in comm hooks" (#47234) Summary: Pull Request resolved: #47234 Revert the diff because of #47153 Original PR issue: C++ DDP Communication Hook #46348 ghstack-source-id: 115720415 Test Plan: waitforbuildbot Reviewed By: mrshenli Differential Revision: D24691866 fbshipit-source-id: 58fe0c45943a2ae2a09fe5d5eac4a4d947586539

…in comm hooks This is almost same as #46959, except that in caffe2/torch/nn/parallel/distributed.py, BuiltinCommHookType is imported conditionally, only when dist.is_available(). Otherwise, this Python enum type defined in caffe2/torch/scrc/distributed/c10d/init.cpp cannot be imported, which is similar to another enum type ReduceOp defined in the same file. See #47153 To review the diff on top of #46959, compare V1 vs Latest. Main Changes in V1 (#46959): 1. Implemented the Pybind part. 2. In the reducer, once the builtin_comm_hook_type is set, a c++ comm hook instance will be created in Reducer::autograd_hook. 3. Added unit tests for the builit-in comm hooks. Original PR issue: C++ DDP Communication Hook #46348 Differential Revision: [D24700959](https://our.internmc.facebook.com/intern/diff/D24700959/) [ghstack-poisoned]

…in comm hooks This is almost same as #46959, except that in caffe2/torch/nn/parallel/distributed.py, BuiltinCommHookType is imported conditionally, only when dist.is_available(). Otherwise, this Python enum type defined in caffe2/torch/scrc/distributed/c10d/init.cpp cannot be imported, which is similar to another enum type ReduceOp defined in the same file. See #47153 To review the diff on top of #46959, compare V1 vs Latest. Main Changes in V1 (#46959): 1. Implemented the Pybind part. 2. In the reducer, once the builtin_comm_hook_type is set, a c++ comm hook instance will be created in Reducer::autograd_hook. 3. Added unit tests for the builit-in comm hooks. Original PR issue: C++ DDP Communication Hook #46348 Differential Revision: [D24700959](https://our.internmc.facebook.com/intern/diff/D24700959/) ghstack-source-id: 115753518 Pull Request resolved: #47270

…I as built-in comm hooks" This is almost same as #46959, except that in caffe2/torch/nn/parallel/distributed.py, BuiltinCommHookType is imported conditionally, only when dist.is_available(). Otherwise, this Python enum type defined in caffe2/torch/scrc/distributed/c10d/init.cpp cannot be imported, which is similar to another enum type ReduceOp defined in the same file. See #47153 Main Changes in #46959: 1. Implemented the Pybind part. 2. In the reducer, once the builtin_comm_hook_type is set, a c++ comm hook instance will be created in Reducer::autograd_hook. 3. Added unit tests for the builit-in comm hooks. Original PR issue: C++ DDP Communication Hook #46348 Differential Revision: [D24700959](https://our.internmc.facebook.com/intern/diff/D24700959/) [ghstack-poisoned]

…in comm hooks Pull Request resolved: #47270 This is almost same as #46959, except that in caffe2/torch/nn/parallel/distributed.py, BuiltinCommHookType should be imported conditionally, only when dist.is_available(). Otherwise, this Python enum type defined in caffe2/torch/scrc/distributed/c10d/init.cpp cannot be imported. See #47153 I tried to follow another enum type enum type ReduceOp defined in the same file, but did not work, because the C++ enum class is defined torch/lib/c10d library, but BuiltinCommHookType is defined in torch/csrc/distributed library. These two libraries are compiled in two different ways. To avoid adding typing to distributed package, which can be a new project, I simply removed the arg type of BuiltinCommHookType in this file. To review the diff on top of #46959, compare V1 vs Latest: https://www.internalfb.com/diff/D24700959?src_version_fbid=270445741055617 Main Changes in V1 (#46959): 1. Implemented the Pybind part. 2. In the reducer, once the builtin_comm_hook_type is set, a c++ comm hook instance will be created in Reducer::autograd_hook. 3. Added unit tests for the builit-in comm hooks. Original PR issue: C++ DDP Communication Hook #46348 ghstack-source-id: 115783237 Differential Revision: [D24700959](https://our.internmc.facebook.com/intern/diff/D24700959/)

…in comm hooks (#47270) Summary: Pull Request resolved: #47270 This is almost same as #46959, except that in caffe2/torch/nn/parallel/distributed.py, BuiltinCommHookType should be imported conditionally, only when dist.is_available(). Otherwise, this Python enum type defined in caffe2/torch/scrc/distributed/c10d/init.cpp cannot be imported. See #47153 I tried to follow another enum type enum type ReduceOp defined in the same file, but did not work, because the C++ enum class is defined torch/lib/c10d library, but BuiltinCommHookType is defined in torch/csrc/distributed library. These two libraries are compiled in two different ways. To avoid adding typing to distributed package, which can be a new project, I simply removed the arg type of BuiltinCommHookType in this file. To review the diff on top of #46959, compare V1 vs Latest: https://www.internalfb.com/diff/D24700959?src_version_fbid=270445741055617 Main Changes in V1 (#46959): 1. Implemented the Pybind part. 2. In the reducer, once the builtin_comm_hook_type is set, a c++ comm hook instance will be created in Reducer::autograd_hook. 3. Added unit tests for the builit-in comm hooks. Original PR issue: C++ DDP Communication Hook #46348 ghstack-source-id: 115783237 Test Plan: buck test mode/dev-nosan caffe2/test/distributed:c10d -- test_builtin_ddp_comm_hooks_nccl //arvr/projects/eye_tracking/Masquerade:python_test USE_DISTRIBUTED=0 USE_GLOO=0 BUILD_TEST=0 USE_CUDA=1 USE_MKLDNN=0 DEBUG=0 python setup.py install Reviewed By: mrshenli Differential Revision: D24700959 fbshipit-source-id: 69f303a48ae275aa856e6e9b50e12ad8602e1c7a

facebook-github-bot added the oncall: distributed Add this issue/PR to distributed oncall triage queue label Oct 31, 2020

robieta added the high priority label Nov 2, 2020

pytorch-probot bot added the triage review label Nov 2, 2020

wayi1 mentioned this issue Nov 2, 2020

Back out "[Gradient Compression] Surface C++ comm hooks to Python API as built-in comm hooks" #47234

Closed

wayi1 self-assigned this Nov 3, 2020

wayi1 closed this as completed Nov 3, 2020

wayi1 mentioned this issue Nov 3, 2020

[Gradient Compression] Surface C++ comm hooks to Python API as built-in comm hooks #47270

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

import torch: AttributeError: module 'torch.distributed' has no attribute 'BuiltinCommHookType' #47153

import torch: AttributeError: module 'torch.distributed' has no attribute 'BuiltinCommHookType' #47153

kshitij12345 commented Oct 31, 2020 •

edited by pytorch-probot bot

jonasteuwen commented Nov 1, 2020

robieta commented Nov 2, 2020 •

edited

import torch: AttributeError: module 'torch.distributed' has no attribute 'BuiltinCommHookType' #47153

import torch: AttributeError: module 'torch.distributed' has no attribute 'BuiltinCommHookType' #47153

Comments

kshitij12345 commented Oct 31, 2020 • edited by pytorch-probot bot

jonasteuwen commented Nov 1, 2020

robieta commented Nov 2, 2020 • edited

kshitij12345 commented Oct 31, 2020 •

edited by pytorch-probot bot

robieta commented Nov 2, 2020 •

edited