More sharded_tensor creation ops: harded_tensor.zeros, sharded_tensor.full, sharded_tensor.rand #63732

bowangbj · 2021-08-22T06:39:14Z

Stack from ghstack:

-> More sharded_tensor creation ops: harded_tensor.zeros, sharded_tensor.full, sharded_tensor.rand #63732

Summary:

Test Plan:

$ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D30472621

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @agolynski @SciPioneer @H-Huang @mrzzd @cbalioglu @gcramer23

….full, sharded_tensor.rand Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py TestCreateTensorFromParams --v $ python test/distributed/_sharded_tensor/test_sharded_tensor.py TestShardedTensorChunked --v Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

facebook-github-bot · 2021-08-22T06:39:21Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/63732
📄 Preview docs built from this PR

💊 CI failures summary and remediations

As of commit 8d5b8a3 (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_xenial_py3_6_gcc5_4_build (1/1)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Aug 26 19:21:32 CMake Error at CMakeLists.txt:1 (cmake_minimum_required):

Aug 26 19:21:31 invalid command 'bad_argument'
Aug 26 19:21:31 + retcode=1
Aug 26 19:21:31 + set -e
Aug 26 19:21:31 + return 1
Aug 26 19:21:31 + [[ pytorch-linux-xenial-py3.6-gcc5.4-build != *libtorch* ]]
Aug 26 19:21:31 + [[ pytorch-linux-xenial-py3.6-gcc5.4-build != *ppc64le* ]]
Aug 26 19:21:31 + [[ pytorch-linux-xenial-py3.6-gcc5.4-build != *clang* ]]
Aug 26 19:21:31 + WERROR=1
Aug 26 19:21:31 + python setup.py bdist_wheel
Aug 26 19:21:32 Building wheel torch-1.10.0a0+git8b2108a
Aug 26 19:21:32 CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
Aug 26 19:21:32   CMake 3.10 or higher is required.  You are running version 3.5.1
Aug 26 19:21:32 
Aug 26 19:21:32 
Aug 26 19:21:32 -- Configuring incomplete, errors occurred!
Aug 26 19:21:32 -- Building version 1.10.0a0+git8b2108a
Aug 26 19:21:32 cmake -DBUILD_ENVIRONMENT=pytorch-linux-xenial-py3.6-gcc5.4-build -DBUILD_PYTHON=True -DBUILD_STATIC_RUNTIME_BENCHMARK=ON -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/var/lib/jenkins/workspace/torch -DCMAKE_PREFIX_PATH=/opt/conda/lib/python3.6/site-packages;/opt/conda -DNUMPY_INCLUDE_DIR=/opt/conda/lib/python3.6/site-packages/numpy/core/include -DOPENSSL_ROOT_DIR=/opt/openssl -DPYTHON_EXECUTABLE=/opt/conda/bin/python -DPYTHON_INCLUDE_DIR=/opt/conda/include/python3.6m -DPYTHON_LIBRARY=/opt/conda/lib/libpython3.6m.so.1.0 -DTORCH_BUILD_VERSION=1.10.0a0+git8b2108a -DUSE_GLOO_WITH_OPENSSL=ON -DUSE_LLVM=/opt/llvm -DUSE_NUMPY=True -DWERROR=1 /var/lib/jenkins/workspace
Aug 26 19:21:32 + cleanup
Aug 26 19:21:32 + retcode=1
Aug 26 19:21:32 + set +x
Aug 26 19:21:32 =================== sccache compilation log ===================

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

bowangbj · 2021-08-22T06:41:56Z

@bowangbj has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

…ed_tensor.full, sharded_tensor.rand" Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D30472621](https://our.internmc.facebook.com/intern/diff/D30472621) [ghstack-poisoned]

….full, sharded_tensor.rand Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 0ec4f29 Pull Request resolved: #63732

bowangbj · 2021-08-22T06:50:46Z

@bowangbj has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

torch/distributed/_sharded_tensor/api.py

torch/distributed/_sharded_tensor/__init__.py

…ensor.zeros, sharded_tensor.full, sharded_tensor.rand" Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D30472621](https://our.internmc.facebook.com/intern/diff/D30472621) [ghstack-poisoned]

bowangbj · 2021-08-24T22:21:06Z

@bowangbj has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

bowangbj · 2021-08-24T22:22:05Z

Thanks Pritam for a great, detailed and fast review. Rebased and fixed all your cmts. PTAL

pritamdamania87

Looks good, just had a few more comments and it looks like there are some lint failures.

test/distributed/_sharded_tensor/test_sharded_tensor.py

torch/distributed/_sharded_tensor/__init__.py

…ensor.zeros, sharded_tensor.full, sharded_tensor.rand" Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D30472621](https://our.internmc.facebook.com/intern/diff/D30472621) [ghstack-poisoned]

bowangbj · 2021-08-25T00:37:22Z

@bowangbj has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

bowangbj · 2021-08-25T00:42:45Z

Looks good, just had a few more comments and it looks like there are some lint failures.

Thanks Pritam. Fixed all the cmts, PTAL

…ed_tensor.full, sharded_tensor.rand" Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D30472621](https://our.internmc.facebook.com/intern/diff/D30472621) [ghstack-poisoned]

….full, sharded_tensor.rand Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 41b7008 Pull Request resolved: #63732

bowangbj · 2021-08-25T01:00:37Z

@bowangbj has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

bowangbj · 2021-08-25T01:20:31Z

The only tricky lint error is

./torch/distributed/_sharded_tensor/init.py:1:1: C101 Coding magic comment not found
Please fix the above Flake8 warnings.
Error: Process completed with exit code 1.

I run $flake8 init.py locally which produced no error.

pritamdamania87 · 2021-08-25T17:33:00Z

The only tricky lint error is

./torch/distributed/_sharded_tensor/init.py:1:1: C101 Coding magic comment not found
Please fix the above Flake8 warnings.
Error: Process completed with exit code 1.

I run $flake8 init.py locally which produced no error.

I think https://pypi.org/project/flake8-coding/ outlines how you need to setup flake8 to repro this error. The page has links in terms of how to resolve the issue as well.

…sharded_tensor.full, sharded_tensor.rand" Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D30472621](https://our.internmc.facebook.com/intern/diff/D30472621) [ghstack-poisoned]

….full, sharded_tensor.rand Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 33fd0d6 Pull Request resolved: #63732

bowangbj · 2021-08-25T18:58:49Z

The only tricky lint error is

./torch/distributed/_sharded_tensor/init.py:1:1: C101 Coding magic comment not found
Please fix the above Flake8 warnings.
Error: Process completed with exit code 1.

I run $flake8 init.py locally which produced no error.

I think https://pypi.org/project/flake8-coding/ outlines how you need to setup flake8 to repro this error. The page has links in terms of how to resolve the issue as well.

OK, added the magic comment (encoding) explicitly to address flake8 complaint. PTAL

bowangbj · 2021-08-25T19:00:06Z

@bowangbj has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

test/distributed/_sharded_tensor/test_sharded_tensor.py

…ops: harded_tensor.zeros, sharded_tensor.full, sharded_tensor.rand" Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D30472621](https://our.internmc.facebook.com/intern/diff/D30472621) [ghstack-poisoned]

….full, sharded_tensor.rand Summary: Test Plan: $ python test/distributed/_sharded_tensor/test_sharded_tensor.py --v Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 44f6851 Pull Request resolved: #63732

bowangbj · 2021-08-26T19:13:04Z

@bowangbj has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

bowangbj · 2021-08-26T19:16:49Z

Thanks Pritam for the great review, added the tests, landing ...

facebook-github-bot · 2021-08-26T23:02:50Z

@bowangbj merged this pull request in 49353e3.

bowangbj requested review from H-Huang, cbalioglu, mingzhe09088, mrshenli, pritamdamania87, rohan-varma, wayi1 and zhaojuanmao as code owners August 22, 2021 06:39

facebook-github-bot added oncall: distributed Add this issue/PR to distributed oncall triage queue cla signed labels Aug 22, 2021

bowangbj removed request for H-Huang, cbalioglu, mingzhe09088, mrshenli, pritamdamania87, rohan-varma, wayi1 and zhaojuanmao August 22, 2021 06:40

bowangbj requested review from pritamdamania87 and wanchaol August 23, 2021 17:27

pritamdamania87 reviewed Aug 23, 2021

View reviewed changes

pritamdamania87 reviewed Aug 24, 2021

View reviewed changes

bowangbj requested a review from pritamdamania87 August 25, 2021 01:14

pritamdamania87 reviewed Aug 26, 2021

View reviewed changes

test/distributed/_sharded_tensor/test_sharded_tensor.py Show resolved Hide resolved

pritamdamania87 approved these changes Aug 26, 2021

View reviewed changes

facebook-github-bot closed this in 49353e3 Aug 26, 2021

facebook-github-bot added the Merged label Aug 26, 2021

facebook-github-bot deleted the gh/bowangb/2/head branch August 30, 2021 14:17

More sharded_tensor creation ops: harded_tensor.zeros, sharded_tensor.full, sharded_tensor.rand #63732

More sharded_tensor creation ops: harded_tensor.zeros, sharded_tensor.full, sharded_tensor.rand #63732

Uh oh!

Conversation

bowangbj commented Aug 22, 2021 • edited by pytorch-probot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Aug 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_linux_xenial_py3_6_gcc5_4_build (1/1)

Uh oh!

bowangbj commented Aug 22, 2021

Uh oh!

bowangbj commented Aug 22, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bowangbj commented Aug 24, 2021

Uh oh!

bowangbj commented Aug 24, 2021

Uh oh!

pritamdamania87 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bowangbj commented Aug 25, 2021

Uh oh!

bowangbj commented Aug 25, 2021

Uh oh!

bowangbj commented Aug 25, 2021

Uh oh!

bowangbj commented Aug 25, 2021

Uh oh!

pritamdamania87 commented Aug 25, 2021

Uh oh!

bowangbj commented Aug 25, 2021

Uh oh!

bowangbj commented Aug 25, 2021

Uh oh!

Uh oh!

bowangbj commented Aug 26, 2021

Uh oh!

bowangbj commented Aug 26, 2021

Uh oh!

facebook-github-bot commented Aug 26, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bowangbj commented Aug 22, 2021 •

edited by pytorch-probot bot

Loading

facebook-github-bot commented Aug 22, 2021 •

edited

Loading