[Lint] Add FSDP/composable API files to ufmt include #90873

awgu · 2022-12-14T22:45:46Z

Stack from ghstack:

[Lint] Add FSDP/composable API files to ufmt include #90873 [Lint] Add FSDP/composable API files to ufmt include

This PR adds FSDP and composable API files to .lintrunner.toml so that (1) lintrunner enforces that those files are formatted and (2) lintrunner f formats those files for you.

There are two requirements here (see https://github.com/pytorch/pytorch/wiki/lintrunner for details):

Install lintrunner:

pip install lintrunner
lintrunner init

lintrunner f before you finalize your PR, which would now be enforced by CI after this PR.

The code changes in this PR outside of .lintrunner.toml are the result of lintrunner f.

I only plan to land this PR if all of the composable API developers agree that this is something that makes sense and is not too intrusive to the workflow.

[ghstack-poisoned]

pytorch-bot · 2022-12-14T22:45:49Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90873

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 4d1e5a2:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]

I am trying to see if we can get ufmt to cover FSDP and composable API files so that we do not need to keep having separate PRs to ufmt the files. [ghstack-poisoned]

This PR adds FSDP and composable API files to `.lintrunner.toml` so that (1) lintrunner enforces that those files are formatted and (2) `lintrunner f` formats those files for you. This should not impede developer productivity because lintrunner comes with PyTorch. The only added step in the development process is the `lintrunner f`, which is enforced on PRs. [ghstack-poisoned]

This PR adds FSDP and composable API files to `.lintrunner.toml` so that (1) lintrunner enforces that those files are formatted and (2) `lintrunner f` formats those files for you. There are two requirements here (see https://github.com/pytorch/pytorch/wiki/lintrunner for details): 1. Install lintrunner: ``` pip install lintrunner lintrunner init ``` 2. `lintrunner f` before you finalize your PR, which is now enforced by CI. The code changes outside of `.lintrunner.toml` are exactly the result of `lintrunner f`. [ghstack-poisoned]

ghstack-source-id: 423fad2 Pull Request resolved: #90873

This PR adds FSDP and composable API files to `.lintrunner.toml` so that (1) lintrunner enforces that those files are formatted and (2) `lintrunner f` formats those files for you. There are two requirements here (see https://github.com/pytorch/pytorch/wiki/lintrunner for details): 1. Install lintrunner: ``` pip install lintrunner lintrunner init ``` 2. `lintrunner f` before you finalize your PR, which would now be enforced by CI after this PR. The code changes in this PR outside of `.lintrunner.toml` are the result of `lintrunner f`. --- I only plan to land this PR if all of the composable API developers agree that this is something that makes sense and is not too intrusive to the workflow. [ghstack-poisoned]

ghstack-source-id: 07be631 Pull Request resolved: #90873

mrshenli

LGTM! Thanks @awgu

rohan-varma

Great step in the right direction of enforcing lint!

awgu · 2023-01-17T23:23:22Z

Failure are related to Dynamo and should be unrelated.

2023-01-17T17:30:37.5734804Z Traceback (most recent call last):
2023-01-17T17:30:37.5735042Z   File "/opt/conda/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 440, in _compile
2023-01-17T17:30:37.5735462Z ##[endgroup]
2023-01-17T17:30:37.5735569Z     check_fn = CheckFunctionManager(
2023-01-17T17:30:37.5735804Z   File "/opt/conda/lib/python3.10/site-packages/torch/_dynamo/guards.py", line 526, in __init__
2023-01-17T17:30:37.5735903Z     local_builder = GuardBuilder(
2023-01-17T17:30:37.5736137Z   File "/opt/conda/lib/python3.10/site-packages/torch/_dynamo/guards.py", line 109, in __init__
2023-01-17T17:30:37.5736278Z     self.scope["__builtins__"].__dict__[name] = package_module  # type: ignore[index]
2023-01-17T17:30:37.5736452Z AttributeError: 'dict' object has no attribute '__dict__'

Edit: Let me rebase to be safe.

ghstack-source-id: 07be631 Pull Request resolved: pytorch#90873

This PR adds FSDP and composable API files to `.lintrunner.toml` so that (1) lintrunner enforces that those files are formatted and (2) `lintrunner f` formats those files for you. There are two requirements here (see https://github.com/pytorch/pytorch/wiki/lintrunner for details): 1. Install lintrunner: ``` pip install lintrunner lintrunner init ``` 2. `lintrunner f` before you finalize your PR, which would now be enforced by CI after this PR. The code changes in this PR outside of `.lintrunner.toml` are the result of `lintrunner f`. --- I only plan to land this PR if all of the composable API developers agree that this is something that makes sense and is not too intrusive to the workflow. [ghstack-poisoned]

ghstack-source-id: abeffc2 Pull Request resolved: #90873

awgu · 2023-01-18T02:09:27Z

@pytorchbot merge

pytorchmergebot · 2023-01-18T02:11:16Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…istributed.py`" As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. [ghstack-poisoned]

…etup ufmt for `distributed.py`" As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. [ghstack-poisoned]

…istributed.py`" As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. [ghstack-poisoned]

…etup ufmt for `distributed.py`" As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. [ghstack-poisoned]

…istributed.py`" As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. [ghstack-poisoned]

…py` (#96597) As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. Pull Request resolved: #96597 Approved by: https://github.com/rohan-varma

…etup ufmt for `distributed.py`" As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. [ghstack-poisoned]

…istributed.py`" As we already enabled ufmt for composable APIs in #90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. [ghstack-poisoned]

…py` (#96597) As we already enabled ufmt for composable APIs in pytorch/pytorch#90873, it seems a good idea to enable ufmt for other distributed APIs as well. This change setup ufmt for DDP. Pull Request resolved: pytorch/pytorch#96597 Approved by: https://github.com/rohan-varma

[Lint] Add FSDP/composable API files to ufmt include

d340aa3

[ghstack-poisoned]

pytorch-bot bot added the topic: not user facing topic category label Dec 14, 2022

Update on "[Lint] Add FSDP/composable API files to ufmt include"

4bb2883

[ghstack-poisoned]

Update on "[Lint] Add FSDP/composable API files to ufmt include"

b5a7d48

[ghstack-poisoned]

Update on "[Lint] Add FSDP/composable API files to ufmt include"

430446b

I am trying to see if we can get ufmt to cover FSDP and composable API files so that we do not need to keep having separate PRs to ufmt the files. [ghstack-poisoned]

awgu added the release notes: distributed (composable) label Dec 15, 2022

awgu pushed a commit that referenced this pull request Dec 15, 2022

[Lint] Add FSDP/composable API files to ufmt include

a967b25

ghstack-source-id: 423fad2 Pull Request resolved: #90873

awgu marked this pull request as ready for review December 15, 2022 09:38

awgu requested review from H-Huang, kwen2501, mrshenli, pritamdamania87, rohan-varma, wanchaol, yhcharles and zhaojuanmao as code owners December 15, 2022 09:38

awgu pushed a commit that referenced this pull request Jan 17, 2023

[Lint] Add FSDP/composable API files to ufmt include

88b15cb

ghstack-source-id: 07be631 Pull Request resolved: #90873

awgu added the ciflow/trunk Trigger trunk jobs on your pull request label Jan 17, 2023

mrshenli approved these changes Jan 17, 2023

View reviewed changes

rohan-varma approved these changes Jan 17, 2023

View reviewed changes

awgu pushed a commit to awgu/pytorch that referenced this pull request Jan 17, 2023

[Lint] Add FSDP/composable API files to ufmt include

4707641

ghstack-source-id: 07be631 Pull Request resolved: pytorch#90873

awgu pushed a commit that referenced this pull request Jan 18, 2023

[Lint] Add FSDP/composable API files to ufmt include

02f750f

ghstack-source-id: abeffc2 Pull Request resolved: #90873

pytorchmergebot added the Merged label Jan 18, 2023

pytorchmergebot closed this in 0d4bbd1 Jan 18, 2023

yhcharles mentioned this pull request Mar 11, 2023

[1/n] Consolidate replicate and DDP: setup ufmt for distributed.py #96597

Closed

facebook-github-bot deleted the gh/awgu/278/head branch June 8, 2023 15:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Lint] Add FSDP/composable API files to ufmt include #90873

[Lint] Add FSDP/composable API files to ufmt include #90873

Uh oh!

awgu commented Dec 14, 2022 •

edited

Loading

Uh oh!

pytorch-bot bot commented Dec 14, 2022 •

edited

Loading

Uh oh!

mrshenli left a comment

Uh oh!

rohan-varma left a comment

Uh oh!

awgu commented Jan 17, 2023 •

edited

Loading

Uh oh!

awgu commented Jan 18, 2023

Uh oh!

pytorchmergebot commented Jan 18, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

[Lint] Add FSDP/composable API files to ufmt include #90873

[Lint] Add FSDP/composable API files to ufmt include #90873

Uh oh!

Conversation

awgu commented Dec 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Dec 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/90873

✅ No Failures

Uh oh!

mrshenli left a comment

Choose a reason for hiding this comment

Uh oh!

rohan-varma left a comment

Choose a reason for hiding this comment

Uh oh!

awgu commented Jan 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

awgu commented Jan 18, 2023

Uh oh!

pytorchmergebot commented Jan 18, 2023

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

awgu commented Dec 14, 2022 •

edited

Loading

pytorch-bot bot commented Dec 14, 2022 •

edited

Loading

awgu commented Jan 17, 2023 •

edited

Loading