Skip to content

[FSDP][state_dict] Add world_size 1 unittest #112669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[FSDP][state_dict] Add world_size 1 unittest #112669

wants to merge 1 commit into from

Conversation

fegin
Copy link
Contributor

@fegin fegin commented Nov 1, 2023 •
edited
Loading

Stack from ghstack (oldest at bottom):

As title

Differential Revision: D50754433

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Nov 1, 2023
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 1, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/112669

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 6e94fc4 with merge base 94f3df2 (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

fegin added a commit that referenced this pull request Nov 1, 2023
@fegin
[FSDP][state_dict] Add world_size 1 unittest
53d8eac 
As title

Differential Revision: [D50754433](https://our.internmc.facebook.com/intern/diff/D50754433/)

ghstack-source-id: 206184496
Pull Request resolved: #112669
wz337
wz337 reviewed Nov 2, 2023
View reviewed changes
test/distributed/fsdp/test_fsdp_state_dict.py
Comment on lines +1226 to +1227
FSDPInitMode.RECURSIVE,
CUDAInitMode.CUDA_BEFORE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QQ for my understanding: what are these init mode?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is explained here:

pytorch/torch/testing/_internal/common_fsdp.py

Line 424 in 6e94fc4

cuda_init_mode (CUDAInitMode): Determines model movement to CUDA.

wz337
wz337 approved these changes Nov 2, 2023
View reviewed changes
Copy link
Contributor

@wz337 wz337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

fduwjj
fduwjj reviewed Nov 2, 2023
View reviewed changes
test/distributed/fsdp/test_fsdp_state_dict.py
return self.outer(i + j)


class TestDummyModel(torch.nn.Module):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessary in this PR, but can we move it to torch/testing/_internal/common_fsdp.py?

@fegin
Copy link
Contributor Author

fegin commented Nov 3, 2023

@pytorchbot merge -f "The failing test is not related."

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@facebook-github-bot facebook-github-bot deleted the gh/fegin/173/head branch November 6, 2023 15:25
xuhancn pushed a commit to xuhancn/pytorch that referenced this pull request Nov 7, 2023
@fegin @xuhancn
[FSDP][state_dict] Add world_size 1 unittest (pytorch#112669)
118f4d3 
As title

Differential Revision: [D50754433](https://our.internmc.facebook.com/intern/diff/D50754433/)
Pull Request resolved: pytorch#112669
Approved by: https://github.com/wz337, https://github.com/fduwjj
Skylion007 pushed a commit to Skylion007/pytorch that referenced this pull request Nov 14, 2023
@fegin @Skylion007
[FSDP][state_dict] Add world_size 1 unittest (pytorch#112669)
d95fbbb 
As title

Differential Revision: [D50754433](https://our.internmc.facebook.com/intern/diff/D50754433/)
Pull Request resolved: pytorch#112669
Approved by: https://github.com/wz337, https://github.com/fduwjj
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fduwjj fduwjj fduwjj approved these changes

@wz337 wz337 wz337 approved these changes

@mrshenli mrshenli Awaiting requested review from mrshenli

@zhaojuanmao zhaojuanmao Awaiting requested review from zhaojuanmao

@rohan-varma rohan-varma Awaiting requested review from rohan-varma

@H-Huang H-Huang Awaiting requested review from H-Huang

@awgu awgu Awaiting requested review from awgu

@kwen2501 kwen2501 Awaiting requested review from kwen2501

@wanchaol wanchaol Awaiting requested review from wanchaol

Assignees

No one assigned

Labels

Merged topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@fegin @pytorchmergebot @fduwjj @wz337