Skip to content

Conversation

mori360
Copy link
Contributor

@mori360 mori360 commented Jun 2, 2025

Copy link

pytorch-bot bot commented Jun 2, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/154897

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

⏳ No Failures, 1 Pending

As of commit e4ba9f0 with merge base 4014297 (image):
💚 Looks good so far! There are no failures yet. 💚

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added ciflow/inductor oncall: distributed Add this issue/PR to distributed oncall triage queue release notes: distributed (fsdp) release notes category labels Jun 2, 2025
@mori360 mori360 added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 2, 2025
@mori360 mori360 requested a review from weifengpy June 2, 2025 23:29
@mori360 mori360 marked this pull request as ready for review June 2, 2025 23:29
mp_policy = MixedPrecisionPolicy(
torch.bfloat16, torch.bfloat16, torch.bfloat16, True
)
model = Model().cuda()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for .cuda(), because fully_shard moves model to cuda

@mori360 mori360 marked this pull request as draft June 3, 2025 17:11
super().__init__(*args, **kwargs)
self._layer = nn.Linear(10, 10)

def forward(self, input: torch.Tensor):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

input: torch.Tensor -> input: Input ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and address linter

@mori360
Copy link
Contributor Author

mori360 commented Jun 4, 2025

@pytorchmergebot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

iupaikov-amd pushed a commit to ROCm/pytorch that referenced this pull request Jun 4, 2025
angelayi pushed a commit to angelayi/pytorch that referenced this pull request Jun 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged oncall: distributed Add this issue/PR to distributed oncall triage queue release notes: distributed (fsdp) release notes category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mismatch of mixed precision cast_fn in FSDP and FSDP2

3 participants