Skip to content

[DDP][FSDP2] add unit test to showcase DDP mixed precision with FSDP2 mixed precision #156130

@weifengpy

Description

@weifengpy

🚀 The feature, motivation and pitch

DDP has mixed precision: #92882, code example can be found in torch/testing/_internal/distributed/distributed_test.py _test_ddp_native_mixed_precision

FSDP2 has ignored_param, fully_shard(ignore_params=). We need DDP mixed precision for ignore_params

action: improve test_fully_shard_ignore_params.py to cover the above usecase

Alternatives

No response

Additional context

No response

cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k

Metadata

Metadata

Assignees

No one assigned

    Labels

    oncall: distributedAdd this issue/PR to distributed oncall triage queue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions