[DSD] Add unittest to verify HSDP1 + broadcast_from_rank0 (#128755)

[fegin]

HSDP1 + broadcast_from_rank0 actually behaves differently from FSDP1 + broadcast_from_rank0. So we need an unittest to cover this use case.

This test relies on the fix from #128446.

Differential Revision: D58621436

Pull Request resolved: #128755
Approved by: https://github.com/Skylion007, https://github.com/wz337
ghstack dependencies: #128685

(cherry picked from commit fe8558b)

Fixes #ISSUE_NUMBER

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @wconstab @yf225 @chauhang @d4l3k @LucasLLC @MeetVadakkanchery @mhorowitz

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/129255

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures

As of commit 77266b6 with merge base b66e3f0 ():

NEW FAILURES - The following jobs have failed:

• pull / linux-focal-py3_8-clang9-xla / test (xla, 1, 1, linux.12xlarge) (gh)
  test_repeat_truncated
• pull / linux-focal-py3.12-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  dynamo/test_misc.py::MiscTests::test_outside_linear_module_free
• pull / linux-focal-py3.12-clang10 / test (default, 2, 3, linux.2xlarge) (gh)
  dynamo/test_dynamic_shapes.py::DynamicShapesMiscTests::test_outside_linear_module_free_dynamic_shapes
• pull / linux-focal-py3.8-clang10-onnx / test (default, 2, 2, linux.2xlarge) (gh)
  onnx/test_fx_op_consistency.py::TestOnnxModelOutputConsistency_opset_version_18_model_type_TorchModelType.TORCH_EXPORT_EXPORTEDPROGRAMCPU::test_output_match_nn_functional_huber_loss_cpu_float16
• trunk / macos-py3-arm64-mps / test (mps, 1, 1, macos-m1-13) (gh)
  'test/test_mps.py::TestMPS::test_cumprod_all_dtypes'

This comment was automatically generated by Dr. CI and updates every 15 minutes.