Support PyTorch Lightning's FSDP optimizer states saving and loading #7123

Job	Run time
pl-cpu (macOS-11, lightning, 3.8, 1.11)	16m 56s
pl-cpu (macOS-11, lightning, 3.9, 1.12)	26m 39s
pl-cpu (macOS-11, lightning, 3.10, 1.13)	25m 43s
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	11m 56s
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	10m 2s
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	10m 18s
pl-cpu (windows-2022, lightning, 3.8, 1.11)	17m 34s
pl-cpu (windows-2022, lightning, 3.9, 1.12)	16m 55s
pl-cpu (windows-2022, lightning, 3.10, 1.13)	15m 54s
pl-cpu (macOS-11, lightning, 3.10, 2.0)	19m 16s
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	13m 56s
pl-cpu (windows-2022, lightning, 3.10, 2.0)	17m 40s
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	22m 29s
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	15m 12s
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	16m 11s
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	24m 3s
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	11m 47s
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	16m 58s
	5h 9m 29s

Provide feedback