Multi-GPU Meta Issue

[wsmoses]

• Allocating across multiple gpus [and ideally auto sharding]
• Ensuring multiple GPUs are used for computation

x/ref #380

[mofeing]

one question: multi-GPU should be already managed by PjRt right? or do we need IFRT?

[wsmoses]

This I do not know