Document the difference between `device=` vs `.to(device)`

[tengyifei]

📚 Documentation

There's a subtle difference between torch.foo(device=xla) vs torch.foo().to(xla) and we should document this in a FAQ section or similar. The first one runs the foo on the TPU. The second one runs the foo on the CPU and then moves the buffer to the TPU.

[bfolie]

@tengyifei do you have more context? This doesn't strike me as something that is specific to PT/XLA or TPU, and the PyTorch documentation discusses creating and moving tensors on devices. Or is this not the behavior one finds on GPU?

We also don't have cohesive introductory documentation where this tidbit would fit in nicely.

[yaoshiang]

I agree with Brendan here. I don't think this is actually subtle, and, it's not unique to PT/XLA.

I do both all the time when I want repeatability torch.random(..., device="cpu").to("xla") vs performance torch.random(..., device="xla")

pytorch is all about constantly knowing where data and op exist.