Handle Duplicated tensors in sync list and interop tensors

[JackCaoG]

This pr is to address the corner case brought up by Alex in here, note that add lowering is removed. I used a slightly different example here

import torch
import torch_xla

torch.manual_seed(42)

device = 'xla'
x = torch.tensor([1,2,3], device=device)
y = x + x
z = torch.masked_select(y, y.eq(0))

There are two problems in this code

1. in y = x + x, we will try to sync the value of x twice (since default implemantion of add is used in this case), this is unnecessary. We can use a unordered_set in sync and a unordered_map in GatherTensorsXlaData to avoid execute the same ir_graph twice.

2. during torch.masked_select(y, y.eq(0)), y has ir_value and tensor_data but does not have xla_data.
   Here is the debugging output when z = torch.masked_select(y, y.eq(0)) is executed

Length of the tensors to be sync = 2

tensorID: 3
ir value: s64[3]{0} xla::device_data, location=<module>@example.py:20, device=CPU:0
with xla data: False
tensor data: 
 2
 4
 6
[ CPULongType{3} ]

tensorID: 4
ir value, pred[3]{0} aten::eq, location=<module>@example.py:20
with xla data: False
with tensor data: False

Currently in CollectSyncTensors we only check ir_value

xla/torch_xla/csrc/tensor.cpp

Line 1119 in 44268fa

if (ir_value) {

but in fact we should only sync the tensor if it

1. does not have xla_data
2. does not have tensor_data
3. has ir_value

[davidel]

Thanks @JackCaoG !
I'd add a test ... given you already have one 😉

[davidel]

Thanks @JackCaoG !