consistent_tensor_to_api

[clackhan]

ConsistentTensor 支持方法 .to("cpu")：
0号进程演示

>>> import oneflow as flow
>>> import numpy as np
>>> import os
>>> if int(os.environ.get("RANK")) == 0:
...     ndarr = np.asarray([[1, -3, 2], [-3, 7, -4]])
... else:
...     ndarr = np.asarray([[6, -5, 9], [-4, 8, -3]])
...
>>> a = flow.Tensor(ndarr)
>>> b = a.to("cuda")
>>> p = flow.placement("cuda", {0:range(2)})
>>> c = b.to_consistent(p, flow.sbp.split(0))
>>> print(c.to_local())
tensor([[ 1., -3.,  2.],
        [-3.,  7., -4.]], device='cuda:0', dtype=oneflow.float32)
>>> d = c.to("cpu")
>>> print(d.to_local())
tensor([[ 1., -3.,  2.],
        [-3.,  7., -4.]], dtype=oneflow.float32)
>>>

1号进程演示：

>>> import oneflow as flow
>>> import numpy as np
>>> import os
>>> if int(os.environ.get("RANK")) == 0:
...     ndarr = np.asarray([[1, -3, 2], [-3, 7, -4]])
... else:
...     ndarr = np.asarray([[6, -5, 9], [-4, 8, -3]])
...
>>> a = flow.Tensor(ndarr)
>>> b = a.to("cuda")
>>> p = flow.placement("cuda", {0:range(2)})
>>> c = b.to_consistent(p, flow.sbp.split(0))
>>> print(c.to_local())
tensor([[ 6., -5.,  9.],
        [-4.,  8., -3.]], device='cuda:1', dtype=oneflow.float32)
>>> d = c.to("cpu")
>>> print(d.to_local())
tensor([[ 6., -5.,  9.],
        [-4.,  8., -3.]], dtype=oneflow.float32)
>>>

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 146.1ms (= 7305.8ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 126.4ms (= 6319.4ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.16 (= 146.1ms / 126.4ms)

PyTorch resnet50 time: 83.7ms (= 4187.5ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.1ms (= 3703.8ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.13 (= 83.7ms / 74.1ms)

PyTorch resnet50 time: 56.1ms (= 2802.9ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.7ms (= 2383.3ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.18 (= 56.1ms / 47.7ms)

PyTorch resnet50 time: 48.1ms (= 2403.9ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 42.7ms (= 2136.5ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.13 (= 48.1ms / 42.7ms)

PyTorch resnet50 time: 45.0ms (= 2250.0ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 42.2ms (= 2109.6ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.07 (= 45.0ms / 42.2ms)

[daquexian]

input_local_tesnor 和 output_local_tesnor 有 typo

[clackhan]

已修改

[daquexian]

这里直接判断 device == input.placement.device_type 吧，后续如果加了新设备类型也不用改这里了

[clackhan]

已修改

[daquexian]

一号进程的输出是不是贴错了

[clackhan]

一号进程的输出是不是贴错了

是的，粘错了，已更新

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 138.9ms (= 6947.4ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.1ms (= 6402.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.09 (= 138.9ms / 128.1ms)

PyTorch resnet50 time: 83.8ms (= 4191.3ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.4ms (= 3721.9ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.13 (= 83.8ms / 74.4ms)

PyTorch resnet50 time: 58.2ms (= 2910.3ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.9ms (= 2394.5ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.22 (= 58.2ms / 47.9ms)

PyTorch resnet50 time: 48.9ms (= 2446.6ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 45.7ms (= 2284.0ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.07 (= 48.9ms / 45.7ms)

PyTorch resnet50 time: 43.9ms (= 2196.8ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 41.6ms (= 2080.0ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.06 (= 43.9ms / 41.6ms)