Import oneflow as torch #6076

BBuf · 2021-08-27T03:12:44Z

此PR将TorchVison的常见模型加入CI测试，确保可以import oneflow as torch和 import torch as oneflow可以跑同一份模型代码(兼容性)。目前已经支持如下模型的测试，评价指标是100个iter的loss相似度：

resnet50
alexnet

jackalcooper · 2021-08-27T03:15:00Z

有个png图片被提交了

daquexian · 2021-08-27T03:15:18Z

python/oneflow/test_utils/models/oneflow_alexnet.py

@@ -0,0 +1,66 @@
+import oneflow as flow
+import oneflow.nn as nn


https://github.com/Oneflow-Inc/models/blob/main/scripts/compare_speed_with_pytorch.py#L67-L87 这里实现了把 pytorch 模型文件里的 import torch 覆盖为 import oneflow as torch，这样不需要维护两套模型文件了

不仅仅是import，里面有一些op torchvsion直接使用的torch.xxx，需要改成oneflow.xxx，这里可以做到吗？

因为已经 import oneflow as torch 了，所以 torch.xxx 就是 oneflow.xxx

哦，对，好的，我修改一下

jackalcooper · 2021-08-27T03:15:49Z

python/oneflow/test_utils/models/pytorch_resnet.py

@@ -0,0 +1,450 @@
+"""


这个文件是放错目录了？

有个png图片被提交了

那个图片在我本地是没有的了，而且我在pr里点删除是灰的，很奇怪

没有放错，我开了一个models来存所有的模型文件

jackalcooper · 2021-08-27T03:54:33Z

python/oneflow/test_utils/oneflow_pytorch_compatiblity_test.py

@@ -0,0 +1,228 @@
+"""


文件太长，上个comment写错地方了。是这个文件好像放错地方了？test util 里面不应该放 unittest.main() 的东西吧？

那这个文件应该放在哪里呢？modules吗

那这个文件应该放在哪里呢？modules吗

嗯，不然不会被测试的

我的理解 test util 下提供一下用例会用到的公共函数，自身不应该是直接运行的测试

和auotest类似调整了目录结构，将oneflow_pytorch_compatiblity_test需要的公共函数当成一个包导入，在modules下进行测试。

…Inc/oneflow into import_oneflow_as_torch

jackalcooper · 2021-08-27T09:14:57Z

python/oneflow/test/modules/oneflow_pytorch_compatiblity.py

@@ -0,0 +1,30 @@
+"""


note 去掉oneflow export之后这种hack的文件可以不用写了，直接import oneflow.test_utils

已删除。

jackalcooper · 2021-08-27T09:21:11Z

python/oneflow/test_utils/oneflow_pytorch_compatiblity/oneflow_pytorch_compatiblity_test.py

+                new_parameters[k] = flow.tensor(w[k].detach().numpy())
+
+        try:
+            shutil.rmtree("/dataset/imagenet/compatiblity_models")


note
不应该直接写 /dataset 目录，应该用 python tempfile 创建临时目录

好的，已解决。

…Inc/oneflow into import_oneflow_as_torch

…low into import_oneflow_as_torch

jackalcooper · 2021-08-27T09:43:53Z

python/oneflow/test_utils/oneflow_pytorch_compatiblity/oneflow_pytorch_compatiblity_test.py

+            with open("/tmp/tmp_model.py", "w") as new_f:
+                new_f.write(buf)
+
+        python_module = import_file("/tmp/tmp_model.py")


note
import_file 这个函数应该改成直接接受源码，在里面处理零时文件创建和回收的逻辑

jackalcooper · 2021-08-27T10:37:22Z

python/oneflow/test_utils/oneflow_pytorch_compatiblity/oneflow_pytorch_compatiblity_test.py

+
+
+def import_file(path):
+    spec = importlib.util.spec_from_file_location("mod", path)


note
记得加一下flush

github-actions · 2021-08-27T17:53:11Z

CI failed, removing label automerge

…low into import_oneflow_as_torch

github-actions · 2021-08-28T04:29:08Z

CI failed, removing label automerge

github-actions · 2021-08-29T06:28:05Z

Speed stats:

GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 128.2ms (= 6407.9ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 141.2ms (= 7059.3ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 1.10 (= 141.2ms / 128.2ms)

OneFlow resnet50 time: 74.4ms (= 3722.2ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 85.1ms (= 4255.5ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 1.14 (= 85.1ms / 74.4ms)

OneFlow resnet50 time: 47.6ms (= 2381.2ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 60.0ms (= 3001.6ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 1.26 (= 60.0ms / 47.6ms)

OneFlow resnet50 time: 41.9ms (= 2096.5ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 49.8ms (= 2491.5ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 1.19 (= 49.8ms / 41.9ms)

OneFlow resnet50 time: 37.4ms (= 1871.2ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 42.6ms (= 2132.3ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 1.14 (= 42.6ms / 37.4ms)

OneFlow resnet50 time: 141.7ms (= 7085.8ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 146.8ms (= 7340.5ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 1.04 (= 146.8ms / 141.7ms)

OneFlow resnet50 time: 90.2ms (= 4507.7ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 90.0ms (= 4502.0ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 1.00 (= 90.0ms / 90.2ms)

OneFlow resnet50 time: 68.8ms (= 3439.2ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 65.8ms (= 3291.3ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 0.96 (= 65.8ms / 68.8ms)

OneFlow resnet50 time: 64.0ms (= 3199.8ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 55.1ms (= 2753.0ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 0.86 (= 55.1ms / 64.0ms)

OneFlow resnet50 time: 58.9ms (= 2947.1ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow GPU used (rank 0): 0 MiB
PyTorch resnet50 time: 48.0ms (= 2400.0ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
PyTorch GPU used (rank 0, estimated): 0 MiB
Relative speed: 0.81 (= 48.0ms / 58.9ms)

BBuf added 4 commits August 26, 2021 21:19

add oneflow_pytorch_compatiblity_test

bafd0f6

add compatiblity test

6b205e1

align init model

c5d3c8a

add alexnet

a8e81d2

BBuf requested a review from jackalcooper August 27, 2021 03:12

daquexian reviewed Aug 27, 2021

View reviewed changes

jackalcooper reviewed Aug 27, 2021

View reviewed changes

fix comments

2c62837

jackalcooper reviewed Aug 27, 2021

View reviewed changes

BBuf and others added 4 commits August 27, 2021 15:45

restruct code structure

56b92d5

add resnet50 and restruct structure

9b978a3

Delete loss_compare.png

fdd53a1

Merge branch 'import_oneflow_as_torch' of https://github.com/Oneflow-…

b235352

…Inc/oneflow into import_oneflow_as_torch

jackalcooper reviewed Aug 27, 2021

View reviewed changes

fix comment

528a8e5

jackalcooper reviewed Aug 27, 2021

View reviewed changes

jackalcooper and others added 5 commits August 27, 2021 17:24

make dataset and modelzoo read only

7c62def

Merge branch 'import_oneflow_as_torch' of https://github.com/Oneflow-…

261e82f

…Inc/oneflow into import_oneflow_as_torch

fix comments

f3c4b1b

Merge branch 'import_oneflow_as_torch' of github.com:Oneflow-Inc/onef…

53eb985

…low into import_oneflow_as_torch

refine

d7b4e9c

jackalcooper reviewed Aug 27, 2021

View reviewed changes

BBuf added 2 commits August 27, 2021 18:38

fix bug

09877fc

fix comments

0f7b5a7

jackalcooper approved these changes Aug 27, 2021

View reviewed changes

BBuf requested a review from oneflow-ci-bot August 27, 2021 11:00

BBuf added api automerge labels Aug 27, 2021

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 27, 2021 14:36

Merge branch 'master' into import_oneflow_as_torch

434413b

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 27, 2021 16:30

github-actions bot removed the automerge label Aug 27, 2021

oneflow-ci-bot removed their request for review August 27, 2021 17:54

BBuf added 2 commits August 28, 2021 10:20

fix ci error

5b2b9a9

Merge branch 'import_oneflow_as_torch' of github.com:Oneflow-Inc/onef…

d06abb3

…low into import_oneflow_as_torch

BBuf requested a review from oneflow-ci-bot August 28, 2021 02:23

BBuf added the automerge label Aug 28, 2021

BBuf and others added 2 commits August 28, 2021 10:23

Merge branch 'master' into import_oneflow_as_torch

00f956e

Merge branch 'master' into import_oneflow_as_torch

9594f88

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 28, 2021 03:26

github-actions bot removed the automerge label Aug 28, 2021

oneflow-ci-bot removed their request for review August 28, 2021 04:31

BBuf requested a review from oneflow-ci-bot August 29, 2021 02:05

BBuf added the automerge label Aug 29, 2021

Merge branch 'master' into import_oneflow_as_torch

0c6e340

oneflow-ci-bot removed their request for review August 29, 2021 03:52

Merge branch 'master' into import_oneflow_as_torch

be60e54

oneflow-ci-bot self-requested a review August 29, 2021 03:52

oneflow-ci-bot removed their request for review August 29, 2021 06:28

oneflow-ci-bot merged commit 01c55a0 into master Aug 29, 2021

oneflow-ci-bot deleted the import_oneflow_as_torch branch August 29, 2021 06:29

BBuf mentioned this pull request Mar 21, 2022

[feature] OneFlow和PyTorch兼容性测试 Oneflow-Inc/vision#185

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import oneflow as torch #6076

Import oneflow as torch #6076

BBuf commented Aug 27, 2021 •

edited

jackalcooper commented Aug 27, 2021

daquexian Aug 27, 2021

BBuf Aug 27, 2021

daquexian Aug 27, 2021

BBuf Aug 27, 2021

jackalcooper Aug 27, 2021

BBuf Aug 27, 2021

BBuf Aug 27, 2021

jackalcooper Aug 27, 2021

BBuf Aug 27, 2021

jackalcooper Aug 27, 2021

jackalcooper Aug 27, 2021

BBuf Aug 27, 2021

BBuf Aug 27, 2021

jackalcooper Aug 27, 2021

BBuf Aug 27, 2021

jackalcooper Aug 27, 2021

BBuf Aug 27, 2021

jackalcooper Aug 27, 2021

BBuf Aug 27, 2021

jackalcooper Aug 27, 2021

BBuf Aug 27, 2021

github-actions bot commented Aug 27, 2021

github-actions bot commented Aug 28, 2021

github-actions bot commented Aug 29, 2021

		@@ -0,0 +1,66 @@
		import oneflow as flow
		import oneflow.nn as nn



		def import_file(path):
		spec = importlib.util.spec_from_file_location("mod", path)

Import oneflow as torch #6076

Import oneflow as torch #6076

Conversation

BBuf commented Aug 27, 2021 • edited

jackalcooper commented Aug 27, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 27, 2021

github-actions bot commented Aug 28, 2021

github-actions bot commented Aug 29, 2021

BBuf commented Aug 27, 2021 •

edited