[primitive] add binary tests #8109

guo-ran · 2022-04-27T10:36:18Z

binary
- elemwise: test_type 0
- broadcast: test_type 1
- left_scalar: test_type 2
- right_scalar: test_type 3

…Inc/oneflow into dev_add_primitive_tests

guo-ran · 2022-05-01T05:56:48Z

oneflow/core/ep/cpu/primitive/broadcast_elementwise_binary.cpp

-#define CPU_PRIMITIVE_BINARY_ONEDNN_TYPE_SEQ                                   \
-  OF_PP_MAKE_TUPLE_SEQ(dnnl::memory::data_type::s8, DataType::kInt8, int8_t)   \
-  OF_PP_MAKE_TUPLE_SEQ(dnnl::memory::data_type::u8, DataType::kBool, bool)     \
-  OF_PP_MAKE_TUPLE_SEQ(dnnl::memory::data_type::u8, DataType::kUInt8, uint8_t) \


onednn实现的int8和uint8类型过不了primitive单测 @luqiang-guo

计算策略不同，onednn在溢出的时候是截断处理，torch 直接溢出

…o/oneflow-repo into dev_add_primitive_tests

guo-ran · 2022-05-01T07:00:03Z

oneflow/core/ep/cpu/primitive/broadcast_elementwise_binary.cpp

-  OF_PP_MAKE_TUPLE_SEQ(dnnl::memory::data_type::s8, DataType::kInt8, int8_t)   \
-  OF_PP_MAKE_TUPLE_SEQ(dnnl::memory::data_type::u8, DataType::kBool, bool)     \
-  OF_PP_MAKE_TUPLE_SEQ(dnnl::memory::data_type::u8, DataType::kUInt8, uint8_t) \
-  OF_PP_MAKE_TUPLE_SEQ(dnnl::memory::data_type::f32, DataType::kFloat, float)  \


float类型过不了left_scalar和right_scalar的单测，是不是实现时没考虑位置不同计算的区别 @luqiang-guo

guo-ran · 2022-05-01T07:00:52Z

ready for review

github-actions · 2022-05-01T09:25:18Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8109/

…o/oneflow-repo into dev_add_primitive_tests

…Inc/oneflow into dev_add_primitive_tests

…dev_add_primitive_tests

MARD1NO · 2022-05-05T03:19:26Z

@luqiang-guo 麻烦看下guoran在PR中提到的onednn相关数据类型过不了Primitive测试的问题

github-actions · 2022-05-06T11:11:22Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8109/

github-actions · 2022-05-06T11:18:24Z

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.2ms (= 12918.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 147.2ms (= 14723.0ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.14 (= 147.2ms / 129.2ms)

OneFlow resnet50 time: 77.3ms (= 7727.2ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.2ms (= 8420.4ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.09 (= 84.2ms / 77.3ms)

OneFlow resnet50 time: 58.0ms (= 11603.0ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 61.1ms (= 12229.4ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.05 (= 61.1ms / 58.0ms)

OneFlow resnet50 time: 40.6ms (= 8122.3ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 44.0ms (= 8793.9ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.08 (= 44.0ms / 40.6ms)

OneFlow resnet50 time: 35.4ms (= 7072.0ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 37.8ms (= 7565.9ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.07 (= 37.8ms / 35.4ms)

OneFlow swin dataloader time: 0.253s (= 50.691s / 200, num_workers=1)
PyTorch swin dataloader time: 0.153s (= 30.633s / 200, num_workers=1)
Relative speed: 0.604 (= 0.153s / 0.253s)

OneFlow swin dataloader time: 0.067s (= 13.385s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.318s / 200, num_workers=4)
Relative speed: 0.621 (= 0.042s / 0.067s)

OneFlow swin dataloader time: 0.059s (= 11.868s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.397s / 200, num_workers=8)
Relative speed: 0.371 (= 0.022s / 0.059s)

❌ OneFlow resnet50 time: 147.3ms (= 14731.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 169.4ms (= 16941.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.15 (= 169.4ms / 147.3ms)

OneFlow resnet50 time: 99.0ms (= 9900.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 109.7ms (= 10965.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.11 (= 109.7ms / 99.0ms)

OneFlow resnet50 time: 74.8ms (= 14962.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 87.7ms (= 17544.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.17 (= 87.7ms / 74.8ms)

OneFlow resnet50 time: 63.1ms (= 12615.1ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.5ms (= 14900.1ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.18 (= 74.5ms / 63.1ms)

OneFlow resnet50 time: 55.4ms (= 11079.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 69.4ms (= 13876.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 69.4ms / 55.4ms)

github-actions · 2022-05-06T12:06:41Z

CI failed when running job: cuda-module. PR label automerge has been removed

github-actions · 2022-05-07T17:04:58Z

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 131.4ms (= 13141.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 148.5ms (= 14851.2ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.13 (= 148.5ms / 131.4ms)

OneFlow resnet50 time: 82.3ms (= 8226.7ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 89.2ms (= 8916.6ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.08 (= 89.2ms / 82.3ms)

OneFlow resnet50 time: 56.2ms (= 11239.1ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 61.0ms (= 12201.8ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.09 (= 61.0ms / 56.2ms)

OneFlow resnet50 time: 44.9ms (= 8971.7ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 48.2ms (= 9635.8ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.07 (= 48.2ms / 44.9ms)

OneFlow resnet50 time: 38.9ms (= 7784.1ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 38.8ms (= 7763.0ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.00 (= 38.8ms / 38.9ms)

OneFlow swin dataloader time: 0.250s (= 50.032s / 200, num_workers=1)
PyTorch swin dataloader time: 0.151s (= 30.162s / 200, num_workers=1)
Relative speed: 0.603 (= 0.151s / 0.250s)

OneFlow swin dataloader time: 0.108s (= 21.510s / 200, num_workers=4)
PyTorch swin dataloader time: 0.041s (= 8.226s / 200, num_workers=4)
Relative speed: 0.382 (= 0.041s / 0.108s)

OneFlow swin dataloader time: 0.063s (= 12.535s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.402s / 200, num_workers=8)
Relative speed: 0.351 (= 0.022s / 0.063s)

❌ OneFlow resnet50 time: 146.6ms (= 14655.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 170.7ms (= 17070.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 170.7ms / 146.6ms)

OneFlow resnet50 time: 97.1ms (= 9705.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 112.3ms (= 11234.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 112.3ms / 97.1ms)

OneFlow resnet50 time: 79.4ms (= 15888.0ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 90.5ms (= 18091.2ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
❌ Relative speed: 1.14 (= 90.5ms / 79.4ms)

OneFlow resnet50 time: 65.1ms (= 13013.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 86.2ms (= 17241.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.32 (= 86.2ms / 65.1ms)

OneFlow resnet50 time: 55.7ms (= 11136.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.8ms (= 15557.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.40 (= 77.8ms / 55.7ms)

github-actions · 2022-05-07T17:49:37Z

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8109/

liujuncheng and others added 3 commits April 26, 2022 18:49

[Primitive] Unit test

1f2cd7d

Merge branch 'master' into dev_primitive_unit_test

a5fd294

add copy_nd test

9a62b31

guo-ran added enhancement primitive labels Apr 27, 2022

guo-ran requested a review from liujuncheng as a code owner April 27, 2022 10:36

guo-ran added 12 commits April 28, 2022 21:26

add test

38475ce

add softmax test

518684a

add test

71579b5

binary test

5d7cd02

Merge branch 'master' into dev_add_primitive_tests

f92d31a

add broadcast

3205ac7

fix softmax test

d5c82b9

add

3eff841

refine

8f1e03e

add test

194a208

add test

adc9904

Merge branch 'dev_add_primitive_tests' of https://github.com/Oneflow-…

426dc34

…Inc/oneflow into dev_add_primitive_tests

guo-ran commented May 1, 2022

View reviewed changes

guo-ran added 2 commits May 1, 2022 14:55

add test

eab44fe

Merge branch 'dev_add_primitive_tests' of work24:/home/guoran/git_rep…

b94388a

…o/oneflow-repo into dev_add_primitive_tests

guo-ran changed the title ~~[WIP] add primitive unit tests~~ add copy_nd binary softmax primitive unit tests May 1, 2022

guo-ran commented May 1, 2022

View reviewed changes

Merge branch 'master' into dev_add_primitive_tests

e719b91

guo-ran requested a review from oneflow-ci-bot May 1, 2022 07:00

guo-ran added the op label May 1, 2022

guo-ran added 2 commits May 1, 2022 23:36

fix of_tidy

f7723ea

Merge branch 'dev_add_primitive_tests' of work24:/home/guoran/git_rep…

919447b

…o/oneflow-repo into dev_add_primitive_tests

guo-ran added 2 commits May 1, 2022 23:37

Merge branch 'dev_add_primitive_tests' of https://github.com/Oneflow-…

5e13dc6

…Inc/oneflow into dev_add_primitive_tests

Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …

1b07bff

…dev_add_primitive_tests

guo-ran changed the title ~~add copy_nd binary softmax primitive unit tests~~ [primitive] add binary tests May 5, 2022

revert copy_nd softmax

5f05ff1

MARD1NO approved these changes May 5, 2022

View reviewed changes

fix float (#8146)

78bd8a8

liujuncheng approved these changes May 6, 2022

View reviewed changes

guo-ran added the automerge label May 6, 2022

guo-ran and others added 2 commits May 6, 2022 15:19

Merge branch 'master' into dev_add_primitive_tests

4aed021

Merge branch 'master' into dev_add_primitive_tests

cb3ee1f

github-actions bot removed the automerge label May 6, 2022

guo-ran added the automerge label May 6, 2022

mergify bot and others added 6 commits May 6, 2022 14:38

Merge branch 'master' into dev_add_primitive_tests

8f37c28

Merge branch 'master' into dev_add_primitive_tests

8d6ac76

Merge branch 'master' into dev_add_primitive_tests

6fcdada

Merge branch 'master' into dev_add_primitive_tests

6bb3d34

Merge branch 'master' into dev_add_primitive_tests

dd18461

Merge branch 'master' into dev_add_primitive_tests

b416375

mergify bot merged commit 58d0052 into master May 7, 2022

mergify bot deleted the dev_add_primitive_tests branch May 7, 2022 18:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[primitive] add binary tests #8109

[primitive] add binary tests #8109

guo-ran commented Apr 27, 2022 •

edited

Loading

guo-ran May 1, 2022 •

edited

Loading

luqiang-guo May 6, 2022

guo-ran May 1, 2022

luqiang-guo May 6, 2022

guo-ran commented May 1, 2022

github-actions bot commented May 1, 2022

MARD1NO commented May 5, 2022

github-actions bot commented May 6, 2022

github-actions bot commented May 6, 2022

github-actions bot commented May 6, 2022

github-actions bot commented May 7, 2022

github-actions bot commented May 7, 2022

[primitive] add binary tests #8109

[primitive] add binary tests #8109

Conversation

guo-ran commented Apr 27, 2022 • edited Loading

guo-ran May 1, 2022 • edited Loading

Choose a reason for hiding this comment

luqiang-guo May 6, 2022

Choose a reason for hiding this comment

guo-ran May 1, 2022

Choose a reason for hiding this comment

luqiang-guo May 6, 2022

Choose a reason for hiding this comment

guo-ran commented May 1, 2022

github-actions bot commented May 1, 2022

MARD1NO commented May 5, 2022

github-actions bot commented May 6, 2022

github-actions bot commented May 6, 2022

github-actions bot commented May 6, 2022

github-actions bot commented May 7, 2022

github-actions bot commented May 7, 2022

guo-ran commented Apr 27, 2022 •

edited

Loading

guo-ran May 1, 2022 •

edited

Loading