Add einsum #7526

Ldpe2G · 2022-02-17T13:40:13Z

DIM_SEQ 拓展到7维，因为 einsum 在运算过程中会对输入张量进行维度扩充，一个5维的张量可能会拓展到7维，而其中可能会执行 reduce sum 的操作，7维的张量执行reduce_sum 会报错。

local 测试已覆盖的 case

TODO List:

速度测试
global 测试

文档

…dev_add_einsum

github-actions · 2022-02-17T17:32:16Z

Speed stats:

github-actions · 2022-02-18T02:30:02Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 129.2ms (= 12918.4ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 140.4ms (= 14038.3ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.09 (= 140.4ms / 129.2ms)

✔️ OneFlow resnet50 time: 76.7ms (= 7667.2ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.8ms (= 8481.6ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.11 (= 84.8ms / 76.7ms)

OneFlow resnet50 time: 51.6ms (= 10325.8ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.8ms (= 11950.7ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.16 (= 59.8ms / 51.6ms)

OneFlow resnet50 time: 43.3ms (= 8654.6ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 47.0ms (= 9397.2ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.09 (= 47.0ms / 43.3ms)

OneFlow resnet50 time: 38.5ms (= 7709.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.5ms (= 7898.5ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.02 (= 39.5ms / 38.5ms)

✔️ OneFlow resnet50 time: 141.6ms (= 14155.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 161.6ms (= 16157.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.14 (= 161.6ms / 141.6ms)

OneFlow resnet50 time: 88.5ms (= 8850.5ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 102.3ms (= 10233.9ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 102.3ms / 88.5ms)

OneFlow resnet50 time: 60.8ms (= 12154.7ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 75.5ms (= 15092.5ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.24 (= 75.5ms / 60.8ms)

OneFlow resnet50 time: 50.9ms (= 10187.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.9ms (= 13173.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.29 (= 65.9ms / 50.9ms)

OneFlow resnet50 time: 48.1ms (= 9613.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 60.1ms (= 12021.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.25 (= 60.1ms / 48.1ms)

github-actions · 2022-02-18T08:12:25Z

Speed stats:

github-actions · 2022-03-05T14:04:23Z

Speed stats:

GPU Name: GeForce GTX 1080 

✔️ OneFlow resnet50 time: 128.4ms (= 12841.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.1ms (= 14105.7ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.10 (= 141.1ms / 128.4ms)

✔️ OneFlow resnet50 time: 76.8ms (= 7682.9ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.2ms (= 8419.5ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.10 (= 84.2ms / 76.8ms)

OneFlow resnet50 time: 50.2ms (= 10030.8ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 61.1ms (= 12221.9ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.22 (= 61.1ms / 50.2ms)

OneFlow resnet50 time: 43.8ms (= 8756.7ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 50.5ms (= 10093.4ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.15 (= 50.5ms / 43.8ms)

OneFlow resnet50 time: 38.2ms (= 7637.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 41.4ms (= 8273.5ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.08 (= 41.4ms / 38.2ms)

✔️ OneFlow resnet50 time: 137.7ms (= 13770.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.9ms (= 16093.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.17 (= 160.9ms / 137.7ms)

OneFlow resnet50 time: 87.2ms (= 8722.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 106.1ms (= 10612.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.22 (= 106.1ms / 87.2ms)

OneFlow resnet50 time: 58.7ms (= 11730.2ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 85.6ms (= 17126.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.46 (= 85.6ms / 58.7ms)

OneFlow resnet50 time: 53.7ms (= 10735.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 66.2ms (= 13248.9ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.23 (= 66.2ms / 53.7ms)

OneFlow resnet50 time: 46.7ms (= 9338.5ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 52.5ms (= 10500.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.12 (= 52.5ms / 46.7ms)

github-actions · 2022-03-05T14:10:04Z

CI failed when running job: cpu-misc. PR label automerge has been removed

* add einsum implementation * reformat * fix ci docs build error * add more test cases from DALLE-pytorch and alphafold repos * fix docs format * add detailed explaination for better understanding how einsum works * fix error * refine * add eager global tests * merge master * try to fix ci 2n4d error * move test_einsum.py to expensive folder * add more test cases * add testcases from openflod repo Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Ldpe2G added 3 commits February 17, 2022 21:23

add einsum implementation

043e47e

Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …

a631682

…dev_add_einsum

reformat

6c2c661

Ldpe2G added feature op system api python labels Feb 17, 2022

Ldpe2G requested review from doombeaker and jackalcooper February 17, 2022 13:40

Ldpe2G requested review from lixinqi, hjchen2 and daquexian as code owners February 17, 2022 13:40

Ldpe2G requested review from oneflow-ci-bot and removed request for lixinqi, daquexian and hjchen2 February 17, 2022 13:40

oneflow-ci-bot removed their request for review February 17, 2022 17:33

Ldpe2G requested a review from oneflow-ci-bot February 18, 2022 01:20

oneflow-ci-bot removed their request for review February 18, 2022 02:33

fix ci docs build error

a3ca9a8

Ldpe2G requested a review from oneflow-ci-bot February 18, 2022 03:24

oneflow-ci-bot removed their request for review February 18, 2022 04:44

Ldpe2G requested a review from oneflow-ci-bot February 18, 2022 07:43

oneflow-ci-bot removed their request for review February 18, 2022 08:13

Ldpe2G added 2 commits February 18, 2022 16:42

add more test cases from DALLE-pytorch and alphafold repos

b85d308

fix docs format

73c4e8a

Ldpe2G removed request for doombeaker and jackalcooper March 4, 2022 23:49

oneflow-ci-bot removed their request for review March 4, 2022 23:56

Merge branch 'master' into dev_add_einsum

a26e828

oneflow-ci-bot self-requested a review March 5, 2022 10:10

Merge branch 'master' into dev_add_einsum

bbe3ea7

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 5, 2022 10:49

Merge branch 'master' into dev_add_einsum

83aefcb

oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot March 5, 2022 11:37

Merge branch 'master' into dev_add_einsum

0bf0b16

oneflow-ci-bot self-requested a review March 5, 2022 12:13

Merge branch 'master' into dev_add_einsum

06091b7

oneflow-ci-bot removed their request for review March 5, 2022 12:19

mergify bot requested a review from oneflow-ci-bot March 5, 2022 12:19

github-actions bot removed the automerge label Mar 5, 2022

Merge branch 'master' into dev_add_einsum

a73d30c

Ldpe2G added the automerge label Mar 6, 2022

Merge branch 'master' into dev_add_einsum

3a44f35

oneflow-ci-bot removed their request for review March 6, 2022 01:54

Ldpe2G requested a review from oneflow-ci-bot March 6, 2022 01:58

oneflow-ci-bot merged commit bee3a32 into master Mar 6, 2022

oneflow-ci-bot deleted the dev_add_einsum branch March 6, 2022 04:13

Ldpe2G mentioned this pull request Mar 7, 2022

einsum functor #5985

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add einsum #7526

Add einsum #7526

Ldpe2G commented Feb 17, 2022 •

edited

Loading

github-actions bot commented Feb 17, 2022

github-actions bot commented Feb 18, 2022

github-actions bot commented Feb 18, 2022

github-actions bot commented Mar 5, 2022

github-actions bot commented Mar 5, 2022

Add einsum #7526

Add einsum #7526

Conversation

Ldpe2G commented Feb 17, 2022 • edited Loading

local 测试已覆盖的 case

TODO List:

文档

github-actions bot commented Feb 17, 2022

github-actions bot commented Feb 18, 2022

github-actions bot commented Feb 18, 2022

github-actions bot commented Mar 5, 2022

github-actions bot commented Mar 5, 2022

Ldpe2G commented Feb 17, 2022 •

edited

Loading