BiasAddkernel use primitive

[liujuncheng]

No description provided.

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8512/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.1ms (= 12906.8ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 142.7ms (= 14271.6ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.11 (= 142.7ms / 129.1ms)

OneFlow resnet50 time: 75.5ms (= 7554.4ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 84.8ms (= 8480.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.12 (= 84.8ms / 75.5ms)

OneFlow resnet50 time: 48.2ms (= 9630.3ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 54.5ms (= 10890.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.13 (= 54.5ms / 48.2ms)

OneFlow resnet50 time: 39.7ms (= 7943.9ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 45.2ms (= 9037.7ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.14 (= 45.2ms / 39.7ms)

OneFlow resnet50 time: 35.4ms (= 7077.5ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.1ms (= 7821.8ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.11 (= 39.1ms / 35.4ms)

OneFlow swin dataloader time: 0.254s (= 50.804s / 200, num_workers=1)
PyTorch swin dataloader time: 0.159s (= 31.739s / 200, num_workers=1)
Relative speed: 0.625 (= 0.159s / 0.254s)

OneFlow swin dataloader time: 0.075s (= 14.988s / 200, num_workers=4)
PyTorch swin dataloader time: 0.043s (= 8.538s / 200, num_workers=4)
Relative speed: 0.570 (= 0.043s / 0.075s)

OneFlow swin dataloader time: 0.040s (= 8.083s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.399s / 200, num_workers=8)
Relative speed: 0.544 (= 0.022s / 0.040s)

❌ OneFlow resnet50 time: 144.6ms (= 14463.8ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 168.3ms (= 16833.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 168.3ms / 144.6ms)

OneFlow resnet50 time: 93.4ms (= 9336.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 112.1ms (= 11212.7ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.20 (= 112.1ms / 93.4ms)

OneFlow resnet50 time: 68.6ms (= 13725.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 88.4ms (= 17674.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.29 (= 88.4ms / 68.6ms)

OneFlow resnet50 time: 57.0ms (= 11408.8ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 78.2ms (= 15647.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.37 (= 78.2ms / 57.0ms)

OneFlow resnet50 time: 50.2ms (= 10042.8ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.8ms (= 14965.7ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.49 (= 74.8ms / 50.2ms)

[github-actions]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/8512/

[github-actions]

Speed stats:

GPU Name: NVIDIA GeForce GTX 1080 

❌ OneFlow resnet50 time: 129.5ms (= 12946.3ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 143.8ms (= 14376.8ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.11 (= 143.8ms / 129.5ms)

OneFlow resnet50 time: 75.9ms (= 7593.1ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 87.2ms (= 8718.2ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.15 (= 87.2ms / 75.9ms)

OneFlow resnet50 time: 51.8ms (= 10368.8ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.7ms (= 11941.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.15 (= 59.7ms / 51.8ms)

OneFlow resnet50 time: 41.6ms (= 8327.3ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 47.1ms (= 9416.3ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.13 (= 47.1ms / 41.6ms)

OneFlow resnet50 time: 36.1ms (= 7210.6ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 38.7ms (= 7741.3ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.07 (= 38.7ms / 36.1ms)

OneFlow swin dataloader time: 0.417s (= 83.458s / 200, num_workers=1)
PyTorch swin dataloader time: 0.156s (= 31.178s / 200, num_workers=1)
Relative speed: 0.374 (= 0.156s / 0.417s)

OneFlow swin dataloader time: 0.078s (= 15.508s / 200, num_workers=4)
PyTorch swin dataloader time: 0.042s (= 8.436s / 200, num_workers=4)
Relative speed: 0.544 (= 0.042s / 0.078s)

OneFlow swin dataloader time: 0.063s (= 12.614s / 200, num_workers=8)
PyTorch swin dataloader time: 0.022s (= 4.496s / 200, num_workers=8)
Relative speed: 0.356 (= 0.022s / 0.063s)

❌ OneFlow resnet50 time: 144.3ms (= 14430.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 168.7ms (= 16871.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.17 (= 168.7ms / 144.3ms)

OneFlow resnet50 time: 92.3ms (= 9234.6ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 112.1ms (= 11213.3ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.21 (= 112.1ms / 92.3ms)

OneFlow resnet50 time: 69.7ms (= 13932.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 94.5ms (= 18894.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.36 (= 94.5ms / 69.7ms)

OneFlow resnet50 time: 57.5ms (= 11503.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 80.8ms (= 16168.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.41 (= 80.8ms / 57.5ms)

OneFlow resnet50 time: 51.0ms (= 10192.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.0ms (= 14799.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.45 (= 74.0ms / 51.0ms)