Faster decorator

[lixinqi]

No description provided.

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.5ms (= 7025.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.2ms (= 6412.4ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.10 (= 140.5ms / 128.2ms)

PyTorch resnet50 time: 85.5ms (= 4273.6ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.7ms (= 3733.5ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 85.5ms / 74.7ms)

PyTorch resnet50 time: 58.6ms (= 2930.7ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.3ms (= 2366.7ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.24 (= 58.6ms / 47.3ms)

PyTorch resnet50 time: 48.9ms (= 2444.8ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 37.8ms (= 1888.0ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.29 (= 48.9ms / 37.8ms)

PyTorch resnet50 time: 43.5ms (= 2176.1ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 37.2ms (= 1861.1ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.17 (= 43.5ms / 37.2ms)

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.5ms (= 7026.0ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.4ms (= 6419.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.09 (= 140.5ms / 128.4ms)

PyTorch resnet50 time: 84.7ms (= 4234.4ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.4ms (= 3719.3ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 84.7ms / 74.4ms)

PyTorch resnet50 time: 58.6ms (= 2930.7ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.4ms (= 2370.3ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.24 (= 58.6ms / 47.4ms)

PyTorch resnet50 time: 48.6ms (= 2431.0ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 37.5ms (= 1876.4ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.30 (= 48.6ms / 37.5ms)

PyTorch resnet50 time: 45.9ms (= 2296.7ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 37.9ms (= 1893.7ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.21 (= 45.9ms / 37.9ms)