Try to fix the problem that the insightface cannot converge。

[BBuf]

Try to fix the problem that the insightface cannot converge 。

[yuanms2]

这里为什么需要memset呢？ 感觉每一个位置都会被赋值的啊

[hjchen2]

dx可以不用memset，alpha_diff是需要的，alpha_diff的kernel实现是累加

[BBuf]

是的，是这里的问题，忘记清0了，梯度累加炸了。我现在本地验证了resnet50，可以正常收敛。

[yuanms2]

按后江的回复，可以删掉dx的memset

[BBuf]

已修改。

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 137.8ms (= 6887.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 126.1ms (= 6305.0ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.09 (= 137.8ms / 126.1ms)

PyTorch resnet50 time: 82.6ms (= 4128.2ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 73.0ms (= 3649.9ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.13 (= 82.6ms / 73.0ms)

PyTorch resnet50 time: 61.0ms (= 3049.9ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.3ms (= 2367.1ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.29 (= 61.0ms / 47.3ms)

PyTorch resnet50 time: 48.3ms (= 2416.9ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 39.2ms (= 1961.6ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.23 (= 48.3ms / 39.2ms)

PyTorch resnet50 time: 38.4ms (= 1919.3ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 38.7ms (= 1934.5ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 0.99 (= 38.4ms / 38.7ms)