Local dep object pool

[lixinqi]

解决cuda_h2d导致的内存开销过大问题。

[lixinqi]

LocalDepObject的复用不会造成任何问题，甚至还有好处。比如cuda_h2d device上只准备了2个LocalDepObject，这样整个计算流就在double buffer的模式下工作。
可以认为LocalDepObject就是流控机制。

[daquexian]

这里把 pool size 设置成 GetInstructionHighWaterMark()，相当于并没有顺序化？

[lixinqi]

当然不是。你可以认为之前这里是无穷大，那种情况下都有顺序化呀。顺序化是device的LocalDepObject成员控制的。

[daquexian]

哦哦，我想错了

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.8ms (= 7041.6ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.0ms (= 6400.7ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.10 (= 140.8ms / 128.0ms)

PyTorch resnet50 time: 85.8ms (= 4289.6ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.4ms (= 3718.9ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.15 (= 85.8ms / 74.4ms)

PyTorch resnet50 time: 57.8ms (= 2889.4ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.1ms (= 2353.2ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.23 (= 57.8ms / 47.1ms)

PyTorch resnet50 time: 48.1ms (= 2404.0ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 41.6ms (= 2082.1ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.15 (= 48.1ms / 41.6ms)

PyTorch resnet50 time: 41.7ms (= 2084.5ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 39.1ms (= 1957.2ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.07 (= 41.7ms / 39.1ms)