Recurrent 3d interface uses a lot of memory

[mkschleg]

After making the recurrent performance measure in #1871 , I noticed that the memory usage of the 3d interface from #1686 uses significantly more memory than the normal interface. This likely has to do with creating views to walk across the temporal dimension. This wasn't an issue for smaller networks (which is what we mostly used to test), but starts becoming problematic as things get larger/the sequence is longer. Below are the benchmark results

Normal Interface:

CPU n=1000, t=64
  forward
  1.682 s (384 allocations: 728.63 MiB)
  backward
  2.197 s (5928 allocations: 2.13 GiB)
  forw and back
  4.545 s (13591 allocations: 3.32 GiB)

3d Interface:

CPU n=1000, t=64
  forward
  1.765 s (389 allocations: 976.58 MiB)
  backward
  14.547 s (7973 allocations: 32.41 GiB)
  forw and back
  17.122 s (16308 allocations: 33.61 GiB)

[ToucheSir]

Thanks for testing this, I think it's a good time to think about eachslice again as I noted in #1678 (comment) :)

[mkschleg]

That definitely fixes this. With eachslice

CPU n=1000, t=64
  forward
  819.105 ms (397 allocations: 976.58 MiB)
  backward
  1.586 s (4208 allocations: 2.37 GiB)
  forw and back
  2.658 s (6147 allocations: 3.57 GiB)

[ToucheSir]

Those are some pretty good improvements over the normal interface! I'm curious why the amount of memory allocated is slightly higher given that the count is lower (especially for forward + backward, what in the world is going on there!) but figuring that out now is not important.

[mkschleg]

Yeah. Learning about the issues using views in zygote has really opened my eyes to some things I wasn't paying attention to before. Thanks for pointing to eachslice again :)!