deprecation of genericLSTM

[apbard]

I have seen that in the last version genericLSTM has been "softly" deprecated. Could you please explain me the reason why this choice has been made?

[tmbdev]

I've run a lot of benchmarks that have shown that (1) peephole optimizations don't help (and usually hurt), and (2) any other choice of non-linearity other than the default one performs considerably worse.

I'll publish that as an Arxiv tech report soon.

[BestSonny]

Is the arxiv link of your tech report available now?

[tmbdev]

I got publication clearance, but it's not quite out yet. I'll try to push it in the next few days.

[BestSonny]

Thank you! Have you ever try to push a GPU version of LSTM since the community seems to have more interests in the RNN field?

[tmbdev]

I'm planning on GPU support via Eigen; however, that will still be a few months before the necessary dependencies are out.

[amitdo]

@BestSonny, here it is:

Benchmarking of LSTM Networks
Thomas M. Breuel

http://arxiv.org/abs/1508.02774

[BestSonny]

Thank you！

[futurely]

@tmbdev , mshadow may be a strong alternative to Eigen. Both cxxnet and mxnet are built upon it.

[tmbdev]

Thanks. Looks like Eigen and mshadow have similar goals. Is there any performance data comparing the two?

[futurely]

@tmbdev , mshadow has very detailed tutorials and other documentations including multi-GPU support while Eigen documents almost nothing related to GPU on its official website.

More importantly, mshadow exploits the powerful NVIDIA cuDNN library. It's impossible for Eigen to catch up mshadow in performance on GPUs.