Add pre-requisits / hardware requirements for the LSTM sample

[dchichkov]

I've tried running LSTM example on my workstation with 24GB of GPU memory, 512 hidden units RNN, it somehow still runs out of memory. Please, could you add the pre-requisits / hardware requirements to reproduce your results for the LSTM example?

Also I would suggest adding a download path for the dataset, which I assume is:
https://einstein.ai/research/the-wikitext-long-term-dependency-language-modeling-dataset
https://s3.amazonaws.com/research.metamind.io/wikitext/wikitext-103-raw-v1.zip

[scott-gray]

It's not clear to me how you're running out of memory. The only time memory was an issue was for the 100M param model with 18k hidden size and 256 time steps. For that I implemented memory efficient back prop (recompute mode).

That is indeed one of the text datasets I was using.

[scott-gray]

So I am noticing a bug in 1.5.0rc1. Some kind of issue with vector::reserve. Not the same as your out of memory error but maybe the issue is with tensorflow. Have you tried it with 1.4.1 and cuda8?

[scott-gray]

So I fixed the issue I found with 1.5 code. TF was somehow passing me an uninitialized shape handle and so I wasn't able to properly do shape inference. I'm not sure if this is related to your out of memory issue.

[dchichkov]

Yes, that fixed the issue!

I'm guessing that it would still be helpful to other people to know minimum hardware requirements for the LSTM example. My 2 cents, with default settings and --ngpu=1 it runs on a single Quadro M6000 24GB.

Wow, 32k nodes LSTM. I think I'll have a lot of fun with this toy. Thank you!