Understanding Keras Recurrent Nets' structure and data flow (mainly LSTM) in a single diagram.
Actually as I was working on understanding how Recurrent Neural Networks really work and what gives these special network architectures this high power and efficiency, especially when working with sequence datasets, I found many difficulties to get the whole concept clearly into my mind. I was very familiar with Convolutional Neural Networks and fully-connected Feed-Forward architectures, where I had been working with these networks for a long time ago, and I can simply describe it as an easy-to-cut cake. However, when talking about Recurrent Nets (e.g. LSTM's, GRU's …) things gets much more complex.
So, when it came into building up my own LSTM model for the first time, I decided to use Keras framework (because I was very familiar with), and I realized then that the worst and the hardest part was about understanding how to prepare and transform my input data to match the Keras model input expectations, and how to transform my training and validation labels to match the output of the network for validation and testing. Besides, when working with LSTM's you will find that there are a lot of network-special parameters that you must understand and take care of in order to start the game.
Finally after having a clear and sharp view of this problem, and because all of what I passed through, I decided to summarize the main flow of Keras LSTM networks in the this diagram for the public benefit.
Please note that the following diagram describes Keras LSTM layers, however the same diagram, as it is, is applicable for GRU's, with a minor difference between both structures in the inner shape of the processing units (nodes).
For Keras LSTM code example, I think that Mr.Jason Brownlee has a great blog here that worth to check.
Please try to read the diagram from bottom to top for better flow coherence.
This project is licensed under GNU General Public License.
Please consider supporting my work
- patreon
- ETH: 0x8c64753c1678D1823Fc73B8990052Ed9a106f641
- BTC: 1JE2811s3NWwDEvL8EbB4U1QB8GoPgJKDL