a simple rnn using OpenCV

CMakeLists.txt

@@ -0,0 +1,7 @@
+cmake_minimum_required(VERSION 3.2)
+project( rnn )
+find_package( OpenCV REQUIRED )
+aux_source_directory( src/. src_list )
+add_executable( rnn ${src_list} )
+target_link_libraries (rnn ${OpenCV_LIBS})
+

README.md

@@ -0,0 +1,70 @@
+#recurrent-net-version-1.0
+=====================
+
+Simplest uni-directional recurrent neural network (C++ / OpenCV).
+
+To run this code, you should have 
+* OpenCV.
+
+##Compile & Run
+* Compile by running:
+```
+cmake .
+make
+```
+* Run: 
+```
+./rnn
+```
+##Structure and Algorithm
+See [my tech-blog](http://eric-yuan.me/). TODO...
+
+##TODO
+log file
+blog post
+
+##Config Files
+
+####General Parameters Config
+* gradient checking (automatically disable dropout, and use tiny dataset)
+* store parameters into log file (for debugging, should be faster if not using it) TODO...
+* batch size
+* non-linearity method (sigmoid, tanh, ReLU)
+* training epochs
+* iteration per epoch
+* learning rate
+* ngram
+* training percent (for cross validation)
+
+####Layers Config
+* layer type
+* amount of hidden neurons
+* dropout rate
+* weight decay
+* amount of output classes (in **Softmax** Layer, initialized to be 0, because it depends on dataset)
+
+####Multi-Layer
+This network supports multiple hidden layers.
+
+The MIT License (MIT)
+------------------
+
+Copyright (c) 2015 Xingdi (Eric) Yuan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

config.txt

@@ -0,0 +1,41 @@
+/*
+*	Config File
+*/
+
+/*******************************************************
+*	
+*	General Parameters Config
+*
+*******************************************************/
+
+IS_GRADIENT_CHECKING = false;
+USE_LOG = false;
+NON_LINEARITY = NL_RELU;
+BATCH_SIZE = 50;
+
+TRAINING_EPOCHS = 50;
+ITER_PER_EPO = 100;
+LRATE_W = 3e-3;
+LRATE_B = 1e-3;
+NGRAM = 3;
+TRAINING_PERCENT = 0.80;
+
+/*******************************************************
+*	
+*	Layers Config
+*
+*******************************************************/
+
+$
+LAYER = HIDDEN;
+NUM_HIDDEN_NEURONS = 512;
+WEIGHT_DECAY = 1e-6;
+DROPOUT_RATE = 0.5;
+&
+
+$
+LAYER = SOFTMAX;
+NUM_CLASSES = 0;
+WEIGHT_DECAY = 1e-6;
+&
+