reorganized files and ToC

docs/activation_functions.rst

@@ -0,0 +1,44 @@
+.. _activation_functions:
+
+====================
+Activation functions
+====================
+
+.. toctree::
+  :maxdepth: 1
+  :titlesonly:
+
+ELU
+===
+
+Be the first to contribute!
+
+
+LeakyReLU
+=========
+
+Be the first to contribute!
+
+
+ReLU
+========
+
+Be the first to contribute!
+
+
+Sigmoid
+=======
+
+Be the first to contribute!
+
+
+Softmax
+=======
+
+Be the first to contribute!
+
+
+Tanh
+====
+
+Be the first to contribute!

docs/algorithms.rst → docs/basics.rst

@@ -1,17 +1,14 @@
-.. _algorithms:
+.. _basics:
 
-===============================
-Algorithms
-===============================
-
-.. toctree::
-  :maxdepth: 1
+======
+Basics
+======
 
 Fundamental machine learning algorithms and concepts
 
 
-Linear Regression
-===================
+Linear regression
+=================
 
 When a model's predicted output is continuous and has a constant slope.
 At its most basic, it takes the form of:
@@ -52,15 +49,8 @@ References:
 * <https://en.wikipedia.org/wiki/Linear_regression>
 
 
-Logistic Regression
+Logistic regression
 ===================
 
-The bread and butter of neural networks is *affine transformations*: a
-vector is received as input and is multiplied with a matrix to produce an
-output (to which a bias vector is usually added before passing the result
-through a nonlinearity). This is applicable to any type of input, be it an
-image, a sound clip or an unordered collection of features: whatever their
-dimensionality, their representation can always be flattened into a vector
-before the transformation.
-
+Be the first to contribute!
 

docs/terms.rst → docs/glossary.rst

@@ -1,8 +1,8 @@
-.. _terms:
+.. _glossary:
 
-===========
-Terminology
-===========
+========
+Glossary
+========
 
 Definitions of common machine learning terms
 

docs/index.rst

@@ -13,8 +13,8 @@ Topics
 .. toctree::
   :maxdepth: 2
 
-  terms
-  algorithms
+  glossary
+  basics
   math
   nn
   deep_learning
@@ -23,7 +23,7 @@ Topics
 Contributing
 ============
 
-Join the team! Check out our `github <http://github.com/bfortuner/ml-glossary/>`_ for more information.
+Join the team! Check out our `github <http://github.com/bfortuner/ml-cheatsheet/>`_ for more information.
 
 Indices and tables
 ==================

docs/loss_functions.rst

@@ -0,0 +1,54 @@
+.. _loss_functions:
+
+==============
+Loss functions
+==============
+
+
+Cross-Entropy Loss
+==================
+
+Be the first to contribute!
+
+
+Hinge Loss
+==========
+
+Be the first to contribute!
+
+
+Kullback-Leibler divergence
+===========================
+
+Be the first to contribute!
+
+
+L1 Loss
+=======
+
+Be the first to contribute!
+
+
+L2 Loss
+=======
+
+Be the first to contribute!
+
+
+Maximum Likelihood
+==================
+
+Be the first to contribute!
+
+
+Mean Squared Error
+==================
+
+Be the first to contribute!
+
+
+
+
+**References**
+
+* http://rishy.github.io/ml/2015/07/28/l1-vs-l2-loss/

docs/nn.rst

@@ -1,48 +1,18 @@
 .. _nn:
 
-===============================
+===============
 Neural networks
-===============================
-
-.. toctree::
-  :maxdepth: 1
-
-Basic neural network concepts
-
-Basics
-======
-
-Be the first to contribute!
-
-Forward propagation
-===================
-
-.. literalinclude:: ../code/nn.py
-    :language: python
-    :pyobject: MyClass
-
-Backpropagation
 ===============
 
-Be the first to contribute!
-
-Activation Functions
-====================
-
-Be the first to contribute!
-
-Loss Functions
-==============
-
-Be the first to contribute!
+Neural networks are a class of machine learning algorithms used to model complex patterns in datasets using multiple hidden layers and non-linear activation functions. A neural network takes an input, passes it through multiple layers of hidden neurons (mini-functions with unique coefficients that must be learned), and outputs a prediction representing the combined input of all the neurons. Neural networks are trained iteratively using optimization techniques like gradient descent. After each cycle of training, an error metric is calculated based on the difference between prediction and target. The derivatives of this error metric are calculated and propagated back through the network using a technique called backpropagation. Each neuron's coefficients (weights) are then adjusted relative to how much they contributed to the total error. This process is repeated iteratively until the network error drops below an acceptable threshold.
 
-Optimizers
-==========
+** Topics **
 
-Be the first to contribute!
-
-Layers
-======
-
-Be the first to contribute!
+.. toctree::
+  :maxdepth: 1
+  :titlesonly:
 
+  nn_concepts
+  activation_functions
+  loss_functions
+  optimizers

docs/nn_concepts.rst

@@ -0,0 +1,30 @@
+.. _nn_concepts:
+
+==============
+Basic concepts
+==============
+
+Basic concepts in neural networks
+
+Neurons
+=======
+
+Intro to neurons
+
+
+Hidden Layers
+=============
+
+Explanation of how layers work
+
+
+Forwardpropagation
+==================
+
+Intro to forwardpropagation
+
+
+Backpropagation
+===============
+
+Intro to Backpropagation

docs/optimizers.rst

@@ -0,0 +1,52 @@
+.. _optimizers:
+
+==========
+Optimizers
+==========
+
+
+Adadelta
+========
+
+Be the first to contribute!
+
+
+Adagrad
+========
+
+Be the first to contribute!
+
+
+Adam
+====
+
+Be the first to contribute!
+
+
+L-BFGS
+======
+
+Be the first to contribute!
+
+
+Momentum
+========
+
+Be the first to contribute!
+
+
+RMSProp
+=======
+
+Be the first to contribute!
+
+
+SGD
+===
+
+Be the first to contribute!
+
+
+**References**
+
+* http://sebastianruder.com/optimizing-gradient-descent/

docs/resources.rst

@@ -30,7 +30,7 @@ Papers
 
 Be the first to contribute!
 
-Libraries and Frameworks
+Libraries and frameworks
 ========================
 
 Be the first to contribute!