Tutorial AE on natural images added

docs/tutorial.rst

@@ -22,6 +22,8 @@ In this section you will find tutorials for several algorithms like PCA, ICA, RB
 
    GRBM on natural images<tutorials/GRBM_natural_images.rst>
 
+   AE on natural images<tutorials/AE_natural_images.rst>
+
 The tutorials show how to reproduce results described in the following publications
 
 - `Gaussian-binary restricted Boltzmann machines for modeling natural image statistics. Melchior, J., Wang, N., & Wiskott, L.. (2017). PLOS ONE, 12(2), 1–24. <http://doi.org/10.1371/journal.pone.0171015>`_

docs/tutorials/AE_natural_images.rst

@@ -0,0 +1,84 @@
+.. _AE_Natural_Images:
+
+Autoencoder on a natural image patches
+==========================================================
+
+Example for Autoencoders (`Autoencoder <https://en.wikipedia.org/wiki/Autoencoder>`_)
+on natural image patches.
+
+Theory
+***********
+
+If you are new on Neural networks and Autoencoders , visit `Autoencoder tutorial <http://ufldl.stanford.edu/wiki/index.php/Autoencoders_and_Sparsity>`_.
+
+Results
+***********
+
+The code_ given below produces the following output.
+
+Visualization of 100 examples of the gray scale natural image dataset.
+
+.. figure:: images/SAE_natural_images_data_raw.png
+   :scale: 75 %
+   :align: center
+   :alt: 100 gray scale natural image patch examples
+
+The corresponding whitened image patches.
+
+.. figure:: images/SAE_natural_images_data.png
+   :scale: 75 %
+   :align: center
+   :alt: 100 gray scale natural image patch examples whitened
+
+The learned filters from the whitened natural image patches.
+
+.. figure:: images/SAE_natural_images_filter.png
+   :scale: 75 %
+   :align: center
+   :alt: ICA filter on natural images
+
+The corresponding reconstruction of the model, that is the encoding followed by the decoding.
+
+.. figure:: images/SAE_natural_images_reconstruction.png
+   :scale: 75 %
+   :align: center
+   :alt: ICA filter on natural images
+
+To analyze the optimal response of the learn filters we can fit a Gabor-wavelet parametrized in angle and frequency, and plot
+the optimal grating, here for 20 filters
+
+.. figure:: images/SAE_natural_images_grating.png
+   :scale: 75 %
+   :align: center
+   :alt: ICA filters with fitted Gabor-wavelets.
+
+as well as the corresponding tuning curves, which show the responds/activities as a function frequency in pixels/cycle (left) and angle in rad (right).
+
+.. figure:: images/SAE_natural_images_tuning_curves.png
+   :alt: ICA  fiter's tuning curves
+   :align: center
+
+Furthermore, we can plot the histogram of all filters over the frequencies in pixels/cycle (left) and angles in rad (right).
+
+.. figure:: images/SAE_natural_images_histogram.png
+   :scale: 75 %
+   :alt: ICA histogram of frequency and angle
+   :align: center
+
+We can also train the model on the unwhitened data leading to the following filters.
+
+.. figure:: images/SAE_natural_images_filter_unwhitened.png
+   :scale: 75 %
+   :alt: ICA histogram of frequency and angle
+   :align: center
+
+.. _code:
+
+Source code
+***********
+
+.. figure:: images/download_icon.png
+   :scale: 20 %
+   :target: https://github.com/MelJan/PyDeep/blob/master/examples/AE_natural_images.py
+
+.. literalinclude:: ../../examples/AE_natural_images.py