NEW: added new notebook and renamed chapters for proper sorting

dalmia · Aug 12, 2018 · 6fddb66 · 6fddb66
1 parent ac5ee90
commit 6fddb66
Show file tree

Hide file tree

Showing 8 changed files with 33 additions and 1 deletion.
diff --git a/Chapter 2 - Linear Algebra.ipynb → 02 - Linear Algebra.ipynb b/Chapter 2 - Linear Algebra.ipynb → 02 - Linear Algebra.ipynb
diff --git a/... Probability and Information Theory.ipynb → ... Probability and Information Theory.ipynb b/... Probability and Information Theory.ipynb → ... Probability and Information Theory.ipynb
diff --git a/Chapter 4 - Numerical Optimization.ipynb → 04 - Numerical Optimization.ipynb b/Chapter 4 - Numerical Optimization.ipynb → 04 - Numerical Optimization.ipynb
diff --git a/... - Regularization for Deep Learning.ipynb → 07 - Regularization for Deep Learning.ipynb b/... - Regularization for Deep Learning.ipynb → 07 - Regularization for Deep Learning.ipynb
diff --git a/...timization for Training Deep Models.ipynb → ...timization for Training Deep Models.ipynb b/...timization for Training Deep Models.ipynb → ...timization for Training Deep Models.ipynb
diff --git a/Chapter 9 - Convolutional Networks.ipynb → 09 - Convolutional Networks.ipynb b/Chapter 9 - Convolutional Networks.ipynb → 09 - Convolutional Networks.ipynb
diff --git a/11 - Practical Methodology.ipynb b/11 - Practical Methodology.ipynb
@@ -0,0 +1,32 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 2",
+   "language": "python",
+   "name": "python2"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/Appendix.ipynb b/Appendix.ipynb
@@ -116,7 +116,7 @@
    "source": [
     "**Explanation of how large weights cause symmetry breaking during initialization**\n",
     "\n",
-    "Suppose the eigen-value decomposition of W is given by: $ W = Q V Q^{-1}$ where V is the diagonal matrix of eigen values. Now, if a noise of $\\epsilon$ is added to the input, upon doing W \\* x an extra term W * $\\epsilon$ appears at the output. This $\\epsilon$ term scales the diagonal matrix V. So, if the eigenvalues of W are $\\lambda_1$, $\\lambda_2$, etc., it becomes $\\lambda_1 \\epsilon$, $\\lambda_2 \\epsilon$, etc. Thus, if W had similar eigenvalues for all its eigen directions, i.e. $\\lambda_1 \\approx \\lambda_2$, etc., then $\\lambda_1 \\epsilon \\approx \\lambda_2 \\epsilon$, which means that using different eigen directions didn't give anything extra. However, if the eigen values differ a lot, then multiplication with $\\epsilon$ will increase that difference. This is making a much better use of different eigen directions and thus, has a symmetry breaking effect."
+    "Suppose the eigen-value decomposition of W is given by: $ W = Q V Q^{-1}$ where V is the diagonal matrix of eigen values. Now, if a noise of $\\epsilon$ is added to the input, upon doing W \\* x an extra term W * $\\epsilon$ appears at the output. This $\\epsilon$ term scales the diagonal matrix V. So, if the eigenvalues of W are $\\lambda_1$, $\\lambda_2$, etc., it becomes $\\lambda_1 \\epsilon$, $\\lambda_2 \\epsilon$, etc. Thus, if W had similar eigenvalues for all its eigen directions, i.e. $\\lambda_1 \\approx \\lambda_2$, etc., then $\\lambda_1 \\epsilon \\approx \\lambda_2 \\epsilon$, which means that using different eigen directions didn't give anything extra. However, if the eigen values differ a lot, then multiplication with $\\epsilon$ will increase that difference. This is making a much better use of different eigen directions and thus, has a symmetry breaking effect./"
    ]
   },
   {