machinelearning/Dimensionality.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Machine Learning: Dimensionality Reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Material for the [2018 Asterics and Obelics School](https://indico.in2p3.fr/event/16864/): [Table of Contents](Contents.ipynb)\n",
    "\n",
    "Content is maintained on [github](https://github.com/Asterics2020-Obelics/School2018/tree/master/machinelearning) and distributed under a [BSD3 license](https://opensource.org/licenses/BSD-3-Clause)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![sponsor-logos](img/sponsor-logos.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns; sns.set()\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "We will use a new sklearn module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import decomposition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Some of the code below generates [annoying warnings](https://github.com/mwaskom/seaborn/issues/1392). You usually want to be aware of warnings, especially when they give advance notice of something that will break in a future version of a package you depend on.\n",
    "\n",
    "However, to keep a clean notebook, you can always silence specific expected warnings like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore', category=UserWarning)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "from mls import plot_image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dimensionality of Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We call the number of features (columns) in a dataset its **dimensionality** $D$. In order to learn how different features are related, we need enough samples to get a complete picture.\n",
    "\n",
    "For example, imagine splitting each feature at its median value then, at a minimum, we would like to have at least one sample in each of the resulting $2^D$ bins.  This is a very low bar and only requires 8 samples with D=3, but requires $2^{30} > 1$ billion samples with D=30!\n",
    "\n",
    "To get a feel for how well sampled your dataset is, compare your actual number of samples $N$ with the target $2^D$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Our image dataset with $D=256$ is heavily undersampled:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset has 50000 samples with 256 features.\n",
      "2^D is 115792089237316195423570985008687907853269984665640564039457584007913129639936.\n"
     ]
    }
   ],
   "source": [
    "df_img = pd.read_hdf('data/sources_img.hf5')\n",
    "N, D = df_img.shape\n",
    "print(f'Dataset has {N} samples with {D} features.')\n",
    "print(f'2^D is {2 ** D}.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Dimensionality Reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dimensionality reduction algorithms attempt to discover a subspace of undersampled data that best explains the data. The features in this subspace are often referred to as **latent variables**.\n",
    "\n",
    "The notion of \"explaining the data\" is poorly defined, with different choice leading to different algorithms.\n",
    "\n",
    "Dimensionality reduction is an unsupervised learning method (like clustering).\n",
    "\n",
    "The key idea of dimensionality reduction is that not all features carry equal information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "As an extreme example, consider the following 2D data which is effectively 1D since one feature has a constant value (zero):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWYAAAFgCAYAAACIf9MLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFX5JREFUeJzt3H+Q3PV93/Hn3a3uTkKLOJklgMk48Ri9k4kNEaEuGJGQGseBqROSOGlCW9sEO47Tjjv2tK5Tu24ydcdTMh5PkgnO2IXG1E3b1DFjNwm0STzMgDBDW4QrOvFHVpu2ARtzQoe04tCddHf94/bEcd5fp1t9932652OGYW8/3+/38/58v7uv++hz392RpaUlJEl5jA67AEnSKxnMkpSMwSxJyRjMkpSMwSxJydSGXcBGTU83t8xtJVNTO5iZmR12GZVyzFtDP2NuNOojFZUzdM6YN5FabWzYJVTOMW8NW3HM3Wz6GfNW8dCTz1DfOUnzxMlhl9K3m37w1cMuQdqUnDFLUjIGsyQl41KGzpmHnnxmw8eoevnG5Rdl4IxZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGT9gIq0yiA/FbNR6P1Tjh2LOP86YJSkZg1mSkjGYJSkZg1mSkjGYJSkZg1mSkjGYJSkZg1mSktmSHzDJ8CECSerEGbMkJWMwS1IyW3IpQzqfbMalOb/foztnzJKUjMEsSckYzJKUjMEsSckYzJKUjMEsSckYzJKUjMEsSckYzJKUjMEsSckYzJKUjMEsSckYzJKUjMEsSckYzJKUjMEsSckYzJKUzMjS0tKwa5AkreKMWZKSMZglKRmDWZKSMZglKRmDWZKSMZglKRmDWZKSMZglKZnasAvYqOnp5pb5hMzU1A5mZmaHXUalHPPW0M+YG436SK/jbKY86DYeZ8ybSK02NuwSKueYt4atOOZuDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkalV2FhGjwN3A1cAc8O5SyuE22/wx8KVSyu9WWZ8kZVD1jPk2YLKUcj3wYeCTbbb5OLC70qokKZGqg3kf8CBAKeUx4NrVjRHxdmAReKDiuiQpjUqXMoALgWOrfl6IiFop5XREvB64HXg78LF+Dzg1tYNabWzAZebVaNSHXULlHPPWMIgxny95UHUwHwdWn/3RUsrp1uN3AK8GvgJ8DzAfEf+nlPJgtwPOzMyeizpTajTqTE83h11GpRzz1tDPmPsJ7s2UB93GU3Uw7wfeBvxBRFwHHFxpKKV8aOVxRPwa8GyvUJak81HVwXw/8JaIeBQYAe6IiA8Ch0spX664FklKqdJgLqUsAr+85umvt9nu1yopSJIS8gMmkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpRMrcrOImIUuBu4GpgD3l1KObyq/QPAz7d+/JNSyq9XWZ8kZVD1jPk2YLKUcj3wYeCTKw0R8VrgbwNvAq4Hfiwirqq4PkkauqqDeR/wIEAp5THg2lVtfwX8eClloZSyCGwDTlZcnyQNXaVLGcCFwLFVPy9ERK2UcrqUcgo4EhEjwG8AB0oph3odcGpqB7Xa2DkqN59Goz7sEirnmLeGQYz5fMmDqoP5OLD67I+WUk6v/BARk8C9QBP4lX4OODMzO9ACM2s06kxPN4ddRqUc89bQz5j7Ce7NlAfdxlP1UsZ+4FaAiLgOOLjS0Jopfwn4WinlvaWUhYprk6QUqp4x3w+8JSIeBUaAOyLig8BhYAz4EWAiIm5pbf+rpZSvVlyjJA1VpcHc+qPeL695+uurHk9WWI4kpeQHTCQpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpKpDbsASRqUh558Ztgl9HTTD7665zbOmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpLZcl9iNHdqgWMn5ti1c4KJbWN9t/far1d/2ydqvDR3uq/9m7PzPP3cCa64ZCf1HeNnjvOtIy+ycGqh4/7d+po7tcD0zCynTi+yrTZKY2rHK9rWjq3beFeOxcgIjYu2M7Ft7BV9T78wy0xznp07trGtNsr42Cv762fc49vGOHZijvHt4zw3M3umjtV1zZ9aaHuezvY6rYxp1wXjvDR3mu0TNY69OA9LS33Vv7Z/oG0tzdl5/vKbx5gcr7FjssaunRN9vzY2au356XWd13Mu+9m+3WtH36nyYI6IUeBu4GpgDnh3KeXwqvb3AO8FTgMfL6X80SD6XVhc5D985TAHDk1z9Pgcuy+cYO+eBn/rb7yOsdHRju1vv+m1fOGh/91xv376e/74HKMjsLgEu+vjXBOXtN1//vRp/sV9T/DM9AkWl2B0BC6/+AL2fPcuvnb4eY4259hd/87+V/p6ojzH0eb8mb5edeEEV195MUtLS3z1qWc5Ob94pq+JbaO86Q2XMjIywte+ceTM2K6+8mJGgCdXPbfSH8C/+/Nv8OjBb5051sS2URpT23np5GmePz7X8XxMbBvhhqsu5xfefGXPcQOMjsLi4sv/310f54Lt48yePMXR43OMjo6w0Np4dAQub1zA667YxcHDz6/7Oq0dUzuT46O86Q2Xta1/9TVYeZ1MjI8BS5ycX+RVrVp+6oe/l098/gmefu7Ftn2sbPf3f25vxzrO1tr6ptacz7XXudt7pdex223f7jxPjo9xwxsu5efffOXAx7vZjSwtLVXaYUT8NPATpZR3RcR1wK+WUn6y1XYp8KfAtcAk8AhwbSml4zt+errZ1wB+/88O8Wf/7enveP7ma6/g9pv3dGz/7kt28lfPnei433r767b/P7v38bZ99dq/V1+DcPO1VwBsuJ+NjnsQ/a223nPX6Xj9HGfn9honXjrds4+fuPG13HbD9/RdUz/6HWe367zesff7Gr352iv4B7/wQ0xPN7vW1mjUR3rV/x//9OvVBtpZWPnaz27j6bnGHBF/bYA1AewDHgQopTzGcgiveCOwv5QyV0o5BhwGrtpoh3OnFjhwaLpt24FDR2jOzndsf2a6fWAcOHSEuVML6+6v0/7N2fmOfXXbv5++BuHAoWn++9e/veHjPFGmNzTu9ep1nZ4oz63reGvrXzlOP9egn1AGeOypb3Ws+Wys5zVy4NB0x3PS7lz2em+tvEa7necDh6Y5Od/fudkq+lnKuCsiLgbuA/5NKeXZDfZ5IXBs1c8LEVErpZxu09YEdnU72NTUDmq17utU3zryIkeb7SfdM82TNOcXO7Yvdvj9O9M8ydj4NhoXX7Cu/jrt/81vTHfsq9v+QM++BuFoc45B/ONqpjm3oXGvv79e12l+nceb+47j9XO91+PICy91rPlsrKe+bte53bns9d56+TXa+Twfbc4xc3yOyxr1vmrs5oId44x2WbrKoNHHOHsGcynlRyPiNcDfBf5LRPw/4PeAL5VSTp1FXceB1ZWNtkK5XVsdeKHbwWZmZnt2uHBqgd31ibZroFP1Serjox3bV9Zr2+23MH+q7T+/uvXXaf/6+GjHvrrtD/TsaxB21ydYXFxk5sTZXPKXTdUnNjTu9ffX6zqNryuc19b/8nEGdw0uvmh7x5rPxnrq212fYGlpqe05aXcue723Xn6Ndj7Pu+sTTF040c9SRs/6X5xd3y/aYVgZZ7fx9PWrpZTyf1meMf8+8Hrg/cBTEfFTZ1HXfuBWgNYa88FVbY8DN0bEZETsAr4feOos+niFiW1j7N3TaNu2d8/F1HeMd2x/dWNnx/06/UW5W3+d9q/vGO/YV7f9++lrEPbuafBD3/ddGz7ONdHY0LjXq9d1uiYuWdfx1ta/cpx+rsHO7f39rf2611820LsV1vMa2bun0fGctDuXvd5bK6/Rbud5754Gk+Nb7gaxrnqejYi4E3gHcBnwOWBfKeXpiLgcOADcv84+7wfeEhGPAiPAHRHxQeBwKeXLEfFbwMMs/9L4SCnl5DqP39bKX5sPHDrCTPMkU/VJ9u65+Mzzndpfviuj/X69+1t7V8YE10Sj7f4fecc1Xe7KONqx/5XHT5RpjjbnzuKujOfPHPvqK1/Vuivj+bb9LS4t8ejBZzk5v3DmOP3dlTHKDVdd1te4ocddGc05Rkc63ZXR+Tx1uk5rx9TO5PgYb3rDpR2Pt/b1M75y2+H8ArsvXK6l37syfvFtP8DRo+23OVtr67to5wQXbN/G7MlTzDTn2p6vfl/zvd5bK9usPc8rd2X0ukZbUc+7MiLiPuDeUspDbdp+ppTyh+eotr70e1fGis18H/PY+DYW5k9tqfuYr7j8Ip7+5gtb6j7mRqM+sGWMbvVluo+5nzFvpbsyKr9dbtDWG8yb2bl8w2blmLcGg/mVcv/5UpK2IINZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGYNZkpIxmCUpGb9rT9J5Y+V7KDY7Z8ySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJGMySlIzBLEnJ1KrsLCK2A58HLgGawDtLKdNrtvkNYF+rts+UUj5bZY2SNGxVz5jfBxwspdwI3Ad8dHVjRPwo8LpSyvUsh/M/joipimuUpKGqdMbMctje1Xr8APBP17R/FXiy9XgJGANOdTvg1NQOarWxQdaYWqNRH3YJlXPMW8Mgxny+5ME5C+aIuBP4wJqnvw0caz1uArtWN5ZSTgInI2Ib8DmWlzJOdOtnZmZ2MAVvAo1Gnenp5rDLqJRj3hr6GXM/wb2Z8qDbeM5ZMJdS7gHuWf1cRHwRWKmmDrywdr/W0sUXgIdKKZ84V/VJUlZVrzHvB25tPb4FeHh1Y+uPg38O3FtK+ecV1yZJKVS9xvxp4HMR8QgwD9wOEBF3sTxLvgF4LfCeiHhPa587Sil/WXGdkjQ0lQZzKWUW+Nk2z3+o9fBx4FNV1iRJ2fgBE0lKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGQMZklKxmCWpGRqVXYWEduBzwOXAE3gnaWU6Tbb7QAeBT5cSnmwyholadiqnjG/DzhYSrkRuA/4aIftfgdYqqwqSUqk6mDeB6zMgB8Abl67QUT8Q5Zny1+rsC5JSuOcLWVExJ3AB9Y8/W3gWOtxE9i1Zp83A1eWUt4bETf008/U1A5qtbGNlrtpNBr1YZdQOce8NQxizOdLHpyzYC6l3APcs/q5iPgisHL268ALa3a7E3hNRDwEfB9wTUQ8W0p5slM/MzOzA6s5u0ajzvR0c9hlVMoxbw39jLmf4N5MedBtPJX+8Q/YD9wKPA7cAjy8urGUcvvK44j4PeDfdwtlSTofVb3G/GngByLiEeCXgF8HiIi7IuKNFdciSSlVOmMupcwCP9vm+Q+1ee5dVdQkSdn4ARNJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkDGZJSsZglqRkRpaWloZdgyRpFWfMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpRMbdgFqLOI2A58HrgEaALvLKVMr9nmy8CrgFPAS6WUWyovdAAiYhS4G7gamAPeXUo5vKr9PcB7gdPAx0spfzSUQgeojzH/FnADy9ce4CdLKccqL3TAIuKvA/+ylHLTmuffBnyM5Wt8bynls0MoLwVnzLm9DzhYSrkRuA/4aJttXgfsK6XctFlDueU2YLKUcj3wYeCTKw0RcSnwfpZD6q3AJyJiYihVDlbHMbdcA7y1dW1vOk9C+UPAvwIm1zy/DfgU8GPAjwC/1LruW5LBnNs+4MHW4weAm1c3RsR3ARcB/ykiHomIv1lxfYN0ZqyllMeAa1e1vRHYX0qZa4XTYeCq6kscuI5jbs2mrwQ+ExH7I+IXh1PiwP0v4KfbPP/9wOFSykwpZR54BLix0soSMZiTiIg7I+Kp1f8Bu4CVWVKz9fNq4yzPsm5j+cX+qYi4pLKiB+tCXh4rwEJE1Dq0tTsXm1G3MV8A/Dbwd4AfB34lIjb9L6NSyh+yvOy21vl6jc+Ka8xJlFLuAe5Z/VxEfBGot36sAy+s2e1Z4HdLKaeB5yLiABDAc+e43HPhOC+PFWC0Na52be3OxWbUbcyzwG+WUmYBIuIrLK9F/49qS6zM+XqNz4oz5tz2A7e2Ht8CPLym/WbgDwAiYifweuAvKqtusM6MNSKuAw6uanscuDEiJiNiF8v/7H2q+hIHrtuY9wCPRMRYa/11H/BE9SVW5i+AKyNid0SMAz8MfHXINQ2NM+bcPg18LiIeAeaB2wEi4i7gC6WUByLirRHxGLAI/JNSypHhlbsh9wNviYhHgRHgjoj4IMvrjl9u3aHwMMuTiY+UUk4OsdZB6TXmfws8xvI//e8rpfzPIdZ6TkTE7cDOUspnWmP/zyxf43tLKc8Mt7rh8Ws/JSkZlzIkKRmDWZKSMZglKRmDWZKSMZglKRmDWZKSMZglKRk/YKJNIyLeD/wMcBPL3zT3r4G9pZQTw6xLGjRnzNpMfpvlTzi+j+WvjnyXoazzkZ/806YSEd/L8vdk3F1K+UfDrkc6F5wxa7N5DctfCXlNRIwMuxjpXDCYtWm0vkHvs8DbgJdYXtKQzjsGszaTu4A/LqX8V+DvAR9rLW1I5xXXmCUpGWfMkpSMwSxJyRjMkpSMwSxJyRjMkpSMwSxJyRjMkpTM/wfaVlSfMLxCdwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "gen = np.random.RandomState(seed=123)\n",
    "data = pd.DataFrame()\n",
    "data['x'] = gen.uniform(-1, +1, 50)\n",
    "data['y'] = np.zeros_like(data['x'])\n",
    "sns.jointplot('x', 'y', data, stat_func=None, size=5);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Is this data is still 1D if we add some small scatter in the 2nd dimension?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAFgCAYAAACfXUPCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuYXHWd5/F3dXe6O006oQOV4eaVkK+uLBhAJWOQiEEMI0tmGBeNwzAR0IHZwZHd8cERdZxlxx1m0UVmYVYuKwwTHxw0DLqCIwZ8kkCWAYMkLnwhuKjcTCfpdFfodPWt9o+qDpVKXburzqlf9ef1PDxUnUuf76+q+1O/+p3zO0lkMhlERKT5tcVdgIiIVEeBLSISCAW2iEggFNgiIoFQYIuIBKIj7gIaob8/Nasufenr62FgYDjuMiKlNre+atubTPYmIiinKaiH3QI6OtrjLiFyanPrm23trUZL9rBnk4effIneed2k9o3EXUrVVrzz2LhLEAmSetgiIoFQYIuIBEJDIhK5h598acY/I+phIA3jSDNQD1tEJBAKbBGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEJs6IVKEek31mqtbJQprs03rUwxYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhiTN5mmFyhIhIKephi4gEQoEtIhIIDYmItKgQh/h0/5Py1MMWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQlEIpPJxF2DiIhUQT1sEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRALREcdBzew9wN+4+4qC5ecBXwTGgdvd/RYzmwvcBSwCUsDF7t4fcckiIrGLvIdtZp8FbgW6C5bPAb4GfBA4E/ikmR0FXA5sc/czgDuBa6KtWESkOcTRw34e+D3gHwqWvx3Y4e4DAGa2CTgDWA5cl9vmfuALlQ4wPj6R6ehor1vBItLUEpU2CCwTSrYn8sB29++Y2ZuLrJoPDOY9TwELCpZPLStrYGB4hlWGJZnspb8/FXcZkVKbW1+17U0meytuE1ImlGtPM510HALyK+0F9hYsn1omIjLrxHLSsYSngRPMbCGwD3gf8N+ANwHnAo8Bq4CNsVUoIhKj2APbzNYA89z9G2Z2FfBDsj3/2939JTO7GbgjN6Y9CqyJsVwRkdi05N36+vtTrdeoMmbb2CaozbNBDWPYFU86hpQJ5drTTGPYIiJShgJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAtER9QHNrA24CTgZSAOXuvuO3Lp3Av89b/PTgdXAY8CzwPbc8vXufkNkRYuINIHIA5tsAHe7+zIzOx24HjgfwN2fBFYAmNlHgJfd/QEzWwl8y93/NIZ6RUSaQhyBvRx4AMDdt5jZaYUbmNlhwJeB9+UWnQqcYmY/AXYCV7r7K6UO0NfXQ0dHe90Lb2bJZG/cJURObW599Wpvq2RCHIE9HxjMez5hZh3uPp637BLgn9x9V+75M8AT7v6gmX0cuBH4/VIHGBgYrnfNTS2Z7KW/PxV3GZFSm1tfte2tJtRDyoRy7YnjpOMQkF9RW0FYA3wcuDXv+Qbgodzj9cDSxpUnItKc4gjszcC5ALkx7G35K81sAdDl7r/OW3wrcEHu8QeAJyKoU0SkqcQxJLIeONvMHgESwFozuwrY4e73AUuAFwr2uRq43cyuAF4DLo2wXhGRppDIZDJx11B3/f2p1mtUGbNtbBPU5tmghjHsRKVtQsqEcu3RxBkRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJREfUBzSzNuAm4GQgDVzq7jvy1n8deC+Qyi06H5gDrAPmAi8Da919OMq6RUTiFkcPezXQ7e7LgKuB6wvWnwKc4+4rcv8NAl8E1rn7GcBW4FORViwi0gTiCOzlwAMA7r4FOG1qRa73fQLwDTPbbGafKNwHuB9YGV25IiLNIfIhEWA+MJj3fMLMOtx9HDgMuBH4KtAOPGRmjxfskwIWlDtAX18PHR3tdS+8mSWTvXGXEDm1ufXVq72tkglxBPYQkP8utOXCGmAYuGFqfNrMNpAd657aZ3/u/3vLHWBgYHYNbyeTvfT3pypv2ELU5tZXbXurCfWQMqFce+IYEtkMnAtgZqcD2/LWLQE2mVm7mc0hOxTy0/x9gFXAxujKFRFpDnH0sNcDZ5vZI0ACWGtmVwE73P0+M/tHYAswBtzp7j83s2uBO8zsMmAXsCaGukVEYpXIZDJx11B3/f2p1mtUGbPtqzKozbNBDUMiiUrbhJQJ5dqjiTMiIoFQYIuIBEKBLSISCAW2iEggFNgiIoFQYIuIBEKBLSISCAW2iEggFNgiIoFQYIuIBEKBLSISCAW2iEggFNgiIoFQYIuIBEKBLSISCAW2iEggFNgiIoFQYIuIBEKBLSISCAW2iEggFNgiIoFQYIuIBEKBLSISCAW2iEggOqI8mJm1ATcBJwNp4FJ335G3/jPAR3NPf+DuXzazBPAi8Fxu+aPu/rkIyxYRaQqRBjawGuh292VmdjpwPXA+gJm9Ffg48B4gA2w0s/XAMPBTdz8v4lpFRJpK1EMiy4EHANx9C3Ba3rpfAx9y9wl3nwTmACPAqcCxZvaQmf3AzCzimkVEmkLUPez5wGDe8wkz63D3cXcfA3blhkD+Ftjq7s+a2VHAV9z9n8xsOXAX8K5yB+nr66Gjo71RbWhKyWRv3CVETm1uffVqb6tkQtSBPQTkvwNt7j4+9cTMuoHbgRRwRW7x48A4gLtvMrNjzSzh7plSBxkYGK574c0smeylvz8VdxmRUptbX7XtrSbUQ8qEcu2JekhkM3AuQG4Me9vUilzP+p+Bn7n7p9x9IrfqS8Cf5bY5GfhVubAWEWlVUfew1wNnm9kjQAJYa2ZXATuAduBMoMvMVuW2/xzwX4G7zOx3yPa0/yjimkVEmkKkgZ07mfjHBYufyXvcXWLX32lMRSIi4dDEGRGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEAltEJBAK7FkgPTbBzoFh0mMTlTeWkvQ6Styivr2qRGhicpK7N+xg67P97BlKs3B+F0uXJLnwrMW0t+mzulp6HaVZKLBb2N0bdvDg4y8eeL57KH3g+ZqVS+p6rPTYBIP70iyY10XXnPD/KaZ8UbyOrfz6Sf0osFtUemyCrc/2F1239dldXHDm8RWDoZoQqaX3GWIo1eN1LEe9d6mFAjtGjQywwX1p9gyli64bSI0wuC/Nor6eoutrCZFqep8hh9JMXsdqRPktaDZ7+MmXGvJzV7zz2Ib83FIU2DGoZ4ClxyZ4ZddrTIxNHBT6C+Z1sXB+F7uLhE1fbzcL5nUd2L/wQ6PaEKm29xlyKFX7Ok5Ho3vv0noU2DGYboDlh2tHe+L10E+lWdh7cOh3zWln6ZLkQceZsnTJkXS0J1j34LOHfGisPuOtVYdINb3PBfO6Sv68TU+9wuoz3kJP15yq2hxHeFV6Hautaaodc7s62J8eZ8G8rrKv3+6hEfYMjXD0EYfNqP6QxP1eh0CBnRPVL8t0elXFeuQ93XP49c59B7YpFvoXnrX4wM8dSI3Q19vN0iVHcuFZi0t+aOwfGa96CKCa3me5UBoZnWDdj57j0g//m0PWNdMwSrnXsZL8duweStOWgMkMLOzt5OTFR5Z8/QBuvnc7X1r7rqYfNpqpUu/1f/j3S+MurenM+sCOOhj2DI2U/AMtNSZaLFxL/Yz80G9va2PNyiVccObxB30YDafH2PTUK0X3f+ZXA/T1drInNXrIusIhgGp6n+VCHeCZXw6QLhjOKdXmuIZRSr2O1Shsx2Qm+/89qVEe2voyxyYPA4q/Ni/2v8a6Hz3LRee8baZNaGql3uueuZ2sfu+b4yusCvUcG69mPLy1P7qrMPXLsnsoTYbXf1nu3rCjIcd78IlDw21KsTHRcj3yYqZCP1/XnHYW9fUcCJl1P3qOkdHikz8GUmne9qaFRdcVGwK48KzFrDztOI6Y301bAhb2dvHbJx7F6jPecuDYb3tjX8l69+5LH1JvpW8hcU1cKXwdK6nmvds3PFZ2/dbn4mtvFMq9Rlu2v9LSbZ+OWR3YMw2GWme+pccmeGrHrpLrTzp+4SFhUG5IoZhKJ8LSYxM888s9ZfbvYs3ZJxwUwkfM72blaccVHQKY6n1++ZJ3s+wdR5FIwKPbX+VLtz3GugefZWJyko+dvYTuzuK/asXqrWZsPATVvHdDr40yv6f0GP7gvtFI2lv4uxzVrM5yr9GuvfuDea+jMquHRKZ7ydZ0h1Eq/QGvPO0NhyyrNKRQqNKJsMF9aQaKDHdMedsb++jpmlPzEMC9G3/B5u2vHnheOISx/KRjqj5xN5MrM5rpxFU1793C+d10d7YzVKKnvXD+zK5EqaTwd7mvt5PD5nYyPDIWyRBhudfoyMPnNrTtIYo8sM2sDbgJOJns4N2l7r4jb/1lwKeAceBad/++mR0JrAPmAi8Da919eKa1TDcYpju+Wu54R8zvZuH87kOWlxsnfsOieQyPjNd0IqxcDd2d7Xzs7NfrnxoCqKSaE6m1nLibzpUZU8HzU9/JntQoC3s7OcUWNSRoqv1QKNeOKScdv5Cf1fitq54Kf5f3pEYPOn/R6HMHHe0JerrnFP19PP3Eo2P/0G02cfSwVwPd7r7MzE4HrgfOBzCzo4ArgdOAbmCTmf0I+CKwzt2/aWZXkw30r820kOkEw0yunZ3uJWLlwm58IkN75xwmRseqnrl40uIjeeinh54sWX7S0fR01f4rUe03lQvOPJ73nXwMZDIkK4wF57d5z9AIC+Z1svSE0h9I3/rxc2x44vU27UmN8uDjLzKZyfAHZ1vNbSpmOt+sXm9H4VUiXZxiSd6/9Fge3vpyyWMW+9ZVL7WcH2nUdeF3b9hx0NVOU96waB6fOO8d7NnzWl2P1wxmMtmm4l+nmb3L3f912kc41HLgAQB332Jmp+Wtezew2d3TQNrMdgAn5fb569w29+cezziwofZLtmY68206l4iVu0qhvQ2SRx5Gf3+q5P7FvvZme+djDKTSNV2mVkylbyrzejqLXvNdLuja29q48KzFTExmePLZXezdl+ap53fT3r7jkDrTYxM8sq34VS+PbHuVj6xYXJegmc43q8L3Lv867K457aTHJmr+1lUvtZwfqceszkLlPjCGR8YZm5is27FaRTXdqetyQxJ3Av/g7q9W2qGC+cBg3vMJM+tw9/Ei61LAgoLlU8tK6uvroaOj+j/QT3/sVEZGxxkYStM3v4vuzoNflvx1vQvmkuyby86B/Yf8nCMPn8vxbz7ikP1rPV45x5VYnkz2ltznlnu3Ff3ae+5vv5nVZy6uuYZi3nvysdy38RdFlh/Dvzz+YtGgmyTB5RecVPLYt9y77aBvAlP7dXZ2cPkFJx9o8wuvDDEyWvyPe2R0gvFEG8eVeX3KmXqfero7eOr53UW3eer53XzqgrkVX8NS71251+64Yw4/aFm597lW5X6XC1X7u11Mqd/1V3a9xp5U6c7PwFCao+vU3sN6OmlrkuvZZ/IeVnz13f39ZvYm4CLgX8zsV8A3gX929/LXJBU3BORX3JYL62LreoG9ecv35y0raWBgesPbHUBqcD9TfdVSX4H/7VsX8uMnDh1SOOn4Iw7av9bjTVcy2Vuyh50em2Dzz4pfK/p/tr/KecveRGpwcsY1nLfsjQzvHz3km8PKU47mszdvKbrPhsd/zZP+m6JjzeXqfuDRFwD43eVvpr2tjYGB8l+bd+4c4rXU/ppORBa+9wvmdbJ3X/GTtbv27uf5F3ZPu/dZ6rU7b9kbD3pfy73P03XS8UeUHWPP367W39VKQ0gTYxMs7C39zaxvfldV7a0mAF8bLn2iPWqV2lSuPVV9XLr7L83sTrInAv+Y7DjzfzGzq919ffWlArAZOA/4dm4Me1veusdyP7cb6ALeDmzP7XMu2Q+KVcDGGo85LaW+Ap916rGsPO24ac18i1q1Qzgzvbqi1LDNrd//vyWv+YbXx5rh4GGFcnVPZuAHj7zA/v2jXHTO21hwWCedc9oYHTu0l93eluB/rN/GQGq0piseCt/7UmENM7+nyEwm5sxU4RDd4fO6OGzunJLDZbX8nlQaQqp0Tqe7s2PGHYlmVGqyTTVj29WMYV8C/CFwNHAHsNzdXzSzY4CtQK2BvR4428weARLAWjO7Ctjh7veZ2dfJBnIb8Hl3HzGza4E7cleQ7ALW1HjMmpUbX/vZc7u59rL3RPoHNt1ArTy+PKfm8eVy8q8sSY9N8PQLxYcRChWe1KrmkriHt77Mcy8Osj89XjSsASYmMweueqjlni21TFZ65wlH1OX9L7wqJ4pLFEt9WBQee2Jysqbfk2pPzs9k2v9sVE0P+0zgS+7+cP5Cd3/ZzK6o9YDuPkm2l57vmbz1twC3FOzzG+BDtR5rJqrtmdbzJEwxM506X6kXc+/G/9eQKeATk5Pc9UNnYF91o2aFJ7WquSQuQ3b6djEL53cxPDJWdGy70hUPg/tKT/0vVUc9xXFvjcIPi8LntZ5wrfbvJ85vFyGq+Bfv7n9YGNZ5675T94qaxFQPr5iZfgWuRT2mzhdOH5+auVjpznwzmeV294YdB02kqaTYa3rhWYt5/ynH0pao7dh987q4YvU7SJc4EVlptuSCeV0lZ2YW87Pndtd1RmCp9/z27/28bseoxXRmBNf691PrtP/ZalbPdCynXrfVnIl63S+5VC9m58BwQ27OX+uQAhR/Tdvb2rjogwaZDA+VuVa50OBraTrndMzwPtbVf0rU85K3SvfWWPXuN0QeatO5lLUZ/n4aJep/tCBfc1zn0qRK9UyjGl+r9z01CnsxtfaCqr2/RK33P+nubD9ws6hi1py9hPcvPabqnnZfbzfJw+eydEmy6Ppqpu+ny5woLXa8en3jasZ7a0z322bcfz+tSD3sMuIeX2vkv3YC1feCah1Hr/X+J6NjE+wbHiv5Dxm0t7VlbzGaSBSdoVmq9ume0Kr3/Vtq0Yz31phubznuv59WpMCuQrX31GjEcRv9tbKaUKv1hFM1JwzzVfvhs2blCbS3Jaq+BG26gVHv+7fUotyx47y3xkyu5ojr76cVKbCbXKMve6oUatMdRy9Wd093R9H7RlT74ZNfa/79Uypd/jadwKh0/5ZG9hhLHTvOe2uot9wcEplMvS9Kil9/f6rlGlUulBoxA27KzoFhPvc/txS9dK0tAX/9ydPLhmHxf4fy0BCs9ZrvRrY5X5y3ay08dlRtbhbVtjeZ7K14diOkTCjXHvWwAxHX18qZjqMX1h1aLy3Or/MaSpBCukpEypoaUy1muuPouuZWZHrUw5aKNH1YpDkosKUinXASaQ4KbKmaxlRF4qUxbBGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEAltEJBAKbBGRQCiwRUQCocAWEQmEAltEJBCR3vzJzOYCdwGLgBRwsbv3F2zzt8DyXG3fcPdbzGwh8CywPbfZene/IbrKRUTiF/Xd+i4Htrn7X5rZR4FrgE9PrTSz9wOL3X2ZmXUBPzeze4BTgG+5+59GXK+ISNOIOrCXA9flHt8PfKFg/aPAk7nHGaAdGANOBU4xs58AO4Er3f2VxpcrItI8GhbYZnYJ8JmCxb8BBnOPU8CC/JXuPgKMmNkc4A6yQyL7zOwZ4Al3f9DMPg7cCPx+qWP39fXQ0TG7brCfTPbGXULk1ObWV6/2tkomNCyw3f024Lb8ZWb2XWDqHegF9hbuZ2Z9wD3Aw+7+ldziDcBw7vF64K/KHXtgYLjc6pYz2/41bVCbZ4Ma/tX0ituElAnl2hP1VSKbgXNzj1cBG/NX5k5K/hi43d3/c96qW4ELco8/ADzR4DpFRJpO1GPYNwN3mNkmYBRYA2Bm15HtVb8XeCtwmZldlttnLXA1cLuZXQG8Blwacd0iIrFLZDKZuGuou/7+VOs1qozZ9lUZ1ObZoIYhkUSlbULKhHLt0cQZEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEB1RHszM5gJ3AYuAFHCxu/cXbHMfcAQwBux391Vmthj4JpABtgN/4u6TUdYuIhK3qHvYlwPb3P0M4E7gmiLbLAaWu/sKd1+VW/ZV4Jrcfgng/EiqFRFpIlEH9nLggdzj+4GV+SvN7LeAw4HvmdkmM/twbtWpwE9K7SciMhs0bEjEzC4BPlOw+DfAYO5xClhQsL4TuB64AVgIbDazx4CEu2fK7HeQvr4eOjraZ1B9eJLJ3rhLiJza3Prq1d5WyYSGBba73wbclr/MzL4LTL0DvcDegt1eBf7e3ceBnWa2FTAgf7y62H4HGRgYnkHl4Ukme+nvT8VdRqTU5tZXbXurCfWQMqFce6IeEtkMnJt7vArYWLB+JfBtADObB5wIPA1sNbMVZfYTEWl5kV4lAtwM3GFmm4BRYA2AmV0H3OPu95vZOWa2hWyv+i/cfZeZ/UfgFjPrJBvg90Rct4hI7BKZTKbyVoHp70+1XqPKmG1flUFtng1qGBJJVNompEwo1x5NnBERCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRAKhwBYRCYQCW0QkEApsEZFAKLBFRALREeXBzGwucBewCEgBF7t7f976DwFX554mgOXAicBc4HvAc7l1N7v73VHVLSLSDCINbOByYJu7/6WZfRS4Bvj01Ep3fwB4AMDM/hzY7O5Pm9mlwFfd/fqI6xURaRpRB/Zy4Lrc4/uBLxTbyMyOAy4C3pVbdGp2sZ1Ptpf9Z+6eanCtIiJNpWGBbWaXAJ8pWPwbYDD3OAUsKLH7VcDX3D2de/4YcKu7P2Fmnwe+BPynUsfu6+uho6N92rWHKJnsjbuEyKnNra9e7W2VTGhYYLv7bcBt+cvM7LvA1DvQC+wt3M/M2oAPA5/PW7ze3ae2XQ/cWO7YAwPD06w6TMlkL/39s+sLh9rc+qptbzWhHlImlGtP1FeJbAbOzT1eBWwsss2JwDPuvj9v2Q/N7N25xx8AnmhciSIizSnqMeybgTvMbBMwCqwBMLPrgHvc/THAgF8U7Hc58HdmNgq8CnwyupJFRJpDIpPJxF1D3fX3p1qvUWXMtq/KoDbPBjUMiSQqbRNSJpRrjybOiIgEQoEtIhIIBbaISCAU2CIigVBgi4gEQoEtIhIIBbaISCAU2CIigVBgi4gEQoEtIhIIBbaISCAU2CIigVBgi4gEQoEtIhIIBbaISCAU2CIigVBgi4gEQoEtIhIIBbaISCAU2CIigVBgi4gEQoEtIhIIBbaISCAU2CIigeiI46Bm9rvAR9x9TZF1lwGfAsaBa939+2Z2JLAOmAu8DKx19+EoaxYRiVvkPWwzuwH4SrFjm9lRwJXAe4FzgK+YWRfwRWCdu58BbCUb6CIis0ocQyKPAJeXWPduYLO7p919ENgBnAQsBx7IbXM/sLLhVYqINJmGDYmY2SXAZwoWr3X3u81sRYnd5gODec9TwIKC5VPLSkomexM1Fxy4ZLI37hIipza3vnq1t1UyoWGB7e63AbfVuNsQkP8O9QJ785bvz1smIjKrNNtVIo8BZ5hZt5ktAN4ObAc2A+fmtlkFbIypPhGR2DRFYJvZVWb279z9VeDrZAN5A/B5dx8BrgU+amabgWXA38VXrYhIPBKZTCbuGkREpApN0cMWEZHKFNgiIoFQYIuIBCKWqekyM2Y2F7gLWET2uvSL3b2/YJv7gCOAMWC/u6+KvNAZMrM24CbgZCANXOruO/LWH3Ibg1gKraMq2vx1sjOBU7lF5+cmmQXPzN4D/I27ryhYfh7Z2c7jwO3ufksM5TUF9bDDdDmwLTdV/07gmiLbLAaWu/uKEMM6ZzXQ7e7LgKuB66dWlLmNQehKtjnnFOCc3Pu6ooXC+rPArUB3wfI5wNeADwJnAp/MvfezkgI7TGWn6pvZbwGHA98zs01m9uGI66uXA+109y3AaXnrSt3GIHQl25zrfZ8AfMPMNpvZJ+IpsSGeB36vyPK3AzvcfcDdR4FNwBmRVtZEFNhNzswuMbPt+f+RnZpfbqp+J9me2WqyfwRfM7NFkRVdP4W3Kpgws44S6yresiAQ5dp8GHAj8AfAh4ArzKwVPqRw9++QHb4r1Krv87RoDLvJFZvib2bf5fUp/MWm6r8K/L27jwM7zWwrYMDOBpdbb4W3KmjLtanYula5ZUG5Ng8DN0zdWtjMNpAd634q2hIj1arv87Sohx2mSlP1VwLfBjCzecCJwNORVVc/B9ppZqcD2/LWlbqNQejKtXkJsMnM2nNju8uBn0ZfYqSeBk4ws4Vm1gm8D3g05ppiox52mG4G7jCzTcAosAbAzK4D7nH3+83sHDPbAkwCf+Huu+Ird9rWA2eb2SNAAlhrZleRHdO8L3fFxEayHY+p2xiErlKb/xHYQnb44E53/3mMtTaMma0B5rn7N3Lt/yHZ9/l2d38p3urio6npIiKB0JCIiEgFLC/KAAAAy0lEQVQgFNgiIoFQYIuIBEKBLSISCAW2iEggFNgiIoFQYIuIBEITZyR4ZnYlcAGwguzd+/4XsNTd98VZl0i9qYctreBGsjM6Lyd7i84/UlhLK9JMR2kJZvYWsvcSucnd/zzuekQaQT1saRVvInvrzVPMLBF3MSKNoMCW4OXuSHgLcB6wn+zQiEjLUWBLK7gO+N/u/q/AnwBfzA2RiLQUjWGLiARCPWwRkUAosEVEAqHAFhEJhAJbRCQQCmwRkUAosEVEAqHAFhEJxP8H2o4Yz6vLd90AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Add some scatter in the 2nd dimension.\n",
    "data['y'] = gen.normal(scale=0.04, size=len(data))\n",
    "sns.jointplot('x', 'y', data, stat_func=None, size=5);\n",
    "plt.ylim(-1, +1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The scatter adds new information in a second dimension, but we can approximately ignore it under two assumptions:\n",
    "- The relative scaling of the x and y columns is meaningful (which is almost certainly not true if these columns have different dimensions - recall our earlier comments about normalizing data).\n",
    "- The origin of the scatter is due to measurement error or some other un-informative process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "What if we also rotate x and y so the data has some size in dimensions?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWYAAAFgCAYAAACIf9MLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuUnHWd5/F3V1+q0+nupJIUKybjhYT+MrMMIRgYImEIGFQYOWYPKk48XhAQ0DmuMC6j6310hjOwOOrsoEeEEWXR4AVFR2FlIkoCbAQTAw75hsYbBGI6nUpf0unq6/5RVUmlUl1d3VX11FNdn9c/VD2/qqe+9aT59q+/z+/SMDk5iYiIhEek2gGIiMixlJhFREJGiVlEJGSUmEVEQkaJWUQkZJqqHUCpenoGSh5WEou1kUgMlSOckimW/BRLfvUUSzze0VCxk4eMesxAU1NjtUM4QrHkp1jyUyxzU833mKWyHtqxp9ohANDR3srA4PCR5+tOX1rFaEQqSz1mEZGQUWIWEQkZJWYRkZBRYhYRCRklZhGRkFFiFhEJGSVmEZGQUWIWEQkZJWYRkZDRzD+pSdWckZg7CzFDsxGlXNRjFhEJGSVmEZGQUSkjRB7asWfKP5OroaO9tdohiNQl9ZhFREJGiVlEJGSUmEVEQkaJWUQkZJSYRURCRolZRCRkNFxOpEyqMRuxmOGVmpFYe9RjFhEJGSVmEZGQUSlDZI4LqsQyk1mrKq8Uph6ziEjIKDGLiIRMXZYycv+0C9PCQSIi6jGLiISMErOISMgoMYuIhIwSs4hIyCgxi4iEjBKziEjIKDGLiISMErOISMgoMYuIhIwSs4hIyCgxi4iEjBKziEjIKDGLiISMErOISMgoMYuIhEzD5ORktWMQEZEs6jGLiISMErOISMgoMYuIhIwSs4hIyCgxi4iEjBKziEjIKDGLiISMErOISMg0VTuAUvX0DJQ8QyYWayORGCpHOCVTLPkplvzqKZZ4vKNhuteUIx8EpdD3UY8ZaGpqrHYIRyiW/BRLfoplblJiFhEJGSVmEZGQUWIWEQkZJWYRkZBRYhYRCRklZhGRkFFiFhEJGSVmEZGQUWIWkUAkR8fZlxgiOTpe7VBCr+anZItIuI1PTLBpczfbd/dwoD/Jos4oq7riXHbBChoj6hvmo8QsIhW1aXM3Dz7+/JHnvf3JI883ru+qVlihpl9XIlIxydFxtu/uydu2ffd+lTWmoB6ziFRM32CSA/3JvG0HBobpG0xyQqytbJ/30I49JZ9j3elLyxBJadRjFpGKWdAeZVFnNG9bA/DAtj8wPjERbFA1QIlZRCom2tzIqq543raJSfjp9hfYtLk74KjCT4lZRCrqsgtWcP4ZS4lMsSy8as3HU2IWkYpqjER43Zl/wsQUe4sk0rVmOUqJWUQqbkF7lMVT1JpjHa0saM/fVq8CHZVhZhHgVmAlkASudPfurPYPAn8NTAD/6O73BhmfiFRGptacPZ45Y1XXEqLN2pYqW9DD5TYAre6+xszOBm4B3ghgZguB9wMrgPnADkCJWWSOuOyCFUCqppwYGCbW0cqqriVHjstRQSfmtcD9AO7+mJmtzmo7BPyeVFKeT6rXLCJzRGMkwsb1XVx63nL6BpMsaI+qpzyFoBNzJ9CX9XzczJrcfSz9/DngP4FG4MZiThiLtZVld954vKPkc5SLYslPseRXi7Esq9Dnz29rIVLi+hthuJ5BJ+Z+IPtbR7KS8kXAicAr088fMLOt7r6t0AkTiaGSg4rHO+jpGSj5POWgWPJTLPnVUyzFJMxDQyMlf05Q17PQ9wl6VMZW4GKAdI35yay2BHAYSLr7MHAQWBhwfCJSBlriszRB95jvBS40s0dIzci83MyuB7rd/T4zWw88ZmYTwBbgJwHHJyIl0BKf5RFoYnb3CeCanMO7sto/AXwiyJhEpHy0xGd56FeYiJTF8MiYlvgsEyVmESmLRP/US3xq2vXMKDGLSFnEOqde4lPTrmdGiVlEyqK1pWnKJT417XpmtIOJiJSNpl2XhxKziJSNpl2XhxKziJRdtLmxrHv51RvVmEVEQkaJWUQkZJSYRURCRolZRCRklJhFpChaMS44GpUhIgVpxbjgKTGLSEFaMS54+nUnUucKlSiSo+NaMa4K1GMWqVPFlCj6BqdfMU4TScpPPWaROpUpUfT2J5nkaIli0+buI69Z0K4V46pBiVmkDhVboog2N2rFuCpQKUOkziRHx/nNnr6iSxRaMS54SswidSK7ptzbnyTSAJOTx78ut0ShFeOCF2hiNrMIcCuwEkgCV7p7d1b7RRzdjPWXwPvcPc+PjojMVO6wt4kp/s+aqkShFeOCE3SNeQPQ6u5rgA8Bt2QazKwDuBl4g7ufDfwOWBJwfCJzUqGacqQBGhpgcWcr61cvU4kiBIIuZawF7gdw98fMbHVW26uBJ4FbzOwk4Cvunv8nSURmpNCwt8lJ+OBbT+ekpQtUogiJoBNzJ9CX9XzczJrcfYxU7/h84HRgEHjYzB51992FThiLtdHUVPoPUzzeUfI5ykWx5KdY8ismlo4F84jH5rEvcfj498fmcdbKpbS2lJ4Oqn1d5re1EClxmni1vwMEn5j7gexvHUknZYBe4BfuvhfAzH5OKkkXTMyJxFDJQcXjHfT0DJR8nnJQLPkplvxmEstpyxcfU2POPj7Qd5hSv1Glr0sxCfPQ0EjJnxPUv22h7xN0Yt4KXALcY2ZnkypdZDwBnGpmS4CDwNnAbQHHJzJnadhb7Qg6Md8LXGhmjwANwOVmdj3Q7e73mdmHgQfSr73H3Z8KOD6ROUvD3mpHoInZ3SeAa3IO78pq/ybwzSBjEqk3GvYWfpqSLSISMkrMIiIho8QsIhIySswiIiGjRYxERLI8tGNPwfZ1py+teAzqMYuIhIwSs4hIyCgxi4iEjBKziEjIKDGLiISMErOISMgoMYuIhIwSs4hIyCgxi4iEjBKziEjIKDGLiISMErOISMgoMYtUWHJ0nH2JIZKj43mfi+TS6nIiFTI+McGmzd1s393Dgf4kC9ubaW1pJjk6RmJghEWdUVZ1xbnsghU0RtRHkqOUmEUqZNPmbh58/PkjzxODo8Dokee9/ckj7RvXdwUdnoSYfk2LVEBydJztu3uKeu323ftV1pBjBNpjNrMIcCuwEkgCV7p7d57X/DvwfXf/UpDxiZRL32CSA/3Jol7b2z/Mgf5hTlw8/8ix5Og4fYNJFrRHiTY3VipMCamgSxkbgFZ3X2NmZwO3AG/Mec1ngEUBxyVSVgvaoyzqjNJbZHJ+8Inneftr7bi6tOrQ9SnoxLwWuB/A3R8zs9XZjWb2JmAC+HGxJ4zF2mhqKr1HEY93lHyOclEs+dVaLOesXMp9D/+mqPP9+rcH6Fgwj6//6Olj6tKZOnTbvBau2vDns44lKNWOZX5bC5EK/wIL4jsGnZg7gb6s5+Nm1uTuY2Z2KrAReBPw8WJPmEgMlRxUPN5BT89AyecpB8WSXy3GcsmalzF0eIQtO19keKRwDXn/wcM83d3D1l/l329u669e4KKz/uS4skYtXpdSzj+dQ0MjFfv8jHJ9x0LfJ+i/jfqB7Ggi7j6WfvwOYCmwGXgXcL2ZvT7Y8ETKIzk6Tm/fMJeet5z/9b5zOOfUl7CoMzrl62MdrTA5OWVdOjEwTN9gcWURqX1B95i3ApcA96RrzE9mGtz9hsxjM/sksNfd7w84PpGSTFUjftfFpzA2PsnXH3AeeWrvce9b1bWEeKxtyrp0rKOVBe1TJ3aZW4JOzPcCF5rZI0ADcLmZXQ90u/t9AcciUna5Y5dzxypffvEptLU2sX33fhIDw8Q6WlnVteTIzb1VXfFj3p+xqmuJRmfUkUATs7tPANfkHN6V53WfDCQgkTIaGBrh8V378rZt372fS89bTrS5kY3ru7j0vOV5h8NddsGKI6/PTdxSPzTzT6REmfLFE7t6ODiY/+ZTpkZ8QqwNgGhz45HH2RojkYKJW+qDErNIiXLLF/nMtEY8VeKW+qAR6yIlKHbqtWrEMhPqMYuUYLqp1wvbW1h9ygmqEcuMKDGLlKDQ1OtYe5RPvvtMOtpaqhCZ1DKVMkRKEG1uZFVXPG/bq06JKynLrKjHLFIiDXGTclNiFimRhrhJuSkxi5SJhrhJuajGLCISMkrMIiIho8QsdS85Os6+xJD23ZPQUI1Z6pa2cZKwUmKWujXdEp0i1aJugdSlQmtcbN+9X2UNqSolZqlLhda4yN3GSTVoCZpKGVKXCq5xkV6ic6oa9N+8ZVUVIpZ6oh6z1KVCa1xklujM1KB7+5NMcrQGfccPfh1ssFJ3lJilbl12wQrWr17G4s5WIg2wuLOV9auXcdkFKwrWoB976kWe3zeg0oZUjEoZUrcKrXHR2zc0ZQ16X+IwH7/jFyzW8DqpkEATs5lFgFuBlUASuNLdu7ParwPemn76I3f/VJDxSX3Kt8ZFoRp0hobX1aeHduwp+rXrTl86q88I+tf8BqDV3dcAHwJuyTSY2UnA24BXA2uA15rZaQHHJwIUrkHn0vA6KbegE/Na4H4Ad38MWJ3V9hzwencfd/cJoBkYDjg+ESA1RO78VUs5/4ylLO5spaFh6tfmDq8TKVXQNeZOoC/r+biZNbn7mLuPAvvNrAG4Gdju7runO2Es1kZTU+lr38bjHSWfo1wUS35BxDI+PsEdP/g1jz31Ij0HD7NkQSunrljMX736Fdx81xP0HDy+r7Bk4TyWv2IxrS3VuWVTb/9GhcxvayESonr/bK9H0D9J/UB2pBF3H8s8MbNW4A5gAHhvMSdMJIZKDioe76CnZ6Dk85SDYskvqFjufnD3MdO0ew4O87Nf7uH/PbWX+MJ5ed9z2vLFDPQdphpXqp7+jYpJcoeGRir2+bNR6HoU+j5BJ+atwCXAPWZ2NvBkpiHdU/4+sNnd/ynguEQKDpEbHhnnuX2D/MkJ7SRHx9l/8LC2kJKKCTox3wtcaGaPAA3A5WZ2PdANNALnAVEzuyj9+g+7+6MBxyh1qtA07Yyh4TE+/7freP6Fg9pCSiom0MScvql3Tc7hXVmPWwMMR+QYxQyRSwwMMzQ8pi2kpKLCUyUXqbJihsjFOlqJdUYDikjqlWb+iWTJ1Iu37HyR4ZHjxyav6lpCa0tTVW70Sf1QYhbJkpmmveHck/jGT3az6w8JEgNJ3eiTQCkxi+TRFm3iijf8GcnR8ePW0RCpNCVmkQLyraMhUmm6+SciEjJKzCIiIaPELCISMkrMIiIho8QsIhIySswypyVHx9mXGNJC9lJTNFxOatpU44zHJybYtLmb7bt7ONCfZJH255MaosQsNSk78fb2J1nY3sKqk5ew8cIuGiMRNm3uPmZdZe3PJ7VEXQepSZnEm1kJ7uDgCD/d/gJ//9XHGUqOTbmusvbnk1owbWI2szODCESkWIUWtH9u3yB3PbBrynWVtT+f1IJiShk3mdkS4GvA1919b4VjEimobzBZcM3kXX84SKyjhQMDx28zFOtoZUG7lu2UcJu2x+zu5wNvAKLA/zWzH5rZm8ysueLRieSxoD3KwvaWKdv7Do1wyssX5W1b1bVEixFJ6BVVY3b335PqMd8NnAq8H3jKzP5bBWMTySva3Miqk5dM2b6oo5WNF57M+tXLWNzZSqQBFne2sn71Mi3bKTVh2lKGmV0BvAM4EbgTWOvuz5vZS4HtpPbxEwnUxgu7eOb5Pp7vOXRc28qTF9MWbWbj+i4uPW+5lu2UmlNMjfk84BPu/lD2QXd/wczeW5GoRKbRGIlgL1uYNzE3ZD3Wsp1Si6ZNzO7+jgJt35npB5pZBLgVWAkkgSvdvTur/SrgamAM+Iy7/3CmnyFzX3J0nB3P7M/btuOZXt60blw9ZKlZ1RjHvAFodfc1wIeAWzINZvYSUvXrc4DXATeamW6h14GZTp3uG0xqSJzMWdWY+bcWuB/A3R8zs9VZbWcBW909CSTNrBs4DfhF8GFKEGYzdTo5Os7I6DiLOqN5h81pSJzUumok5k6gL+v5uJk1uftYnrYBYEGhk8VibTQ1lf4nazzeUfI5yqWeYrnte0/mnTrdNq+Fqzb8OQDDI2O8uP8Qne1R7n7AeeypF+k5eJjWlvz/7uesfCnLXrqwonHX07/RTFQ7lvltLURCtBbKbK9HNRJzP5AdbSSdlPO1dQAHC50skRgqOaB4vIOennBsSF9PsSRHx9n6qz1527bs2MOqFYv5+Y497Hy2lwMDSaLNEYZHJo685nAyVfZobWlkZHT8yE7Wl6x5WUXjrqd/o5modCzFJLlDQ8dPKqqmQtej0PepRmLeClwC3GNmZwNPZrVtA/7BzFpJTWj5U+Cp4EOUIBSqEx8YSPKpfzu2gpWdlLO1RZv4n29/FfGF83TDT+aEaiTme4ELzewRUiObLjez64Fud7/PzL4APEzqxuRH3H24CjFKABa0R6esE8/EwcEkLU0RJWWZMwJPzO4+AVyTc3hXVvttwG2BBiVVEW1uZFVX/Jga82zoZp/MNVqPWaoqM0V6++79HOgfZnIW59D6FzLXKDFLVTVGIkemTvckhvj8t3dOW9rIvdmn9S9krlFillCINjey7ISOgqWNxZ2pRLzh3FcyODSq9S9kzlJillDJLm0kBoaJdbRy2orFvOVCg7Gj06zbolp1VuYuJWYJlezSRvaqcGEarytSaUrMUhbZu1UDJS+1qVXhpJ4pMUtJcte6iLY0ApMMj0ywuIh1L0TkeErMUpLMbtUZwyNHV4fLrHsBsHF9V+CxidQqdWNk1grtVp1t++79R5bznOnyniL1SD1mmbVCa11kSwwMc6B/mJ9u3zOj5T1F6pUSs8xasWtddM5v4YFtv+fnv9p75JjKHFIPHtpx/OqJ605fOu371FWRWcusdTGdg4MjbNm5N29bdplDRFLUY5aS5E4IaUkPj8u+CQgwMcUiGJltoDQ0TuQoJWYpSvY45eyxyfkmhIyMjvPJO35Booh991qaG2lva6lk6CI1R4lZCip2T77sCSF9g0kOFrkZ6vDION97+DeqM4tkUY1ZCsqMU+7tTzLJ0Zt2mzZ3Tzn0LXNTsFiqM4scSz1mmdLwyNiU45S37Hxxyl50tLmRU14WY+tT+W/45VKdWeRY6jHLlBL9U49THh4Zz9uLznjLa06mscifLu1AInIsJWaZUqxz9iWJH2z9LeP59049jnYgETmWErNMqbWlqahxyhmZkkShqdqNkQYWdbQQaUgtfL9+9TLtQCKSQzVmKSh3nPLC9ihDybHjxinD0ZJEoanak5OTfOAtp9PSFNEOJCJTCDQxm9k84C7gBGAAeKe79+S85mZgbTq2L6d3zZYqyTdO+Ts/ezbv9k+ZkkShqdqxjlbiC+cpIYsUEHQp41rgSXc/F/ga8NHsRjM7H1jh7mtIJee/M7NYwDFKHplxytHmRi67YAXrVy9jcWdr3pJEoanaqieLTC/oUsZa4Kb04x8DH8tpfxTYkX48CTQCo8GEJsWaavunbPn27tOO1iLFaZicnGIRgxKZ2RXAdTmH/wj8jbs/bWYR4A/uvizPe5tJ9ah3uvuNhT5nbGx8sqlJPbCwGh4ZI9GfJNYZpbVFtzSkJA3TveBHW38zGQn5MrKvX/OKzMMpv0/F/k9x99uB27OPmdl3gY700w7gYO770qWLbwMPTZeUARKJoZJjDdNGn3MxliZgoO8wpZxpLl6XcqinWOLxjmlfc2hopGKfXy6Za1To+wT9q2UrcHH68UXAw9mN6ZuD/wHc4e6fDji2uqPdRETCKei/Lb8I3GlmW4ARYCOAmd1Eqpd8DnAScJWZXZV+z+Xu/tuA45zTil2YSESqI9DE7O5DwJvzHL8h/XAb8M9BxlSPcjdQ1W4iIuGi7lGdKTQrT6u8iYSDEnOdKTQrLzOlWkSqS4m5zhRaK7nYVd5001CksjSwtM5kZuUVmlI9Fd00FAmGEnOdyN6zb7az8nTTUCQYSsxzXKFebqEp1bmmu2l46XnLtQaGSJkoMc9x0/Vyi93OqZibhtoaSqQ8VBicw4aSo2zZ+WLetpkOjSvHTUMRKY4Scw2bbnTE3T95Ju+C9jDzoXFaylMkOCpl1KBiRkckR8fZ9fsDU54j1hE90svNvjFYKMFqKU+RYCgx16BiRkf0DSZJDEy90tYpL4vR1NjA3Q/uLnr4WzHrMItI6VTKqDHFTqkuVBNubWnkry/sOpLge/uTTHI0wW/a3F0whuzdTESk/JSYa0yxU6oL1YTXnnYijZEGrZkhElIqZdSY6TY6zR4dUagm3Ns3PG2CP25rGREJhBJzjZnJlOrGSIRLz1vOX552IjQ0HLM79UwSvIgES4m5BhUzOmK6kRulrJkhIpWlxFyDihkdUczIDQ1/EwknJeYalhkdkavYdS00/E0knDQqYw6a6WL4Gv4mEi5KzHOQ1rUQqW2BljLMbB5wF3ACMAC8092P+5vbzNqAR4APufv9QcYYRsVOmc7QjT2R2hZ0jfla4El3/6SZvRX4KPDf87zuX4HJQCMLoVJ2DNGNPZHaFXRiXgvclH78Y+BjuS8wsw+S6i03BBhXKJWyY4hu7InUroolZjO7Argu5/Afgb704wFgQc57XgOc7O5Xm9k5xXxOLNZGU1PpCSce7yj5HOUSj3cwPDLGzmd787bvfLaXqy+dR2tLcf98pczgC9t1CQvFkl+1Y5nf1kIk5PtPFnONKpaY3f124PbsY2b2XSATVQdwMOdtVwAvN7OHgFOAM8xsr7vvmOpzEomhkmONxzvo6Rko+TzlkIllX2KInsThvK/Zf/Awz/6ut+I7hoTxuoSBYsmv0rEUk9AODU29omJYZK5Roe8TdCljK3AxsA24CHg4u9HdN2Yem9lXgW8WSspzmaZMi9SvoPv8XwT+q5ltAd4DfArAzG4ys7MCjiWUhkfG2Jf+K0A7hojUp0B7zO4+BLw5z/Eb8hx7VxAxhUVmBMbOZ3vpSRxmUWeUlScv4TWvWsqOZ3o1skIkYOtOX1q1z9aU7JDINwJj8xN7WL96GZ+56i80skKkjoT79mWdmG5tC0BTpkXqiBJzCMx0bQsRmduUmENAa1uISDYl5hAotD+fRmCI1B/d/AuJzEiLnc/2sv/g4bwjMGa6mJGI1CYl5pDIrG1x9aXzePZ3vcck31IWMxKR2qPEHDKtLU3HTbUuZTEjEak96m5VWHJ0nH2JIZKj47N+f6GhdLM9r4iEl3rMFVKu8kMxQ+kqvZiRiARLPeYKyZQfevuTTHK0/LBpc/eMzqOhdCL1R4m5AspZftBQOpH6o1JGBZS7/KBtokTqixJzBSxojxLraOHAwPGLdi9sj864/KBtokTqi0oZFRBtbmT+vJa8bfPnNc86qUabG7WYkUgdUGKugOToOEPDo3nbhoZHNcRNRApSYq6AwjXmpFaLE5GClJgrQEPcRKQUSswVoCFuIlIKjcqoEA1xE5HZUmKuEA1xE5HZCjQxm9k84C7gBGAAeKe79+S85l3AtUAj8H13/3SQMZZbZoibiEixgq4xXws86e7nAl8DPprdaGbL069ZB5wFtJhZc8AxiohUVdCljLXATenHPwY+ltO+HngcuBM4EfgHd88/IDgtFmujqan0EkE83lHyOcpFseSnWPJTLEfNb2shUqbNI6r5XSqWmM3sCuC6nMN/BPrSjweABTntS4C/BF4NzAO2mtmZ7n5wqs9JJIZKjjUe76CnZ6Dk85SDYslPseRXT7EUkygPDR2/DMJsVfq6Fvo+FUvM7n47cHv2MTP7LpCJpgPITbi9wEPuPgAMmNl/Al3AtkrFKSISNkGXMrYCF5NKtBcBD+dpf5+ZtZK6+fdnwMwWMBYRqXFBJ+YvAnea2RZgBNgIYGY3Ad92921mdjupBN0AfNrdDwQco4hIVQWamN19CHhznuM3ZD3+HPC5IOOaieTouMYli0hFaYJJkcq1h5+IyHSUmIuU2cMvI7OHH8DG9V3VCktE5iB19YpQzj38RESmo8RchGL28BMRKRcl5iJofWURCZIScxG0vrKIBEk3/4qk9ZVFJChKzEXS+soiEhQl5iLkTirR+soiUklKzAVoUomIVIMScwGaVCIi1VB33b7k6Dj7EkPTTgrRpBIRqZa66TEXKkvkU8ykEtWaRaQS6qbHnClL9PYnmeRoWWLT5vzLPWtSiYhUS10k5unKEsMjY8cd16QSEamWuihlTFeWSPQn814ITSoRqS3rTl9a7RDKoi4Sc6Ys0ZsnOcc6Wol1RhnoO3xcmyaViEg11EUpY7qyRGtL4d9PmUklSsoiEoS66DGDyhIiUjvqJjGrLCEitSLQxGxm84C7gBOAAeCd7t6T85rPAmuBCeBv3X1rOWPQWhciEnZB15ivBZ5093OBrwEfzW40s5XAq4G/AN4OfCHg+EREqi7oxLwWuD/9+MfA+pz2PcAQEAU6gdHgQhMRCYeKlTLM7ArgupzDfwT60o8HgAU57WOkShi70m1XTfc5sVgbTU2l14rj8Y6Sz1EuiiU/xZKfYjmqXPmg2iqWmN39duD27GNm9l0g8y/XARzMeds7gL3A69LtW8zsUXffM9XnJBJDJccaj3fQ0zNQ8nnKQbHkp1jyq6dYikn65cgHQSn0fYIuZWwFLk4/vgh4OKc9AQy6+zipHnUSaA8uPBGR6gt6uNwXgTvNbAswAmwEMLObgG8DdwPnmNkjQCPwf9zdA45RRKSqAk3M7j4EvDnP8Ruynl4TXEQiIuFTF1OyRURqScPk5GS1YxARkSzqMYuIhIwSs4hIyCgxi4iEjBKziEjIKDGLiISMErOISMgoMYuIhEzd7GAy3SL9ZvZ64EPppw2klig9FZgH/AB4Jt32RXffVMlY0q+5D1hMaunTw+5+kZmtAL4KTAJPAe9z94kAYrmZ1PVoAr7s7reZ2SJgdzoOgHvd/fOzjCEC3AqsJLU+ypXu3p3VfhVwNanVBz/j7j80syWkpvDPA14ALk/PLC1JEbFcB7w1/fRH7v4pM2sAnufoz8ij7v7hCsfxBeAcUv9mAG8Emgn4mpjZ6cDnsl5+NrAB2EaZfj7qUT31mAsu0u/u97v7OndfB/wQ+Cd3fxoJ1dE7AAAErUlEQVQ4A/hspq3UpFxMLGkrgLXpz7wofeyzwEfT72sg9T9jRWMxs/OBFe6+hlRy/jszi5G6Lt/Iui6l/E+3AWhNf8aHgFuyPv8lwPtJJaHXATeaWRT4OHB3Ou7tpBJ3ORSK5STgbaQ2c1gDvNbMTgOWA7/MuhYlJeXp4kg7A3hd1mf2UYVr4u47sv6/+Vfgu+5+P+X9+ag79ZSYp1ukHwAzW0Zq95RPpQ+9CvgrM/u5md1uZuVYcLZgLGb2X4CFwA/MbIuZvSErlp9N9x3KGQvwKPDu9ONJUotLjaZjOcPMfmZm3zKzE8sRg7s/BqzOajsL2OruyXTy6QZOKyLuSsTyHPB6dx9P/6XSDAyTuhZLzeynZvYjM7NKxpHuwZ4MfNnMtprZu3PfQ3DXJBPTfFL/z7w/faicPx91Z06WMma5SH/G9cA/u3sy/Xwb8BV3f8LMPgJ8AvhghWNpIdUr+TywCNhqZtuABnefLPC+ssfi7sPAsJk1A3eSKmUMmtku4Al3f9DM3gb8C/CmmcSTpTMrBoBxM2ty97E8bZkYs4/P+FrMJhZ3HwX2p0sXNwPb3X13uld/o7t/y8zWkioNnVmpOID5pK73Z0n9ovypmT1OFa5J1rErgG+5+/7083L+fNSdOZmYZ7lIf6Yn8gbgI1mH73X3zGvvJfUDVulY9gJfSv/g7zOz7YCR2t2FAu+rRCykSxffBh5y9xvThzeT2gYMUtfl72cSS47+rBgAIln/0+e2ZWLMHD88VdwViAUzawXuIJX43ps+/Dip+jfuvsXMlppZ9i/RcscxBHw+Uz82s82k6r9VuSZpb+PYxFvOn4+6U0+ljOkW6YfUzb5d7n4469gDZnZW+vFrgCcCiGU9cA+AmbWn43oa2G5m6wq8r+yxpG8O/gdwh7t/OqvpK8Cl6celXpcjMZjZ2cCTWW3bgHPNrNXMFgB/SuqGUjH/nmWNJd1T/j7wK3e/Or2hA6T+ivpA+jUrgT+UmJQLxgF0kdrdpzH9l8xa4JdU4Zqkjy0Aou7+XNbhcv581J26WV3OzNpI/Sl+IulF+t19b2aRfnffZmZvBs5x9w9kve8M4H+n37MXeI+79wcQy+dI3eGeAG5y9++ZWRdwG6lSx9PAVVnJoSKxkLrp9glgR9bbLk//9w5SNyEPkbpT/+IsY8jc9T8tfb7LSSWCbne/Lz0q4z2kOhL/6O7fSdfh7yTVk9ufjvvQbD6/2FhIlQ2+ATyW9ZYPk/qz/S5Su+2MkRots6tScaSvyQ2k1jYfBb7m7l+qxjVJx3Im8BF335D1nldSpp+PelQ3iVlEpFbUUylDRKQmKDGLiISMErOISMgoMYuIhIwSs4hIyCgxi4iEjBKziEjIzMkp2TI3mdn7Sc0mW0dq4su/AavcfbCacYmUm3rMUkv+hdRMyGtJTfl9l5KyzEWa+Sc1JT3V9yngVnf/H9WOR6QS1GOWWvNyUiu7nZFeVEhkzlFilpqRXmnvNuASUktbXlvdiEQqQ4lZaslNwL+7+y+A9wEfT5c2ROYU1ZhFREJGPWYRkZBRYhYRCRklZhGRkFFiFhEJGSVmEZGQUWIWEQkZJWYRkZD5/zSzxrsekY3aAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "theta = np.deg2rad(30.)\n",
    "rotated = data.copy()\n",
    "rotated['x'] = np.cos(theta) * data['x'] - np.sin(theta) * data['y']\n",
    "rotated['y'] = np.cos(theta) * data['x'] + np.sin(theta) * data['y']\n",
    "sns.jointplot('x', 'y', rotated, stat_func=None, size=5);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The rotation does not change the effective dimensionality of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Finally, what if we put the data along a curve instead of a line?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAAFgCAYAAACfXUPCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt0XGd57/GvNJJGUnSxZI9LSEpTbPSUReLETgCncYhxzSEJpHg1cEKcRWkuQKGrgWSdw4FCSemh5RBWgJSWcgjJ4ZKGhktNCSWhDU44iSErxFaahIMfoRRKrliyZWlkWSNppPPH7FFG45nRSJrb1vw+a2VlZu+5PLMl/+bVu9/9vg1zc3OIiEjta6x2ASIiUhwFtohISCiwRURCQoEtIhISCmwRkZBoqnYB5TA0FC/J0JeennZGRiZK8VIrplpqtw5QLflUopZYrLOhrG9QQ9TCLqCpKVLtEuaplhPVSh2gWvKppVpWg1XZwpbyu//RZ6pdAp0drcTHJ+fvbz/rlCpWI1J+amGLiISEAltEJCQU2CIiIaHAFhEJCQW2iEhIKLBFREJCgS0iEhIKbBGRkFBgi4iEhK50lFWjmldfZl91mUlXYEqpqIUtIhISCmwRkZBQl0iNS/+ZX+hP7krr7GitdgkidUktbBGRkFBgi4iEhAJbRCQkFNgiIiGhwBYRCQkFtohISGhYn0iZVfoKzGKGgOrqy3BSC1tEJCQU2CIiIaEuEZE6VKlumqVeoauumsLUwhYRCQkFtohISKhLJEP2n4m1NOGSiIha2CIiIaHAFhEJCQW2iEhIKLBFREJCgS0iEhIKbBGRkFBgi4iEhAJbRCQkFNgiIiGhwBYRCQkFtohISCiwRURCQoEtIhISCmwRkZBQYIuIhETD3NxctWsQEZEiqIUtIhISCmwRkZBQYIuIhIQCW0QkJBTYIiIhocAWEQkJBbaISEgosEVEQqKp2gWUw9BQvCRXA/X0tDMyMlGKl1ox1VK7dYBqyacStcRinQ2LPaZUmVAJhT6PWtgFNDVFql3CPNVyolqpA1RLPrVUy2qgwBYRCQkFtohISCiwRURCQoEtIhISCmwRkZBQYIuIhIQCW0QkJBTYIiIhocAWkapKTCc5NDJBYjpZ7VJq3qq8NF1Eal9ydpY79w7SPzDEkbEEvV1RfuclPVz+uj7ao4qmXHRURKQq7tw7yL2PPD1///BYgn1PPM/+gUNs2/RiLtuxkUijOgEy6WiISMUlppP0Dwzl3Dc5Ncu9jzzNnXsHK1xV7VMLW0QqbnQ8wZGxRMHH9A8Mc+kFG4g2r3wCqfsffeaEbdvPOmXFr1tpVQlsM3s18Al33561/XrgaiD91fsu4FfA7cB6IA683d1zfzWLSCh0d0Tp7YpyuEBoj8QnGR1PsL6nvYKV1baKd4mY2fuBLwKtOXZvAf7Q3bcH/znwbuBxdz8f+Arw4cpVKyLlEG2OsLkvVvAxPZ2tdHdEK1RROFSjD/tJ4A/y7Dsb+KCZPWhmHwy2bQPuCW7fDewsc30iUgGX7djIznNOpbUld5fH5r51JekOWU0q3iXi7t8ys9Py7P5H4O+AMWCPmb0R6AJGg/1xoHux9+jpaS/ZxOmxWGdJXqcUVMuJaqUOUC35FKrlvZefzTt2TfGFbz/BY4NDHB6dZN2aNraefjJXXfIKIpHStClPam+hMWvESS0do2LVzElHM2sAPuPuo8H9fwE2kwrv9JHtBI4u9lqlWpIoFutkaChektdaKdVSu3WAasmn2Fqu2Pky3nzBSxkdT9DdESXaHOHIkWNFv8dijk1MnbCtVo5RtkKfp2YCm1RL+gkzezlwDNgB3AZMABcDDwMXAQ9UrUIRKZtoc0QnGBdR9cA2s91Ah7t/wcz+DLgPSAA/cPfvmdn9wJfN7EFgCthdvWpFRKqnKoHt7r8Etga378jY/lXgq1mPnQDeUsn6RERqka50FBEJCQW2iEhIKLBFREJCgS0iEhIKbBGRkFBgi4iEhAJbRCQkFNgiUjFav3Flqn6lo4isfrnWb9zcF9MyYEukwBaRssu1fmP6/u6dfdUqK3T01SYiZVVo/cb+gWF1jyyBAltEyqrQ+o3pZcCkOApsESmr9PqNuWgZsKVRYItIWRVav1HLgC2NTjqKSNldtmMjkOqzHolP0tPZyua+dfPbpTgKbBEpu0hjI7t39nHpBRsWLAMmS6PAFpGK0TJgK6M+bBGRkFBgi4iEhAJbRCQkFNgiIiGhwBYRCQkFtohISFRlWJ+ZvRr4hLtvz9p+OfA+IAk8BrzH3WfNrB8YDR72C3e/spL1isjqsf2sU6pdwrJVPLDN7P3A24BjWdvbgI8BZ7j7hJl9DXijmf0rQHa4i4jUm2p0iTwJ/EGO7Qngd919IrjfBEwCZwLtZvavZrbXzLZWqE4RkZrSMDc3V/E3NbPTgH9095zha2Z/Clwc/Hc6sBX4IvAy4G7A3H0m3+vPzCTnmpp02atInWhY7AHf2/cfc43ByjYXnntauetZqbyfp6YuTTezRuBGoA+41N3nzGwAGHT3OWDAzA4DJwNP5XudkZGJfLuWJBbrZGgoXpLXWinVUrt1gGrJpxK1xGKdiz7m2MTU/O1aOTb5FPo8tTZK5H8DrcCujK6Rq4CbAMzsxUAX8Fx1yhMRqZ6qt7DNbDfQATwCXA08AOw1M4CbgVuBL5nZg8AccFWh7hARkdWqKoHt7r8k1S+Nu9+RsStfi393uWsSEal1tdYlIiIieSiwRURCQoEtIhISCmwRWZHEdJJDIxMkppPVLmXVq/ooEREJp+TsLHfuHaR/YIgjYwl6u6Js7otx2Y6NRBrVFiwHBbaILMudewe595Gn5+8fHkvM39+9s69aZa1q+hoUkSVLTCfpHxjKua9/YFjdI2WiwBaRJRsdT3BkLJFz30h8ktHx3PtkZRTYIrJk3R1ReruiOff1dLbS3ZF7n6yMAltElizaHGFzXyznvs1964g2a7bMctBJRxFZlst2bARSfdYj8Ul6OlvZ3LdufruUngJbRJYl0tjI7p19XHrBBkbHE3R3RNWyLjMFtoisSLQ5wvqe9mqXURfUhy0iEhIKbBGRkFBgi4iEhAJbRCQkFNgiIiGhwBaRE2jK1NqkYX0iMk9TptY2BbaIzNOUqbVNX5kiAmjK1DBQYIsIoClTw0CBLSKApkwNg6r0YZvZq4FPuPv2rO2XAB8BZoDb3P0WM2sDbgfWA3Hg7e6e++82EVm29JSpmX3YaZoytTZUPLDN7P3A24BjWdubgU8Drwz27TOzu4DdwOPu/hdm9lbgw8B7K1u1SH1YzVOmbj/rlGqXsGLVaGE/CfwB8NWs7S8HBt19BMDMHgTOB7YBNwaPuRv488XeoKennaam0rQGYrHOkrxOKaiWE9VKHRDuWianZhgZS9DT08Z7Lz/7hftdUVpbVhYTtXBcTmpvqYk6Vqrige3u3zKz03Ls6gJGM+7Hge6s7eltBY2MTKywypRYrJOhoXhJXmulVEvt1gHhraXQuOumxkbio8dZyaeqxHEpJoiPTUzVzM9nMYU+Ty2Nwx4DMivtBI5mbU9vE5ES0LjrcKmlUSI/A15mZr1m1gK8BvgxsA+4OHjMRcADVapPZFXRuOvwqXpgm9luM3unu08D1wPfJxXUt7n7M8DfA68I+rTfCXy0etWKrB4adx0+VekScfdfAluD23dkbL8LuCvrsRPAWypZn0g9SI+7PpwjtDXuujZVvYUtItWRHnedi8Zd16ZaOukoIhW2msddr0YKbJE6FmlsZPfOPi69YAOj4wm6O6JqWdcwBbaIEG2OsL6nvdplyCLUhy0iEhIKbBGRkFBgi4iEhAJbRCQkFNgiIiGhwBYRCQkFtohISCiwRURCQoEtIhISCmwRkZBQYIuIhIQCW0QkJBTYIiIhocAWEQkJBbaISEgosEVEQkKBLSISEgpsEZGQUGCLiIRExdd0NLNG4HPAmUACuMbdB4N9ZwGfyXj4VmAX8DAwADwRbN/j7jdXrGgRkRpQjUV4dwGt7n6umW0FbgLeBODujwLbAczsLcCz7n6Pme0Evubuf1qFekVEakI1AnsbcA+Auz9kZudkP8DMTgI+Crwm2HQ2sMXMfggcAq519+cqVK+ISE2oRmB3AaMZ95Nm1uTuMxnbrga+4e7Dwf2DwH53v9fMrgA+C7w53xv09LTT1BQpSbGxWGdJXqcUVMuJaqUOUC351EItJ7W3sH/wcN79F557WuWKWYFqBPYYkPkTbMwKa4ArWBjIe4GJ4PYe4C8LvcHIyESh3UWLxToZGoqX5LVWSrXUbh2gWvKpRC3FfCEcm5gquL9WjhcU/jzVGCWyD7gYIOjDfjxzp5l1A1F3fypj8xeBS4Pbvwfsr0CdIiI1pRot7D3A68zsR0ADcKWZXQ8Muvt3gD7gl1nP+QBwm5m9BzgGXFPBekVEakLFA9vdZ4E/ztp8MGP/T0iNJMl8zi+A15a/OhGR2qULZ0REQkKBLSISEgpsEZGQUGCLiISEAltEJCQU2CIiIaHAFhEJCQW2iEhIKLBFREJCgS0iEhKLBraZvbIShYiISGHFzCVyo5mtA74CfNXdny9zTSIiksOiLWx3fy3wRiAK/KuZfdfM3mxmzWWvTkQWSEwnOTQyQWI6We1SpAqKmq3P3f/TzL4CzJCaae9a4K/M7APuvqecBYoIJGdnuXPvIP0DQxwZS9DbFWVzX4zLdmwk0qhTUfWimD7sq4O1FO8FIsA2d38NqelOP1/m+kTqRqHW8517B7n3kac5PJZgDjg8luDeR57mzr2DlS9UqqaYFvYFwA3ufn/mRnd/NlhQQERWYLHWc2I6Sf/AUM7n9g8Mc+kFG4g2l2YNU6ltiwa2u/9hgX3fKm05IvUn3XpOS7eeAXbv7GN0PMGRsUTO547EJxkdT7C+p70itUp1qfNLpEoS00mePhRn/8Ff59x/wIdITCfp7ojS2xXN+Ziezla6O3Lvk9WnGms6itS1dBfIAT/EkXj+1byPxBPzrefNfbEFrfC0zX3r1B1SRxTYIhWW3QWST2MDtEVT/0Qv27ERSPVZj8Qn6elsZXPfuvntUh8U2CIVVOgEYrbZOTiemKGzvYVIYyO7d/Zx6QUbGB1P0N0RVcu6DimwRSqo0AnEbGu7oif0T0ebIzrBWMd00lGkggqdQMy2uS+mVrQsUPEWtpk1Ap8DzgQSwDXuPpix/2+A84B4sOlNQDNwB9AGPAtc6e4TlaxbpBSizZG8JxDTWlsibNt0svqn5QTV6BLZBbS6+7lmthW4iVQop20BXu/uw+kNQYjf4e5fMrMPAO8CPl3JokVKJR3EB3yII/EEjQ2p/uq1XVF+5yU9XP66Ptqj6q2UE1Xjt2IbcA+Auz9kZuekdwSt75cBXzCz3wBudffbguf8dfCwu4PbCmwJpewTiG3RJo4nZnQiURZVjcDuAkYz7ifNrMndZ4CTgM8CnyI1b8l9ZvZI1nPiQHehN+jpaaepqTS/+LFYZ0lepxRUy4lqpQ5YvJbJqRlGxhL0dEVpbUn90zu1SrVUUi3UclJ7C40FJsmqhRqLUY3AHgMyj05jENYAE8DN6f5pM9tLqq87/Zzjwf+PFnqDkZHSdG/HYp0MDcUXf2AFqJbarQMK11LpmfbCclxK+R6LOTaR/wIloGaOFxT+PNUYJbIPuBgg6MN+PGNfH/CgmUWC+ba3AQcynwNcBDxQuXJFVkYz7UmpVCOw9wCTZvYjUv3Q15nZ9Wb2++7+M+AfgIeAHwJfcfefAh8D3mpm+4Bzgb+tQt0iS7bYTHtaiECWouJdIu4+S2oRhEwHM/bfCNyY9ZxfAxeWvzqR0hoameCwZtqTEtHYIZEyyOy3zkcz7clSKbBFyqCYCZ40054slQJbpMQWm+CptzPKFovpSkZZMgW2SIkVmuCpoQHe91/P5NRYR4WrktVAkz+JlFihCZ56O1uJrWmrcEWyWiiwRUosPcFTLuq3lpVQl4hIGWiFGCkHBbZIGWiFGCkHBbZIGWmFGCkl9WGLiISEAltEJCQU2CIiIaHAFhEJCQW2iEhIKLBFREJCgS0iEhIKbBGRkFBgi+SRmE5yaGRCy3hJzdCVjiJZKr3KuUix9NsnkqWYVc6zW9+TUzNqjUvZqYUtkmGxVc53nf/bfPuBXyxofbe3NjM5NcPw0Um1xqWsFNgiGQqtFjMSn+SOf/s5P3ri+flth8cSC1ZFT7fGAXbv7CtvsVJ31AQQyVBotZiezigH//NIUa/TPzCs7hEpuYq2sM2sEfgccCaQAK5x98GM/dcBbw3ufs/dP2pmDcDTwM+D7T929w9WsGypI+nVYnKteP47L+lZ0LouZCQ+yeh4QlOrSklVuktkF9Dq7uea2VbgJuBNAGb2UuAK4NXAHPCAme0BJoAD7n5JhWuVOpVvtZhd57+Ug78aWdAFkk9PZyvdHblb6iLLVenA3gbcA+DuD5nZORn7ngIudPckgJk1A5PA2cApZnYfcBy4zt29smVLPSm0Wky+1nc2rd0o5VDpwO4CRjPuJ82syd1n3H0aGA66QD4J9Lv7gJm9CPi4u3/DzLYBtwOvLPQmPT3tNDWV5h9LLNZZktcpBdVyonLXcWrW/Wt2ncEsDTw+OMTw0UnWrWmls72F+MQUh0cnWbemja2nn8xVl7yCSKR6p4hq5ecDtVHLSe0tNBYYtVMLNRaj0oE9BmQemUZ3n0nfMbNW4DYgDrwn2PwIMAPg7g+a2Slm1uDuc/neZGRkoiTFxmKdDA3FS/JaK6VaqltH+mKaA36II/EpGhtS/XbJ2Tle+uIurtl1Br98amS+NX7kyLGK1JVLrfx8oDK1FBO2xyamCu6vleMFhT9PpQN7H3AJ8PWgD/vx9I6gZf3PwF53/0TGc24ADgM3mtmZwK8KhbVIOaQvpkmbDX4DjwTD+NrbWth13mnVKU7qRqUDew/wOjP7EdAAXGlm1wODQAS4AIia2UXB4z8I/C/gdjN7A6mW9h9VuGapc4Uupkl76InnuOhVv6l+65C6/9FnFn3M9rNOqUAlhVU0sN19FvjjrM0HM2635nnqG8pTkcjiCl1MkzZ89LiG8UnZ6cIZkUUUupgmbd2aNg3jk7JTYIssIn0xTSFbTz9Z3SFSdppLRKQI6YtpDvgQR+IJGhtSJx7XBpM9XXXJK6o6MkTqgwJbpAjZF9O0RZs4npiZH8ZXzTHXUj8U2CJLEG2OzJ9Y7GxvqXI1Um/ULBARCQkFtohISCiwRURCQoEtIhISCmwRkZBQYIuIhIQCW0QkJBTYUpcS00kOjUxooVwJFV04I3UlvRBB/8AQR8YS9AaXll+2YyORAiuSiNQCBbbUleyFCA4HCxAA7N7ZV62yRIqiJoWsSonpJE8fivP00Ph8t0ehhQj6B4bVPSI1Ty1sWVWSs7N87Qc/50ePP8fk1CwArS0RzjvjRezYcmrehQhG4pNagEBqngJbVpU79w6yd//C5Z4mp5L8YP8zzM5Bb1eUwzlCu6ezVQsQSM1Tl4isGonpJAf8UN79jw0Os2nD2pz7Nvet0wIEUvPUwpZVY3Q8wZH4VN79R+IJdp7zm0QijfQPDDMSn6Sns5XNfevmFygQqWUKbFk1ujui9Ha25A3t3s4ovV2tCxYiSC9AIBIG6hKRVSPaHOGsAmsvbu6LzYdzeiEChbWEiQJbVpWGPNtPXX+Suj0k9CreJWJmjcDngDOBBHCNuw9m7H8H8C5gBviYu3/XzNYBdwBtwLPAle4+UenapbYlppM8+vPhnPuOTyaZSc6hpRclzKrx67sLaHX3c4EPADeld5jZi4BrgfOA1wMfN7Mo8BHgDnc/H+gnFeiyyi11vo/R8cSi46xFwqwaJx23AfcAuPtDZnZOxr5XAfvcPQEkzGwQ2BQ856+Dx9wd3P505UqWSip2vo/JqRkOjUzMj5+emk5qnLWsatUI7C5gNON+0sya3H0mx7440J21Pb0tr56edpqaSnMyKRbrLMnrlEK91HLLtx/POd9He1sL79h1BsnkLLfd9VMeeuI5Do0cpy0aARqYnJqhtSX3z/28M1/MqS9eU7aaoX5+PktVC7Wc1N5C4won96qFz1GNwB4DMj95YxDWufZ1Akczth/P2JbXyEhpurdjsU6GhuIlea2VqpdaEtNJ9v37Mzn3PfjoM5zTt477DjzNff3Pzm8/nkiecLu1JcLUdHJ+nPUl576krMevXn4+S1WJWooJ0mMT+cfnF6tSx7TQ56lGYO8DLgG+bmZbgccz9j0M/JWZtQJR4OXAE8FzLga+BFwEPFDJgqVyCvVDH4knuOHWh2nINxQkQ3u0iT9729nE1rRp6J6sGtU46bgHmDSzH5Hqh77OzK43s9939+eBvyEVyHuBD7n7JPAx4K1mtg84F/jbKtQtFdDdEaW3K39f8xwwO7f46xwdT9DS1KiwllWl4i1sd58F/jhr88GM/bcAt2Q959fAheWvTqot2hxhc19sQR/2cugko6xGujRdyioxnWR0PEFbtInjiZmiLgVPX+DSPzDMkbFJimhQn0CTOUmp3f9o7nMrpbL9rFMWfYwCW8oic2je4bEEjQ2prozezha22PqCS3JFGhvn5/sYGpng5m8+lnOoXmMjzM4yPzIk8ySjrmqU1UiBLWWRvRRXut/5SHyq6CW5os0RTl3fmbeL5MKtp/GaM1403/WhyZxktdOFulJyhZbiSlvKklyX7djIznNOZW1XK40NsLarlZ3nnMo7d50xP4GTJnOSeqAWtixbun86u1VbaGhe2lKW5MrsIsl8v4gmBpE6o8CWJUsmZ7nj3oG8l46nh+bl6ndOyx7FkS/8M6Vb0SL1SoEtS3bbXT/Neek4pPqlixmat2lDL6PjCTram/n2A79YdN4QEVFgyxIlppM89MRzOff1Dwxz6QUbiDZHMobmLRwl0tPRQkd7C489eZj7+58l2hJhcuqFvuzs8BeRFyiwZUlGxxMMHT2ec1+6X7q7I8roeIJLL9gw3++cHof9/Z88xX0HXhjPmhnWmTLDX0RSFNiBYvpQJXXpeGxNG4dGTgztNR1Rvv+Tp3hscDhn90ZLc4THBnMvMJBtKSclRepF3Qd2obmXw2olXz6LPTfaHGHr6SfznQf+44R9J7U1L2g9Z3dvjI4nCp6IzNTSHKGjvWVJtYusdnUf2NkXeGSGzHsvP7taZS1LsRP/r/S5V13yCiaOT9E/MMxIfJKezlY2bejlsScP53ztdPdGW7Rpvi97MZNTSb79wH+oH1skQ10HdqELPPoHhpmcmsm5r1YV+vJZLPiW8txI5MRx0aPjCe7PmKM6U7p7Izk7V1RYp6kfW2Shuh43tdgagCNF/vleCxb78sl1VWF6zcT4xNSSnwssuLqw0LSo6THX9+5f2gx8WodRZKG6bmEXusCjp7OVnq4o8dHcIyJqTTEL0KZP4GV3f6zpiDKSJxiLPflXaOz15r51AEWfcEzTFKkiC9V1CzsdMrls7ltHa0v5vs+WuiL4Yopp4aaluz8OjyWYg7xhneu5heSb8+OyHRuLulw9m6ZIFVmorlvYsHDu5fQJtHJOz7mSE4PZskd0FGrhpoOvmImZ8j13Mfnm/IDCf82s7YqyacNaHnvySEV+BiJhVfeBXShkymG5JwYnp2Y4NDJBd0eUpkhDztB/8/aXAoW/fBYbWremo4WxY1MrCs1cc34U/kKJsXtnn8bCiyyi7gM7bSUTCxUbNIudGMw1IiLdIn/sycMMjRyntytKe2szTx0an39MdugX+vIpNLSusQE+9LazSc7OlSU0F/trRpM7iRSmwF6BpXZvFGrdHhnLfXIvV4s832tkhn6+4DuemMk7tG52DpKzc2ULzUr/NSOy2tT1SceVyj55l27p3rl3MOfjuzuitLbkPuTRlsgJJ/eW2t9czDC47o4ovZ25ryDs7YxWZFSGFhsQWR4F9jItZ9xzSkPR77HUkRXFjOiINkfYYutz7ttisRWFaKlHvojIQuoSWaaho8eLHvecNjqeIJFndrqpoB888znFLASQqdgRHZft2Ehydo5HB4Y5eixBb54TjMX2zZdy5IuI5KfAXqJ0OB3wQ+S7yjpfS3exC3Wyn1NoZMVvru9gYnJmycPg5k9iDg4zMp5gTUcLmzb0LgjXpQbwSi6JF5HiVTSwzawNuB1YD8SBt7v7UNZjPglsC2r7grvfYma9wADwRPCwPe5+c+Uqf0F2OOWSr6Vb7FjpTOkQfuzJwwwfPb4gnGeSc0s+eZdd/9HxKe7rf3Z+fpBcjykUwMsZ+SIiy1PpFva7gcfd/S/M7K3Ah4H3pnea2WuBje5+rplFgZ+a2TeBLcDX3P1PK1zvAoudBFxbxNSsS71QJz2y4l2XtvHkLw8vCOdII0sa0VFMuKZuL/6YtKVcEi8iK1PpwN4G3Bjcvhv486z9PwYeDW7PARFgGjgb2GJmPwQOAde6e+51qoCennaamkrTqovFOudvPzd8jCPx3OHUAPzFO3+X007uWvQ133v52UxOzTAylqCnK1r0JfCv6PuNoh6XT6H6R+KTRFqaAYp6TPq4dHa3EevJvaDBujVtbDhtbVkv8c/8+VSbasmtFmo5qb2Fxho/n1LMcSrbvyQzuxq4Lmvzr4HR4HYc6M7c6e6TwKSZNQNfJtUlMm5mB4H97n6vmV0BfBZ4c773HhmZKMlniMU6GRqKz99PTifp7czdB93b1UrT3OyCxy+mCYiPHqeYZ2TXshyF6u/pbCU5NQ1Q1GMya9m0YW3Obp5NG9YW/fmWoxTHpFRUS26VqKWYoDs2MVXWGkohfZwKfZ6yfeW4+63ufnrmf6TCOl1NJ3A0+3lm1gPcA/w/d/94sHkvcF9wew+wuVx1F7LYZFHV6KtdylC6YupfzmcsNOmTiJROpbtE9gEXAw8DFwEPZO4MTkr+ALjJ3f8hY9cXgW8BXwd+D9hfkWpzqPRkUfksdyhdMfUvt59dVzCKlFfD3NwSlgBZITNrJ9XVcTIwBex29+fN7Ebgm8B5wA280I8NcGXw/9tIdRUfA64p1Ic9NBQvyYcq9OdcpScqyq7ljnsHcnZD7Dzn1KKG0hVTf77H1Mqf3LVM6UclAAAGhklEQVRSB6iWfCrUJbLo1Wjf+LeDlQu6Zdp+1ilA4c9T0Ra2u08Ab8mx/f3BzYeBT+d5+mvLVddyVHOiolIMpSumfk3GJFJbdOFMCGkoncjSpFuvYVfb41wkp6WsLiMiq4cCO4RqcbSKiJSfukRCqlZGq4hI5SiwQ0pD6UTqjwI75DSSQ6R+qA9bRCQkFNgiIiGhwBYRCQkFtohISCiwRURCQoEtIhISCmwRkZBQYIuIhERF58MWEZHlUwtbRCQkFNgiIiGhwBYRCQkFtohISCiwRURCQoEtIhISCmwRkZCo+wUMzKwNuB1YD8SBt7v7UMb+C4EPBHcbgG3A6UAbcBfw82Df37v7neWsJXjMd4C1wDRw3N0vMrONwJeAOeAJ4E/cfbbMdXyS1LFoAr7g7reYWS8wENQAsMfdb15mDY3A54AzgQRwjbsPZux/B/AuYAb4mLt/18zWAXeQ+tk8C1zp7hPLef8l1nId8Nbg7vfc/aNm1gA8zQu/Hz929w9WoJa/Ac4j9XMDeBPQTIWPi5mdBXwm4+FbgV3Aw5Tod6QeqYUN7wYed/fzga8AH87c6e73uPt2d98OfBf4hLv/DNgCfCq9b6VhXUwtgY3AtuA9Lwq2fQr4cPC8BlL/SMtWh5m9Ftjo7ueSCu3/YWY9pI7J1zKOyUr+Ie4CWoP3+ABwU8b7vwi4llQwvR74uJlFgY8AdwR195MK9FIoVMtLgSuA3wXOBf6LmW0CNgAHMo7FisN6sVoCW4DXZ7zvKFU4Lu7+aMa/m78D/snd76G0vyN1R4GdCpx7gtt3AztzPcjMTgXeBnw02HQ28AYz+79mdquZdZa7FjP7DWANcJeZPWhmb8yo5YeLfYZS1QH8GLgquD0HREi1+M8GtpjZD83sG2Z2cilqcPeHgHMy9r0K2OfuiSCQBoFNRdRdjlqeAi5092TwV00zMEnqWJxiZveZ2ffMzMpdS9DifRnwBTPbZ2ZXZT+Hyh2XdE0nkfo3c22wqZS/I3WnrrpEzOxq4Lqszb8GRoPbcaA7z9OvBz7t7ong/sPAF919v5l9CLgB+G9lrqWFVCvmZqAX2GdmDwMN7j5X4HklrcPdJ4FJM2sGvkyqS2TczA4C+939XjO7Avgs8OZia8nSlVEDQNLMmtx9Jse+dI2Z25d0HJZbi7tPA8NBF8gngX53Hwj+Cvi4u3/DzLaR6mJ6ZTlrAU4idcw/RepL9D4ze4QqHJeMbVcD33D34eB+KX9H6k5dBba73wrcmrnNzP4JSLeOO4Gj2c8LWi5vBD6UsXmPu6cfu4fUL165a3ke+HzwD+KQmfUDBmT2V+f8DCWug6AL5JvA/e7+8WDzXiDdN7oH+Mti68hhLKMGgMaMIMjel64xvf14vrrLUAtm1grcRioM3xNsfoRU/zru/qCZnWJmmV+s5ahlArg53T9tZntJ9S9X5bgErmBhIJfyd6TuqEsE9gEXB7cvAh7I8ZjTgYPufjxj2/fN7FXB7d8D9leglp3A1wHMrCOo62dAv5ltL/C8ktYRnJT8AXCbu//PjF1fBC4Nbq/0mMzXYGZbgccz9j0MnG9mrWbWDbyc1EmsYn6WJa0laFn/M/Dv7v4ud08Gu24A3hc85kzgVyUI64K1AH3Ag2YWCf762QYcoArHJdjWDUTd/amMzaX8Hak7dT9bn5m1k/qz/mRgCtjt7s+b2Y3AN939YTN7C3Ceu78v43lbgL8NnvM88E53H6tALZ8hdcZ9FrjR3b9tZn3ALaS6TH4GvCMjOEpeB6mTfTcAj2Y87crg/7eROvF5jNSogeeWWUN6BMKm4PWuJBUOg+7+nWCUyDtJNTr+2t2/FfTxf5lUq284qPvYct6/2FpIdT18DXgo4ykfJPWn/+1AB6mW9p+4+8Fy1hIcl/cDbyF1TuEr7v75ahyXoJZXAh9y910Zz/ltSvQ7Uo/qPrBFRMJCXSIiIiGhwBYRCQkFtohISCiwRURCQoEtIhISCmwRkZBQYIuIhERdXZouq5OZXUvq6rntpC7q+T/AZncfr2ZdIqWmFrasBp8ldeXnu0ld+vxHCmtZjXSlo6wKwSXPTwCfc/f/Xu16RMpBLWxZLX6L1Gx5W4IJmURWHQW2hF4wc+EtwCWkphB9d3UrEikPBbasBjcC/+LuPwH+BPhI0EUisqqoD1tEJCTUwhYRCQkFtohISCiwRURCQoEtIhISCmwRkZBQYIuIhIQCW0QkJP4/82zjKmGgi1kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nonlinear = rotated.copy()\n",
    "nonlinear['y'] = rotated['y'] + rotated['x'] ** 2\n",
    "sns.jointplot('x', 'y', nonlinear, stat_func=None, size=5);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "A non-linear relationship between x and y also does not change the underlying dimensionality since we could, in principle, perform a non-linear change of coordinates to undo it.  However, we can expect that non-linear relationships will be harder to deal with than linear ones."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Linear Decompositions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of a linear decomposition is to automatically identify linear combinations of the original features that account for most of the variance in the data. Note that we are using variance (spread) as a proxy for \"useful information\", so it is essential that the relative normalization of our features is meaningful.\n",
    "\n",
    "If we represent our data with the $N\\times D$ matrix $X$, then a linear decomposition can be represented as the following matrix multiplication:\n",
    "![Linear Decomposition](img/LinearDecomposition.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The $N\\times d$ matrix $Y$ is a reduced representation of the original data $X$, with $d < D$ new features that are linear combinations of the original $D$ feature.  We call the new features \"latent variables\", since they were already present in $X$ but only implicitly.\n",
    "\n",
    "The $d\\times D$ matrix $M$ specifies the relationship between the old and new features: each column is unit vector for a new feature in terms of the old features.  Note that $M$ is not square when $d < D$ and unit vectors are generally not mutually orthogonal (except for the PCA method)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "A linear decomposition is not exact (hence the $\\simeq$ above) and there is no \"best\" prescription for determining $Y$ and $M$. Below we review the most popular prescriptions implemented in the [sklearn.decomposition](http://scikit-learn.org/stable/modules/decomposition.html) module (links are to wikipedia and sklearn documentation):\n",
    "\n",
    "| Method | sklearn |\n",
    "| ------ | ------- |\n",
    "| [Principal Component Analysis](https://en.wikipedia.org/wiki/Principal_component_analysis) | [PCA](http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html) |\n",
    "| [Factor Analysis](https://en.wikipedia.org/wiki/Factor_analysis) | [FactorAnalysis](http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.FactorAnalysis.html) |\n",
    "| [Non-negative Matrix Factorization](https://en.wikipedia.org/wiki/Non-negative_matrix_factorization) | [NMF](http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.NMF.html) |\n",
    "| [Independent Component Analysis](https://en.wikipedia.org/wiki/Independent_component_analysis) | [FastICA](http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.FastICA.html) |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "All methods require that you specify the desired number of latent variables $d$ (but you can easily experiment with different values) and are called using (method = PCA, FactorAnalysis, NMF, FastICA):\n",
    "```\n",
    "fit = decomposition.method(n_components=d).fit(X)\n",
    "```\n",
    "The resulting decomposition into $Y$ and $M$ is given by:\n",
    "```\n",
    "M = fit.components_\n",
    "Y = fit.transform(X)\n",
    "```\n",
    "except for FastICA, where `M = fit.mixing_.T`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "A useful visualization of how effectively the latent variables capture the interesting information in the original data is to reconstruct the original data using:\n",
    "$$\n",
    "X' = Y M\n",
    "$$\n",
    "and compare rows (samples) of $X'$ with the original $X$.  They will not agree exactly, but if the differences seem uninteresting (e.g., look like noise), then the dimensionality reduction was successful and you can use $Y$ instead of $X$ for subsequent analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "We will use the function below to demonstrate each of these in turn (but you can ignore its details unless you are interested):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "def demo(method='PCA', d=2, data=df_img, npix=16):\n",
    "    \n",
    "    X = data.values\n",
    "    N, D = X.shape\n",
    "    \n",
    "    if method is 'NMF':\n",
    "        # All data must be positive.\n",
    "        assert np.all(X >= 0)\n",
    "        # Analysis includes the mean.\n",
    "        mu = np.zeros(D)\n",
    "    else:\n",
    "        mu = np.mean(X, axis=0)\n",
    "    fit = eval('decomposition.' + method)(n_components=d).fit(X)\n",
    "    \n",
    "    # Check that decomposition has the expected shape.\n",
    "    if method is 'FastICA':\n",
    "        M = fit.mixing_.T\n",
    "    else:\n",
    "        M = fit.components_\n",
    "    assert M.shape == (d, D)\n",
    "    Y = fit.transform(X)\n",
    "    assert Y.shape == (N, d)\n",
    "    \n",
    "    # Reconstruct X - mu from the fitted Y, M.\n",
    "    Xr = np.dot(Y, M) + mu\n",
    "    assert Xr.shape == X.shape\n",
    "    \n",
    "    # Compare a few samples from X and Xr.\n",
    "    fig, axes = plt.subplots(2, 4, sharex=True, sharey=True, figsize=(12, 6))\n",
    "    for col in range(4):\n",
    "        img = df_img.iloc[col].values\n",
    "        rec = Xr[col]\n",
    "        for row, image in enumerate((img, rec)):\n",
    "            plot_image(image, ax=axes[row, col])\n",
    "    plt.tight_layout()\n",
    "    return Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Principal Component Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PCA is the most commonly used method for dimensionality reduction. The decomposition is uniquely specified by the following prescription (more details [here](https://en.wikipedia.org/wiki/Principal_component_analysis#Computing_PCA_using_the_covariance_method)):\n",
    "- Find the eigenvectors and eigenvalues of $C = X^T X / (N-1)$ which is an [empirical estimate](https://en.wikipedia.org/wiki/Covariance#Calculating_the_sample_covariance) of the covariance matrix for the data.\n",
    "- Construct $M$ out of eigenvectors ordered by decreasing eigenvalue (which are all positive) and solve the resulting linear equations for $Y$. At this point the decomposition is exact with $d = D$.\n",
    "- Shrink $Y$ and $M$ from $D$ to $d$ rows ($M$) or columns ($Y$), which makes the decomposition approximate while discarding the least amount of variance in the original data (which we use as a proxy for \"useful information\").\n",
    "\n",
    "![PCA decomposition](img/PCAdecomposition.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The full $M$ matrix (before shrinking $D\\rightarrow d$) is orthogonal $M^T = M^{-1}$ and satisfies $X^T X = M^T \\Lambda M$, where $\\Lambda$ is a diagonal matrix of the decreasing eigenvalues. Note that this description glosses over some details.\n",
    "\n",
    "The resulting latent variables are *statistically uncorrelated* (which is a weaker statement than *statistically independent* -- see below), i.e., the [correlation coefficients](https://en.wikipedia.org/wiki/Correlation_coefficient) between different columns of $Y$ are approximately zero:\n",
    "$$\n",
    "\\rho(j,k) = \\frac{Y_j\\cdot Y_k}{|Y_j|\\,|Y_k|} \\simeq 0 \\; .\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3UmMZdd9HvD/fe/V3FVd3c1udrNFkZRIUdRAybLkUXZkWR4SG1FGw14EgeHEi8BAsvPGRpwsDATJIog3ySbIxs6MeBIEeYjsOIjo2JasgaLYpkhKJNUju6u7a37DzcI7w8TJV2BVsanfb/3de+5795x77ldvUV3f9wUAAMBrGxz3BQAAALzRKU4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANIyOaqAfGPzd/qjG+v8xXD+ZHbCwEI/Rnz+THTDNvqLpanZNw+39KN8Psl492Bsf7vk3t6N8VVW/cTvKT8N86rdn/7VL8m+0dQPHIV03VW+8tTO6cD7K9ytL8Ri7D5/O8vfNRfm1Z+9G+eGVV6P85PKVKE/bm2HtwHF4rbXjFycAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgYXTcF3Bcphu3o/zw8UfjMbqt3Sg/vrAe5adLwyi/vz4f5QfjWZQfbWXTabA7ifL9TvZ9ArxRbH7rW6P81v3Z872qanyii/LTxez8eyfXovypSwtRfv70ySifmj797KGeH3jz84sTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANIyO+wKOy+gtF6P8dHUxHmO6NBfld+6fj/KThS7K98Ms3836KD/cy6bT8uW9KN+dPR3lq6oGG3ezAzZuR/Hh+sns/MCbwv4PfyjKz21NsvOv59vz5tumUX714p0of/PqiSi/cy7bN9deyPbA009djfLDRx+J8tPnXojywJufX5wAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgYXTcF3Cv6HYn8THjs0vZAX0W31/rovx0IcvPbUXxmg2z84/X5qJ8P8p7/vxsFuWH95+L8tOr16I88Oaw/PmXovzdb3trlJ/fCDeEqjpx8U6Uf3B9I8r/5Ls+EeV/4ekfjfIbo/Uof+KVU1F+/lq4qcGb1HBt7VDPP72TPYvuJX5xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGF03Bfwehmun4zy/Xgc5WfLc1G+qmrv1DDKj5e6KL97JsvPwo8wm8/Ov3Crj/L9MDt/N83OX1VV+9l9nl69lo8BfNOZXL4S5efvPBDlr34w2z+qqt59+kaU/6kH/iDK/8jybpQfv+uTUf4Xux+O8tPPnIjy++dWovzCzfNRviqfF/B6GK6tRfnuvtNRfra8GOVHm+tRvqpq8uLX42OOg1+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGF03BfweulOnIjy0/OnsvzKXJSvqposdll+OcuPT/RRvsviVVvh9axk+b21rLcPxvl0nRsOo/xw/WQ8BveW+B4vLGT51ZUo3u3uR/nJy69EeQ7H6ML5KL+/kD2LVr8exauq6mu3s33tpfvORPlbC38W5VeHi1H+LSdvR/lrD65H+ZXL2R4Vrnw4NtM7d6L87H1vj/K33pGt5aVb+bvU6s2NKJ9+5teLX5wAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgYXTcF/Bahusno3y/uRnlu/FalN9fy7+qbpblxyey/HA/y0+W+yy/1EX5xe0oXqO97HoWbuxkA1RVTadZfON2lB/efy7Kc/y6E9lCm53JnhWbb8/ylS2Dmn/H+Si/eOlKNkBVTV5+JT7mm83kcva9zj+UPStWRtnzt6rq8hfPRPnfXH0yyn95+4Eof27+bpS/vpWtzfFy9h31wyhe/ensPaSqqsJ5Aa+H0YVsX7jyxGKUv/t92Qve3eeXo3xV1dLlR6L86OkXovz0zp0o/1r84gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANo6MaaLh+8nAHeOD+LD/to/juqWF2/qq687YsP9jP8uPV7DPM3z7cntx3h3r6mi7m03UwzO5bOk+nV69FeY7f5C1novzNd61E+RsfnEb5weo4yteNhSj+0CcvZOevqoUza1F+9vln4jG+6Tz1hSi+8u7H4yHWz5yO8s/vPBLln374YpSfW87m9vTyUpRfy5Zazd+ZZAfshZtyVY0unI/yk8tX4jHgL+pPZc/suw9n5/9XH/yvWf70D2UDVNXk09nzq7tzJx7j9eAXJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGgYHdVA043bUX54/7ko393ZivKb33oxyt/40CzKV1Wde+TVKL80N47yL76YfUeDa1lPntuM4rWw0Uf50Vb2nQ53J1G+qqpfnIvy3cJClB+un4zyvP7SezCbZvP0ziNRvL7zfX8W5X/+4iei/Cc33xPlf2nxY1G+quqR/7ES5Rcfzb6k6XMvRPk3g9GF89kBN7M9s6rqvj/O8ie/uhTlb7x/OcoPxtnzd/d0F+WXbmR7yPy17D0B7hXd7n6UH5/N3qf++sp2lH/24heifFXVfz//A1H+uN6+/OIEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADaPjvoDXMr16LcoP3/WOKD9ZzDrj0vm7Ub6q6l+/8z9H+dXBfpT/2bm/FeUvvfpwlD/5Qh/l5+9Mo/zi9Z0o3+1OonxV1exLX4mP4d4y3bgd5Qf72TxavtpF+Zt7y1H+ifksXye+FMV/6/EnsvNX1dXHHoryF56Ph/imM7l85fAHCcfov+8DUf7UV/ai/GRlGOUXb2ZrbfW5zSjf3cyeFQdxJPcZ/oJ+lK214e0s/9Ru9n73yt6pKF9VtfhqNsZx8YsTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANIyO+wJey3D9ZJTv7mxF+aUbq1H+8u2lKF9V9dTO26P8Pzn1YpT/6H3PRvmvzD0U5ScLXZRf3p5G+enSXJSfv3IryldVzeIjeLPrvvaNKH/6vuUo/2dfeDDK/9j890f5R1euR/lLL90f5auq7tvs42MS6fOdwzH89Gej/MKF81G+X8n2zW5r51DPP7l8JcrDvaKbZO9f689k73e/+NKPRPkvvfhAlK+qenicvbENz56N8tPr2d75WvziBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2j476A1zLduJ0dEObnzq5H+XOfXovyVVX/9sT3RPlrj2Vj/OnGW6L8aKuL8ss3Jtn5b+9F+eGtu1G+39yM8lVVw/WTUT6ed9xz0nu8+OyVKP+2ybko/8zLj0f5Lw+z/MlsmVVV1drXd6N8N5lG+T5K80YxuZythdTowvko323tHNKVwL1l8vyLUf7EO89G+Rd+821R/vRG/pRfeu6VKD+5fj0e4/XgFycAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaRsd9Acel/9zTUf7U8D35IN1qFP+N+z8c5afzUbzOXJpG+bnb4yg/vL4R5ftxdv7pxu0ofxSG6yeP+xI4ZJOXX4nyc5ubUf7BV05H+cn92ZwbbGfrrKpqsLUb5Scvfj0eA/6iyeUrx30JcE8arq1F+ZWvXI/yc1unonzNsnjVvbOP+MUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGkbHfQH3iv6PvxQfc+buo1F+9cJalO8msyjfj7KePHd9Mzv/eBzlp1evRfnh+skoX1U13bgdH/NGOj/3nnRODMPzj+5uZQfs7YUjmNcA95LpnTtRPt135seT7IBJmK+q/Ijj4RcnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaBgd9wW8mU2ffS7Kz2+cywbY24vi043b2fnvP9zrScXXD/cA8xqAozS9cyc7IMwP19ay899D/OIEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADaPjvoB7xXD95KGPMb167VDPH3+Gvb3DuRAAAN6UpnfuHPclHBq/OAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAEBD1/f9cV8DAADAG5pfnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoGB3VQD84/xPZf9odDqN4F+bz8x+gY3bhMYMuPH+YP+x/djwLzz+dRvE+zB9sjFl2/j7L/9b+f4xu2g+MfvxQb1oXz7kj+FtLek2hLl03g/Azp8+W9HpC34zrpqrqh5b/XrZ25uaieDcf5sPz18J8lq+qPrymGoX7ZjpX03kRzrtubxzl+/39KF/72fkPNMYhr7Xf2vvleO3E72uH/K4Tv98d8jP1SIT7TrqP9If+Phiu/ap8LaSfIXxHfa214xcnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaBgd2Uhd1tG64TDLj8KPMj+XnX9hITt/VdVceE2DN1iP7fssP5tl+f1xmN/P8lXVh2N0leX7aRQ/dN2gSw/I8uH5uy68ngPo03karrMufFZU+Kzo0nWfft7JJMtX5WszHKMfH+CaUnPhMz7dE5aWony/kuVnq4tRvqpqcmI+yk8Xs322Tx8Xk2yuDneyB+ro7l6U7zZ3svxWlj+IPt3X0vV/EOn712G/r6XvUgfZd6bh+0sf5sPvKP4M4bzoZkcwj1LT8IUqzPcH2Qv/Em+wN3UAAIA3HsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaBgd1UDdMOxow2GWX1iI4t3KUpSfrS1H+aqq6Up2Tf3ocHtsN+uzA8L8YDzN8tv7Ub7b2onyVVW1tR3Fw2+ouhqHR4TnH3TZAeG66dL8KHxkpNdfVf3uXpSfvvWPsvxsFuUXb39/lK/VlSjeL8xn5w+vv9vN1llVVb+drbVuZzcbIPwMB5HO1W5xMcr3q9meMD6TzYud89n+UVW1dS5bz/sns/P34bY8DKfewqvZE/jE5eweL30jilc3nmQHVFVNsn2w68O10Ke7VC7eF+bnsny41mopzHcH2HfuboX5u1F+cOZ0lK9B9j7Yb2bXv7X+u1F+5dWPRPn4nlVVzcJn9uQA6/N14BcnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaBgd2UiDrKN1o+zSuoX5KD87sRzl986fiPJVVTv3ZZ9hsthF+W4axWsw7bMDwvhoLztg4eZClJ+/lvf8wXSWHRDm+2l4E1JduG6Gwyw/PxflayG7Z12Xzemqqpsrn4ryH7j4WJRfmMs+8zOfvh3l66HzUXyyln2n6bqc29jNDqiqwSxcB5NJNsBhr5uqqmH4vAjXwnR1McrvnM/u863HsrVcVbXz+F6Uf/uD16L8qYXtKP/S3fUof/Wr90X59BVm+WvZvOtvhWu/Kn5GVvjMrmG4px1Eek3hZ+5XV6L89FSWP4gL9/+fKH//yZNR/ovPX4zys5Xs+fLAg38U5R8+974o//zVr0T5a9e/O8pXVXWzcHPbyfe214NfnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGgYHdlIXXe4559OD/X0k+W8Y26dz44Zr4UD9GH+kO/B3N0svzrIrmfu7lw2QFXV3fC+hdd06NLrSe/xcBiePjt/36eTtGp9dTXKry4uRvmlhYUof/r9X43yt3YfjfKDvcN9dtUB7kE8jwbhPArn3YGEY/SjLD+bz/LjpexZtHdmFuWrqr7tsRei/D+9+JtR/h1z2Vr7g93sFePnur8R5TefPR/lP/DIF6L85K352vzCZy5kB8zNR/FueMjPi6rqhtlc7eayvXm2ks2j/dNZfjDO186HHsie20vz2X373N1sX9tfy77Tx09cjPJn17IXzqeefWeU33n7yShfVTV3ZxzlRzfC97tZPi/+Mn5xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgIbRcV/Aa+pnUXx2ayPKX3jkc1H+ytPfHeWrqrbO3R/l905l558u9Vl+OftO01o9Gw2j/ImXs/N3O+PsgKqqyTTLz7LvtLouy4e69PyD8KZNs+/n0p3/EOU/8OSTUb6q6u7zH47yX7q7lA2wMB/F+8FjUf7JR/8kyj9+8WKUn4T37KlLl6J8VdXV298e5bv5uSjfTyZR/kj02dof7mX3YenV7DMvXcu+06qqb2yejPJ3+2yMYZc9X1YHu1F+YZR9R5vh4/HE4mKU39nfzwaoqhpm+2A3zL7TfnIEf+8OP0OFe38/l+Vnc9mNHu6F+3hVnVpZifL74TNs+/xClE8/w/n19Sj/73/nd6L8jR//mSg/CF+9qqpOPh++Q46zQYb7B3iH/Ev4xQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaRkc1UDccZgeMwksbTqL4A6dPh/lnonxV1e9unovyo53D7bH9MDv/bHEW5btpF+Xn72T3rH/2+ShfVdUvLWYHDLLPcM+b9VH8nY8/HuWX5uejfFXV3XDt9yeWo/zkvhNRfu909hn+cPqRKP/FP7gZ5ftXrkT52Xt/KMpXVXVns3kxuLWZDTDJ1v6BhHO7m0yj/GBjK8qvbP2XKL/84sejfFXVV9fOR/mfH/3NKP8d970Q5a/vr0b5yxtrUX6wFMXr019/T5QfvXQ9G6CqutXwmTcO18JR7FFdOEaaD3XTbC2PNsfxGLM+G+OrV69G+cnJbO9c/cTno/zaT5+J8jd+4mei/NmPfCPKv3T1VJSvqhptZwt68dVsrQ02Xp/K4xcnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaBgd2UjDYRTvFhez8589HcX/781Ho/ytdy5H+aqqm+/po/xseRLl529k3+nitawnTxez/PydKF6jnWmUv3Hm09kAVXVu96/Fx3xTmcseAbMb3xvlr9+ei/JVVVsnfj3Kf89j747yn/3KO6P8le+4EOXvvns/yq+eno/yW5vviPLzl5aifFXVmS9na/PEZBblh7vZd3QgfXZNNck+cxfm3/HwxSh/du1SlK+q2v7Mh6L812YPRvlLF7K10C1k31G/lT2PlrMtsLYezPbxlTqbDVBVw43tKJ/Oo5pl7xVHog+vaZatzcF+mN/cjfJVVbvjcZT/k2cej/Kj92Xf0d//ybUov7W3F+UXPvZqlP/nj/5qlP83Sx+L8lVVX76U7W2TxewBMB++77wWvzgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAw+jIRhp0WX5+LoqPz6xE+VtPLEX5V79zHOWrqn7kvV+M8utz21H+V59/Msr3f3Qyyi9f7qN8au90do/vu/DT8RjdzTtRvt/K7kFNZ1k+1PfZPQhXWVUXHrG0GMVna8vZ+avqxMbHo/yZ1atR/om3fSHKP3f2fJT/+Pv+NMr/o/t+P8p/Zf9slP/Zhb8d5auqtm6sRvmlK9laHm4c3dZzaEbDKH7pi49F+We3d6J8VdXck9k+deqZ7DPsXs3u8/5alk8tXc+ejwu3JlF+sJ3v+91kmh0wDfNHIdx3YuG2OZge8vVU1d44u9f98kKUn7+b3ecTD2XvqL/wK78S5T/8Dz8S5b832/rrN1ZezQ6oqi8uZPe5H4XvL+n7zmvwixMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0jI5spK6L4v0w63Sz+WGUH5/Irue+c3eifFXVP7jvf0X5J+azz7w63I3y/+6Vj0b5E69k19OHNXy8nN2DyfpCNkBVDbbmswO2s+/00M36KN7v7ET5O6f+Z5R/YP1MlN++8l1Rvqpq8uDZKP/Ln/5PUf573vveKD/cyebpieFelH/H3EqUX+yuRfnTq1tRvqpqc2E1yvejcPGH+8GB9NnaiU8/F26fK0vZ+c+dys5fVYPJLMqvvpg9L1Yup/ts9h114fNu5ekrUf5b3vtclH9xOVtrVVW3bnwwyvfpPO2ze3wkptk1ddNplh+Hn/nm7SxfVXPDbG530+y+zeayZ972XraPvOeRR6L8N3ZORvln9rej/OXd7PxVVYNx9h2l9+D12hP84gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANoyMbqe+jeDeZRvnh7iTKz9+ej/Kv3jwR5auqXpycifLvX9iM8k8svhLla20cxefvzkX5xRv7UX68mk2/wV42J6ryedT3s2yAWZgPfbX+W5R/8KG3RvnH739LlH/7+fNRfvTotShfVfXpF89G+bUn/3GUv7v9iSh//qlsXv/yQ98e5cffMozyi4NsHV+5cTLKV1Wtb2fP68H4jbVu/nyM7DOke1ScH2d7VD/99ez8VfXE2YtRfjjM5t7lmzej/O0XPhzl+63tKP8d33M9yi/Nr0T5xfnsPaGqqk/nRTpP0/xBxNeUredufLjvdysPPBXlq6r6/sko323tRPnp/HqUv7KxEeV/8H3vi/I/97m3Rfl/NvjRKP+5l7N3i6qqpVtZfrQTvhOGz+DX4hcnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaBgd2UjTaZbfH0fx0cZOlF97MfvoO08vRvmqqn957gej/PNv+dMo/+z2/VG+bs9F8ZWXt6P8d13Mrv+p3zsd5bv1k1G+qqp297L8ZJLl+z7Lh5ZXV6P8/Fx2j+dH2TpYCM+/upivm+H121G+P5Otg0HXRfmfevjFKP8r/2I9yv/Gj31XlN+9bxbll67kfx9bvp49r4db+9kA43CdHUSffU99uPa7Sbh9DrJnxbmz+fNudXk5yvfh82t3nO3L/eJCll8/EeU/8+VTUb4L3yuqe0uWr6puMVs7/fZuPMahC9/X4rWzl92H9Jm9Md6M8lVVt7a2sgPGa1F8uJs9j762dT3KXzydvU+99VPZ9fzJ9uNRfm4ru2dVVStXsmsa3c32nXTevRa/OAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANAwOqqB+uksO2B/HMW7rZ0ov3C1i/KnLuVf1Y2F81H+lx74WJTv9rPPsPbcMDv/eBrlb29vR/m3vPfVKP+/P/vZKF9V9fDwx6J8P80+c83CeR0a7+1F+f1xtm62w/PfvHv3UM9fVdXfzP6es3jlRJTfHGXPinRef/T9G1H+tz+1FOU3H1qO8sP9SZSvqlq8lt23bjP7Tvvw+X4Q6Z7TpWt/HH6vc9ke8urXvjU7f1VdevnXovypWx+N8oPzj0f52amFKN/PZXtULc1F8W6SzYnBVv786jbDeTHL5l28Rx1AOkaXrufBbnb+cJ+d2/y+KF9V9eWXPp8d0D0QxRevZfvI01tfj/Jn19ai/PKlxSh/bvlslO8O8Gq0fCVbb4Pb2Xfa7+9H+dcc93U5CwAAwJuY4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0jI5spOk0y4/3o3i/20X5bpB1xuVXtqN8VdXZ2VKU3/9qdk3dLIrX4q1xdsAkG+CZpy5G+a0zvxfl3zr5eJSvqur7SXbALPvM/TS8CaHxfrYOtnd3o/y127ej/CRcx4NwnVVV9fsr2RhXb0b5W7efiPJ/OLwU5TfDezDcejDKr4Zzrpv2Ub6qqtvcCfPZ87Hf24vyB5LuOZPsWZF+q134bKlRvj2f6f5OdsDZbN/sB1l+sJ09v+oAz4vIJJsT3V54/VXVh8/seJ72+XqOhdcUf+ZUuna6fB7dePbd2RCL2fNieONOlO+3vjfK//6nsnvQXcjW8uoLW1H+IPN0cDfbO49r3/GLEwAAQIPiBAAA0KA4AQD71D7NAAAC4klEQVQANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADSMjmqgfjrNDtjP4l0Wr+qyI0Y34hFqZXsc5ZcXhtkAsyw+2Muup9vaifL9/FyUX77xV6J8DcM5VFU1y76kvu/zMQ7R2yYfj/IvbXzykK7kz+3s7UX5Sbruq2q1fzTK91vb2QDDbJ1d//ITUb4bZY/VbjFbZ8PxJMrXNHxQVFU/zp4VfTgv+kn4GQ4iXfv74fNxlj0r4mfLINwPqqoG2T7VhftgNwnX8264kR+28HnUp5+3qmqcfeZ03sXvUgfQh8+M9O2o3w/nRfi86IaH/5tAv3e4nyHVLS9l+e3dKD8M5+mB9p1wXsT7TvoZXoNfnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBhdGQjTadRvE/Pv58eEJpM4kMG2zvZAcNhPEYkvQfpZ94fZ/n0esJ8VVXNwpnUz/Ix3kAe3P6rUf75/V87pCv5c2+bfDw/aJTds34/XPyD8O9FXRfF43WzE8653b0sf4A53U/DY45iLYfiz9DHu04m/MzdQfaDQThX0/N39/jfWtN5epA5cdhr4QjWTvrM6NNLCr/XLn0GT49gnobXlO476WeuQbbvxPtUKn33qjr0d9TXa9+5x5+CAAAAh09xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABq6vu+P+xoAAADe0PziBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADT8PzERzxH/u/o7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('PCA', d=16); # 16x compression from D=256"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The top row shows the original binned data and the bottom row shows the PCA reconstruction. Although the original data is always positive (yellow-green), the reconstruction has some negative values (brown)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3UmMrfl93+ffe6aa69adhx7Yk5uTRGqgAsmmJEsyIzpyJCQwhSTe2Mgmy+wSIHAQBIiDAAESIJsA2QRwFnGsQDYiiZZlhVFkW7RIa6DZJLvZI3u481BznTpTFt4EQi7++Ra67u3bep71533/b513Or97FrdbLBYFAADAw/Ue9wEAAAB81BmcAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0DB7VQl/qfWXxqNb6/6O/dSbbYGkpXmNx5Xy2wSz7iGYb2TH1D46jftHL5ureeHK6+987iPqqqsWD7aifhX3qd+Z/v0v6j9p9A49Det9UffTuncHVK1G/WFuJ1zh67lzWXxhG/earu1Hfv3E36qfXb0Q9bR+Hewceh4fdO35xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0DB73ATwuswfbUd//5EvxGt3+UdRPrm5F/WylH/XHW6Oo703mUT/Yzy6n3tE06heH2ecJ8FGx9+PPRv3+5ez5XlU1We+ifrac7X98ZjPqz762FPWjc2eiPjV75dVT3T/w8ecXJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfC4D+BxGTz9VNTPNpbjNWYrw6g/vDyK+ulSF/WLftZ380XU98fZ5bR6fRz13cVzUV9V1Xuwm23wYDvK+1tnsv0DHwvHX/6JqB/uT7P9b+Wv570XZlG/8dRO1N+7uR71h5ey9+bmW9k78NzXb0Z9/6Xno372+ltRD3z8+cUJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgYPO4DeFJ0R9N4m8nFlWyDRZYfb3ZRP1vK+uF+lNe8n+1/sjmM+sUgn/NH83nU9y9fivrZzVtRD3w8rP7pu1G/+288G/WjB+ELoarWn9qJ+me2HkT93/rMb0b9f/HKX4v6B4OtqF9//2zUj26FLzX4mOpvbp7q/mc72bPoSeIXJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQMHjcB/Bh6W+difrFZBL189Vh1FdVjc/2o36y0kX90fmsn4d/wnyU7X/p/iLqF/1s/90s239VVR1n53l281a+BvDnzvT6jagf7VyL+ptfyN4fVVWfPXcn6v/Da78f9b+0ehT1k898Ner/TvflqJ/9wXrUH19ai/qle1eiviq/LuDD0N/cjPruwrmon68uR/1gbyvqq6qmb/8g3uZx8IsTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAweNwH8GHp1tejfnblbNavDaO+qmq63GX9atZP1hdR32V51X54PGtZP97M5vbeJL9ch/1+1Pe3zsRr8GSJz/HSUtZvrEV5d3Qc9dP33o96Tsfg6pWoP17KnkUbP4jyqqp6Zzt7r7174XzU31/6ftRv9Jej/ukz21F/65mtqF+7nr2jwjsfHpvZzk7Uzz//YtTffzm7l1fu59+lNu49iPr0b/6w+MUJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgYPO4DeJj+1pmoX+ztRX032Yz64838o+rmWT9Zz/r+cdZPVxdZv9JF/fJBlNdgnB3P0p3DbIGqqtksyx9sR33/8qWo5/Hr1rMbbX4+e1bsvZj1ld0GNXr5StQvv3YjW6Cqpu+9H2/z5830eva5jj6RPSvWBtnzt6rq+r86H/W/sfG5qP/OwbWovzTajfrb+9m9OVnNPqNFP8prcS77HlJVVeF1AR+GwdXsvXDj08tRv/tz2Re83TdXo76qauX681E/eOWtqJ/t7ET9w/jFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGDyqhfpbZ053gWuXs362iPKjs/1s/1W180LW946zfrKR/Q2j7dOdkxfdqe6+Zsv55drrZ+ctvU5nN29FPY/f9OnzUX/vM2tRf+cLs6jvbUyivu4sRfknvno1239VLZ3fjPr5n343XuPPna9/K8rXPvvJeImt8+ei/s3D56P+leeeivrhanZtz66vRP1mdqvVaGeabTAOX8pVNbh6Jeqn12/Ea8CftTibPbN3n8v2/99+4e9n/blfzBaoqunXsudXt7MTr/Fh8IsTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAweFQLzR5sR33/8qWo73b2o37vx5+K+js/MY/6qqpLz9+N+pXhJOrffjv7jHq3sjl5uBfltfRgEfWD/ewz7R9No76qarE8jPpuaSnq+1tnop4PX3oO5rPsOt15Psrrpz7//aj/20/9ZtR/de+Hov5/WP4rUV9V9fyvr0X98kvZhzR7/a2o/zgYXL2SbXAve2dWVV34ZtafeWMl6u/8yGrU9ybZ8/foXBf1K3eyd8joVvY9AZ4U3dFx1E8uZt+nfnntIOpffepbUV9V9b9f+VLUP65vX35xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaBo/7AB5mdvNW1Pc/83LUT5ezmXHlym7UV1X995/6e1G/0TuO+v9k+O9G/Wt3n4v6M28ton60M4v65duHUd8dTaO+qmr+7e/F2/BkmT3YjvrecXYdrd7sov7eeDXqPz3K+lr/dpT/409+Ott/Vd38C5+I+qtvxkv8uTO9fuP0FwnXWPzcj0X92e+No3661o/65XvZvbbx+l7Ud/eyZ8VJPJLzDH/GYpDda/3trP/6Ufb97v3x2aivqlq+m63xuPjFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGDzuA3iY/taZqO929qN+5c5G1F/fXon6qqqvH74Y9f/x2bej/ucvvBr13xt+IuqnS13Urx7Mon62Moz60Y37UV9VNY+34OOue+eDqD93YTXqv/+tZ6L+V0e/EPUvrd2O+tfevRz1VVUX9hbxNon0+c7p6H/tj6J+6eqVqF+sZe/Nbv/wVPc/vX4j6uFJ0U2z719b382+3/2dd38p6r/99rWor6p6bpJ9Y+tfvBj1s9vZu/Nh/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMHvcBPMzswXa2QdgPL25F/aWvbUZ9VdX/uP7TUX/rL2Rr/MmDp6N+sN9F/eqdabb/7XHU9+/vRv1iby/qq6r6W2eiPr7ueOKk53j51RtR/8L0UtR/971PRv13+ll/JrvNqqpq8wdHUd9NZ1G/iGo+KqbXs3shNbh6Jeq7/cNTOhJ4skzffDvq1z91Merf+o0Xov7cg/wpv/L6+1E/vX07XuPD4BcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAweNwH8Lgs/viVqD/b/6F8kW4jyv+Py1+M+tkoyuv8a7OoH25Por5/+0HULybZ/mcPtqP+UehvnXnch8Apm773ftQP9/ai/pn3z0X99HJ2zfUOsvusqqq3fxT107d/EK8Bf9b0+o3HfQjwROpvbkb92vduR/1w/2zU1zzLq56c94hfnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACAhsHjPoAnxeKb3463Ob/7UtRvXN2M+m46j/rFIJuTh7f3sv1PJlE/u3kr6vtbZ6K+qmr2YDve5qO0f5486TXRD/c/2N3PNhiPwxVc1wBPktnOTtSn753RZJptMA37qsq3eDz84gQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAwe9wF8nM1efT3qRw8uZQuMx1E+e7Cd7f/y6R5PKj5+eAK4rgF4lGY7O9kGYd/f3Mz2/wTxixMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0DB43AfwpOhvnTn1NWY3b53q/uO/YTw+nQMBAOBjabaz87gP4dT4xQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaOgWi8XjPgYAAICPNL84AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGDyqhf7Npb8R/U+7XT+c6fr9KO+G4Z8+HGV9VXWjYbhGdkyLQfY3Vy/8TMP/HLmbzbP9T6ZRvjg+zvZfVTUeh2tMsn42i/p/fPi/dEn/pd5XopPw2uLXouN5ufvrUV+98D4L78sT6UUfadU8u64X0+yaeHf1q1E/XFqK+kl4TR/u7UV9VdVglD/vTtMb43F4kqt+ceNvZu+c5ew8dMvLUb9YW4n6+Vq2/6qq+Ur4Dgnfs1147/SOsnunt5dd291Odm3Pt3ei/oPl34z6k0jv59T3dnfje+dLg38vu3fSZ/ApW4TXaVXVm/1fj/oXpr+SLdBl99q7K6d/7Z2mZw7+arxN+u6cnOQ7YeBh7x2/OAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADYNHtlKvy/p+P8q7YfinLC1l+19ZzvZfVYvVbJv5yjDrl7K/edEP5+TFIsp703nWHxxn/d5R1FdVVRded6nsT4i9tvi1qB+MRlF/Y+NrUZ+6uvOlfKP03k/PcT/sF9nxrG5sZPsPHezuRv3Lva/Ea7w7+mq8zUdN+k7ohtnzd7G2EvXT8+tRP76QvaOqqo62smt1Fi7RzbJ+aTdbYOVm9vwaTaZR/87O/xz1T52/FvVVVcfT7Jj2Dw+jfnJ8yi+dqupO+/ta+MxehN9FapZ/RmfOn882uJnlXfgZpTa3tqJ+Ostu5gt3fjbqa5D/LpNe28Pw+84w/N7/MH5xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaBo9qoa7fP9W+Btmf0o1GUT9fX436qqrp2ZWon2xmf8PxevYZzYdRXt0s6wdHi6gfbWcHNOq6qK+q6s3n2QbTadbPwv2HXu7+etS/O/pHUb++vh71g/C+XDzIromqqt5S+FiahRfqMLwRwnN8dmMj6ncPDqL+xd5Xor5byu+bZw7+atTfOPt/xmucuvQdEl4Xi5XsHXJ8Luu3n89fz3vPZPfbbCt93mXX0tKt7BxsvboU9efuZp/pyy+8EPWzRf78Sk3D7yKT4+NTOpL/l/T7Wvr9axj28+w8nOStvLm2FvV7z/xhtv87Pxv1q+F7ZHV5OeoPjo6iPv3OHD9/q6qyQ6qz589H/Sz9rvAQfnECAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoGj2ylrsv6fv90+/k8yhfDcP9Vdbw1jPrxZjbHHp3P+tlSlFd/nPXDvUXU96bZZzrYyy/XbhhuE15HXXpdp055/0vD7BpdW1mJ+kl6X1ZVF26z/NKfRv3RWz8W9dPnvhn1o/AzPXv3L0d9t5qdgxM5nkT5SnhdTGezqD+RXvoOCf8dcZE97yrMx+eyvqrq7KfvRv2Xnvpe1F8a7UT9P3j/R6L+5uxa1G++vRr13Zs/E/WLaX6dHqz8VtQPwufdcDSK+pNI32vxe3Yp+zKSHs/w2W9EfVXVan896ncPDqK+G2Sf0YU7P5vtfye7LpbC78DdSvgFcpS9B6uqrl14OurXw/fO8SR7rz2MX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQMPgUS3U9fvZBoPs0LrhMNv/dBrli2E+Y07Wsm3mgy7b/3qU1zz8iHqTrF+EH1H69y76J5jze9k2XdgvetnfEOuy45kcH0f9wdFR1G+urkZ9eAn9a8Ps3l9dWor6o5XlqF9bzvrj6z8V9eOnfjfqV8K/d/7OF6K+qmr+4h9H/UZl18XBeBz1J9Glz4suu5e7o+zqXnlvN+p7nz8X9VVV60vZ5/qT669H/S+vHUT9lcF21P/n9/7tqN/9fnbdbdSVqO/v5dfp3rt7UX/p/PmoH6ffpT4GFuPsvbYRPrOrqrbD5/ZB/x9G/Zmorjrz+eze3PvgJ7MFwu/Y8zPZF84uPGdVVZfWt6J+Kfze/2G9d/ziBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0DB73ATxM1w9nutEwyhdn1qN+sjmK+qqq47UuW2Mj6483F1Gf6h1nx1OLrO+Ps3M8Wskv18Ggn23QD/vudP/toetln+mLva9k+3+QfabHu9nnc+f8V6O+qursxkbU33vtp7MFBrMof+/27ai/eOVi1C+9+Rejvjazz+f+1j/M9l9VL519KuoPxuOon83nUX8iXXbvLNJ3TtgvwmdLL7tMq6rqcJK9B+/NsvfgnVl4L4TP35evZPu/vvFc1KcWvfz5/vSVK1HfC9dI77VHYTGZZhvMsvt/euWfRf32QXYfVFXVLLvhPv38s1H/wXg56vcOD6N+ET5Tp0+fj/p7n12N+rUb+QNs8upe1L949Wq8xofBL04AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQMPgUS20WCyyDWbzKO/C/S+G/aifj04wY4abzMOz0WV5bD7K+m4v64eH2TnuTbK+qqrS6y7tn3S98CIdZhfp9t272f6r6vK5c1Efn7HwHF945j+K+r2nV6N+8ckfjvrR7izqz/6zX4j6qqrl59+M+vFkEq/xkdNlT9T0HdLbP4r6K38QPoCr6p2zF6P+745+Muq/e/5a1Pcre2bfOViL+tlSds4OrixF/frb+XV9dn096g/G43iN07YIv38twr9h/tKfRn2/n91ri5t/Keqrqjae+xdR/8G3PpctsJLl6XV0fz+7tu99JntPbf/8YdTvvhH+wVX1/M2/EfXT+T+J+kF4HT2MX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgIbBo1povn8Q9b35POq7fjYDzi+fifq9a/lHdXCli/rZaBH1i3Ds7aZZH8v+3OqyU1zdLNygqmo6y/rwuqvFCY7pNM2za+jgwu9F/fnNzay/cynqq6oW1/9iuEV2DubPXY362z+6HvX3P58dz+DcUdTP31uN+iubPxT1VVUfvHIt2+DW3ShfCt8HJ7FI3yHps2Ia3vvD7B3SP8of2Ge/k/UfjJ+O+v/t6ex+Hq5Non56eznqzx5lz7vhfnbOem9fj/qqqsHL/agfT7LPaHx8HPUnsZhmx7T3zB9G/eXVs1F/MB5HfTcaRX1V1Tx9XmxuRP3lZ78Z9Te+/2NRX+eWonz3hWz3/9UXfj3q/7uzfyVboKq2v3Mx6g+//n7UP3/lStQ/jF+cAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAICGwaNa6OgT34j69ZWVqF/c/pmoP7yyHPX3fngR9VVVz376etTPF13Uv/ve+ahffWMU9Uv3s795cBDl1Ztk++9N5tkCVdVNplG/mM6yBeb5dfFRMp1lf+8k7M8d/VtRX1U1v/z7Ub8RPivub7wY9ftPZfflL3zh21H/n1757aj/nf1PRv1/s/FLUV9VNe9nz5Yzr/Sjvrt5N+pPZBre+3vZA2xx607UP/Njb0X9e9/OznNV1fmDS1G/8e5a1N9/eSnqpyvZO2cRfiMZHGbP3+UPdqN+vrcf9VVVt175VNQvFtnfcHEyifqT6PrZ/by3txf1W+vrUZ8+4/fv5J/R0nCYrXF0FPXH4fOoW87utToaR/nx+exd/qvr21H/zy+9E/VVVb939nLUnzv+a1E/nf2LqH8YvzgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA2DR7XQytJS1K+G/X5UV403+1G/8YkH4QpV/+VL/yDqL/YOov4/W/mVqH/l+ktRv/Z+lNf6+8dRP3owjvredvb5VFXVUbZGTadRvpjNsv2nuuzfNrp+1m/e/ctRP/kg+zy70VHUV1XVGz8a5fub61E/fCq7jlZuL0f9/eOVqH9xmB3/5c03ov7vPncn6quqDi5cifqN9VHUDx5k/UksDrNr7/bm70T95z7/QtQvD7PrqDt7Juqrqg6P/l7U96d/M+rXbmbPu/54EfXjM+F7+a3szb/e/62o33/hF6O+qqom2Tuk283+hsUJXoOpbpTdn9cmvxz183ejvO5e+v2oXwrv/aqq2994Nur7G5Oov/f9z0X9hZe/HvW3j74U9cP72b32h+Ps771xtBn1VVW9afa8qPk8XuPD4BcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBh8LgP4GF6vWym21//7ahfvvd81N+6sRH1VVXf/+SVqP+ZMzei/ovn3oj6P9l4Mep7syiPzVaGUd/b6eI1FtPwj5idcp/qhX9zvx/l3TB7BHyw9BtR/+Izz0R9VdXhqz8U9fOLvx/1vXe/GPVbr2f3/h9/46Wo/1vDcdRfXdqO+ht3zkR9VdXZ40W2wTzsH4HFZBr1g/De2T08jPrrf5S9c3rXoryqqj4V3m/vvH0v6s8d/t9Rv7vz81G/MsreCXXnfpTPrsyjvjvM7s2T+OjdOVXVZe+dbhB+lQzP8/CtH4/66Yt/EPVVVcP7Pxf11z71nai//vrnon48zZ5f3TS7tjdfj/L622/+O1H/2jvZ99+qqmevZ9+nZmd+N1xhM+z/v/nFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGDyqhWazWdSPj4+jfnNtLeqnb+9E/fl/eTbqq6r+680vR/0bn/6jqP/W9lNRP9ruon75/jTql24fRH1vJ+vnd+5F/b/eaJ71i0WYZ/1Hzjw7/mvHvxT1h9+P8qqq6tZXon5pOIz67MlStfLNN6P+2Xoh6r/5wQ9H/WwpymvzftZXVa2/P4n6/n74qU6zZ8uJ9LLn3dbNL0b97p1+1PdWshN3/uI/jfqqqndu7cbbJM6srkb97kH2GX326X8Z9a/MfzTqjz74qai/9slvRH1V1Xd/8IOo3zz4+WyB8LvUIxHea12X9fMX/zjqr547F/VVVXePsvfO9v5+tkD4nhr2s3tn8YMPov7shexefu+fPBv152/l343WX7ke9ZevZOf5KJwrHsYvTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfCoFjqeTrMNjo6ifDafR/3K/v2ov/Anw6ivquqP16P+17/3xajvTaK8tn6QfUbDneycdYfHUT9b+q2oX31hKeqrqt54992ov7r/5WyB+SLrU+n+Z7MoXyyy/b8++V+j/rnnnov6qvxenv2rH4n6/oXs34vOvfAnUT+++Y2oX3nnF6O+etnxz0f5Y753HN77O/tRvxhnz4pHoRuNsg36/XCB7Lx989vfzvZfVc8t/wdRP7uXPR8PLo2jPr0u3rhxI9v/UXYd9S7/06hfHV2K+qqqpy9ejPp3Zr8T9Vs3s+8JJxK+F+L3TtdF/TD8m69Xdp6rqoa7u1E/O5e9p+7Ofy3q925nz6P1zY2oH945iPpr4Uc6vLOXbVBVFy//86hPvx4dTcIvzQ/hFycAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoGHwqBYaHx+fap969+7/FPUv7f778RoX7lyI+rPf2Yj6xSCbe3tH06jvxpOsn2T77+38dNQf7OxFfVXV1fpMtkH6TwmLebjB6e5/MQv3P83O2aVrV6N+NBxG/UlMuy7cIPub777yqWz/oW71Qdb3s4u03+9HfVVVzbILaTHJnhWLU36+PxLz8N4P7+Xn+r+a7b+q5ts7UT/67Pei/mCcXUuzO3f6yDXkAAADeklEQVSj/ujgJ6K+ervZ/gfjqL+zk32eVVUH42yN9P5chM+vk1iE939Nsmdw+MSu6mVbDD74qXSFurf121F/cXY2689m/e3796N+7eAw6tNzMLqbvafq3P8VrlA1na1H/Th876T9w/jFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGDyqhQ4PD6N+OBpF/aDfj/prTz8d9b2Vb0b9SVx/80bUPzX9lWyB0TDrw890MV9k+5/NorxLj7+qFtNptkH6N5yy+DOt7DOtedbPwnO2F973J7EyyK6LxWwe9V14H1Svy/rwGl3Mw/2Pj7O+Kr43F2Gf7v9EwntnEd47XZedh8UiPJ4TfEZ3Lvxe1G8ersVrJEbpOYif19m9vHtwEPWzcP9VVePJJOr39vai/oPFr0X9iZz2eQvvhUdhGt5vk/BvTt+d/fC9M9/bj/puPI76D5Z/M+qvzi9FfVXVdnh/pufgOLw3H8YvTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAw+BRLTQZj093gdEoyvuzWdQfnuD4R8Nh1F86fz7qt8e/G/Ub1/9S1Hej7Phji0WWh+esqqrm2Rq1mId5uP/TFh5/aufBg6hf3diI15gcH0f98nSSLXDcZX2/n/WpLjyeVHifVVXVPLwP0jVOci+n0nthnv074qLSvzk8nhPcy3evX882uHo1ytN7M9t71SLcf+rM+z8Z9TfOfS1eI/2u88z+l6P+yik/Lqrq9O+d+TTbfy/8o0/wXk6v7Qe7u1HfD98js/AZeed8fq0mVvorUX+S78wn2SYx/pCeL35xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAausVi8biPAQAA4CPNL04AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABo+H8AcwC9P5CBZn0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('PCA', d=64); # 4x compression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the reconstructed samples are in some sense *better* than the originals since the original Poisson shot noise was associated with a small eigenvalue that was trimmed!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Factor Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Factor analysis (FA) often produces similar results to PCA, but is conceptually different.\n",
    "\n",
    "Both PCA and FA implicitly assume that the data is approximately sampled from a multidimensional Gaussian. PCA then finds the principal axes of the the resulting multidimensional ellipsoid, while FA is based on a model for how the original data is generated from the latent variables.  Specifically, FA seeks latent variables that are uncorrelated unit Gaussians and allows for different noise levels in each feature, while assuming that this noise is uncorrelated with the latent variables.  PCA does not distinguish between \"signal\" and \"noise\" and implicitly assumes that the large eigenvalues are more signal-like and small ones more noise-like.\n",
    "\n",
    "When the FA assumptions about the data (of Gaussian latent variables with uncorrelated noise added) are correct, it is certaintly the better choice, in principle.  In practice, FA decomposition is more expensive and requires an iterative Expectation-Maximization (EM) algorithm.  You should normally try both, but prefer the simpler PCA when the results are indistinguishable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Umspel93/f/mc+db81VPbC7yeYo2bItSg6NCKbi2BbiCYiTGEayMQJk7V1WBrzyygsD2SRANlkEQWAkgWErMpTEhKHYoilFFJuiyG622NXd7K656s7TGd4sDG8ENR7/CnXrdrU+n/X3vO+5532f9zn/OovqdV1XAAAAfLL+Rb8BAACATzuDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACAhuHzOtFf7P/n3fM617+PwfZW9oLJJD5Hd/NK9oJF9hEtNrL3NDg6i/qun83V/dPZ+R7/4Cjqq6q6nd2oX4R96v9a/uNe0n/a1g1chHTdVH361s7w1s2o79ZW4nOcvH4566+Oon7z7f2oH9x9FPXzO3ejnrbPwtqBi/BJa8cvTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACAhuFFv4GLstjZjfrBl9+Mz9E7PIn62a3tqF+sDKL+bHsc9f3ZMuqHh9nt1D+ZR313nH2eAJ8WBz//uag/vJE936uqZuu9qF9Ms+Ofbm1G/aV3JlE/vrwV9anFD94+1+MDn31+cQIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGoYX/QYuyvCVl6N+sTGNz7FYGUX98Y1x1M8nvajvBlnfW3ZRPzjNbqfVO6dR37t2Oeqrqvo7+9kLdnajfLC9lR0f+Ew4+5VfiPrR4Tw7/na+PR98fhH1Gy/vRf3je+tRf3w92zc338v2wMvfvhf1gzffiPrFu+9FPfDZ5xcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBheNFv4EXRO5nHr5ldW8le0GX52WYv6heTrB8dRnktB9nxZ5ujqO+G+Zw/Xi6jfnDjetQv7t2PeuCzYfV7H0b9/i9+LurHO+GGUFXrL+9F/avbO1H/d772q1H/93/wV6N+Z7gd9esfXYr68f1wU4PPqMHm5rkef7GXPYteJH5xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANw4t+A8/KYHsr6rvZLOqXq6Oor6o6vTSI+tlKL+pPrmT9MvwTluPs+JMnXdR3g+z4vUV2/KqqOsuu8+Le/fwcwB878zt3o36891LU3/t6tn9UVf3M5YdR/1+/9BtR/1dWT6J+9rVfi/p/0PuVqF/85nrUn11fi/rJ45tRX5XfF/AsDDY3o7539XLUL1enUT882I76qqr57Q/i11wEvzgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA3Di34Dz0pvfT3qFzcvZf3aKOqrqubTXtavZv1svYv6XpZXHYbvZy3rTzezub0/y2/X0WAQ9YPtrfgcvFjiazyZZP3GWpT3Ts6ifv7Tj6Ke8zG8dTPqzybZs2jjgyivqqr3d7N97cOrV6L+yeTHUb8xmEb9K1u7UX//1e2oX7uT7VHhyocLs9jbi/rlz30h6p98KVvLK0/y71Ibj3eiPv2bnxW/OAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADcOLfgOfZLC9FfXdwUHU92abUX+2mX9UvWXWz9azfnCW9fPVLutXelE/PYryGp5m72fy8Dg7QVXVYpHlO7tRP7hxPeq5eL31bKEtr2TPioMvZH1ly6DGX7oZ9dN37mYnqKr5Tz+KX/PHzfxO9rmOX8ueFWvD7PlbVXXn+1ei/p9t/Mmo//2jl6L++ng/6h8cZmtztpp9Rt0gyqu7nH0Pqaqq8L6AZ2F4K9sX7n51GvX7v5x9wdv/yWrUV1Wt3Hkj6oc/eC/qF3t7Uf9J/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMn9eJBttb53uCl25k/aKL8pNLg+z4VbX3+azvn2X9bCP7G8a75zsnd71zPXwtpvnt2h9k1y29Txf37kc9F2/+ypWof/y1tah/+PVF1Pc3ZlFfDydR/tqv3cqOX1WTK5tRv/zeD+Nz/LHz7beifO1nvhyfYvvK5aj/yfEbUf+D11+O+tFqdm8v7qxE/Wa21Gq8N89ecBpuylU1vHUz6ud37sbngD+su5Q9s/dfz47/D7/+j7P+8l/OTlBV829lz6/e3l58jmfBL04AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQMPweZ1osbMb9YMb16O+t3cY9Qc//3LUP/yFZdRXVV1/41HUr4xmUX/7dvYZ9e9nc/LoIMprstNF/fAw+0wHJ/Oor6rqpqOo700mUT/Y3op6nr30GiwX2X2690aU1zd+7sdR//de/tWo/7WDn436/276H0d9VdUb/8da1E/fzD6kxbvvRf1nwfDWzewFj7M9s6rq6m9n/dYfrET9wz+1GvX9Wfb8Pbnci/qVh9keMr6ffU+AF0Xv5CzqZ9ey71N/fe0o6t9++a2or6r6327+xai/qG9ffnECAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABqGF/0GPsni3v2oH3ztS1E/n2Yz48rN/aivqvpHX/lfo36jfxb1/+3oP436dx69HvVb73VRP95bRP30wXHU907mUV9Vtfy9H8Wv4cWy2NmN+v5Zdh+t3utF/ePT1aj/6jjra/33ovzXv/zV7PhVde+Lr0X9rZ/Ep/hjZ37n7vmfJDxH98t/Juov/eg06udrg6ifPs7W2sa7B1Hfe5w9K57Gc7nO8Id0w2ytDXaz/tsn2fe7j04vRX1V1fRRdo6L4hcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBheNFv4JMMtreivrd3GPUrDzei/s7uStRXVX37+AtR/3cv3Y76/+jq21H/o9FrUT+f9KJ+9WgR9YuVUdSP7z6J+qqqZfwKPut6738c9Zevrkb9j996Ner/i/FfiPo31x5E/Tsf3oj6qqqrB138mkT6fOd8DL71O1E/uXUz6ru1bN/sHR6f6/Hnd+5GPbwoevPs+9f2D7Pvd//gw78S9b93+6Wor6p6fZZ9Yxtcuxb1iwfZ3vlJ/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQML/oNfJLFzm72grAfXduO+uvf2oz6qqr/fv2Xov7+F7Nz/O7OK1E/POxF/erDeXb83dOoHzzZj/ru4CDqq6oG21tRH993vHDSazx9+27Uf35+Pep/+NMvR/3vD7J+K1tmVVW1+cFJ1Pfmi6jvoppPi/mdbC2khrduRn3v8Pic3gm8WOY/uR3161+5FvXv/bPPR/3lnfwpv/LuR1E/f/AgPsez4BcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAwvOg3cFG67/4g6i8NfjY/SW8jyv/pjf8w6hfjKK8r7yyifrQ7i/rBg52o72bZ8Rc7u1H/PAy2ty76LXDO5j/9KOpHBwdR/+pHl6N+fiO75/pH2TqrquofnkT9/PYH8TngD5vfuXvRbwFeSIPNzahf+9GDqB8dXor6WmZ51Yuzj/jFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGF70G3hRdL/9e/Frruy/GfUbtzajvjdfRn03zObk0YOD7PizWdQv7t2P+sH2VtRXVS12duPXfJqOz4snvScG4fGH+4fZC05PwzO4rwFeJIu9vahP953xbJ69YB72VZW/4mL4xQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaBhe9Bv4LFu8/W7Uj3euZyc4PY3yxc5udvwb5/t+UvH7hxeA+xqA52mxt5e9IOwHm5vZ8V8gfnECAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABqGF/0GXhSD7a1zP8fi3v1zPX78N5yens8bAQDgM2mxt3fRb+Hc+MUJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGjodV130e8BAADgU80vTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGobP60R/aeW/iv6n3d4we2u98TjrJ1lfaV9V3Sj8eAeDsM/m3q4fzsnnPFb35svsBbN5fo6Ts6jvzrK+Tk+j/J8//h97Sf/NL34xWjcrk0n0fiajUdQPwntosQyvcVU92tuL+rVH38xOEH5G1Y8uWdU8u0+745Os/9xvRf0ofJY+jVn4Nx+H6+Y3/uAPwotQ9StX/ptsz0mf8dPsPurC48f7R1XVIPyYuugjqt4i62u+yI6fPuNPw+f70XHUL4+Oor6qaufqv4z6Qbjvn4Z71O98/HG8dtJ9Z21lJTr+KPyb02dY+jyqeop958kvR31vOo36+PtguNd2s1nWh/vUg41fj/qqqkn4PT61WGTPo+98+OEfuXb84gQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAyf14l6w+xUvekk61dWor5by/rlevZ+qqqW01HULyaD7PijXtR3g7DvZ32v67J+nvWjg3nUV1UN9k+ivr9/HPXdMvsbUiuT7L5bD9fBWnj8cbiOT2azqK+qOnn/57MX3LgS5d1knB0/XDe92SLq+/tHUX+2yI5/dXMz6quqlstl1KfXeRH+DU+jl17n1WztLLbWon6+la21+Wq2H1Tlz/gKH1+9RfaCwVl2Hw2Os2f84PAs6vu72fOr/xT36WQc3nehxSC/L1LpvrMa9hvhPpXuO0enp1Ff9RT7zvWrUb5czT6jeC1nS636J+HaCfepaw//QtRXVZ2+/O34NdHxn9Fx/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMn9eJeuNx1q+sRH23tR71Z9fWov7k6ijqq6pON7K5dL7Si/pl9pHWMvwTunMeq4fHWT99nN+ua3cGUR9+pNWfL8JXZCaj7KKtTSZRv7WWrYPxIPs8H927F/VVVcsvfi7qT65Po362lv0NXbYsa3DWRf3kSfbsmnzw16O+3/t/o76qajrNPtPUaXhfP5VpthaWm6tRf3oj6w9uZc+v08vhjVdVi+xPrspu1Rqchf1J1o8OsyfwZDe7T1fuZx/QqJdfg9U7fy7qj2796/gc5+3c953VbO1Mhtna2Ts6ivqqfN85vpl9R52tZV+oluHXnf4860eHy6ifPMqu2WiUf18b//TPRv3xS78Zn+NZ8IsTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAwfG5nGo+ivFudRv3ZtbWo33t9EvX7r/Wivqrq9Poie8HaPMqHk7AfZe9nOAzff2hvbyXqT25n1+zfyu67wXF2jt7RadSnBv3s3zaGg0HUj8P+8DT7e3c2/2bUV1U9+lq29o9uZmtzvraM+gqX/uAku2aTx9nfu3n1RtR33/9m1FdVbaz/3/FrEotleA2eQhfuOfOtbO0f3sy2z90vR3nV6wfhC6q2N46jfrHMbu7j03HUHxxlfe1l12zyMHt+rX2U7Tlbk+z4VVXZaq5avf9LUf/Bwf8UniGX7jvjYbYWRuG+czrPvuvcH/21qK+qevgns3vj6FZ2/Nl6F/XdMOv7s2wtj/aza7Z6J7tmW6v5eDEdZPfd2v1vRv3y6CjqP4lfnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAw/B5nag3GkX9cnUS9aeXs+Pvf64X9d3P7Ed9VdXXX/o46t9cexD1V0cHUb81OIr6tf5p1J91g6j/ncPXo/6f9v5E1FdVnTxYifrV1WxJDEfnu4Rm83nUn5ydRf3bOztRf3j5b0X93V/MPv+qqoOvZn/DjZvZ33B9LVs3/eqifuc0+5s/frQV9SdXVqN+Obwa9VVV9f2/nPUf3onyO8N/kh3/aYRrc76S9SeXsz2kXjuM8v/yq7+dHb+qvr72k6hfdNm/nd6db0f9B6dXov6dg+tR//bDrH+yna21Wo6zvqqGR9k5hrPsGf+V/t+J+qexWC6j/mQ2i/oPHmTfdR5O/kbU3/lG9oysyvedmy89ifqX13ejftxfRP3RPLtXP9zL7tPHP83W/nKQfYevqro6y84xnmefUb8fPrM/6TjP5CgAAACfYQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAEDD8LmdaZSdarEyivqz9WwGPL2yjPo/dfNu1FdV/SdXvx/1Xxnfifprg+Oo3+r3on7Uyz7To+Ui6lP/z/qX4tcspitRvxyF/5bQP99/e3hycBD1Zz/9s1G/fPOVqH/0tWnUH3zlLOqrqn7uCx9G/S9euh31r00eRv20N4v6B/ONqH/r0qtR/y/HX4j63YOtqK+qGu9uRv3qbB71Lz/6m1H/NLpB9rxbDsN+HOW1vnYS9T+3+kF2gqr689OdqB/1BlH/eHE/6j+eZh/Sj1ZuRf2/mX4+6v/FMttDjh7ma2f97iTqB0+yPap3mj9TUzvhvnPy3p+J+uWXPhf1j342+4yeat95M9t3vnH5J1H/xiRbO5v97Hmxt8z25h9uvxz1vz76StQ/eHIj6quq1u5lz4vhzlrU95/R2vGLEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQMHxeJ+oG2YzWDbN+MYny6lYWUf/Syl52gqp6dfQo6l8ZHkf95f446lfDPjXrDqN+Gc7ty2Uv6quq+l38kk+VtUffzF5wfTvKT65Po/74RnYNrt3cjfqqqq9fej/rV38S9a8Os/c07S2jfnc0ivqNwUnU3722EfXfC/uqqtNLg6ifbmT30eBoJeqfSnbZqrfMHhbhbVGzRfaZniyz+6iqalHZ37Dey85xNdzHq7I9bTa6H/U7q6tR/8Otm1F/e2Mz6quqZmvZde6m2TXoD7PjP43J/V/KXnDjSpSfXM/W/9HNcN+59enbd14P953V8OvOUfhdZ7Of7Tu3t7Nr/PH21aivqjrbyO7t5Uq4dibP5juwX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgIbhcztTr5f14UjXpSNgv4vyYX8RnqBq2ptF/SA8/iD8TBfdMurnlf3Nu8vsM30w34j609NR1FdVrZ9lfX+WfUa1DPvUOPybJ+Mon69kC2e+kl3jyytHUV9V9cr4cdTfHO5nfbjQJr3sM93uz6N+UXei/o21R1H/3Y3Xor6qaraW3XeLlawfjM5/6+ktsudXf57d24OTKK+D4+w+ev/sanaCqvp48lH2guFxlA8q23PSPWray9bORj97/2vDbENYjrJ7oqpqGT5fun745SXtn0LvBd93Lk2z+6Kq6tZoJ+qvDQ6j/uoguzFWw31n1mXPu5Mu+3tfnmZ9reTfmReT7DNahn03TL9l/9H84gQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAyf25m67pyPn/a9KJ8vB+EJqmZd9vEuwuOfdPPwFVm/u8ze0e35VtS/f3w16hcHo6ivqhoeZzfG4HSZnWCWXoPQp23dfAqNKrtmk162Llf74+z44bq/NjiK+puT3ajvT9MnS9VinP3N3Sj7N7humD9PY+HaHBxl/fAoex4t9rLP9J3DG1FfVfXjafZMrXoY1Wv97F6ahc+Xwy77THcXa1F/PM+O359n3xOqqvqL7I/uLcM9Z5Gv53N33vvUp9Ag3DxHlT3zRr2s71d2r270z6J+a5jtU/1ReF9X1XKY/Q1dP1yfvXw9/1H84gQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAyf14l6yy7rZ8uo78+ivGrWi/K9+SQ8QdWjxXrUb/ePo/60y/7oRWV/893w/f/+yStR//be9agf7uS363g/u+8GR2dR3ztNb7xMd5Ydv390EvWjw2ydjQ4GUf/oaC3qq6ruzbai/vF4GvXXumydrdY46ge97N+jRlFdtdrP7tHBILvGVVVddpmr62XPlkr7p9A7Po364UH2uU53sj1h/Cj7UH//yY2or6p6Y/VzUb8M/+302mAv6lMfzq5E/Qenl6P+wWH2PBoe5vfp8GgR9b2T7Bnfzc53z6mq6k6ytdMP19p57zsPj1ajvirfdx5NVqL+WncY9ZPuuX09//cyqPA7/NM84s9/W3gm/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMn9uZZvMo758ton503EX9cH8Q9R/sX476qqofrr0U9bMue08bg+OoP1mOo/79s6tR/53d16P+vbvZ8Vfv9aK+qmrlwVnU93ePor47yvrU/Oa/ivrBB78Q9dPN1ahf/Xg76h9+tBX1VVX/ZvP1qF8fnET9oG5H/Ul3EPXjXnafPlhmj+HdeXbNFov838d62eO3el32/K20fwrLvf2o74fXbXVlFPXrV7LrdvfKlaivqvrW6EtR/+RS9p5enT6O+tRHp5ei/nefvBL1T+5uRv2le/l9On2YPY96u9l9utjLnkdPo3v1O1H/3ocfRv1rG3836tc+zu6Lhx9l+1RV1bc334j6jXjfeS/q9wfZd4tR+PXo4/lK1D+crUf94izfd/qzbL315hez7/jFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGD63M81mUd4/zvrx3iLqpw9HUf/+R1eivqrqny+/FvUvrb8c9WvDs6g/nI+j/uODray/vx3103emUb/97jzqq6qm7z2K+uXd+1H/fu9/j/qq/yGqN1ZXo377529HfVXWd2/9paifbW5EfVXVd0evRf3+LLuPfrx9I+pvjXejftLPnl0Hi+z9f/vxG1HfPcnWfVXV6LCL+v7ZMup7i6x/Gv1XvxP16ysrUX96kD1/e9/7z6J+OVyP+qqq9+a3ov7eS9n6vLp+GPWpx0fZNdi/m73/jXeyrzyX3z6J+qqq/lvvRv3ki29F/dH2adQ/jY1wLfy5P/2no37Y/42o7773zag/28i+u1RVfX/4StTvn02i/u3tm1F/c5LtO/3KntlP5tl3i996mO3Lg0fZd+yqp9h3Ztn3/t486z/xvM/kKAAAAJ9hBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoGH4vE7UzWZR3zs8jvrJw3HUb74/iPrqJllfVffv3Yj6j1evRX037KK+f5bNycODXtRv3s/67T/I7om1/++DqK+quvbGd6P++NJZ1H/u8NWoT42G2RKdjsN1sLIS9ZcW/zrqF7/1S1FfVdU/W4362/dfifp3L9+M+v7KPOp74T9Hdcuw38mu8frt8FlXVdMn2d88OMzWTc2y4z+NyWgU9RvhWnj9Wva8rvrNqH7rX/1CePyq0cFW1B++lPUfb2xGfWoU7jlXHmR74KV3su8VL5/9atRXVW3+B9l9tHecPe+Wy/CB8RT6/ewhNg73qcvr61G/vfZbUb/8zjeivqpqEO47Hz14KepvX8q+D/ami6ivXrYWunl2jQdPsmu8/n7+u8z0cfadsH9wMfuOX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgIbh8zpRdzbLXnB4HOXDh9kMuB7VVaP9afiKqrPNQdQvJtnf0PWivPrzLupHR8uonz48i/rx7QdRf+vNt6L+38quwafNbD6P+pOz7BqsjMdRPxpkn+f48f8S9VVV1373b0f92r1J1J9uZo+9xSTru0G4MLNlWcPj7AUrj7N7qKpq5d5J1PfPwkUPAAAD1klEQVQOs74L79OnsVxmz6/5YhH1Z2G/MhpF/cvr34r6qqr67p+P8rW72U44Xz3f5+nwOLtmkwdHUX+28z9H/fbrr0d9VdU8vO/S+3TRhQ+Mp5DuO2dhfxyu/8kwewZfOvsnUV9VVb/7N6J89X72nfBsI/0+mPVd+DNIL7vtapgttVp5FH7nr6rp/ex7fz+cE7rT06j/xPM+k6MAAAB8hhmcAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANw+d2ptksyrtzehv/znCxiPrBwUp8junKKOq7QTjH9ntR3psvs8MfZ9est3cY9WfT/zPqT84uRX1V1XKZ/c1Hp6dRfxbe16nT8PiH4ftPjYfZI2NrdTU+x+5P7kb9xv521K+uTaJ+ORlEfZeuyy572vVm2T09OMzv0f7BcdT3Do6ivjvn+7Sq6iRdOycnUd/vZdf5OFw783CPqqqqdz+I8tX9a1HfrWRrJ9U7OYv67slu1N/8QraH7B1n66Aqv24H4X03m8+j/mmk+87+U3xOiWE/+240GmTP7Kqq0XvpvpPdS8v1cdaPwn1nEO47i2zf6c+y+7p/lK3lqqr+fnYfdUfhfXf2bL6v+cUJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgYPq8TdfN51PfS4y+X2QvS93N6lh2/qgbD8OMdnPMcu8g+o242i/rl0XHUL65n72f36Cjqq6qW4X1xfHoa9afhZ5Q6OM4+09QsXAeDfnaPnjzF57N4/CTq++E1Hq1Mo74bDqK+ws8o1ZsvshfMsmtcVdWdZc+7Llw33VO8p1S6llPp2hmF+8Ei3dOqav7yt6N++PE3or43nUR9anmWPS/2Lv2LqB+cbEb90zy/Zotsfab30Xnf18/jHOe976THr6qa338Y9YOui/rhfrh2Btm+0/XTb82Z57LvpPdd+LxI55BP4hcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKCh13XdRb8HAACATzW/OAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDh/wcQetG3kL7HUQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('FactorAnalysis', d=4); # 16x compression from D=256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Umspfl93vffe885d55q7KrqruqRbLJJmeYgwbIoU6IoS5QU2hRC2kiyCQLYgBeGFw68SYAsAgcBsgiQTbILnMROQlkDRIaKaYtSLIltijItiU2y57GmW1Vdd6o7nSkLLQIILvzzFPpWdbc+n/X3vO+59573Peeps6huOp0WAAAAdzfzoJ8AAADAu53hBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAEBD/36d6GdnvjS9X+f6/6O3vpY9YG4uPsf03KnsAePsVzReyZ5Tb+8o6qcz2a6eORwe7/F396K+qmq6uRX147BPfWPylS7p323XDTwI6XVT9e67dvrnz0X9dGkhPsfBYyez/vQg6lef34n63rVbUT+6ei3qaXs/XDvwINzt2vGNEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANDQf9BP4EEZb25Ffe/pp+JzdHcOon54fj3qxwu9qD9an436meEk6vt3spfTzMEo6qf72e8T4N1i95OXov7OQ9n9vapquNxF/Xg+O/7h2mrUn3hhLupnT65FfWr83PPHenzg/c83TgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANDQf9BP4EHpP/Jw1I9X5uNzjBcGUb//0GzUj+a6qJ/2sr6bTKO+d5i9nBavHkZ9d+Zk1FdVzWzuZA/Y3Iry3vpadnzgfeHo53806gd3Rtnx1/O3590nxlG/8vB21L99fTnq989m75urr2bvgSefvR71vacej/rxS69GPfD+5xsnAACABsMJAACgwXACAABoMJwAAAAaDCcAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaOg/6CfwXtEdjOLHDM8sZA+YZvnRahf147msH9yJ8pr0suMPVwdRP+3nO392Mon63kNno358fSPqgfeHxT9+M+p3fuxS1M9uhm8IVbX88HbUX1zfjPr/9JmvRf1/9dwvRf1mfz3qly+fiPrZjfBNDd6nequrx3r88XZ2L3ov8Y0TAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADf0H/QTeKb31taifDodRP1kcRH1V1eGJXtQPF7qoPziV9ZPwR5jMZsefuz2N+mkvO343zo5fVVVH2d95fH0jPwfwF87o6rWon92+EPXXP5W9f1RVfeTkzaj/zy7866j/xcWDqB8+8/Wo/8fdz0f9+FvLUX90dinq594+F/VV+esC3gm91dWo706fjPrJ4nzU93fXo76qavTaG/FjHgTfOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAEBD/0E/gXdKt7wc9eNzJ7J+aRD1VVWj+S7rF7N+uDyN+i7Lq+6Ez2cp6w9Xs90+M8xfroNeL+p762vxOXhvif/Gc3NZv7IU5d3BUdSP3roc9RyP/vlzUX80l92LVt6I8qqqen0re1978/SpqL8992LUr/Tmo/6Rta2o37i4HvVLV7P3qPDKhwdmvL0d9ZOPPRn1tz+YXcsLt/PPUitvb0Z9+jO/U3zjBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA39B/0E7qa3vhb1093dqO+Gq1F/tJr/qrpJ1g+Xs753lPWjxWnWL3RRP78X5dU/zJ7P3M397ARVVeNxlm9uRX3vobNRz4PXLWcX2uRUdq/YfTLrK7sMavaD56J+/oVr2QmqavTW5fgxf9GMrma/19lHs3vFUj+7/1ZVXf3TU1H/1ZW/FPXf37sQ9Wdnd6L+xp3s2hwuZr+jaS/Ka3oy+xxSVVXh6wLeCf3z2fvCtQ/PR/3OT2cf8HZeWYz6qqqFq49Hff+5V6N+vL0d9XfjGycAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABo6N+vE/XW1473BBceyvrxNMoPTvSy41fV9hNZP3OU9cOV7GeY3TrenTztjvXwNZ7PX64zvezvlr5Ox9c3op4Hb/TIqah/+5mlqL/5qXHUz6wMo75uzkX5o18/nx2/quZOrUb95I9/EJ/jL5xn/yTKlz7ydHyK9VMno/6V/cej/rnHHo76wWL22h5fXYj61exSq9ntUfaAw/BNuar6589F/ejqtfgc8OdNT2T37J3HsuP/d5/6Staf/LnsBFU1+mZ2/+q2t+NzvBN84wQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAAN/ft1ovHmVtT3Hjob9d32najf/eTDUX/zRydRX1V19vFbUb8wGEb9a69lv6OZjWwnD3ajvOY2p1Hfv5P9TnsHo6ivqprOD6K+m5uL+t76WtTzzkv/BpNx9jrdfjzK68c/9mLU/5cPfy3qv7770aj/H+Y/F/VVVY//2lLUzz+V/ZLGL70a9e8H/fPnsge8nb1nVlWd/k7Wr728EPU3//Ji1M8Ms/vvwcku6hduZu8hsxvZ5wR4r+gOjqJ+eCb7PPWFpb2of/7hP4n6qqp/fu5no/5BffryjRMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA09B/0E7ib8fWNqO8988GoH81nm3Hh3E7UV1X99x/6P6J+ZeYo6v/R4Jej/oVbj0X92qvTqJ/dHkf9/I39qO8ORlFfVTX53g/jx/DeMt7civqZo+x1tHi9i/q3Dxej/sOzWV/L34vyf/H0h7PjV9X1Dzwa9edfiU/xF87o6rXjP0l4julPfyLqT/zwMOpHS72on387u9ZWXtqN+u7t7F5xL+7L3xn+nGk/u9Z6W1n/7EH2+e7y4Ymor6qav5Wd40HxjRMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA09B/0E7ib3vpa1Hfbd6J+4eZK1F/dWoj6qqpn95+M+n9w4rWo/+zp56P+h4NHo34010X94t446scLg6ifvXY76quqJvEjeL/rXr8S9SdPL0b9i39yMeq/PPszUf/U0o2of+HNh6K+qur07jR+TCK9v3M8et/8t1E/d/5c1E+XsvfN7s7+sR5/dPVa1MN7RTfKPn+t/yD7fPeP3/zFqP/eaxeivqrqsWH2ia135kzUj29k75134xsnAACABsMJAACgwXACAABoMJwAAAAaDCcAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaOg/6CdwN+PNrewBYT84sx71Z7+5GvVVVf/j8k9G/cYHsnP8u81Hor5/p4v6xZuj7Phbh1Hfu70T9dPd3aivquqtr0V9/LrjPSf9G88/fy3qnxidjfofvPV01H+/l/Vr2WVWVVWrbxxEfTcaR/00qnm3GF3NroVU//y5qO/u7B/TM4H3ltErr0X98ofORP2rX30i6k9u5nf5hZcuR/3oxo34HO8E3zgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQ0H/QT+BBmX73uag/0ftofpJuJcp/86FPR/14Nsrr1AvjqB9sDaO+d2Mz6qfD7Pjjza2ovx9662sP+ilwzEZvXY76we5u1F+8fDLqRw9lr7mZvew6q6qauXMQ9aPX3ojPAX/e6Oq1B/0U4D2pt7oa9Us/vBH1gzsnor4mWV713nkf8Y0TAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANPQf9BN4r5h+53vxY07tPBX1K+dXo74bTaJ+2s928uDGbnb84TDqx9c3or63vhb1VVXjza34Me+m4/Pek74meuHx+zt3sgccHoZn8LoGeC8Zb29Hffq+MzscZQ8YhX1V5Y94MHzjBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA39B/0E3s/Gz78U9bObZ7MTHB5G+XhzKzv+Q8f7fFLx84f3AK9rAO6n8fZ29oCw762uZsd/D/GNEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADT0H/QTeK/ora8d+znG1zeO9fjxz3B4eDxPBACA96Xx9vaDfgrHxjdOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0NBNp9MH/RwAAADe1XzjBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAEBD/36d6EMrK9H/tDs8OoqO/+TMl6J+ZnEx6rvlpaivqpouzmf9/CDrB72or67L+skkO/xwnPUHw6zfO4j6qqrp7m7UT3bvRP0bc1+N+h/u7ER/hJ/tfTm6bl6YfCV6PgvLy1Gfurj3+fgxXS99XR/zv/9Ms+sgfU2srq9HferWxkb8mCfGXzyGZ/L/eaX3a1H/8uFhePOq+sknn4yunTv7+9Hxz46+EPV15mSUD0/l7znDlew9pMLfan8vu8fPbmT33+lrb0X9zGN/FPW3traifn3jr0V9VcX3i6ur34j6c7c/G/XfGP+f8bXzI6dPR9dO+py6fvY67XrH/2/803H2d8tPcLzvI4O5uag/7r/ZvXi5fiXqF1dWon5vZyfq7/a+4xsnAACABsMJAACgwXACAABoMJwAAAAaDCcAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaOjfrxMN5uaOta+dadb3ss04nZ/Njl9Vk9WFqB+uZOcYz4e7twvzcdb3DrIH9HePon5mMon6qqo6CF/ivV6UX9z7fHb8VJf9jT8486Wovzb321E/mA2vgzvhdVlVb8x9NerTe8W5zc9FfTfIXkOXDn8p6m+Ofzfq93Z2ov7J8DVRVfE/qU2Ho6iPX0f3wenbn4367vxS1B+dXo76nUvhe2BV7Z/N/nCTQXb8wXZ2LawuZffTxXF2j59uZO+xJ8P7ey0dZn1VXZ759ahPr4Uu/RnuwcJC9nt98/DrUf/o+G9G/XSavY/sX/hW1FdV9Waya+fatWtRf3H/F6M+tbyc3V92l78d9SvXPx3192JxaeVYj//E6G+8I8fxjRMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA09B/0E3jHTCdZ32WbcTo3mx2/qoYr2WMOT2R/juFS9jNMwr/2zDDrZ++Ev9OZLjv+0Sjqq6pm9uey/ij7ocNXXawLf0fTSfY3GB4eRv25zc9F/cxi9vuvqrp09IWoH1/4TtRPTn036i9fuRL1j/Z+OepP3/5s1Nd4HOXdbC87flVNw3NcW/+XUb84uxL19+Lw6Cjql1aWo36yuhj1B6ez94OdS/m/a+59IPuZF9f2o35zZz7q989m1/+p+VNRv/TmQtT37mS/n7p2I+ur6pGHzkX9ra2t7AS9/HpOzc1mr9XBXHifP8x+hp1TvxP1T549H/VVVdt7e1E/3/ty1I8n21H/6DR7H6mN7NNI/6k/jvrhlfR9J//MvLyc3YN3d3ejvusPov5ufOMEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADf37daJztz8b9a/0fyPqu7m5rJ/P+smgF/V/9pgu6kfzWX+0Gh5/Icqrd5j10164w6fZy693kP3Nqqq6vfAxh9kP3Y3H2fFD08n0WI+fmgmvm+rl1003mUR9//Jfifof3vknUf/0hz8c9fVmdl1W+Bpa+9hLUX94dBT1VVW3d3ejfnm8HPX38pxSa1d/Iuq7hwZRn74ndNPsWu7u4dI/d/521P8nj347P0ngVx/5eNS/Nfdw1M+Eb2pLLw+jvn/+W1FfVbUyvxj1m+G11t3DPTU1Cu9J53d+Luq7tfmoX34ru8cvP7ER9VVVN29/JnvAI9nf4fHTL0f9G9//UNR3qytRvzL/fNRvLi9FfXX59zLry9n7yO1bt6K+G7wzk8c3TgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADT0H/QTuJsnp78c9d0g+1GmywtRPzwxH/VVVUervaxf6aL+cD3Ka7wwjfreXvZ8uknW1zTb7YM7g+z4VdW/lf0Nqhf2M8f7bw+v9H4t6tdOnYr61V72IppcPYz62We+H/VVVb0uex0Nt38m6j984T+P+rd3/lnUrx8No753Ovubbb+Q3bteOvjfor6q6qnel6N+Op5E/fJ4HPX3YvLkd7P+2k9E/czuctQvXMnuFUsr2fGrqvYOZ6P+I3OXo/6nFrK/84/Mvxn1f2/7P4768R+vRH13kN2/JsPPRX1V1evbvxr1qxs/mZ1gEL5H3YPDo6OoX5kJ3/vD+0X3wT+J+hcv5/eX6SNrUT+7cSfqn3744ah/47ns89o0/AzcDz/rTD5wKepntvaivqpqcW4u6gdh343emcnjGycAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABo6N+vE3Wzs1nfddkJ+tmPMu31on68kG/Mw/XsMQensp/56MQk6qfhr7R3kD2gG0+jfibsu1HWV1XVeJz1o9HxHj80CK+b4dFR1J/e+bmon1ldiPp7cXJlJeqv38h+5vAyqKcefjjqry9+J+pnui9GfbeyFPUf3P07UV9VNQ1fR3U0DPvw+PdgdjCI+rknvhv1B69+IuqHFz8Q9dtPpq/UqpPzh1F/bbQW9XuT61F/sZf9nS+duB31V89mz3/hkRNR378Tvq6rau21X4j6yXQ76rtjfs+pqjp9+7PZA8Ln1Hv42ag/t3oy6q99/0NRX1VVm29G+c//9Z2o39jK+uEzH4n680e/GfU3t3866jc+sxz1y1fv4bPC730qO8fyraif3r6Hz5D/Hr5xAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABoMJwAAAAaDCcAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgIb+/TpR1+tl/ewg6+fmon4aTsZunPX3opuE/aiL+ulgGvXj2ayf9LPn042z48+Mw19QVXWT7BxZffwu7n0+e8Cd7CfoFsNbwGA2ysfX/mp2/Kr6oyu/EvUXDp6M+m5hPupvnvz5qN/7+ErUT8I/wfpzW1F/6fy/yU5QVddu3476t27ciPqtg1tRX/W/hn3V7v5+1B8Nh1E/PvP/RP1g+9Gon72dvadVVV25fDLq/8ncj0f9K6dejvrBzCjqbx8sRP1RdqnVaCn7HDJ4ey87QVVtLfxfUb9+Zjnqb21l1/+9SD+vpe+b84Ps893lb12I+pmPPRz1VVUfO/+dqF+YPRf139rP3gtH69nf4FOrT0X9/7xyOup3P5NdC3svLEV9VdXC9Ueivn89+x3dWPqtqL8b3zgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAQ/9BP4G76odPbW42ykcnFqJ++1L+q9q9mPWTwTTqZ0bZ8btpF/XTXnb88VzWd9mPW9Mue/5VVdO5QdR3g6yfHg2jPtaF/7YR/ooOz/9B1C8thNfNnTtRX1X1yP7fjvrp9CjrL56L+mufXov6rY9nz2flZPY7uv7GatRv/cF/EPVVVevXbkT9qY3Xon7p0T+M+nuxuPGTUd8tzEf9YJTdgJ+49LtR/9EfZPeiqqo/fO1nov7Vjz8W9T+4dCHqB4vZtTC6ld1fVsPbSzfK3nS6cfgmVVVr25+L+v1B9rro9cI35vthkv2ehuG10z/3UHb8xfzaefnatag/DH+Gw/UPR/3a778W9Zf+1umov/2x7LPLf/vJX4/6/3rhF6K+qurOn2bvbct/9FeifvXpP436u/GNEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADT0H/QTuKvpNMsH2Y+yd3Y26m//yCTqq6o++pdej/rZmVHUf/e1i9nxX17I+q0or8F29jeb3RlHfW9vGPVVVTXJntO7zjR73XW9XtTfun076pcWstfQ4vVPR31VVZ2dy/pRdt2Ml7Jrf+9C9hr63Ed+EPX/4KF/GfXPPXMu6v/R8peivqpqZnQ66teOsmvz8Oj3o/6+GIf3+PBae+WPHov6rfV/FfVVVWvLn4z6ae9E1O/eyK6d4UrWzx9FeS1uZH+z2a3wBIdhX1Xd8nLUz9/4a1E/2NmJ+nsxHWfvzTdPfTPqH6kzUV8nsuP3X8jfdz7yE5ei/ub2dtSnn3e++AsHUT+dZp+Bn3r0etR/eTn7QPirZ7LjV1W9vLoe9atL2eeR+cEg6u/GN04AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQ0L9fJ5qOx9kDjoZR3k0mUT9ayDbj4sPbUV9V9V9c+s2ov9g7jPr/ZuGno/7rlz8V9ae+l/1OewfTqO/fyV4Tva39qK+q6rZ2on56cJCdYJi9TlMvTL4S9R/s/lbUXzj8pagfvZ5dN93SfNT/2Umy18X6Y/826rfeWIj6/u6lqJ9Mu6j/yGz2fD4yuxX1/9Oljaivqrp95uGoX1lejPoTL/1U1N+LK3NfjfrzZ89G/drSUtTvbGX364WLfzfqq6qGw+yePfd2dv+aGWb3+Ek/uxamvSiv/l74HvXqtahfv/iHUV9Vtfnqx7MHzGS/o+ruw793h5/Xbl29GvWnb34m6mcWs/vLnVPfiPqqqtn+J6J+Pbz+K/x8dPX27ahfmc/ea69sno76bx9m94ob+8tRX1XVhTMhvRZmZt6Za8c3TgAAAA2GEwAAQIPhBAAA0GA4AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANDQv29nmkyzfjyO8u7gKOrntrPj39iZi/qqqs3xYtT/2Nwg6j+x/HrU/+aJvxz16a6eGYV/41C3dxA/5lbvN7IHLIXHv7ORPaD+l6hePXUq6t88/FrUXzr8pag/evQ7Ub++vBz1VVVvXL8e9TMHp6P+oYt/GPXbP3g46n/7kWei/u/3D6P+iYUbUf/WrfWor6pazZ5SddPs2p/OHP+/2S0sLER977if0+2tKF+Y/634FJ988smof+nq1ai/dfuvR3137VbUTy49FPXjxewjzPy5P4j6kysno76q6t8d/NOovzT6G1G/+8izUX8vrq5+I+qf3v7bUd/1w4+ec9nnr8WV/yg7flX9/rPbUf/xj78a9b2D7DPnv97/RNSfXnkx6us72b3iHy5/OerffOVM1FdVPXpzFPXjzc2oPzzKdsLd+MYJAACgwXACAABoMJwAAAAaDCcAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGvr360TT8fh4j7+zE/XLz9+O+rV/cybqq6r+4dp/GPWff/QHUf/iTvaceju9qJ8ZTaJ+dvMw6nvbB1E/2dyK+qqq0VL2utu6dSs7/tFR1KeGh9nvdDA3F/Vv1Fej/szkbNTf2t6O+qqqs6MvRP1kayHqr13PfqcrL7wU9Y9vPxb1v/Pij0b9/31iGvULN7qor6paeSP7HXW7e1E/GY6i/l6sX/901E9uZW+HOwvZ665bXoz6UysrUV9VNZ1mr43bb3wq6rvV7H73iR99M+pvbj8X9W/czP7GB1eya+3Z69n9sarqsd6Xo35j5etRv9TLXnf3Q/r5bv/it6P+yfPno/7ayx+L+qqqyZmT8WMSg++9GvXDjz4e9dc2N6P+wivZPftKZX+DUzeye1FV1cLr2efywye+G/XD7Ee+K984AQAANBhOAAAADYYTAABAg+EEAADQYDgBAAA0GE4AAAANhhMAAECD4QQAANBgOAEAADQYTgAAAA2GEwAAQIPhBAAA0NC/Xyd6c+FrUX9x/xejfrJ7J+pr+utRfv53vpgdv6o2Dk9F/W9c+qtR302ivE68OY36+RtHUd/f2I768eWrUX916beivqrqwp0vRP2ZwTjqpzWK+tT+7m7Up9dNN8huAd31XtRXL+yrajo5yPr9rK9x9jeeeeK7Ub9w5fei/uIrn4/68dpS1HfT7LqvqprZ3ov66VZ27d889c2ovxddP3x7m8n+HXF6lN0fayt73V2/9VR2/Kp68fTvRv36QnaO7mgY9a9tbET93uFh1He3NqN+c+VfRP3ZzezarKqqmS7K+/dwj3y36fqDqF9488ei/s7Jl6J+6cKzUV9Vtbfx6ai/svp21F/4aPZavfryQtS/OrytzsIgAAAE4UlEQVQe9bPXs+d/7tvZ63SwGb4vV1W9fjnKD09k96OVKz8e9XfjGycAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABo6N+vEw3m5qL+laNfi/onp78c9Vc3NqL+4dG1qK+qOvutrD/x/GL2gF6X5XujqJ/Z3o/6OjyK8q6fvfz2dnaivqqqesP8Me8iH5z5UvaA6STrx+Mof2nylag/deJs1FdV9Xq9qJ8bDOJzJGZf/LGon1ldjvpukF1nvb2sn47D10RVTQ8Po36yfxD1w5nsXnEv0p87/Zm7p78X9TMz2b9TTl/5eNRXVa3e+qmov77yT6N+bnY26qcvfC7qK3xPmO5l7wknn1iN+rfrX0V9VdXqzc9E/dr2T0T91vnfj/p3oy78PLj9g2ei/vaJ3476qqoTm1tRf3Xzqah/vf551J849UbU725k75tL4T17/uUbUT/d2Y36qqq3V78R9etXPp2dIPxscTe+cQIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABoMJwAAAAaDCcAAICG/v060fDwMOoHs7PZCbLD14W9z0f9dGaUnaCquivXo35way47ftdFfaX9dJrlo/B3FB7/XlxZ/HrUn9v83DE9k/vjhclXon5hcTnq11ZORf325mbUV1VdDK/Nrj+I+unwKOtnxlE/2d6N+m4Q3oYnkyif3st1Ns5+5mnY7+3sRP29SJ/TrYd+L+qXDhei/vAoe92dmLmHf9cMf+bT6+tR3wuf0/jqftTPLMxH/eQge+Mfhu9R4/D3WZW/7rpeL+pvbWxE/f0wHQ2zB8yEn0Um2T1sfz973VVV3Tr6Z1H/VO/LUf9o/c2of/3Wr0f9angtL+5k71PTvex3Og2vzaqq/Wl2juwnfuf4xgkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABoMJwAAAAa+vfrRMOjo6h/YviFqJ/OTKO+plnfTSbZ8atqsn+QPSDsu14vO37Yd10X9dPwdzodj6N+cWUl6quqBrOzUf/mwteifn93N+qr/vewP17pdbl161bUPzH+YtRXVXX98HUdHz+77V1d/UbULy8vR/1xG4fXWVXV/v5+1A8PD6N+tJu97u7JNLtnn7r+6fD42f1uYW4uO/7xXgZVVbW5sxP1c+H99Er9StQ/efilqO9msveom5ubUb8X/n6qqtbT663L/v36Xu6pqfR6fnPx68f0TP7MILx2FhYW4nOkj5lcDj+vhe87x339X5nPPuucu/3ZqL+X18RwJ3tfmE7Cz/GVvxf++/jGCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABoMJwAAAAaDCcAAIAGwwkAAKDBcAIAAGgwnAAAABq66XT6oJ8DAADAu5pvnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABoMJwAAAAaDCcAAIAGwwkAAKDBcAIAAGgwnAAAABoMJwAAgAbDCQAAoMFwAgAAaDCcAAAAGgwnAACABsMJAACgwXACAABoMJwAAAAaDCcAAICG/xcmxOoZc9ivmgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('FactorAnalysis', d=64); # 4x compression from D=256"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Non-negative Matrix Factorization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most linear factorizations start by centering each feature about its mean over the samples:\n",
    "$$\n",
    "X_{ij} \\rightarrow X_{ij} - \\mu_i \\quad , \\quad \\mu_i \\equiv \\frac{1}{N} \\sum_i\\, X_{ij} \\; .\n",
    "$$\n",
    "As a result, latent variables are equally likely to be positive or negative.\n",
    "\n",
    "Non-negative matrix factorization (NMF) assumes that the data are a (possibly noisy) linear superposition of different components, which is often a good description of data resulting from a physical process.  For example, the spectrum of a galaxy is a superposition of the spectra of its constituent stars, and the spectrum of a radioactive sample is a superposition of the decays of its constituent unstable isotopes.\n",
    "\n",
    "These processes can only **add** data, so the elements of $Y$ and $M$ should all be $\\ge 0$ if the latent variables describe additive mixtures of different processes.  The NMF factorization guarantees that both $Y$ and $M$ are positive, and requires that the input $X$ is also positive.  However, there is no guarantee that the non-negative latent variables found by NMF are due to physical mixtures.\n",
    "\n",
    "Since NMF does not internally subtract out the means $\\mu_i$, it generally needs an additional component to model the mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3cmPZWl+FuDfuVMMmRGRQ+XU1VNVt7vdA25jt5EsQMhAYyNbXiBAsDFC/APsWCGx8ooFkjds2IEEsoCFbRnaCy8worGNJ3ru6q7qri5n5VBZkZEx3bj33MOCnUX64w1lRGRmPc/6PcM953zfd944i+iGYSgAAACebnTRJwAAAPC8U5wAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaJic14G+NPp7w3kd6//H+MpOtsHaWnyM4fb1bIM+u0T9VnZO48OTKD+Msl49mi/Odv/7h1G+qmrYfRzl+zCf+q3Vr3ZJ/nkbN3AR0nFT9fyNncmd21F+uLQRH+P449ey/CvTKL/9rSdRfvzue1F+effdKE/byzB24CI8bez44gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0TC76BC5Kv/s4yo8//cn4GN3BcZRf3LkS5fuNcZQ/uTKL8qPFKspPDrLHaXS8jPLDUXY9AZ4X+z/50Sh/cCub36uqFpe7KN+vZ/uf72xH+avfXovys2s7UT7Vf+1bZ7p/4OXnixMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0TC76BC7K5MOvRvl+az0+Rr8xjfJHt2ZRfrnWRflhnOW71RDlx/Pscdq8O4/y3Y1rUb6qarT7JNtg93EUH1/ZyfYPvBROfu6novz0YJnt/0q+PO+/3kf5rVf3ovyje5ej/NHNbN3cfjNbA6995V6UH3/ytSjfv/FmlAdefr44AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQMPkok/gRdEdL+NtFjc2sg2GLH6y3UX5fi3LTw+ieK3G2f4X29MoP0zynj9braL8+NbNKN/fux/lgZfD5h+/HeWf/KWPRvnZbrggVNXlV/ei/Eeu7Eb5f/zZ34jy/+JrvxDldydXovzld65G+dn9cFGDl9R4e/tM99/vZXPRi8QXJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABomF30Cz8r4yk6UHxaLKL/anEb5qqr51XGUX2x0Uf74epZfhT9hNcv2v/b+EOWHcbb/rs/2X1VVJ9l97u/dz48BfOAs774b5Wd7H4ry976YrR9VVZ+79jDK/5MP/bco//Obx1F+8dnfjPK/3P1clO//x+Uof3LzUpRfe3Q7ylflzwU8C+Pt7SjfvXItyq8216P8ZP9KlK+qWr71g3ibi+CLEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADRMLvoEnpXu8uUo39++muUvTaN8VdVyvcvym1l+cXmI8l0WrzoIz+dSlp9vZ719tMgf1+l4HOXHV3biY/Biie/x2lqW37oUxbvjkyi//OE7UZ6zMblzO8qfrGVz0dYPonhVVX3/cbauvf3K9Sj//tp3ovzWeD3Kf3jncZS//5ErUf7S3WyNCkc+XJh+by/Kr77wiSj//qeysbzxfv4utfVoN8qnv/lZ8cUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGiYXfQJPM76yE+WH/f0o3y22o/zJdn6pulWWX1zO8uOTLL/cHLL8Rhfl1w+jeE3m2fmsPTzKDlBV1fdZfPdxlB/fuhnluXjd5Wygra5nc8X+J7J8ZcOgZp+6HeXXv/1udoCqWv7wnXibD5rl3ey6zj6WzRWXJtn8W1V1939fj/K/vvVjUf7rhx+K8jdnT6L8g4NsbC42s2s0jKN4Ddey95CqqgqfC3gWJneydeHdz6xH+Sc/k73gPfneZpSvqtq4+1qUn3ztzSjf7+1F+afxxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaJud1oPGVnbM9wIduZfl+iOLHV8fZ/qtq7/UsPzrJ8out7DfMHp9tTx66M9199ev54zoaZ/ctfU77e/ejPBdv+eHrUf7RZy9F+Ydf7KP8aGsR5evhWhT/2G/eyfZfVWvXt6P86o+/ER/jA+crfxLFL33u0/Ehrly/FuW/d/RalP/ax1+N8tPN7Nnu725E+e1sqNVsb5ltMA8X5aqa3Lkd5Zd3342PAX/WcDWbs598PNv/v/zir2b5az+bHaCqlr+dzV/d3l58jGfBFycAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABomJzXgfrdx1F+fOtmlO/2DqL8/k++GuUf/tQqyldV3XztvSi/MV1E+bfeyq7R6H7Wk6f7UbzWdocoPznIrun4eBnlq6qG9WmU79bWovz4yk6U59lL78Gqz57TvdeieP30F74T5f/5q78R5X9z//NR/lfW/2aUr6p67T9fivLrn8wuUv/Gm1H+ZTC5czvb4FG2ZlZVvfL7WX7nuxtR/uGPb0b50SKbf4+vdVF+42G2hszuZ+8J8KLojk+i/OJG9j71i5cOo/y3Xv2TKF9V9R9vfynKX9Tbly9OAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0DC56BN4mv7e/Sg//uynovxyPeuMG7efRPmqqn/1o/8hym+NTqL8P5v+nSj/7fc+HuV33hyi/Gyvj/LrD46ifHe8jPJVVauvfjPehhdLv/s4yo9Osudo814X5R/NN6P8Z2ZZvi5/NYp/+dOfyfZfVfd+5GNR/s734kN84Czvvnv2BwmPMfzMT0T5q9+cR/nlpXGUX3+UjbWtN/ajfPcomytO41zuM/wZwyQba+PHWf4rx9n73Tvzq1G+qmr9vewYF8UXJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGiYXPQJPM34yk6U7/YOovzGw60of/fxRpSvqvrK0Sei/D+9+laU/+uvfCvKf3P6sSi/XOui/OZhH+X7jWmUn737fpSvqlrFW/Cy677/p1H+2iubUf47f/KRKP/3Z38jyn/y0oMo/+23b0X5qqpX9od4m0Q6v3M2xr/9B1F+7c7tKD9cytbN7uDoTPe/vPtulIcXRbfM3r+ufCN7v/vlt38+yn/1rQ9F+aqqjy+yN7bxjRtRvn+QrZ1P44sTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANEwu+gSept99nG0Q5qc3rkT5m7+9HeWrqv715b8a5e//SHaMP9r9cJSfHHRRfvPhMtv/43mUH7//JMoP+/tRvqpqfGUnysfPHS+c9B6vf+vdKP/68maU/8YPPx3lvz7O8jvZMKuqqu0fHEf5btlH+SFK87xY3s3GQmpy53aU7w6OzuhM4MWy/N5bUf7yj96I8m/++utR/tpuPstvvPFOlF8+eBAf41nwxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAICGyUWfwEUZ/vBrUf7q+PP5QbqtKP5rt/5KlO9nUbyuf7uP8tPHiyg/frAb5YdFtv9+93GUPw/jKzsXfQqcseUP34ny0/39KP+Rd65F+eWt7JkbHWbjrKpqdHAc5Zdv/SA+BvxZy7vvXvQpwAtpvL0d5S9980GUnx5cjfK1yuJVL8464osTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANEwu+gReFMPvfzXe5vqTT0b5rTvbUb5brqL8MMl68vTBfrb/xSLK9/fuR/nxlZ0oX1XV7z6Ot3me9s+LJ30mxuH+J08Osg3m8/AInmuAF0m/txfl03VntlhmGyzDfFXlW1wMX5wAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgYXLRJ/Ay67/1RpSf7d7MDjCfR/F+93G2/1tnez6p+PzhBeC5BuA89Xt72QZhfry9ne3/BeKLEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADRMLvoEXhTjKztnfoz+3v0z3X/8G+bzszkRAABeSv3e3kWfwpnxxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaumEYLvocAAAAnmu+OAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQMDmvA/2t2T/M/tNuF3a6UZftvsvyNR5n+aqqUfYbunH4m8/4GlV6jVLpP19eLPND9H22QZgf+lWU//L830UX9UuTfxBdpC6+x2f8t5P0fOoUYzMVjst4HKT7T62yZy4eZ3UO42aVndNvLf99/FD87OV/lI2dSbgczqZRvFtby/Y/zZfnYRKuU8/ZHN+F82k8Fpbhc3qyyPZfVbU4yY4RrmvDMst/+ejfxjf5uVt3zvr9rip+x+vSd8J0PI/S8znbdSd916nwOa3Kn+1aZOPzWb2v+eIEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADZOLPoGnGnVRvJtkP6Wbhj893H9Vfk41Hmf5SZgfhT25y+5BrO+z/MkiP8bJSRQfwmN0dYpzOktdeI/TcZY+E+kzXZU/p6F4XKZzy/hsz3/oV9kGy2V+kHTchLvvKhz7pxDf57W1bP+XN6P86tJGlr88i/JVVatpNt6GSTiewxvd9dkGXfhsj06y56g7zsZCdzyP8lVV3VG4zeFRfIyz1oXrwsuw7sTvhOl8sbEe5YfZNMun74OpZTjW5tkaUlVV6Xg7Cp+LcF17Gl+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGFybkcaj6N4N8lOrZvNsvzmRpSvtWz/VVXDNPsNw1qYn2X51Sy7B8Ooi/Ldaojyo5M+yx/Mo3xVVffkMN4mMaxWZ7r/LrwHld6zcFxWl55P/reZbhxuk84tm5tRfthYy/KT8JqmFsso3h2f5Mc4Cu/B0XEUz2aKU5pNo3h3KVsT+muXo/zxrey5O7qeL8/L7BC1moTjObxx40W2wTic4ifH2f4nh9l8vfZ+PnYmjw6ifJeuIcPZrjmn8hKsO6n0HbK/vhXlF9vZutOvhb85vKSTo+x9bbKXv6+NHp/t+1oNz2bl8cUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGibndaBuPM7y0+zUus2NKD/sXI7y/fZ6lK+q6tez39CvZT12eSm7psv1LsqvJlm+hiw+O1hF+fUH0+wAVZVu0fV9tkGaT3XZM9F14T1L86HVkydnuv+qqsntW1F+dTUb+8udbG7pN7JxWats4IyPs2dusncc5atO8Re1PhvLNYSTxSl002z0ry5vRvnjm1l+9xPZ+Tx5PbymVVWvzKP4aJzdh1UfriGH2Ro43s/GzvgoO5/Z4+x8Lr+T/215K8xPFstsg8UiPMIpWHeaxh/K1p3DD2fzxcGtbCycbGXXtAun4OmTbOxcupfXi+wKVY3TsXPybMaOL04AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0TM7tSONxlp+Ep7Y2i+L99nqUP76R5auqFpeyXtpPuyh/sp3mo3itskta3SrLT/eyZ2LowhOqqvHRWpifZwc4OcnyqVF2j+NxNsqe0dWTJ9n+z0OXXaPlzkaUP7qVPXeLzbP9e9T0IBtoG+H1qaqaHoXP9XE2brq+z/Z/GrNpFF9tZvnj69lYe/J6dt8+9xNvRfmqqi+98o0ovzM+iPIPl9ki8o2DO1H+jb1XovzecTa/P3oQLoKnWHNme9k248fZbxil70anYd1pGsLfsH87nS+ieC2vhXP2kN3jyaPwuevCZ6KqZnvhWHgSjs9wTXjqcZ/JXgAAAF5iihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQMDmvA3WzaZZfX4/yw/osyvfr2U9fXMo75snlLsqvsktUJzth/soQ5fv1VZTv+uz3pr19sZnfg9Vadp/Ho/AYXfqbQ6vsnlXfR/FuPM72/zwKf8PyUvZMzLfC5zQc9114i4cuO5/Zk/weT9ay+XS0Gc7Xq+w5PZVwLK9m2XVabGb3ebg2j/JfeuUbUb6q6pd2vhnld0YbUf5+//0o/wfr70T531n7VJR/tLgU5X+v+2iU37t7PcpXVS03wjVhEq455zFnn/G6k47N51J43052sueiv5nNFzdu7EX5ZZ+d//vdVpQ/eZCtIVVVy83s2Z6uZy/No4NnU3legqcXAADgbClOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQMPkvA7UTadRfri0EeUX1y9F+cPbsyh/9EreMReXs3yfnVItLw3Z/jdWUX4YZ/tPDekl7c7kNJ5vQ3bPVscn2f6Pj7P882icPUjDONt9v57l03E/6rN8DdlAOL6WT/PjeTb/ThfLKN/Nw+f0A2jUZWO/qmp8xpPkNNz/5mge5bfG2Xx01GfvFeNRfk0/kKw7TatZtpCswml4PMsWhp317JrOl9kJ7W1k59NnS0hVVS02s7V8fRZe1Em4+D+FL04AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQMDm/I2WHGjZmUf74lSz/+LWsMx5+ah7lq6pu3NyL8pdmJ1F+73gtyu/uXory3cNs/+OjLspPjqJ4jedDtkFVdYs+22C1yvJDfk7R7pfLM93/S2GV3YPdT0yj/OPPLaL89q39KJ8+QbsPs3G8eCObG6uqJsfZNZo8zo7RHZzD0hOOzdFJNldMD8L9P8yu0a+9+2NRvqpqvsru2+XxcZQ/XGW/4Y3DW1H+e/vXo/zhIjuf+w+3o/zmk2xNq6qaHIUjehmuOekadQrWnbauz+7DYit7Lj584/0o/9dufCfKH4dzxX8fvR7l3373TpSvqlquZ+/lq+k4yo9Hz+ZbkS9OAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0DA5vyONo3i/MY3y8+2sAx692kf5v/zp70b5qqq/e+P3o/zW6CjK/87+p6P8ry7+YpTvv7MR5WePo3jN9oYwn92zqqrRfJltsAjzfX5OPFvDxlqU3//YKsr/5GffjPK/eOOPovwq/PvVf3nv81H+d49+JMpXVW3ezebr9XC+HoXrwamEY3l0uIjyG+9lY3/ru9k1+u7yo1G+qupXrn0oynez7DcMqy7K1zy7z6OjbCyMTrLz2djN8pd/mM0VVVWz3ZMo3x3Po/ywtOY8F8JHY3klu29fvP6DKP8LW38c5Y+H7PV/vsryb23fjPJVVf16doxhHM5HXZh/Cl+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGFybkcaZR1tGHdRvl+L4lXbiyj+01e+Gx6g6m9vvh/l17pplH+y+kGU/0+jL0T5tXtDlN98sIry46MsP9s9ifJVVd3hPMoPy2WW77PfwLO3uLMd5Ydb2TPxizf+KMr/0vbDKH+4yp7rg1U22f3uzsejfFVVv54dI52v0/XgNIZ5dp9H+4dRfv1Po3jdfJTN77f+Zz63dIs+yvdb61F+GGX3eZhka0i6/9E8+71ddjo1mmfrQVXVaO8oyndHZ7tGnUoXjuchvLAvgeVONkeOLmfvnH9h8+0o/+Nr2fmk687/Ws/WtdFG/pyuJmElCdeddH55Gl+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGFybkcahijeZfE4Pyyzzrjfr2cHqKrHq5Mov9Yto/x7y9tR/ng+jfJX97OLOj5eRfnJUZ/t/2Ae5auquqNsm+FkkR2gz35DrOuyfDjOXgbL9XGU70bZOBunk0toVdm4WQ3n8PeuM55/z+U5DcfysH+Q7f/Be1n+INv/aa5Quk04u8T5F92pntIbN7JjrLI1ZFhm89epdOkck81hL8M6tZql1yi7z6vwu8ZiyPY/H7LnaL7K3h/Td+yqqi59nerDXtGHz+lT+OIEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADZNzO9Kyj+Kjkyw/PRii/Pi9aZT/8r3PRPmqqsUwjvKbo5Mo/zuPPhHlT95bj/KjZRSvyWF2z8bH2QG6eXhCVVXLcJs++w21WmX5szbKnrlahb/3OTSMuyi/erQW5X/t4Rei/PHq61H+cJWdz2/c+3yUHz2YRfmqfD5N5+vzGDdDOvbD/Op4nu2fD4azXnPS/Dnoxtm6M6S/Ycjmo/OwCted4f1sHv71Bz8W5Y9X2Tttuu7813ufjfKjh9n5VFXN9sP3+PnFrDu+OAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANAwObcjLZdRfHR4EuXX31uL8ltvzqL89+vVKF9V9W+u3cw2GA1Z/PE0ym+9nfXk2f4iyk8fHUb5bp49E93hcZSvqhoW2W8Y+j7Mr6L8c2c0zvKr7Pqch/E8uwdb383Gze/2n4ryv3ftY1F+WHVRfvQwm7u23sz/PrbxXjZu0vm6TrL9n0Y6lmuVzb/dNFs+h/nzN3b4843W1+NthvBdp+I15zl8jrpsjukmWX5YhPPLORifnO2684f9J6P8H1z9aJSvIV13wvfNU6w7649ejHXHFycAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABomJzXgYaTkyjfHRxF+bX72U+5EqWr1nfzS7XYnMbbJCbHQ5Rf211m+QfHUX60dxjl63gexYfFItt/VQ0n2TbDMrtGz51hleW78G8no3GWT8+nqsbXr0X5bjd7Tq9+O/vNGw+y37zYXI/y1WXx6UE27tffy8fN2oNsLKfz9WnGcmyVXaez3n83nWX7P8XYScfzsMjW5efNeHs7yg9D+EysTnEP+v5sj3HWz/VpnOZZDaRjZ0jvQVWNr2Vvhd2jM153HmbrznJjLcoPZ7zubJxm3bl3xutO2EOexhcnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaJic14GGk0W2wcFRFB9ne6+Nk2WUX3s0C49QtZqlZ5XpFn2UHx1n96A7nEf5mp9E8eH4OMsvsntWVTUsw21WQ3iAVZZ/3qTn32V/a+lm+bhJjR/tR/mNeTZu1h5Mo/xZj/vRMrtno/1sXP7fbQ6j/BDO15WuB6fxvI3NcOzE+aqqURfmw2d1lY2dVDfN5ot4fh+Hv3cI14OqqlX23A19+Jw+b891VQ3hutmlz2n4m0ezbM4+jfFuuO6E72trD7PfMEyzZ3tIp4qT8H3zMJ/jz3zdOcU75P+LL04AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQMDm3Iy0WUXxI97/qo3g3P4ny4/1plK+qGo/OuJeuVlm+z/JDes9Ownx4PtVn97iqqlbhkzSE5/S86c72metG3Znuv6pqCMdm1X6UHoXP6ejJOMrX8zbul/m4GebzMB/es3BuOY0hHfuheCycx9wS3ur0NwwVjoUzNqRrwmnWkNQZrzln/Vw/l854XauqqvT95Yl1588V/t6qU7xzXtC644sTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANHTDMFz0OQAAADzXfHECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgCiy4C4AAAADklEQVQAaFCcAAAAGv4PmysPdNRheRsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('NMF', d=4); # 16x compression from D=256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHstJREFUeJzt3UmMZVl+FvD/fe/FnDFkZlVlZg3uqupyt7vcohsPCIRtZHBjWwZkIRkJNoAQKzbsQGJcIG/wAokFbFkghhZCgI3BLfACLFqWjXFT5e6qLndVd1U556yIyJhevOGyaMSi5eT4C2cMmfX7rb97zn3vnnvP/eItouv7vgAAAHi0wXmfAAAAwEWnOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQMDqrib4w+Jn+rOb6vRhubWYHLC3Fc/TXr2YHzLKvaLaendPw4DjK94OsVw/Gk9Mdf+8gyldV9ds7UX4W5lNfmn+xS/IX7b6B85DeN1UX794Z3bge5fu1lXiOo5evZPlnFqL8xlsPo/zw1v0oP715K8rT9jTcO3AeHnXv+MUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaBid9wmcl9n2TpQffvq1eI5u/yjKT25sRfnZyjDKH28tRvnBZB7lR/vZchocTaN8f5h9nwAXxd73f1eU37+WPd+rqiaXuig/W87GH29uRPnLby9F+cUrm1E+NXvzrVMdH3j6+cUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGkbnfQLnZfTiC1F+tr4czzFbWYjyh9cWo/x0qYvy/TDLd/M+yg/H2XJavTmO8t2zV6J8VdVg+2F2wPZOFB9ubWbjA0+F45/4wSi/sD/Nxt/Kt+e9V2dRfv2F3Sj/4PalKH/4XLZvbryb7YFXvnw7yg9feyXKz955N8oDTz+/OAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAEDD6LxP4EnRHU3jYybPrmQH9Fn8eKOL8rOlLL+wH8VrPszGn2wsRPl+lPf8xfk8yg+vPRflZ7fvRHng6bD6m+9H+Yd/6Lui/OJ2uCFU1aUXdqP8S1vbUf4vv/4LUf7vv/mnovz2aCvKX/rwcpRfvBNuavCUGm5snOr4s93sWfQk8YsTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADaPzPoHHZbi1GeX7ySTKz1cXonxV1fjyMMpPVroof3Q1y8/DjzBfzMZf+qiP8v0wG7+bZeNXVdVxdp1nt+/kcwAfO9Obt6L84u7zUf72D2T7R1XV9165F+X/yvP/Lcr/1OpRlJ+8/otR/me7n4jys/9xKcofP7cW5ZceXI/yVfm6gMdhuLER5btnrkT5+epylB/tbUX5qqrpe9+KjzkPfnECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACAhtF5n8Dj0l26FOVn1y9n+bWFKF9VNV3usvxqlp9c6qN8l8Wr9sPzWcvy442stw8m+XJdGA6j/HBrM56DJ0t8jZeWsvz6WhTvjo6j/PSDD6M8p2N043qUP17KnkXr34riVVX1zZ1sX3v/matR/qOlr0f59eFylH9xcyfK33lpK8qv3cz2qPDOh3Mz292N8vPPfTLKf/Sp7F5e+Sh/l1p/sB3l08/8uPjFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABpG530CjzLc2ozy/d5elO8mG1H+eCP/qrp5lp9cyvLD4yw/Xe2z/EoX5ZcPoniNxtn5LN07zCaoqprNsvj2TpQfXnsuynP+ukvZjTa/mj0r9j6Z5Su7DWrxU9ej/PLbt7IJqmr6wYfxMR8305vZ97r4iexZsTbKnr9VVTf/99Uo//PrfyDK/9bB81H+ucWHUf7ufnZvTlaz76gfRvHqr2TvIVVVFa4LeBxGN7J94dZnlqP8wx/NXvAefmM1yldVrdx8JcqP3nw3ys92d6P8o/jFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABpGZzXRcGvzdCd4/lqWn/VR/OjyMBu/qnZfzfKD4yw/Wc8+w+LO6fbkvjvV4Wu2nC/XwTC7buk6nd2+E+U5f9MXr0b5B6+vRfl7PzCL8oP1SZSve0tR/BO/eCMbv6qWrm5E+flvfjWe42Pny1+J4mvf++l4iq2rV6L8Nw5fifJvvvxClF9Yzdb27OZKlN/IbrVa3J1mB4zDTbmqRjeuR/npzVvxHPCd+svZM/vhy9n4P/cDX8zyV348m6Cqpr+cPb+63d14jsfBL04AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQMDqriWbbO1F+eO25KN/t7kf5ve9/Icrf+8F5lK+qeu6V+1F+ZWES5d97L/uOBneynrywF8VrabuP8qP97DsdHk2jfFVVv7wQ5bulpSg/3NqM8jx+6TWYz7J1uvtKFK8/8rmvR/m/88IvRPlf3PtslP/Hyz8W5auqXvm3a1F++bXsS5q9826UfxqMblzPDniQ7ZlVVc/8Wpbf/O2VKH/v86tRfjDJnr9HV7oov3Iv20MW72TvCfCk6I6Oo/zk2ex96s+sHUT5t174SpSvqvo3178Q5c/r7csvTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANAwOu8TeJTZ7TtRfvj6p6L8dDnrjCvXH0b5qqp/9D3/KsqvD46j/N9Y+LNR/u37L0f5zXf7KL+4O4vyy3cPo3x3NI3yVVXzN74WH8OTZba9E+UHx9k6Wr3dRfkH49Uo/5nFLF+X3ojiv/Tpz2TjV9Xt7/5ElL/xjXiKj53pzVunP0k4R/+j3xflL39tHOWna8Mov/wgu9fW39mL8t2D7FlxEmdyneE79KPsXhvuZPkvH2Xvdx+OL0f5qqrl+9kc58UvTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANAwOu8TeJTh1maU73b3o/zKvfUof3NnJcpXVX358JNR/q9ffi/K//Fn3oryX1v4RJSfLnVRfvVgFuVnKwtRfvHWR1G+qmoeH8HTrvvm70T5K8+sRvmvf+WlKP/nFv9ElH9t7W6Uf/v9a1G+quqZvT4+JpE+3zkdw1/+n1F+6cb1KN+vZftmt394quNPb96K8vCk6KbZ+9fWV7P3u599/6ei/BvvPR/lq6penmRvbMNnn43ys7vZ3vkofnECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACAhtF5n8CjzLZ3sgPC/MKzW1H+uV/eiPJVVf/00g9H+Tvfnc3xv7ZfjPKj/S7Kr96bZuPvjKP88KOHUb7f24vyVVXDrc0oH687njjpNV5+61aUf3X6XJT/6gefjvK/Nczym9ltVlVVG986ivLddBbl+yjNRTG9md0LqdGN61G+2z88pTOBJ8v0G+9F+Uvf82yUf/fnX43yV7bzp/zKOx9G+endu/Ecj4NfnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGgYnfcJnJf+N96M8peHn80n6daj+H+49kNRfrYYxevq27Mov7AzifLDu9tRvp9k48+2d6L8WRhubZ73KXDKph98GOUX9vai/EsfXony02vZmhscZPdZVdVg/yjKT9/7VjwHfKfpzVvnfQrwRBpubET5ta/djfIL+5ejfM2zeNWTs4/4xQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaRud9Ak+K/tfeiI+5+vC1KL9+YyPKd9N5lO9HWU9euLuXjT+ZRPnZ7TtRfri1GeWrqmbbO/ExF2l8njzpmhiG448e7mcHjMfhDNY1wJNktrsb5dN9Z3EyzQ6Yhvmqyo84H35xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgIbReZ/A02z21jtRfnH7uWyC8TiKz7Z3svGvne75pOLzhyeAdQ3AWZrt7mYHhPnhxkY2/hPEL04AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQMDrvE3hSDLc2T32O2e07pzp+/BnG49M5EQAAnkqz3d3zPoVT4xcnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaOj6vj/vcwAAALjQ/OIEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQMPorCb6k4t/PvtPu8NhFO/S/Cj86IsLWf4kc8T57DPXIOzJ6T9Hns6y/GQSxfswX1XVH4fHpOc0m0f5Xxr/8y7Jf2HwM/5D9ePWRZcg9zT8U/H0Ozrlz/yl+Rfji/bjl/5idFJd+IzvlpejfK1k+X5lKRu/qvqFbE/oR9me0M2y69xNwj0hfF53h+Mo3x8eZvmjbPyqqpplnzndQ/pw/C9N/2V879h3zl/8/hi+A8fm2ZJI1+m3D8ruhXjfCfe1L83+9e96gF+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGF0ZjN1WUfrhsMsv7gQ5WtpKRt/OctXVfVLi1l+OfsM/VKWny+E12DeR/nB0TQb/2Cc5Q+z/P89KIpnn7iqq0l4BDwB+vROuIDCPSTdE/pLq1F+fvlSlB9fyfec6Vr2madLXZQfTLN1sbA3j/KLu9nzdLSdPd+72SzK1yTb06ryPSS917o+u2ZnogvP6Wl4vlwwXXoNQn26sufhvXYS6WcOe8ij+MUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGkZnNtOgy/LDYZYfZR+lW1mO8vPNtShfVTVdX4rys9XsM0wuZd/RZDW7Bt0sitfSbnbA4vZilB/uLET5qhP8ZaDvs/h8ns7AeQuvcXXhsyvNp9Lz/5jq0j1hIXu+zJez5/v4arbnbL+aP+8OXsjWxmQze351x9naXrmdfYa138mu2eY3oniNbt2N8vODg2yCquqWsnXRhc+LvvP37o+Dfha+gIXvzOnzsR+Po/xZ6NKe8JjuHXcgAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANIzOaqKu6041X13YAfs+is/WFrPxq+ro2aUoP13OPvPR1ewzH69H8RodZvnZUnb+84Usvxylv62bzbL8dJpNMAnzPHnCZ8XToFvInnf95PiUzuT3YRDuIcPT/TtiN51H+aNn8zlWP/tRlP/hF347yr+6cjfK/7N3/nCU337jcpRfu529wowub0b5wSh/Reof7sXHRNJ1fRY+hs/IiyZ+Zw6v2ej5G9nwB+ELZFX1h+Exw2E4weNZp35xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGF0ZjMNwo42HIbjd1l+3mf5E5guZ+eU5o+uRPGq8Cuqoyw+W8ry3SzL96P0A1T1i9kS79J1l+Zp68Lr3Gf3cjcK18RSuLDTZ11ovn8QHhDeaFVV/Tw/5kkXXrfueBLll+4dRvnRQbjuqura+sMo/+nVW1H+r229H+XrtSz+T45+JMpPf3U5yh+/dDnKL9zNr0E93MuPge/QjRZOdfzBlexemL54NRt/7zjKV1UNPtqN8v10mk0wHmf5R/CLEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADSMzvsEHpdueSnKz9fXovxkfSHKV1VNV7oof7yZ5aerfZRPDcfZ+VRl+fFm1tsX9tLzOYHuDObg/6/P1nW3sBjlB2srUb6WsmdLbVyK4t3D/Wz80GBtNT5meuv2KZzJGZuHz8fJNIr369k6mq5n62h0gmVxZy9be5N+GOXfPD6M8i8v3ovyz1/ZifI7z2f7+NUHkyjfhc+iqqpaWY7i/dE4ynezWZT/WDrJPn6Sa32K+slxlB9eeyHKj1+7FuUPn8v22YWH4b5ZVcsr2Xv2YDt8SO5m8UfO+3iGAQAAeHopTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAEDD6LxP4JH6eZafhfmwMvaDLjugqvpwjvkwy3d9OP5idsBsKfvMo4MoXsNxdj7dJLzGVdVNw2P68Evl3HXLS1l+fT3Kzy9fivIPv3szynf9s1F+cWca5ZffuhXlq6q6UbY19NPsnM5EvIfMonh3eBzlR132PN18N9+eb/3G1Sj/71Y+F+Xf3roe5ddHR1H+/v5qlO8Ws+90sp59p6Pd/G/L6b1Tg0mWD9fpx9JTsI+Prl+L8tMPPozyD/70d0X5hz98GOWH76xE+aqq5349O2Y1fL8bHmXP7EfxixMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0jM5spvk8ivezLN8Nsw44ubIa5fev51/V/vNdlJ+u9lG+H0bxGh5k51NhvE+Hzy5xddnX823huqvZLMv34fi0DbKF3XXZwptdvxzl733+UpS//wezNTTcOo7ydXM5il+//GI2flVt/JeD7IDwvun7k9zMmXQPqckkind74Xe0uxfFlwf53zW33l6P8rfmL0T5925ci/Ld6jTL31+M8ltH2ToajsP3iqNsTVRV1fz01zZPv/l+9nyZ/9Dno/zBjShef+v7/mOU/4dLX8gmqKrj8Pm1vJK9lw9H4UvzI/jFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABpGZzVRP5tH+a4m2QTzPoqPryxE+fvfl51/VdUrn7kZ5SezYZR//5vPRPnF3ewzL+5E8Vrazq7B6ChcE5NZlK+q6sbZOuqn2Rzpuub3YB5e52F238xWs8fe/gtdlP+x738zyv/N6/85yv+n/c9E+Z9b+8koX1XVzT4V5dd/636UH+w8jPInMgnv/XD4fnycHTDL1vXgg/SMqq4cjKP8xnuXovz9z65E+elydq914Ude2Muev4v3DrIJ7mTruqqq+nBPCNdF3+frggugy/aRwTNXovzCg2xtj59ZjPJ/aeNOlP+vz78f5auqvnL5s1F+I/xO02vwKH5xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgIbRmc3Uz7P4NMvXMOuAh5ez/PpL21G+qurvvfrvo/z14X6U/wcbPxnlf/XW90b5pY+ieK3cm4bjj6P88MFelK+qqoPDLD85zvKzWZbnseuWFqP84jfvZ/ntF6P83aNLUf6TC1n+L6y/FeX/xcs/GOWrqvaevx7lV2+uRvnROLzPTqCfZs+jCvPx+KGT/FVzEN4L/bPZ2tt4b5KNP+ii/ORS9qnXfifbQwZ3sk1tfpJrPA/fddI9ZN5n+ZPosutW/Rmc00Vzyt9R/1H2ztmv3YjyS/eGUf5XjrJ1vX2c7QlVVV26jNJ74TGtU784AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQMPorCbq532U74bDbIKui+ILh9n5PPhwI8pXVd16fTPK/8hyNv7n1z+I8v998/Uo3w+y73Q4nmfjD7Pe3oVrqKpq/nAvO2A2i+L9LPvM/B4Msnt/vvswGz58Vlz56nGUf+OlV6L8X13+o1H+yuJ+lL95N3sOVVVtTbJ7bb4QPq8Hp/83uz68l6vPny+nqT/O1l1V1fze/Si/GI6/MM7Oqbu0GuXnayvZ+LfuZuMfHkX5E63T015HvT3nQkivc7iv9ceTKN/dzu79ldtXovzf/cZPR/l3v3YjyldVvfj+NMoPjrJ8TcM94VHzPpZRAAAAnmKKEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANAwOquJukGX5Ydhpzs8iuKbb+9F+cnqepSvqvrbV346yr/5+q9F+d/YfinKL+5k12DlwSwb//5BlO+OJlG+38/Gr6rqj7M58gnmpzv+x9E8W3fzo3E2/nA/iq/8+rtR/oWFT0b5X33/c1F+shbFa2Mny1dVrd7NrsHwaJpNMD+D+6YL95A++8ynrZ+G3+kJjunv3svyx8dRvtteyPLLS1l+cyPK1yxbd/3hYTZ+VfV9nx0wu1jrrqqq0s9AW7ivnfb4z7yRre2b6y9G+at38jW0uJPt5YOj7HlUk8fzPugXJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABpG530Cj9LP5ln+6CjKDz+4G+UvryxE+aqqhYOVKP/FN/9YlO+yr6g23++j/NL9cZQfPDyM8v3ObpSf7+1H+W9PEn5JF03XhfkL9reQ+ezCzdEfZuu0Ztn4a7/y9Si/8uHzUX6wF95nK4tRvqqqXxhG+cHD7PlbR9mz5Sx0o2w77OfZ8/RM7oXwedFPpqd0IifTHx9n+f2DLD/O1l0/Pf3vJ15HZ+GU951uEK7T9Ds6yb7fn/J1GGTP1NMef/G9e1H+pZ31KN+na6iqBuNJlO/SvXCSjf8oF+wtCwAA4OJRnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAICG0XmfwCMNuijez+bZ+JNJFF+4uZ2NX1Vbu8tRfvOtrMf2C8Mo3836KD/YOYjydXCY5efZ+XRdtiaqqvou/NtAn62jPvwMpy48/0q/n/ksy5+BbpQ9xvpZ9hn6o3GUHwyz+3LwrZtRvj88ysbf3IjyVVUVfqfp87QfH2fjn0AX7iHxvVAX7164aPrJ6V7n2XE4fh8+r0+w5zwV4u/pdE7j/7mA+04Nwvev9HkU6o+yfaEbZs+7wf3dKH8i8/D9a5ztzf1xtk89il+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGF0VhP18z7Kd12Wr9ksy0+6LL+zl+WranA4jo/JJgh773wexftxdv798STLh9es78M1wcdCuo5i83CdHh9H+W6UPYa7xYUo3x+d5DkUHpPey6d9zeoEe84gez52g3APGWTXuZ9Os/FPos8+8xOvC6/Zx5XvqSm+/0PpMzI9m/7oKDtgnO1rJxI+j/pJ+Ix8TPuOX5wAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgYXRmM/XzMN9l8ck0Gz81m+XHjIeP/zx+H/q+zw6Yh9cs/I7y8wnzVfG6608yx5MsvS+77L6s9Bqf1RynKVxD/fHxKZ3I78MpPyvie/8k4nv/gv0dMb0P4AnRp+9Tg4v1LnUm0n2kwu/0Am47p76PnOQ9/ndxwXYKAACAi0dxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABq6vu/P+xwAAAAuNL84AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA3/BwC6lXOx9enIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('NMF', d=64); # 4x compression from D=256"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note how the reconstructed images all all non-negative, by construction. NMF is quite slow with this many components."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Independent Component Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final linear decomposition we will consider is ICA, where the goal is to find latent variables $Y$ that are *statistically independent*, which is a stronger statement that the *statistically uncorrelated* guarantee of PCA. The basic idea is that the joint probability of a sample occuring with latent variables $y_1, y_2, y_3, \\ldots$ can be factorized into independent probabilities for each component:\n",
    "$$\n",
    "P(y_1, y_2, y_3, \\ldots) = P(y_1) P(y_2) P(y_3) \\ldots\n",
    "$$\n",
    "\n",
    "ICA has some inherent ambiguities: both the ordering and scaling of latent variables is arbitrary, unlike with PCA. However, in practice, samples reconstructed with ICA often look similar to PCA and FA reconstructions.\n",
    "\n",
    "ICA is also used for [blind signal separation](https://en.wikipedia.org/wiki/Blind_signal_separation), where usually $d = N$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3UmMZVl+FvD/fVPMkZGZVZWZVV1Td3VXDzbtoW2DMGBjjC1sQEIMQrAAWezZsUBIrFggFkhsWHgBQiwsQCyMadlG6gVT23jqdvVQ1e2ualdV55wZGXO86bIwC7BcOnyhjIyq6t9v/d177nv3nnPuF28RXd/3BQAAwHsbXPQFAAAAvN8pTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0jJ7UQD85+Gv9kxrr/8dw51J2wMpKPEZ//Wp2wCL7ihZb2TUNj6ZRvh9kvXpwOjvf8x8cRfmqqn73UZRfhPnUry7/XZfk32/zBi5COm+q3n9zZ3TjepTvN9biMU5eupLlnxpH+e3X96P88Nb9KD+/eSvK0/ZhmDtwEd5r7vjFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGgYXfQFXJTF7qMoP3z1lXiM7vAkys9u7ET5xdowyk93JlF+MFtG+dFh9jgNTuZRvj/Ovk+A94uDH3whyh9ey9b3qqrZZhflF6vZ+U8vbUf5y2+sRPnJlUtRPrX4yuvnen7gw88vTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANAwuugLuCijjzwX5Rdbq/EYi7VxlD++Nony85UuyvfDLN8t+yg/PM0ep/Wbp1G+e/pKlK+qGuzuZwfsPoriw51L2fmBD4XpT/9QlB8fzrPz7+Tb88FHF1F+67m9KP/g9maUP34m2ze338z2wCtfvB3lh6+8HOUX33wzygMffn5xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgIbRRV/AB0V3Mo+PmT29lh3QZ/HpdhflFytZfnwYxWs5zM4/2x5H+X6U9/zJchnlh9eeifKL23eiPPDhsP6lt6P8/g+/EOUnu+GGUFWbz+1F+ed3dqP83/30L0X5f/yVn43yu6OdKL/57uUoP7kTbmrwITXc3j7X8y/2srXog8QvTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADSMLvoCHpfhzqUo389mUX65Po7yVVWnl4dRfrbWRfmTq1l+GX6E5SQ7/8rDPsr3w+z83SI7f1VVTbP7vLh9Jx8D+K4zv3kryk/2no3ytz+X7R9VVZ+5ci/K/9yz/zXK/8z6SZSfffrzUf6fdD8d5Rf/czPKT5/ZiPIrD65H+ar8uYDHYbi9HeW7p65E+eX6apQfHexE+aqq+Vu/Hx9zEfziBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2ji76Ax6Xb3Izyi+uXs/zGOMpXVc1Xuyy/nuVnm32U77J41WF4PRtZ/nQ76+2DWf64jofDKD/cuRSPwQdLfI9XVrL81kYU706mUX7+zrtRnvMxunE9yk9XsrVo6/ejeFVVfftRtq+9/dTVKP9w5RtRfmu4GuU/culRlL/z/E6U37iZ7VHhzIcLs9jbi/LLz34syj/8RDaX1x7m71JbD3ajfPqZHxe/OAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAEDD6KIv4L0Mdy5F+f7gIMp3s+0oP93Ov6pumeVnm1l+OM3y8/U+y691UX71KIrX6DS7npV7x9kAVVWLRRbffRTlh9eeifJcvG4zm2jLq9lacfCxLF/ZNKjJJ65H+dU3bmUDVNX8nXfjY77bzG9m3+vkxWyt2Bhl629V1c3fvRrl/9PWH4vyXz16Nso/M9mP8ncPs7k5W8++o34Yxau/kr2HVFVV+FzA4zC6ke0Ltz61GuX3fzx7wdv/1nqUr6pau/lylB995c0ov9jbi/LvxS9OAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0DB6UgMNdy6d7wDPXsvyiz6Kn1weZuevqr2PZvnBNMvPtrLPMHl0vj2578719LVYzR/XwTC7b+lzurh9J8pz8eYfuRrlH3x6I8rf+9wiyg+2ZlG+7q1E8Rc/fyM7f1WtXN2O8ssvfS0e47vOF78cxTc+82o8xM7VK1H+W8cvR/mvvPRclB+vZ8/24uZalN/OplpN9ubZAafhplxVoxvXo/z85q14DPjD+svZmr3/Unb+f/a5f5flr/xUNkBVzb+QrV/d3l48xuPgFycAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoGD2pgRa7j6L88NozUb7bO4zyBz/4XJS/90PLKF9V9czL96P82ngW5d96K/uOBneynjw+iOK1sttH+dFh9p0OT+ZRvqqqXx1H+W5lJcoPdy5FeR6/9B4sF9lzuvdyFK8/8dlvRPl/9NwvRfnPH3xPlP8Xq38uyldVvfwfN6L86ivZl7T45ptR/sNgdON6dsCDbM+sqnrqN7L8pd9bi/L3vm89yg9m2fp7cqWL8mv3sj1kcid7T4APiu5kGuVnT2fvU39p4yjKv/7cl6N8VdV/uP6TUf6i3r784gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANo4u+gPeyuH0nyg8//YkoP1/NOuPa9f0oX1X1zz/5C1F+azCN8v9g/Fei/Bv3X4ryl97so/xkbxHlV+8eR/nuZB7lq6qWr309PoYPlsXuoyg/mGbP0frtLso/OF2P8p+aZPnafC2K/8qrn8rOX1W3P/5ilL/xrXiI7zrzm7fOf5BwjP7HfyDKX/76aZSfbwyj/OqDbK5tffMgyncPsrXiLJ7IfYY/pB9lc234KMt/8SR7v3v39HKUr6pavZ+NcVH84gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANo4u+gPcy3LkU5bu9wyi/dm8ryt98tBblq6q+ePyxKP/3L78V5f/sU69H+a+PX4zy85Uuyq8fLaL8Ym0c5Se3Hkb5qqplfAQfdt23vxPlrzy1HuW/8eXno/xfn/xElH9l426Uf+Pta1G+quqpgz4+JpGu75yP4Rd+K8qv3Lge5fuNbN/sDo/P9fzzm7eiPHxQdPPs/Wvna9n73T95+2ei/GtvPRvlq6pemmVvbMOnn47yi7vZ3vle/OIEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADaOLvoD3sth9lB0Q5sdP70T5Z76wHeWrqv7l5p+K8nc+no3xO7sfifKjwy7Kr9+bZ+d/dBrlhw/3o3x/cBDlq6qGO5eifPzc8YGT3uPV129F+Y/On4nyX3vn1Sj/1WGWv5RNs6qq2v79kyjfzRdRvo/SvF/Mb2ZzITW6cT3Kd4fH53Ql8MEy/9ZbUX7zk09H+Tf/00ej/JXdfJVf++a7UX5+9248xuPgFycAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaRhd9ARel/+2vRPnLw+/JB+m2ovgvXvvRKL+YRPG6+sYiyo8fzaL88O5ulO9n2fkXu4+i/JMw3Ll00ZfAOZu/826UHx8cRPnn370S5efXsmducJTNs6qqweFJlJ+/9fvxGPCHzW/euuhLgA+k4fZ2lN/4+t0oPz68HOVrmcWrPjj7iF+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGF00RfwQdH/xmvxMVf3X4nyWze2o3w3X0b5fpT15PHdg+z8s1mUX9y+E+WHO5eifFXVYvdRfMz76fx88KTPxDA8/2j/MDvg9DQcwXMN8EGy2NuL8um+M5nNswPmYb6q8iMuhl+cAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoGF00RfwYbZ4/ZtRfrL7TDbA6WkUX+w+ys5/7XyvJxVfP3wAeK4BeJIWe3vZAWF+uL2dnf8DxC9OAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0DC66Av4oBjuXDr3MRa375zr+ePPcHp6PhcCAMCH0mJv76Iv4dz4xQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaur7vL/oaAAAA3tf84gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAw+hJDfTnJ38z+0+7XdbpumHYAQdhfjjM8lXVdV12wCDMp9eUXk94D2L9MsvP5/kQi3CM2excz/8rp/82ugnxvAmfiS59huLzn+EZOsNc+66yDP9p+WIRD9Gnx4T58543VVU/tfV3oi+qm0yyASbjKN6trkT5Pjx/VVWNzntPCPN9+Kym+Xn23HVhvmZn2HNOp+EYWb6fZnvULx/863ju/PmVv5XNnXSdP+99anyGV9vBOe9t6dw57/e71DJbs/szvK+l73jpXEjf73756N/8kV+qX5wAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgYfTERuqyjtYNw043HIbnz/Lp+auqatBl+VF2O7rJJDv/ODt/n96DULdYZgdMZ/kgJydZvgvv2XSa5VPpc33ez9BknJ1/nOWrqmp0hrl2nvr+oq/g/5XOm/k8H2OWzbU+nJtdnWEuh9Jnu1tdifL95nqUX2yvZfnNcG5W1WIlmzvLYbjehbpw6nTz7NkezLMBBqeLKD88PI3yVVWD/eMo3x9l96Bbnv96FL9/het8d977yEo+d/r0/WgQfkfhvtaPw3x6PWE8nZvdSb7Gd0fp+1o417Kzvye/OAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAEDD6EkN1A3DjjYcZucfn+9H6efz/KDlMooPdlajfL+9EeUXWytRfrmS3YO+66L8YLqI8sOD0yhfVTXYCz/DwWE2QN9n+VCXzoPJJMuvr0X5fnM9yi/Ws2euqqofZ2tFl96CeTYvu3Aex8J5k64r3cksO39VdUcn8TGJ/ry/06qqyTiKp8/2/OmtKH98PVvfj57O/64528yepWX2FVWFc63LlvgahI/q8DS7oMlBll+9n62nVVWrd7I1O0tX9YvwSz2LeN/JHqRuLdx3trK5udzI953lJHxXCN9pl5Msv1gN8ytZvg+Xl8EsnGuP8nfm8YPsPT5eIR/T+5pfnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBh9MRGGg6jeDcOLy08/3LvIMp/5EfejvJVVeuTSZT/xhf7KL948ekof3RjNcqfbme9us9uQY1Oss+7dnecDVBV2SeuGswX2QHzeThCKHyua5J9R/3GWpSfPb0Z5U+vZnOgqmox6aL8ILwFo6PsHg+ny2yA0HJ8vn+/Gh3M8mMeZtfU9dlcrkU4z86gC9ffxWY2F46vZ6vLg1ezuXz0yjTKV1XtPJXtaxuT7NmYL7LPcDzN1qPDkyy/OMreE0YPsvNvfjvfc6rWs/RJtoB103w+p7r4fe2c952nwn3nSn7fluNs31kOs/xiJctPt8L8dhSvZbg1D8LlaONm+O5SVVuD7DOvzLJ9pJs9nrnjFycAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaRk9qoG4cDrWyEp5/HOXTxrgZXk9V1ebaWpTv1laj/Mkz2TXtPz+M8qeXo3gtx32UH++ndyF/XMf72Xc0ODjJBhid7xTqui7Lh9ezXM3mzWw7yx9fyZ65qqrlJMsPw1vWh4/d8jQ7YLGa3bPZWnhB2elrZS//+9jaMpvLo9NZNsA0zJ9FuOcsNsJn+2r2vR59NPvMf/GzX4ryVVU/u/M7Uf7q8DDK7y6zPe2t6dNR/s3TLP/uyU6U//LdG1F+f3k1yldVrT7I1rzVe9mCNzzI19TYIFxkhtk19SvZ3JyvZ+efrT+B3wTCr2i2keWPr2Vr8OmNeZSfbJ9m5z8J3y3G2ftsVdXkIBtj/CidO9ka/1784gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANoyc3UjZUt7Ya5ZeXNqL8fPuZKP+79XKUr6paLLJeevLTwyh/dD08/9N9lJ9vLqN8dWG8z65/vhoOUFXLSfad9sPwbwnd++xvD8vsnnWL7JkYTLPzj4+z81dV1XEWHx+Gn3meXdP0UvYMHV/NnonpdhSPLcbZ9VdVjQ7GUX64vxLlu+OTKH8W/Sj73Olaka5Hk+3TKP8jW9+K8lVVf2btKMqvdJMof3P+KMqvdrMoP+7m2fkH2fnvbm1G+Ycbl6N8VdViJZv/y3GWHw7z+XzuFoso3h1Po/zKvWxTWHsne46qqn7oY69F+fWVbM37lTe+P8of3cjm5trl7Dv66FP3o/zeafZO/s7utShfVTXdDOfOarhPjR9P5XmfvfUBAAC8/yhOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQMPoSQ3UjcdRvl9fjfKnT69H+b0XJ1H+6NkuyldVTXeWUX6xnuVrMs/y6UdYZAd0J1kPH0yz8w9mfZSvquoW+THvJ/1ikR1wchrFu/u7UX7twaMo/8JLvxXlq6o+9ZGPRPl+JbvHX7j7vVH+4Sc2o/zBS9k8Xl4K5/E0/XtXvsyv7mbHTB5m+eHoiW0956YLl5bFIrtv35ntZANU1WvTm1F+1mfryxvTl6L8751ei/L3p9lcO1xk+/juyVqU7+b5vl/pltOdYYxz1s+yNamfHUT57ed+Lcp/7mMfi/KfffHFKF9VdfXjPxrlB+NsPn/qS+9E+X948HyUX12dRvlXtu5G+XuTbG6+vXE1yldVzVeznrAM70E/fDy/FfnFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABpGT2ykyTiKL9cnUf70Snb+gxe6KD/4zF6Ur6r6vmu3ovyl8UmUfzhdi/Jv7V7Jzn97O8qPd7Mevno3itfq7iI7oKqGx7Mo382zMfp+GeVT/TS7/tPr/z3Kf/L556P8Z8L8J278qShfVbV1ZTPKHzw8jPK/+iBbK/Y/mt3jZ1+9E+Vf2n4Q5d852Inybx9fj/JVVbN3s7m8XMm2kuFoGOXPpO+j+GCR3efhSXb+5cOVKP9fbn8qyldVfXXj2Sj/zmH2LL378FKUPz3O5tpgmH2nq2vTKH96kl3P8Dh7T6iqGsyzz1DLMB8+12eR7jvjV383yn/8xktR/qPXrkX5pz+Tr3lbr2TvR12XPRvXd0+j/DLbRuq5rUdR/gc23ory74yvRvlf33gxyldVLSfZe38/DOfn4PH8VuQXJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGgYPbGRBllHW65klzbd7KL86bV5lP+ZF78R5auq/uqV/xXltwYnUf53Tl6I8v9h8QNRfvftnSi/+U4Ur8uvZ593/OAoG6CquuNpdsB0luUXyywfunf1C1H+5e0bUf7q1laUv76TPRM7L1yO8lVVK0+tR/nlPLsHj16eRPmrH7sf5f/2C78W5T+5cjPK/4/Nj0f5n791JcpXVc02VqJ8P87W9354/n+z68K52U2z/OSgj/Irt4dR/o3hs1G+quobs+ei/OZb2TU987VsfewH2b788NVs3z94YTXK95PsHq8dZddfVTWcZc/FIFy/qs/OfxY3t345yn9y5eUovzIeR/nN1ew+Ty5l61dV1XgzXPPmiyg/P8zmzvH1bJ/64ctvRfk/vvbtKP/1Yfb+tbaWvW9WVS3HG1G+H4bzM1yP3vM0j+UsAAAAH2KKEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANAwemIjdV0U7wdZfjGJ4jXYnEX579/8djZAVf3J1WyMcZd9iJP+O1F+ffQ9UX54kPXqK185jvI/cf1LUf7W8GGUr6p6/defzw6YjLP8YpHlQyuT7JkYD4fnmp+MsiVjuBZ+n1XVhXP/eD977vZf2ozyP3X9zSj/lzdfj/I3Rtn1HPbZWrS28bkoX1W1mKxG+eUw/BtcuB+cSTg3ByfzKD8+WEb5tbvZZx4d59vz9reza/p721+L8h//kWej/PHRaZT/p/8tmwtvb65H+dl2FK9BtoX/wTHzPjtgmd2z6sPzn8HR/n6UX4afYR7Ozek8m5uLcC5XVc0Pp1F+uncS5T//a78Z5Td/7rNR/kfDfecT440ov7t8FOVXxvk9OA63kf4JbCN/FL84AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQMPooi/gPfV9FO+W4ekXXZQ/Wq5kA1TVwfI0yo+7rMfeX1yJ8g9P16P86DD7jhY3/1WUf+bVH8vOvwxvclW9+8qXovzBG98bj3GeFotFlg/nzXQ+j/LH02l2/gfHUb6qanE0i/K//dZbUX72p69G+avjwyi/Gs7j1KzPlu3lMr+edD3tltlzl67vZzLP5s5gms2F0XF2/sl+dh+GJ1G8qqouvfYgyl/7CztRfrgWvjIcZXtgLH2MwnyX3eL/c0z47hLma5Hvg6mP9X8lyu8fZ/vsw4ODKH9rdzfKX/n6VpSvqpofZHvb/pvZNf3iyz8W5V/YeTPKb3fZXEv3kf3lapQ/neX1YpAtwTWYX8zc8YsTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANIye2EiLRRQfnmb58WEf5Qf3JlH+83c+E+Wrqk767Otd7eZR/nf2n4/y375zJcpv7EfxWjv5C1H+wcGdKH88nUb5qqrFchkekD13NTjfvz3Mw+s5Oj2N8ntHR1H+9qNHUX7yTr7EpPfsV97O5mY37aL8b+6+EOV/YZx9R1vDkyj/Xx5+Osqf3FuL8lVVmwfhejrNntNuHs6zM+jn2Xpa01kUHx5n5x+dZHOh77LntKpqem0zyn/93dei/LWDgyj/9v37Uf7oxg9F+S5c3kdH2Xc6ypbHqqoanmQX1c3CuZDuUWcxHGb53/v+LD74UpQfhHPhLO8Km++sRvl//1+vRvmTvxXF697RRpT/z/t/LMp/ZeVelP+fe69E+b172fVXVV1N952wJzyufccvTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADSMnthI01kUHxxNo/za/dUov/XmOMq/US9E+aqqr125EeW7QR/l+8Ps9q3dzPLrt5dRfnl1O8p/9e3fjPLT+TzKV1V95969KH95EP4tYZl9R6n7N29G+Y21tSh/b28vyo9H2TO0e3gY5c9yzGItm2cbb2f3+LXBS1H+K+G8HwyzeT/fnUT5rd/Ll/n1e9lcGx5m63XN8rkcC9eLLtyj0s+88iC7D8ONYZSvquoHXZT/1d98Psov33w7ynef+WSWfzmK1/rtcM8Ml/e1+4vsgKoa72XPUXd8GuX7eX5Nqa7LnqOaZM92/50/HuXfuJuteW98OYpXVVW3tRHljz9xJcqv3o/ide+rT0X5n7/7J6P8aCVbH2d7K1F+41vZO3ZV1dq97NkeHmZz7XHtO35xAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgIbRkxqon06jfHd4HOVXbmcfZafWs/M/HEb5qqr5+iTK9112/mH2ldZkfxHlV+/Ponw3y85/6/XPRPn+8CjKV1VdHrwaH5Po+/5cz/+J4d+I8m/f+aVzupI/sH9yEuXHw3zenHznT2RjvHga5S+/kV3T2t0sP9tYjfL9IJv4o+PsmVu/N4/yVVWrt7P1d7Cf5ftptracySz73P1J9hwN9rK/O04W2X0br55hew7Xo34yjvKDV17Mzh+lq9buZPdgvJ/NzS68oNFh/pwOH4b71HG2ptYs3PjPIN3XwleXXLhGdpPs3auqqh9n8238KLsPl76VfYb0nXO2sRLmeUO1AAADKklEQVTl+1G2Tw1Psmdi7X72PlhVtXYrmwuD/WyupT3kPcd9LGcBAAD4EFOcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgIbRkxqon86yAw6Po/gwO3utTedRfuX+JByharGSXlWmW/RRfnCSfebByTTKdwfZPeu7Ljv/JL8H/Tz7zLVchvnsHpy3F05/Nsr3386uf7pcRPn5+nqUr6rqNrL88MFBlF+fZZ9h9d44yi/H4bwP/3zVzbJndHiYzeOqqsF+OJcPj7IBTk+z/Bmkc78Lr6kP5/5gFq5FozPsH4PwYUrXu1B3mu373VF2D0bpd9Rn9yy9/qqqOj6J4v1J+Nylz9FZpPvaIltT010ze1PI52ZVVRc+G6Nw7gyPsnU4fefsJ9ncj9+/5um+k8+dwX62j3QHWb5/TPuOX5wAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABoUJwAAgAbFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgYfTERprNonifnn+5iOLd6TTKDw/GUb6qajgIe2nXZfk+/Jbm2XdU83kU78N7XNPwmViE119VtVyGY2T56sP8+0w3CJ+54eR8LuT/dnoaxdO1YhA+d4PRMBzgnP8elc77WTaPq6r6cH2M71m4tpxFvF6EH7lbZvch/szDMzxH3Tk/e+l6EerSPfCcnek5jffNMP8E5k66r/XhVEvvcrrGp3OzqqpP3yHD+9CdZAvMYC/bd/qzrBeB+Ds9y74zzb6jPt13wr3/vfjFCQAAoEFxAgAAaFCcAAAAGhQnAACABsUJAACgQXECAABoUJwAAAAaFCcAAIAGxQkAAKBBcQIAAGhQnAAAABq6vu8v+hoAAADe1/ziBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADQoTgAAAA2KEwAAQIPiBAAA0KA4AQAANChOAAAADYoTAABAg+IEAADQoDgBAAA0KE4AAAANihMAAECD4gQAANCgOAEAADT8b3JIyR34KHDcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('FastICA', d=4); # 16x compression from D=256"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3dmPpXd+3/fvc5baq7p6YXeze0gOOSRnONLMyBIlW8hYgZaxpMiWEcMjGDDsxMg/kJsgAQIHQQDbMBAgAWIDvrcBw7ESyYhsyZZsWbAsjaSxltFwhvtO9t5dVd1dVafO8uReQOPnT4HVTVKv1/X7Wc6znm+di+r6vi8AAAAebPCodwAAAODjzuAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBh9LA29LXB1/uHta3/HMPtU9kCy8vxNvqLZ7MF5tkhmm9m+zTcP4r6fpDN1YPJ9GTXf28/6quq+p3dqJ+HfepXF/+8S/qP230Dj0J631R9/O6d0eMXo75fX423cfjZM1l/bhz1W6/cjfrh1VtRP7tyNepp+zTcO/AoPOje8YsTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBh9Kh34FGZ7+xG/fDzz8bb6O4fRv308e2on68Oo/5oeynqB9NF1I/uZ5fT4HAW9f1BdjwBPi7u/cCTUX//QvZ8r6qabnRRP1/J1j85tRX1p19djvqlM6eiPjV/6ZUTXT/w6ecXJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfSod+BRGX3mctTPN1fibcxXx1F/cGEp6mfLXdT3w6zvFn3UDyfZ5bR2ZRL13WNnor6qarBzN1tgZzfKh9unsvUDnwpHP/WDUT++P8vWv52/nu89M4/6zct7UX/72kbUH5zP3ptbb2XvwDPfuBb1w2efjvr5629FPfDp5xcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBh9Kh34JOiO5zFy0wfW80W6LP8aKuL+vly1o/vR3kthtn6p1vjqO9H+Zy/tFhE/fDC+aifX7se9cCnw9ofvRf1d3/oyahf2glfCFW1cXkv6p/Y3on6v/XFfxn1/+tLfzHqd0bbUb/xwemoX7oevtTgU2q4tXWi65/vZc+iTxK/OAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACAhtGj3oGPynD7VNT302nUL9bGUV9VNTk9jPrpahf1h2ezfhF+hMVStv7lO33U98Ns/d08W39VVR1l53l+7Xq+DeBPndmVq1G/tHcp6q+9mL0/qqq+58zNqP/vLv2HqP+ZtcOon37xl6P+73Y/FfXz396I+qPz61G/fPti1Ffl1wV8FIZbW1HfnTsT9Yu1lagf3duO+qqq2dvvxss8Cn5xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaRo96Bz4q3cZG1M8vns769XHUV1XNVrqsX8v66UYf9V2WV90P92c96ydb2dw+mOaX63g4jPrh9ql4G3yyxOd4eTnrN9ejvDs8ivrZ+x9EPSdj9PjFqD9azp5Fm+9GeVVVvbObvdfeO3c26u8svxb1m8OVqP/Mqd2ov/7EdtSvX8neUeGdD4/MfG8v6hdf+VzU33k+u5dX7+TfpTZv70R9+pk/Kn5xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaRo96Bx5kuH0q6vt796K+m25F/dFWfqi6RdZPN7J+eJT1s7U+61e7qF/Zj/IaTbL9Wb55kG2gqmo+z/Kd3agfXjgf9Tx63UZ2oy3OZs+Ke5/L+spug1p6/mLUr7x6NdtAVc3e/yBe5k+b2ZXsuC49lT0r1kfZ87eq6sofn436X9r8ctR/Z/9S1J9fuhv1N+5n9+Z0LTtG/TDKqz+TfQ+pqqrwuoCPwujx7L1w9YWVqL/7o9kXvLtvrkV9VdXqlaejfvTSW1E/39uL+gfxixMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0DB6WBsabp862Q1cupD18z7KD08Ps/VX1d4zWT84yvrpZvYZlnZPdk7uuxNdfc1X8st1MMzOW3qdzq9dj3oevdlnzkb97S+uR/3NF+dRP9icRn3dXI7yp3758Wz9VbV8divqF3/03Xgbf+p841tRvv49n483sX32TNS/efB01L/02ctRP17Lru35ldWo38putVram2ULTMKXclWNHr8Y9bMrV+NtwJ/Un86e2Xc/m63/f3/xn2f9mZ/MNlBVs1/Pnl/d3l68jY+CX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgIbRw9rQfGc36ocXzkd9t3c/6u/9wOWov/mDi6ivqjr/9K2oXx1Po/7tt7NjNLiezcnje1Feyzt91I/uZ8d0eDiL+qqqfmUc9d3yctQPt09FPR+99Bws5tl1uvd0lNcPf+W1qP/bl/9l1P/yve+N+v9r5Seivqrq6V9Yj/qVZ7ODNH/9raj/NBg9fjFb4Hb2zqyqOvfNrD/1xmrU3/y+tagfTLPn7+GZLupXb2bvkKXr2fcE+KToDo+ifvpY9n3qZ9f3o/6Vy9+K+qqq/+fi16L+UX378osTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAwetQ78CDza9ejfvjF56N+tpLNjKsX70Z9VdX/+YV/FvWbg6Oo/x/HfyXqX7312ag/9VYf9Ut786hfuXEQ9d3hLOqrqhbffjlehk+W+c5u1A+Osuto7VoX9bcna1H/wlLW18a3o/zffP6FbP1Vde25p6L+8TfjTfypM7ty9eQ3Em6j/9Hvj/rTL0+ifrY+jPqV29m9tvn6vajvbmfPiuN4KOcZ/oR+lN1rw92s/8Zh9v3ug8npqK+qWrmVbeNR8YsTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAwetQ78CDD7VNR3+3dj/rVm5tRf2V3Neqrqr5x8Lmo/+9Pvx31P3bulah/efxU1M+Wu6hf259H/Xx1HPVLV+9EfVXVIl6CT7vunQ+j/sy5tah/7VtPRP3PLf141D+7fiPqX33vQtRXVZ2718fLJNLnOydj+Ou/H/XLj1+M+n49e2929w9OdP2zK1ejHj4puln2/Wv7u9n3u7/73s9E/bffvhT1VVWfnWbf2IaPPRb18xvZu/NB/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADSMHvUOPMh8ZzdbIOzHj21H/flf34r6qqp/tPHno/76c9k2/nDnM1E/ut9F/drNWbb+3UnUD+/cjfr+3r2or6oabp+K+vi64xMnPccrr1yN+mdm56P+u+9/Puq/M8z6U9ltVlVVW+8eRn03m0d9H9V8XMyuZPdCavT4xajv7h+c0J7AJ8vszbejfuMLj0X9W7/0TNSf2cmf8quvfxD1sxs34m18FPziBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaRo96Bx6V/g9eivrTw+/NN9JtRvn/d+GrUT9fivI6++o86se706gf3tiJ+n6arX++sxv1D8Nw+9Sj3gVO2Oz9D6J+fO9e1D/xwZmon13IrrnBfnafVVUN7h9G/eztd+NtwJ80u3L1Ue8CfCINt7aifv3lG1E/vn866muR5VWfnPeIX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgIbRo96BT4r+m9+Olzl799mo33x8K+q72SLq+1E2J49v3MvWP51G/fza9agfbp+K+qqq+c5uvMzHaf188qTXxDBc/+ju/WyBySTcgusa4JNkvrcX9el7Z2k6yxaYhX1V5Us8Gn5xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaRo96Bz7N5q+8HvVLO+ezDUwmUT7f2c3Wf+Fk9ycV7z98AriuAXiY5nt72QJhP9zaytb/CeIXJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfSod+CTYrh96sS3Mb92/UTXH3+GyeRkdgQAgE+l+d7eo96FE+MXJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgoev7/lHvAwAAwMeaX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADSMHtaG/sLyX4/+0243DGe64TDKu3H40cdLWV9V3dI43Ea2T/0o+8w1CI9p+M+Ru/kiW/90FuX90VG2/qqqySTcxjTr5/Oo/zcH/6RL+q8Nvn6y/6G6i3anqsuuoS68L49lEH6GRXZI36ifj/rxUvasGC8vR/00vKYP7t2L+qqqUfgZTtobk0l4kqt+cvO/zd45K9l56FZWor5fX436xXq2/qqqxWr4Dgnfs1147wwOs+fp4F52bXd72bW92N2L+ne6/zfqq07+fk69fPdufO98bfTXsnsnfQafsD68Tquq3hz+QtTPwu8jzw++HvVXT/+7qP+4uXj7R+Nl3lv/laifHuc7YeBB7x2/OAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADaOHtqVBl/XDYZR34/CjLC9n619dydZfVf1atsxibSnrl7Jj1A/DObnvo3wwW2T9/lHW3zuM+qqq6sLrLpV9hNir/c+f6Pq3zpw90fU/fvcn84XCZ0WXnuNh1n9u9lei/ubmb0R9ajqZRP3zg6/H23hv6ZfjZT5u0ndCNx5Hfb++GvWzsxtRPzmXvaOqqg63s3fCPNxEN8/65bvZBlavZe/Apeks6l/+8B9E/dPPPhv1VVWT6TTrl7LPPD064ZdOVXXpM3gUfv8aZN9F+nl44c3zY3TqbPYu3L11K+q7Prs3Uxsb2fNlHh7TW9evR318TVR+bY/De2ccfu9/EL84AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANo4e1oW44PNG+RtlH6ZaWon6xsRb1VVWz06tRP93KPsPRRnaMFuMor26e9aPDPuqXdrMdWhrkc/5gscgWmM2yfh6uP/R891ej/r31X4n61dXsGh2m9+Vefny60XLU9+E5S+/99Byf3tyM+v3JJOrPHf1s1HfL+X3zxMHPRP3V7V+L+mn4mY8lvVbH2fOoX82uo6MzWb/7dP56vvdE9gyeb6fPuy7Kl69n52D7lezeP3MrO6Zf+fKXo37eZ8ezqmp4jPdUYnp0dKLrr6oTv3e6YXaMusrWf5y3cvrcXg/fnd218L0TWg7PQfoEfnb4c9kC6TVUVRVe2qfPno36+Tz8UvsAfnECAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABpGD21LXZf1w+HJ9otFlPfjcP1VdbQ9jvrJVjbHHp7N+vlylNfwMOvH9/qoH8yyYzq6l5+Dbpxd4t0ovCWOplmfSu+b0NrKStSvr65G/fRK/reZ9JytPvtHUX/41vdH/eTJ34361XF2o43e+6Go7zayc3Ac3Tx7Pq6G18V4aSnqj2WQvkPCa7XPnncV5pMzWV9VdfqFW1H/tcsvR/35pb2o/8UPvi/qr80vRf3W22tR3735I1Hfz+ZRX1W1v/qvon4Ufnd5GPdOF753uvDe6cL3Tmrlid+LlxkMsmvp7v5+1HdL2ffBS/s/HfU1ya6jpdks6rvV8AvkcthX1fnlx6N+e2Mj6o+mH833Nb84AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAICG0cPaUDccZguMsl3rxuNs/bNZlPfjfMacrmfLLEZdtv6NKK9FeIgG06zvw1Ocft5+eIw5fxAuk16ng+wzxLps/6dHR1G/f3gY9Vtra1E/7RdRX1Xxvb+2vBz1k7XVqE8/89GVH476/cd/Neq31tejfvbOi1FfVXX05O9G/eYgO0b7k0nUH0eXPi+67F7uDrMH5Or7d6N+8JUzUV9VtbGcHdc/t/F61P/s+n7UXxztRv3/cvsvRf3d17LrbrO/EPXD+9nztKpqfnMe9Zvh82USPuOPJX1vhu+pVD/JPnP6Tqiq2k2f28N/EfWnFn3Un/7eV6J+9/0fivpaXoryxWb23unCc1ZVdWHtdNSn5/mjeu/4xQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaBg96h14kG4YznRL4yjvt9ajfra5FPVVVUfrXdRPN7P+aKuP+tTgKNuf6rN+OMnO8dJqfrmORsNsgS78zN3J/u2hG2T787nB17P172TH9OhudjyvbP2LqK+qevzc+ai//dpXsw0MZ1H+6ptvRv1TT/9s1K+9me3/Ymsz6u8c4xw8e/Zy1O9PJlE/Xyyi/ljCe7lP3zlh3w+ze2cwj/KqqjqYZu/B2/ONqL85vxH1j4XP32cv3Iz6a5tPRX0fPk/7Uf58f+z06agfhtfpMLyOHoZ+lj1T03528T9G/e17+TEazrMb7oWnn4z6K0erUb+7vx/1/Tx7ps4unYn629+zFvXrV/MH2OF370X9+VOn4m18FPziBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0jB7Whvq+zxaYL6K8C9ffL2Uffb58jBkzXGQRno0uy2OLpazv7mX9+CA7x4Np1ldVVXrdpf0n3SC8SMfZRXrxzk9k66+q8aXvRP28D6+LeZY/+cX/IervfWYt6vvPfynql+5mH+D0b/5Y1FdVrTz9VtRPptN4Gx87XfZE7cfDqB/sT6L+4m/tR31V1TunH4v6f7z056L+u2cvRX3q9kF278yXs3N2cGE56jfezq/r0+vrUX9wdBT1o2F23R1HPw8fktNZlM8/+82oX1nOztv8g+y6rqraeOZ3ov7Db30528By9p7aXsvuhd21lai//cVw/T92EPV331iN+qqqp6/9jaifLX4t6j+qe8cvTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAw+hhbWhxfz/qB/N51HfDbAZcXDgV9fcu5Ydq/2IX9fOlPur7cOztZlkfyz5udYuwn4cLVFXNsuuoFuE2+mPs00laZNfQ/rnfiPqzW1tR/+HKzaivqjrzwU9nCyyyc7x47lLU3/gzG1F/5yvZNTE6exD1i/fWo/7i1peivqrqw5cuZwtcvxXly+H74Dj68F7u0mfFLLz3R8MoH07C/amq09/N+g8nn4n6//sz56N+tJa9dOY3l6P+zEH2vBvfz87Z4J2rUV9VNXguezEfTqdRPzk6ivpjCb9/7Vz4zai/tHYu6ifhMerWVqO+qmqRPi+2NqP+wpPfjPqrr70Y9XUmu3fuPpOt/u+8+AtR/3+c/olsA1W1+53Hov7gGx9E/VPns+fXg/jFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGD2sDU2f/k9Rv7q8HPX9jR+J+oMLK1F/+0t91FdVPfnClahf9F3Uv/f+2ahfe2Mp6pfvZJ95tB/l1c2z9Q+mi2wDVdVNZ1Hfz+bZBhb5dXGiBtk1NDk6yvrpNOrPHP5XUV9VNT3376N+e2Mj6u9sPhv19y9nx/THX/x21P9PF/911P/qlz8f9X9/82eivqpqMcyeLadeGkZ9d+1W1B/LUXat9vPs3u+v3Yj6y9//VtR/+K3nor6q6uz9i1G/+W527+w8l72Xp2vZO6cPv5EMD7Pn78qVe1Hf7x9EfVXV9Ze+kG0jvO5Oh/2xdNnf1E99+MNRPzn1najfWF2N+rs3snu/qmp1KbtW7x8eRv1h+K7tlrP9qUm2/qOz2XX0cxu7Uf9b59+J+qqq3zh9IerPTP9S1C/6b0T9g/jFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGD2sDS2Nx1G/trwc9fejumpyahj1m0/thFuo+t+e/cWof2ywH/X/8+pfjvqXrjwb9esfRnltfHAU9Us7k6gf7KRnuaoOs23UbBbl/XyerT/VZX/b6Lou6k/v/HjUz69Po74bH0Z9VdXw3R+M+vtra1E/fiK7z1ZvrET9naPVqP/ceCPqL2y9EfX/+LM3o76qav/cxajf3FiK+tGd7H1wHP3BQdTvP/5bUf/cFy5H/Ur4DuzOnYn6qqqj6c9n25j9zahfu54974aTPurj9/Jb2Tvh9PhXon7niez5WFVV4Tuhu5t9hn4/u66Po1vKrtXBavaM7N//s1G/e+m3o747zN87N/74+Wwbo+xduPP6l6P+3Od/J+pvHH4t6sd3snvtdyfZ5716uBX1VVWDWfa8qD7rF4tFtv4H8IsTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAwetQ78CCDQTbT3d/411G/cvvpqL9+dTPqq6pe+/zFqP+RU1ej/qtn3oj6P9z8XNQPZlEem6+Oo36wl8/5/WyeLTAP+77P+tSgy/rhMMq7cfYI+HD5l6L+c088EfVVVQevfG/ULy78ZtQP3vtq1G+/nt37f/B7z0b93xpPov7x5d2ov3rzVNRXVZ0+Cq/rRdh34XV9DIujadQPw3fO3YODqP/g97N3zuBC/mx57vLlqH/n7dtR/9jhf4j6nb0fi/rVUfb8qls7Ub5/PrvXuvAaqqr4nXDCb5CHI3yPdOPs3V+vfyXK+2e/ka2/qgbXsvfCpe95OeqvvP7lqJ/Msi9g3WwR9VuvR3n97Tf/66h/9Z3s+29V1ZNXsu9f881fjfrBYCvqH7iej2QtAAAAn2IGJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAw+hhbWg+n0f95Ogo6rfW16N+9vZe1J/9T6ejvqrq7239VNS/8cLvR/23di9H/dJuF/Urd2ZRv3xjP+oHe1nf37oT9VVVfd9nCywW2frD6/rExZ836y9N/2LUH7wW7k9VdRurUb88Hkf9UXjOVr/5ZtQ/Wc9E/Tc//FLUz5ejvLby26Y2PphG/fB+9ryuWfZsOY5ukD3v3nv33ahfeufFqB9sZifusYu/HfVVVe9c342XSWytrUX9zv3sb7NfevIPov6P689E/dGVH476S1/43aivqnrvP16M+uHmZraB8B11LIMT/pt6l92bB09k5+GpU+ejvqrq1v3sO+Ttu3ezDYyyr9vD8Bj1734Y9afPZffy+7/2ZNSfvZ6/+zdeuhL1Fy6eifrDcK54EL84AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAICG0cPa0NFsli1weBjl88Ui6lfv34n6c384jvqqquFkI+p/4eWvRv1gGuW1/W52jMZ72TnrDo6ifr78r6J+5an8HBy89ELUD5bybZyoRZ/1XXaO+/C+XOzvZ/1z34r6qvxeXnz3xagfbGfn+Mwzfxj1k2u/F/Wrb/2FqK/RMMoXS/ljfnAU3vt796O+n2TPimMZZsfp2e7nsvWPw2fFMPs75dVvfjZbf1UNNtajfnH3w6jfO5fd/+l18dqVK9n6D74Y9cuXfyfq15bORn1V1db3vRH1b4Wf+dytH436Y5nPwz67tvtp9uVl7d5/GfXXl78R9VVVg7t3o36xnb2ndga/mPXXo7y2tzajfnwzu5cv/WaU1/jmvWyBqnrswm+FS3RRPU2v6wfwixMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0DB6WBuaHB2daJ9668o/jPov3P1v4m2cu3ku6k9/ZzPq+1E29w4OZ1HfTaZZP83WP9j781F/eO9+1FdVdeOs7/s+W2A+z/pUv8jydHdm2Tk7fPr3o35reT3qq6oWi+wzz9JzEH7mWy99IVt/qFu9k/Wj7LE9HA6jvqri67qfZM/r/oSf78cyCP+SWVlMAAADlElEQVSOGF6nNc/6wfpqtv6qevvon0b98y88E/XTeXbtzW/eivrD/R+M+hrei/K7o/2ovzkOXyBVdTjN3puj8P7sZ9n6j6NPn6nTLsq7RfieDXVv/0C8zO75X4v6M/1W1m9l/ZWbN6O+nxxEfXbGqpZu7WQLnPn34RaqZvONqJ+G1+n+ZBL1D+IXJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfSwNnRwcBD146WlqB8Nh1H/5DPPRP1g9ZtRfxxX3rwa9ZdnfznbwNI468Nj2i/6bP3zeZR3o/xy7WezbIH0M5yw+JhWdkxfnf+zqL9w9Jmo34vq41lNr9P5Iuq7cP016LI+3J++n2brnxxlfVV8b/Zpn96XxxHeO31473Rddp77ySTrp/kx2jq/HfX3Dw+jfr7IrtWlk74ujrL9uTO6m63+GNfpLPzMezs7UX8qqo8pvXfS49SH77XwujuOeXjepuFnTteffqdd3Lsf9V34nfzG9r+N+nOL7FlUVbW7vx/16Tk4mobvzgfwixMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0DB6WBuaTiYnu4GlpSgfzudRf3CM/V8aj6P+/NmzUb87+bdRv3nlv4j6bhxeHl2X9X2f5eE5q6qqRbaN6hdhHq7/pIX7/3z3V6P+vZ1fifq1zc2or6qaHh1F/Up6XYTrr+Ew61PpfZMK77OqqlqE98E869Pr9FjSbSyyvyP2FR7Xh3CM9nZ24mUSs/BeOxeuv0/vzdDZa1+N+qun/128jfS7zhP7Px1v48Sd9L2zmGXrfwgODg5OdP3DE36P3Dz76ye6/lG4/8f5znycZRKTj+j54hcnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKCh6/v+Ue8DAADAx5pfnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANDw/wP9t8Tet/J6zQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "demo('FastICA', d=64); # 4x compression from D=256"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Dealing with Noisy or Missing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The linear algorithms presented above work fine with noisy data, but have no way to take advantage of data that includes its own estimate of the noise level.  In the most general case, each element of the data matrix $X$ has a corresponding RMS error estimate $\\delta X$, with values $\\rightarrow\\infty$ used to indicate missing data. In practice, it is convenient to replace $\\delta X$ with a matrix of weights $W$ where zero values indicate missing data. For data with Gaussian errors, $X_{ij} \\pm \\delta X_{ij}$, the appropriate weight is usually the *inverse variance* $W_{ij} = \\delta X_{ij}^{-2}$.\n",
    "\n",
    "The [wpca package](https://github.com/jakevdp/wpca) implements two different schemes to incorporate weights into PCA, which give similar results to each other.  Both schemes are used almost identically to sklearn PCA, but with an additional `weights` argument (method = `wpca.WPCA` or `wpca.EMPCA`):\n",
    "```\n",
    "fit = method(n_components=d).fit(X, weights=W)\n",
    "```\n",
    "Unfortunately, `wpca.WPCA` expects `weights=np.sqrt(W)` but this might be [fixed soon](https://github.com/jakevdp/wpca/issues/2).\n",
    "\n",
    "To study these schemes, we will assign weights assuming that each value $X_{ij}$ is the result of a Poisson signal superimposed on a constant (subtracted) background level $B$, so has inverse variance $W_{ij} = \\left(B + X_{ij}\\right)^{-1}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "import wpca"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "def weighted_pca(d, B=100, missing_frac=None, data=df_img, npix=16, seed=123):\n",
    "    gen = np.random.RandomState(seed=seed)\n",
    "    X = data.values.copy()\n",
    "    N, D = X.shape\n",
    "    # Calculate inverse variances assuming Poisson fluctuations in B + X.\n",
    "    W = 1 / np.sqrt(B + X)\n",
    "    # Remove some fraction of data from each sample, if requested.\n",
    "    if missing_frac:\n",
    "        assert 0 < missing_frac < 0.5\n",
    "        num_missing = int(round(missing_frac * D))\n",
    "        missing = gen.randint(D, size=num_missing * N)\n",
    "        for i in range(N):\n",
    "            idx = missing[num_missing * i: num_missing * (i + 1)]\n",
    "            X[i, idx] = 0.\n",
    "            W[i, idx] = 0.\n",
    "    # Perform the fit.\n",
    "    fit = wpca.WPCA(n_components=d).fit(X, weights=W)\n",
    "    Y = fit.transform(X, weights=W)\n",
    "    Xr = fit.inverse_transform(Y)\n",
    "\n",
    "    # Compare a few samples from X and Xr.\n",
    "    fig, axes = plt.subplots(2, 4, sharex=True, sharey=True, figsize=(12, 6))\n",
    "    for col in range(4):\n",
    "        img = X[col]\n",
    "        rec = Xr[col]\n",
    "        for row, image in enumerate((img, rec)):\n",
    "            plot_image(image, ax = axes[row, col])\n",
    "    plt.tight_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3UmsZvl93+ffed/3znPNQ8/d7JZIWTTFIWZC2bLESaBN2bJsCN7YhuGNV17ZmxjxyistAgRBAmSVRWDEQhgooUCLksMIlklaEiVKZIvsid3srq55uLfufN8pC++EFP7+XvSt6mo9z/rznnPee87/nPOrd1HddDotAAAAHqz3qA8AAADg/c7gBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfCwdvS53t+dPqx9/Zfor69lH5ibi/cxvXA6+8A4+xONV7Jj6u8dRf20l83VvcPhyW5/Zy/qq6qmm1tRPw771G9Pfr1L+vfbuoFHIV03Ve+/tTO4eCHqp0sL8T4OnjmV9Wdmon71le2o71+/E/Wja9ejnrYPwtqBR+FBa8cvTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACAhsGjPoBHZby5FfX9l16I99HtHkT98OJ61I8X+lF/tD4b9b3hJOoHu9nl1DsYRf10P/t7Arxf7Hz8qajfPZ/d36uqhstd1I/ns+0frq1G/carc1E/e2ot6lPjl1850e0DH3x+cQIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgaP+gAelcETl6N+vDIf72O8MBP1++dno34010X9tJ/13WQa9f3D7HJavHYY9d3ZU1FfVdXb3M4+sLkV5f31tWz7wAfC0Rc/GfUzu6Ns++v543nnuXHUr1y+H/V3byxH/f657Lm5+mb2DDz17RtR33/h2agfv/5m1AMffH5xAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaBo/6AB4X3cEo/szw7EL2gWmWH612UT+ey/qZ3SivST/b/nB1Juqng3zOn51Mor5//lzUj2/cjHrgg2HxT96J+u1PPRX1s5vhA6Gqli/fj/on1zej/h99+Dej/l+9/DeifnOwHvXL725E/ezN8KEGH1D91dUT3f74fnYvepz4xQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAwe9QG8V/rra1E/HQ6jfrI4E/VVVYcb/agfLnRRf3A66yfhV5jMZtufuzeN+mk/2343zrZfVVVH2Xke37iZ7wP4C2d07XrUz96/FPU3PpE9P6qqPnLqdtT/40v/Ieq/tHgQ9cMPfy3q/3X3xagff2s56o/OLUX93N0LUV+VXxfwXuivrkZ9d+ZU1E8W56N+sLMe9VVVo7fejj/zKPjFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGDzqA3ivdMvLUT++sJH1SzNRX1U1mu+yfjHrh8vTqO+yvGo3PJ6lrD9czeb23jC/XGf6/ajvr6/F++DxEp/jubmsX1mK8u7gKOpHV96Nek7G4OKFqD+ay+5FK29HeVVV/Xgre669c+Z01N+bey3qV/rzUf/E2lbU33xyPeqXrmXPqHDlwyMzvn8/6icffT7q772YreWFe/m71MrdzahPv/N7xS9OAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAEDD4FEfwIP019eifrqzE/XdcDXqj1bzP1U3yfrhctb3j7J+tDjN+oUu6uf3orwGh9nxzN3ez3ZQVTUeZ/nmVtT3z5+Leh69bjlbaJPT2b1i5/msr2wZ1OyLF6J+/tXr2Q6qanTl3fgzf9GMrmV/19mns3vF0iC7/1ZVXfve6aj/6spPR/2f7V2K+nOz21F/azdbm8PF7G807Ud5TU9l7yFVVRVeF/BeGFzMngvXf3I+6rf/evaCt/2jxaivqlq49mzUD15+M+rH9+9H/YP4xQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaBg8rB3119dOdgeXzmf9eBrlBxv9bPtVdf+5rO8dZf1wJfsOs1snOydPuxPdfI3n88u118/OW3qdjm/cjHoevdETp6P+7oeXov72J8ZR31sZRn3dnovyp792Mdt+Vc2dXo36yZ/8IN7HXzjf/tMoX/rIS/Eu1k+fivof7T8b9S8/cznqZxaza3t8bSHqV7OlVrP3R9kHDsOHclUNLl6I+tG16/E+4M+bbmT37O1nsu3/2id+PetPfSHbQVWNvpHdv7r79+N9vBf84gQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAwe1o7Gm1tR3z9/Luq7+7tRv/Pxy1F/+5OTqK+qOvfsnahfmBlG/VtvZX+j3s1sTp7ZifKa25xG/WA3+5v2D0ZRX1U1nZ+J+m5uLur762tRz3svPQeTcXad3n82yuvTH30t6v/l5d+M+q/t/FTU/w/zn436qqpn/8+lqJ9/IfsjjV9/M+o/CAYXL2QfuJs9M6uqzvxh1q+9sRD1t//yYtT3htn99+BUF/ULt7NnyOzN7D0BHhfdwVHUD89m71NfXtqL+lcu/2nUV1X9Hxc+F/WP6u3LL04AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQMPgUR/Ag4xv3Iz6/odfjPrRfDYzLlzYjvqqqv/+J/73qF/pHUX9v5j55ah/9c4zUb/25jTqZ++Po37+1n7UdwejqK+qmnz/h/FneLyMN7eivneUXUeLN7qov3u4GPU/OZv1tfz9KP/6Sz+Zbb+qbnzo6ai/+KN4F3/hjK5dP/mdhPuY/vWfifqNHx5G/WipH/Xzd7O1tvL6TtR3d7N7xXE8lPMMf850kK21/lbWf/sge79793Aj6quq5u9k+3hU/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMHvUBPEh/fS3qu/u7Ub9weyXqr20tRH1V1bf3n4/6f7bxVtT//JlXov6HM09H/Wiui/rFvXHUjxdmon72+r2or6qaxJ/gg6778dWoP3VmMepf+9Mno/7vzf5C1L+wdCvqX33nfNRXVZ3ZmcafSaT3d05G/xt/FPVzFy9E/XQpe252u/snuv3RtetRD4+LbpS9f63/IHu/+9fvfCnqv//WpaivqnpmmL2x9c+ejfrxrezZ+SB+cQIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgaP+gAeZLy5lX0g7GfOrkf9uW+sRn1V1f+8/LNRf/ND2T6+u/lE1A92u6hfvD3Ktr91GPX9e9tRP93Zifqqqv76WtTH1x2PnfQcz79yPeqfG52L+h9ceSnq/6yf9WvZMquqqtW3D6K+G42jfhrVvF+MrmVrITW4eCHqu939EzoSeLyMfvRW1C//xNmof/Orz0X9qc38Lr/w+rtRP7p1K97He8EvTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfCoD+BRmf7xy1G/0f+pfCfdSpT/3+c/E/Xj2Siv06+Oo35maxj1/VubUT8dZtsfb25F/cPQX1971IfACRtdeTfqZ3Z2ov7Jd09F/eh8ds319rJ1VlXV2z2I+tFbb8f7gD9vdO36oz4EeCz1V1ejfumHt6J+Zncj6muS5VWPz3PEL04AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQMPgUR/A42L6h9+PP3N6+4WoX7m4GvXdaBL100E2J8/c2sm2PxxG/fjGzajvr69FfVXVeHMr/sz7afs8ftJroh9uf7C9m33g8DDcg+sa4HEyvn8/6tPnzuxwlH1gFPZVlX/i0fCLEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQMHjUB/BBNn7l9aif3TyX7eDwMMrHm1vZ9s+f7PGk4uOHx4DrGoCHaXz/fvaBsO+vrmbbf4z4xQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaBg86gN4XPTX1058H+MbN090+/F3ODw8mQMBAOADaXz//qM+hBPjFycAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoKGbTqeP+hgAAADe1/ziBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgYfCwdvT83Fz0P+0+N/qlaPu9ubmo7xYWsn4x66uqpgvZMdXsTLb9mX7W97I5uQv/c+RuOI76Ohpm2z84yrZfVdODw+wDh1k/HY2i/rd2/tcu6T83+NXoJHS9aPO57iH8W8sJf4drS/8u6s+dPh31/X62LsfjbN2MJ5OoPxxm6+xhODzK1vIfXb0aXxRfWPmH2dpZXIy2360sRf14LeuHG/NRX1U1XsjW5zRca72j7Nqb3crOc//2dtTXva0on+4fZNvvjnEvCtdz+gx5o/tK1h8exl/iU08+Ga2d9J63GL6vpds/OsY9b+ZHH4v6+J1zPnwfTJ+1/bAfZ2t5srMb9ftPfDvqj2NzK1v/F+9/Luq/fvRv/n/Xjl+cAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAICGwcPa0drp01F//ej/ifqLu1+M+m6QffXp4nzUV1VNVheifrw4m/UL/aif9Luo742nUd8/HGf97jDqe718zu+m2XeYTifZDsLtn7juZP8tZDrOzvH19d+J93Fx+wtR383PRf2lvV+M+um570T9THhv2dzejvr1G5+J+sNL34r6qqq5mZn4M4+7rp+tnel8dr8ensqeIbsX83Owfzr7DpPsK9RgL+sXb2bPqJVhdn/p3duK+m4uu1dUL3tmVlVNDw6j/o3xv4369F3qOPr97Lyl94v58DzMhMdzLLPZYkifOzUJ3xVmsmuvC8/BtMvW2tFTvx/1G4vLUV9VdTDM3gnnNz+b7WDmvbmO/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMHvUBPMjy8nLUby//XtSvHX4p6qeLc1FfVTVcm8/61X7WL2Zz73i2i/reaBr1M7vZ8c912fHMDMdRX1XVHQ2zDwyz79D1T/bfHrpe9jeqsO/62fdNt3/x3Lls+1W1v/btqF/a/ly2g0F22+v1snN8cHgY9aeO/mbU1/okynd2drLtV9VKeN7G43xtnrTpaJT1w/BeERotZGtt73x+b9l5LjsPvY3sWh0fZt/h4MfZc7M/XIr6ldtZX+Ezp8bZWquqmu4fRP2TTz0V9eNJfkzvN/3wPPTDe/Dtzc2or6o6t5K9c6bnee65P4761NG1T0d9NzMT9Rc2NqJ+cT57/62qunL7dtT3lhazHbxHa8cvTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAw+Bh7ejM7b8W9duXvhX1K4uLUV974yifzvSz7VfVeC6bS0fzWX+41mXbX8j6/kHW13SSbf8w+5sOdo9xuXbZd+jCfhrVuTe6r0T985NfyXYwm52D3iA7B/2wr6panJuL+oPN7LqbHh1F/VJ4PFeuX4/6y+GtZbqzG/VPPHch28ExHA2HUT8eZ/ff49g+/3tRPzczE/Xzdz4f9f1LK1E/nsvXzsrl+1H/N5/5ftSvDfai/t89+ZGov1pPRP3s5tmon9k8iPpuZz/qq6rmnv9u1K/Orkf9zn5+TKl7d+5E/VNHX476g172nL1+5nej/uzuF6O+qqoG2XOk9/QfhjvIbvTLCwtRP3z6O1G/c/MzUX8Q3uMH/fyduXfrZ7MPLIb3yP1s/T+IX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQMPgYe2om5uL+rXNX4j66Z1x1Hfr2fFM5vI/1Wgxm0uPlruoP9zI+nH2lWs223wNR9kHeuN+1M9s5+eg1wu/xHSa9ZOwD527eDHq9/vfjvqFdz4V9d3yUtSPfvyJqK+qOtrdi/qti1+L+pUzi1G/8+qno/7y7Mejfv7St6J+dTE7/ndvD6O+qmrm1s9G/dxRto+Zg8OoP47F8JnT72f3oxpnz5zeOLtXTMNbV1XVuZWdqP/ba9+J+o/PzUb9JxfejPp/uvn3o3731eWoX9vOrruuy0/CufX1qB+F19Hh0VHUH8eT+1+K+t7qQraDmew62kjfB5/MnoNVVdM3Phb18+H9Yrj92ai/dz+7X2yc/2bUT1ey58h+9+WoP7h3jHv8xiTKu939qJ+8R2vHL04AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQMPgYe2om5/L+pmZE+2nK4tRP1zJtl9VdbSSzaVHa13UD1emUT8Nx+TxQXY8/dmsn/TD4x9k26+qqn4/yrsu/A7T7Duk5sLrejbs+y9+L+o3d3aifuXqp6O+qqq/vBT1+/v7Ub+x+QtR3zu1EfU1yK65w7s/F/W3bgyjfnZwjNv8wjj/TKAbjk50+1VV23t7UX9q9wvZDs5n18VkJrsB9w+jvKqqNvcXov7d0XrUPze4HfWn+tl3vryxFfW3Tq9G/fLibNR3w3wd3AvvkQuz2TENxye7Nquqti7+x6jfuPNzUd/NZe+DV+e+GvUf23gp6quqbp07E/Vz4XlbPpf9Te+9/YmoT+/z+0+tRf3OpWz785uTqK+qWnllM+qX174R9QvPZOfsQfziBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0DB7annr9rB+Eh9brsr7L+slMuP2qmoRfeZrv4kRNZsM+PGWDw2nU9w7H2Q6q8usiNZmc6ObH4faPhsOo3zs4iPq1q5+O+v5Pfi/qj+PSO1+O+t7qUtSPz6xF/f4T2fZHC9m/Xy1ezc5Z7z/+SdRXVS399GtRv723F/VHG6OoP47R+Bj3i0B3eBT181fuR/3pmey6q6q6cv501P+PCz8f9d8582bUL/ayv9HOUfbQGS9EeQ1Xs4dUN5rLdlBVe29m98jNld/Otn94GPXHceHUqajvnfl+1O+/9tNRf3nxV6J+MsnveTXIXth2rv6VqH/6pZej/t5cthZ6vew5cucjM1G///HsHj99N1ycVTV3L3t2vrh8KerH79EzwS9OAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAEDD4KHtaTTK+n42002H06ifnFqN+t1z/aivqtq71EX9eC78DuHZ6x9lfTfJ+lS8/S77ex7HNLxOJ/sHJ3Qk/9nhUXbS1m98JurnFxejvk7PRPn4tY9m26+qbn4u68N7xfjC6ai/9ansXnHvp7MLe3BqP+onV7JzdvHsp6K+qqr/zbNRv3Lhm1F/OBxG/XEMw7Vzd+m3on5lJjsPgx9l52Fx+mTUV1WdXcuu7bcOs328evl81M8uZud5eGsh6te3s2dmfz9cm3d3o74qf4YM3v0rUb8yDN+ljmFxLrsHL87PR/3WS9+P+r2tn4/60Xgc9VVV5y//ftRff+UjUX8UXhfDS6ei/vTyctRvv5itzV/7xFei/n974r+K+qqqKy+/EPWre9n9Yjx5b15q/eIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMHtqexuMonw6HUd8tLkb93tNLUX/3o5Oor6p69sPXov5wlJ2Od6+civrZt2ajfu5elNfs9jTqB3vh33SSbb+qqkbhdbe3H/XDZ78T9Sftje4rUf/C6Fej/urgN6L+qZcuRX1V1fCHH4n6+5e+FfWLS/8k6ncvdVH/mZ/5QdT/i4u/FfX/796LUf9rK78Y9VVVT3TPRn33n8JHydq/z/pjuLD52ajvzv1p1M8Msu88OX826/v9qK+q2vjjO1E/f2ct6refyJ4hw9W5qJ8Nv/LsdvYMmdk6iPrJj96O+qqqbnYm+0B4nrO70fH0etm/qQ/Cfm4m+xvt9rJvvX2QneeqqoPwnXN68VzUP3suW5tXf7AX9dtz2XdePJNt/+8s34/6W2f+LOqrqv6njQ9F/eAwWzv98Dp9EL84AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANg4e1o8nhYdR343HWr61G/eFKP+oXLt+P+qqq/+65/yvqT/f2o/5fLn456l++/kLUz4ZfeeH2KNv+ZnZN9HaOor6qqtvejfrJU38Q9Qu9uag/aZeeeCLq9+r3ov7i3Lmof+PNN6O+quqZ5U9F/eFRdl2sbh5E/fydxajfG81G/UdmF6L+fP+HUf9vnvlk1FdV7Vy4EPUr6ytRv7uX3euOo5sJH2+vfzTKJy9+L+rvjH496s/WRtRXVS0uZvej0c7fivqlG9m/tU7vRHmN5rLtL1/J1nLvtXeivjt/NuqrqmqUPQen4VqYTiZRfxw7+ye7Prf29qK+28/eFc6ey94Hq6pubG5GfTfMnju3t7ejfrKYPUeu37sX9Xt3s+fadw6z7/vmYb52esOsn06nUT8J+wfxixMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0DB4WDvaPPO7Ud/v96N+fe9vRP3C3VHU37y1FPVVVddHa1H/V5ez7f/CmR9G/XdXn4v6btJFfW84ybY/HGf9wWHUV1VNDw6ivt/L/i1hPMm+80k7PDqK+tE4OweL1/7rqH9mPuurqrZWfyfql/oLUd+7fS/qT7+crf3vPv1C1P+jmey6PjWzG/XXbmf3oaqqUwfT7APTrN/c2sq2fwzd7GzW98O1/9pHo/7sS9+L+uPYfutnon5mNVsLM29k99P0/lvnz0Z5f+Y3o37yRHi/3v181lfVdJithclhtv7fHvxG1B/H9bt3oz59jpy59/NR31vOrqM3rl2L+qqqhbm5qD+zkb3TXrk9jPrezn7U/8SHLkf9a6/NRP0/f/ZXov6d2+tRX1V17np2HR1V9h4/DK/TB/GLEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQMHhYO1qcn4/6fi+b6SZ3dqJ+8bW7UX/m989FfVXVf7v+S1H/zQ/9WdT/cOt81M9uZn/Tua1x1M/cP4r63vZB1Hc7e1FfVTU5OIz63f39eB8n6c7Nm1H/fP1K1Hcz6S1gEtVbq78Tbr9qfXk56q9cvx71C/fvRP38K9nf6MmZS1H/R2//pag/WovyWr2d9VVVq29l66a7vxv162vhlziGruvCD2T3x25+LupHP/pY1PdWV6K+qmq6vx31T33klaj/3ltvRf3a8AtR34X36/H4i1FfW9l7Qnf269n2q2rpVHZdbG9kz5zFzfy6SJ3a/nz2gfQ5spC9W/Qvfivql659Nuqrqrrlpai/e3Ur6t8d/EbUXx5lN+4/eiu7p27MjqL+ev+JqF/MHglVVbXy+mbUb17MdjIcZ9fdg/jFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0DB7Wjvq9bEYbTyZRPwi3393fifozfzgT9VVVveFG1P/Ws5/Ktj+K8lr/cfY3nb99EPWDa/eifn/2q1G/18+Op6pqo34u6hfeyc7BtdXfjvrU89NfjvpucLL/FnJ18WtRf3omWwNVVZPwHFyezdbm5IX/EPUV3lsW/uC/ifonXl2N+vHGUtR342nUV1X1NrP74/j6jajvP33y/2Y3HY+zD4T9rZWvR/2Z9fWov7G5GfVVVRuXV6L+jWvZ9te77H40Obibbf/ZP476zTf+ctQPz/xu1D+5fjbqq6qORtmDeXt/P97Hiet1Yd/P+kl2T1qZn4/6+ZdejvqqqjeuZYth9nz2+vzi8tNRv/tW9jftn8+u1ZU/ye7Z8zez+9dx9O7cj/ors3ey7Xfhdf2g7bwnWwEAAPgAMzgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4ASiZ9kxAAAEcElEQVQAANBgcAIAAGgwOAEAADQYnAAAABoGD2tH48nkRPsK++loFPW9m/eivqrq9LePon79teWonwyyubd/OI763u5h1Ff4N72zm/1N97a3o76qaqOf9d0gWxIXNj+b7SDVZed4Og7XTdgvn8mu0WNJ1/4wu+7G42wd9HvhOTgaRn3t7kV5f/8g6qfp37OqJts7Ub//5O+HO8jyYwnPc2ppYSHq0+vo/KlTUV9VdeX69ai/PPlbUT+dbEX92Z96JepnZ+aj/scrX4/6tdufj/pr7+xHfVXV6+N/G/XPjf921J/phw+1h2EaLujwOXv3u89Hff/F70V9Vb4+e298LOpvPv0HUb988ZtRPxNeF7sH2XNk9s1s7UyXF6P+OO784MNRv3fmd9+T/frFCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGDysHR0Oh1E/6Pej/urcV6P+0t6Xon46mUR9VdXrd/6XqH/h3V+N+v7KctR3s7NRH5tOo/zy6Jeyzc+Oo76qqsbZZ6bhd+jC6zQ2za+7xBvdV6L+9Phc1B9G9X+2t/7vo37t6qfD7WdHdW97O+rPhve6Gs1EeXaFVrwGqqqm4Wd29/fjfZy0dC3XJOuvXrkS9ecuXoz647i094vZB2az7zw9Oor6N65di/rltz8Z9Wvn/1LUd4vZK083lz8zP3T4D6J+OhplO0jvL8cwHWfPna4Ln1OD7N/se+trUT+99VejvqpqPv3ARnaPXLz916J+fJA9p26c/kbUn939YtRX9rpZ3TC8rquqwvfsbpCt54XwXeFB/OIEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQMHtaODo+Oon7c70f9wsJC1N+a/e2oH4THU1X15PxTUb83+b2oX7j66ajvLcxHfaXfeTKN8uk06yvtj7OP8DvUdJL1oWl6PJNxlM8szUb9aJxt/2HYuvStqO9Xdl2na//q4tei/uLOF6O+67qoj9dAVU3D+/Uw7B+KdO2Enp/+cvaBq+EOumP8u2Z4y56ORvk+Aqt3fi7qu/XwlSR95qTf9zjXUPpMmGT9cdZzLLzPp0fUpX+j+P3rGM/lE35XSO/bNT8X5We3Px/13Ux2f5mOwmf/aD/rq+L3l3Q9x+fgAfziBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0dNPp9FEfAwAAwPuaX5wAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQ8P8Bp7bJHnX1PBoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "weighted_pca(d=64) # using weights with no missing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Externally provided uncertainties put more weight on explaining pixels with large values, and correspondingly less weight on on the low-statistics tails."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGtCAYAAAAh7ySEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Uuw5nd95/fvczv3a3errwJJSEhYYMOAzGAMNmCDcYztyvgyTmUxNUklVXFNqpJyqrJITU1WU6kki5nKJpvUFMks7HjwhBpsBxObcQxGYAIjgywQDRJSt/qibvXp06fP7bn8s0hWrii/fDo63ZL8eq3f/8vzPP/b9/wXp9d1XQEAAPDq+vd6BwAAAF7vDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABqGd2tDH+//Wne3tvX/xWBjPVtgfj7eRnf6eLbANPuKpqvZPg12D6O+62dzdf9gfLTr39mN+qqqbutm1E/DPvWF2e/2kv71dt7AvZCeN1Wvv3NneOZ01HfLi/E29h88lvUnRlG/9t1bUT+4fD3qJ5cuRz1tb4ZzB+6FVzt3vHECAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQM7/UO3CvTrZtRP3jskXgbvdv7UT89/1zUTz7xRNQfbsxFfX88i/rh7exw6u9Por7by75PgNeLnfe9NeoXPve1eBvXf+FM1E8XsvUfrK9F/eaz81E/d2w96lPTp797pOsH3vy8cQIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGob3egfuleH956J+uroQb2O6OIr6/vls/bv3ZT9fN+hFfW/WRf3gINufpUsHUd9duRr1VfnvXFs3o3ywsZ6tH3hTOPzkj0f96PYk6i/91gejvqpq523TqF89tx31r1xZifq9k9l9c+25uag/9uSVqB888lDUT88/F/XAm583TgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAw/Be78AbRW9/Ei8zvm8x6ie/8rej/nCtF/XT+awf3Y7ymg2y9Y/XRlE//Mh7o76qanB1J+tPnYz66ZWrUQ+8OSw99WLU33r/W6N+bquL+qqqlXPbUf+Wja2o//uP/37U/5dPfyrqt4YbUb9ycTPq566GNzV4kxqsrUX9dDu7tryZeeMEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABqG93oHXiuDjfWo78bjqJ8tjaK+qupgcxD148Ve1O8fz/pZ+BFmc9n65290Ud8NsvX3ptn6q6rqMPudp1eu5tsA/saZXLoc9XPbZ6N+8bNfi/qqquW/d1/U//tn/yzqf2FpP+rHj/9h1P/j3iejfvqVlag/PLkc9fOvnI76qvy4gNfCYG0t6qfb21Hff9c7sn5nN+qrqibPvxAvcy944wQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAzv9Q68VnorK1E/Pb2Z9cujqK+qmiz0sn4p68crXdT3srzqdrg/y1l/sJbN7f1xfriOBoN4mcRgY/1I189rL/7N5uezfnU5ynv7h1E/uXAx6nl9mM5n16Kt/+An4m1cu3k96l88cTzqb8x/L+r/2WPvifr7v3gz6q++ZSPqly9l96jwzId7Zrq9HfWzD/+tqL/+6ELUL97In41WX9mK+vQzv1a8cQIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGob3egdezWBjPeq7nZ2o743Xov5wLf+qerOsH69k/eAw6ydLXdSf/a+/EvVX/8EHo354kO3P/LW9qK+qquk0XyYxP3+06+c111vJTrTZ8exasfNw1ld2GtTco6ejfuHZy9kGqmpy4WK8DP/v5rYOon552Iu3celbx6P+c6s/FvV/tXs26j/8l7ei/rMvZOfmeCn7jhY+97Wo7975WNRXVdWl/HyD/7+GZ7L7wuTPvhn127/57qi/9YOlqK+qWrz0UNQPn34u6qfb21H/arxxAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAahndrQ4ON9aPdwNlTWT/tonx/c5Ctv6q235b1/cOsH69mn2HuZjYnv/CPPhj1869EeWy6kB+u/UH2u6XH6fTK1ajn3ptcuBj1r3zirVF/7Ylp1PdXx1Ff1+aj/IE/PJOtv6rmj69F/eypZ+Jt/I3z5F9G+fI7H4s3sXH8WNT/YO+hqH/6wXNRP1rKju3ppcWof+Sf/nm2/o++N+oHL96I+qqq4ZnTUT+5dDneBvx13WZ2zX7uP87O/f/2iU9n/bGfi/qqqskXs+tXb3s73sZrwRsnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBheLc2NN26eaTrH66sRP3O+85F/bUfn0V9VdXJh65H/eJoHPXPP38y6vtXszl5tBPlNb/VRf3yv/hq1PeeeFfUV1V1C6Oon4XH6WBjPep57aW/weztb4367YeivH7i3d+L+n947vej/g93svPg8//pWtRXVR184omoX3gk+5Km55+L+jeD4ZnT2QKv5PfME1/P+s1Pfzfqr/6DD0Z9f5xdf/eP9aL+1m98IOo3vnUj6uGNord/GPXj+yZR/0vLu1H/3XN/GfVVVZ85/fGov1dPX944AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANw3u9A6+VyYWLWf+Tb4n6xdO3or6q6p+843eifrV/GPX/+ejvRP2z1x+M+vXnuqhf/e0no773xLuyfn8S9VVVs29/J14mMd26eaTrpy39DfqH2XG0dKUX9dd/8kbU/8hLS1FfK9+O8j/6k1/L1l9VNz4zF/VnfhBv4m+cyaXLR7+RcBvTj7436je/cxD1k+VB1C+8kp1rK7/71ajvnTkd9XfirvzO8Nd0w+xcG9zM+if3p1F/8WAz6quq1v959gx5r3jjBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0DO/1DryawcZ61PdWVqJ+9XeejPpLP/dE1FdVPbn3cNT/J5vPR/3HTnw36r8zeiDqJ/O9qJ9+5L1R35t2UT93+UbUV1XN4iV4s+v98KWoP3ZiKerP/5MPRP2v/+B41D+y/HLUdx+7GPVVVaO/99Z4mUR6fedoDL74jaifP3M66rvlxajv3d7L1v/IQ1E/Of9c1MMbRW8yjfqNZ7Lnu3/84i9E/befPxv1VVUP/my2TwtPvRD105eze+er8cYJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgY3usdeDXTrZvZAmHf+1vvjPqTXxxFfVXVf7/y4ai/+va1qP83W/dH/fB2L+o3P/2VqE+/08GNW1Hf7exEfVXVYGM96uPjjjec9Dde+O7lqH/b5GTUP3Phsaj/q0HWj34zyquq6vgz+1Hfm0yjvotqXi8ml7JzITU8czrqe7f3jmhP4I1l8oPno37lHfdF/XOfe1vUH9vKr/KL5y9G/eTll+NtvBa8cQIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADcN7vQP3SvfNp6N+c/CufCO91Sj/V6c+FPXTuSiv489OswVCg5e3or4bj6N+unUz6u+Gwcb6vd4FjtjkwsWoH+3sRP1bLh6L+smp7Jjr72bnWVVV//Z+1E+efyHeBvx1k0uX7/UuwBvSYG0t6pe/83LUj25vRn3NsrzqjXMf8cYJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgY3usdeKPovv7teJnjtx6J+tUza1Hfm8yivhtmc/LgsWz/u63tqJ9euRr1g431qK+qmm7djJd5Pa2fN570mBiE6x/eup0tcHAQbsFxDfBGMt3Onr/S+07/B89H/fDM6XALVZN4iXvDGycAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoGF4r3fgzWz63fNRP7d1MtvAwUGUT7duZus/dbT7k4r3H94AHNcA3E3T7e0jXX93e/dI138veeMEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQM7/UOvFEMNtaPfBvTK1ePdP3xZzg4OJodAQDgTWm6vX2vd+HIeOMEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADT0uq671/sAAADwuuaNEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACAhuHd2tDHB7+e/afdXjbT9edG2eoXF7N+eSnqq6q6xflsgfAzdKNB1g96UV+zLO8fTrIFDg6jvLd3kK2/qrr9/aw/HGcbGGf953f/p+hHeKDXi86bRwd/N9qf3iA7hqqfHkP5P9i+tPr5qN9YX4/6xfnsvBz0s2vRYXhM3NrdjfqNKx+K+hfmPxf1VVVrGxtRP5lO420kvvHSS+GBV/Vz6/9edPD1lo72njDdXI768fpC1FdVTReyYzW9J/TH2fk8upld44fXd6K+bmxHeXfrVtZPwnta5efb0upq1I8Ps+/0W9euxefOx/u/lp07w+xR8tLaF6I+tbKyEi8zP8qev+bCPnb+3VHeXwqfUe/CvTzVdeE2ZuFDarj+z+98+v/xS/LGCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGN6tDV3e/JOoHx8cRP1bD38p6ntzo6jvlhaivqpqtrYY9dPlbJ+m84Oonw17Ud+bdlE/3J9G/WAnO/z6vWz/q6p6s1m2wCz7zN00+8ypR/u/FvW9fvgdpX2oN8j/NnPm1s9F/eHm16J+0M/2aTTIzrPD8Tjqj936RNTXUpa/dfdT2QJVtTeXfafz4foPDg/DJXK98HrRC4+LWXhPONzM+t3T2f2gqmr/WPYZpnPZ+of7Wb90NTt3VifZ9bp/YzvqD849GfXzFz8Q9XdibXk56nfD69Ed6Wfb6ML75tn9X4j6lxZ+P+pXl8KLZFXthc+c6X1kfpSdz99f+0LUn+v/atTHJpOsD4+JqqrefHhB6sLntcPs3vxqvHECAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABqGd2tDi4uLUb+yshL10/mvR33/xkejvluaj/qqqsnKXNSPV7Of43Alm3snC72oH4y7qB/dHkT9fD/bn9FkFvVVVb3xJFtgPM76QfaZU71w/d0s+816vbBfyM6D7vAw6u/Ewss/FfUXhp+N+vtPn4760ZUPRX0tLmT9JDumDx/Iro1VVYuj7He+tbsb9ZPpNOrvii47F9I/O04Xs3N591T+d82dB7PvtX/8IOqn+9k9au/CKOr746WoX3s5e65Y2PlE1PeO549I55bORv3cKPuOprP8PnjU+uF9Ib1vnpv8ctTf3P3jqK+q2jj8VNR3N/az9T/+nah/9IEHon73WnYuXBn/TtS/5x2PZOvf2or6qqppeO/sDdOL8Gtz3/HGCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGN6tDW1c+VDUTx/6Rrb+lZWo33lpP+q70SDqq6omi9ky0/le1B9sZP1kKeuHu1Fe1WX54DD7fgZ7o2wDVdXbDQ/xQf47H6VrJ/406ndv3TqiPfm/PDj8d6L+xtoX4m0cX1+P+u76T0f9uemvRv386KtRf3B/1qf7P3f2yahfnduI+qqq3YODqF958f1Z3zv6v9l1k0m2wHgc5f2bt6N+dGsp6rtBfnteect21H/qgaejfn24F/X/61sej/rLs3NRv/Dysagf7KW/cXoTrBqE95Cl+fl4G0etv5DtU28YHqujrL/YfSbq37p2Nuqrqi5d+72oP7X+G1H/5FNPRf2jb3tb1HeHh1F/cvKLUf/I6RtRP5ceE1X1w1sL2QKTadbPwofUV+GNEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGN6tDfUWF6N+9PKHo/7rz/4PUf/4W38q6sejfMY8XBtk/Wov6vePZf1svov6qmz9vWm29v40+35G21lfVTUYZYd4r5d95kr70HJ43pzY2Ij6S1evRv1s+1bUb04+HvVVVdPVJ6N+4eSXon7/2R+N+r3z7476bjKJ+sP7/yjqN+Y3o/7GM49HfVVVb24u6gfhtajb24/6OzKbZX16Lg+y69Hw1kG2+v2FqK+qOr22HfW/vvEXUf+e+fmo/4nl70X9f7T970b9zrOrUb/+vfAmdQfX9/Xl5agfhcfR/ngc9XdD12XPFjcW/iDqz1z7ZNSvL52P+qqq0alTUb/XZb/zQ73/MOpfvvmZqL+x8z9G/YNnfzPq/+wvss87W30s6quq+tPs+aK7fTvqe/d/NepfjTdOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAEDD8G5tqDfMNtVbXor6xzd/K+pny4tRP14dRX1V1XS+F/UHG1k/We2iPqurpnNZ31/I9r/bCffoTsb8LtxGL/wM6fpDg/7R/m1jcTE7D/qT7LysUX6J6W99LOpvr34hW/94EvWD+05EffqZFye/GPVbz+9GfX9zIeqrqmo4iPLu5q2sn06j/o6Msmt2es+ZHl+N+sPN+ajvsp+gqqpe2l6L+hcmm1H/6Ggr6u/rZ+fa/Rs3o/7qsfWoX53LvtTeYngTrKr58FknNZvNjnT9//dGjnT1q0vZuTY4lh2nNy7+7aivqpptZcded3A+6tff8/2or2u/EuUPvfPLUX9x81jU33gsOxeWL+fH0Po3DqL+0cd+GPWT2amofzXeOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADcO7tqV+72jX38vW3+u6qJ8u5DPmdC5eJNKbZn03CvtB1qf6k6zvTbLfrKqqZuEy4XFx1A7G46jfvn076k/u/nzU39z446h/4OTJqK+q+uHVq1G//vLPRn3v7ErUz06sR/3OQ6tR34WXlsVrh1F/bOdfZhuoqvEkOzlnq7Oo3w+P6zuSbmOSXVB7B9n6R1tRXhvn83vOxfuzY/W/W/2ZqH/y2A+j/tz8jajfPpyP+mmW1/592U15aT8/Tq99951Rv/zQN+NtHLUuPP+3Tn0p6jcG2TVya/Svov7hM2eivqrq2v6PxcsklhYWov5muP7xNLt+XXt3di7svWc36m9fWIz6qqrli9n169hqdhxNZ9l96tV44wQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANAzv1oa6vf1sgfEk6+dGUT5bX4763RODqK+qun2uF/WzURf10/ms74+z/Ul14Rjem2b7fzd001m2wHR6NDtyh25evx71Jxez9a/v/VtRf/nGF7INVNXajY9F/dyDfxH1461s/S+/bz3qX3lvdgwNNg+ivntpIer3vvJ3o76qau0Lz0T9xqPfirdx1HbP/HnU397bi/rj3/6pqD/3gQtRf/lr7476qqrjGw9H/Qt790f99+8/GfX9uez62F3Nju217NSpua3suaK/Hz6HVFVvMbuoPnfpUtQvh+u/E5fWsuv2mWufjPoT585Hff/Kh6L+5avZuVxVtfIjX4/63RsfifqXvpc9c/aWsue1xbm5qL/9eHby/FdP/Muo/+373x/1VVUvfO+RqB9czx4654avzcjjjRMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA0GJwAAgAaDEwAAQIPBCQAAoMHgBAAA0DC8Wxs6OPPnUb84Px/13fWfjvrdB5aj/pV3z6K+quqhxy9F/cEk+zkuXjgW9fPfn4v6uVtRXnPbXbj+adT3xllfVVXTcJnJJMpn+wfZ+kPDwSDqH3zwwagf15eifvcg+7zzNYr6qqr+2mrUb+/uRv3wLcej/taDUV4//b6/ivr/7PQfRf0Xbv9I1P/T1Y9HfVVVb5Zto/fkUtSPTvzvUX8n5obZ9fT4qVNRf2v1m1F/5aVPRX13OrtHVVUd/8YrUd+fbkb9wcXsHjJd6EX9ZCHKa34ruy/Pv5B9P9Vl97Q7Wea+rZ/JVn/jDvbpiB3e/9Wov/7Mh6O+f2It6ntLi1FfVdXvZ+8RusXsGfWnnngx6v/k4o9F/an19ahfXN2P+l9fuRn1uyefivqqqv/mxNujfnw1e75L7wmvxhsnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKBheLc2tLy4GPWjwSDqDxYXon5/PVv/4rntqK+q+ocPfS7qTw12ov4frfxi1D91+dGoX7kY5bV4fRL1o62DqO/vjqO+qqo3mUb97PAw6m+e+XLUpwb97G8bc6NR1Kfn2e5B9pvdiaWTX4r67vmPZv1i9hkWry5H/d40+w3eOZddG+8bPB31/+KB90Z9VdXO2dNRv3LfRtRfvv5K1N+Jw0l2PUqdOXYs6q89vx/1K5v/OuqrqlbXs2Np/1p2D1m82kX9dCG7vkyWsuvd0kvZd9oNwr8Vb93K+qqq8BrZW5jP1r9/9NfgjfX1qJ/OZlHfhffZ2st+53Nv/1a2/qraPciO1f3D3436P/3Ob0T9cD975pytZefm7s3sWvH04V7Uv3S4GfVVVb3sI1TXhdej8Dh9Nd44AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANw7u1odlsFvXjdAOTaZTPb2f78/K1paivqro+XYn6jyxm+/TTx56N+q+vPxL1/Ukv6oe72W/Qm2aftzeeRH1VVYXLHJz+ctTPz+aiPrW7vx/127dvR/2J6x+N+vWN9ajvjQdRX1V1+2Z49g+zv//0LlyK+mPPLEf9X3zt0aj/+4Ps8x4bZb/x5evZb1ZVtbHfxcskNm987EjXX1W1dPXDUb9/PTtWdw8Oov7ke74R9dfOPxH1VVVbx/511C9evBL13W52PerOnoj63uXrWd/L7lHdODvXxuH3WVV1OMnuOUtXPpRtoP/6+3v32nJ2jZwMsnPtwuFvR3331C9HfVVV12XXvMWV7DNvHv9C1L/S/WzUP/21t0T9yruzZ5ffevBXo/7FGxtRX1W1+mL4TBie/+Np9oz6al5/ZyAAAMDrjMEJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANAwvFsbOhyPo3734CDqNyaTqF/5/s2oP/HkZtRXVf0Xm78c9X/28DNR//TWmaifu5HNyaPb06jvH4T9XnZM1CvZb1ZVtbX4B1E/nA3ibRyl9Us/GfX95aWo7y1nx8Tsxo2oH73j6aivqlpdWIj69Fox7bqoX/zOStQ/2J2O+m8+96NRf7AR5bV6Leurqtafy77T/o2dqH+u93tRX/XpsK/qLcyHfXbc9UfZ7bPfz861dH+qquamvxT1x09/Kepn4blzkN73lz4a9amFU1+O+pNLJ+JtPHf5ctRvn/jTqF/f+pmovxMvXbiQLXD//VG+9rZvRv1DvXNRf3g5O/erquZPZ8fG+JWPRP2pjfDCXf9bVJ9fuhj1933jbVF/of9A1A/3o7yqqjaevRX1h/dl15fpbBb1r8YbJwAAgAaDEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQMLxbG9o9OIj61Zd+Iuq7zVnU96/eiPoT/0eUV1XV4HAj6j//yPujvjeN8tr8QfYdLb1wO+r7F65G/cLZr0T94P58zp/tLUX9pavZZxjNzUV9qtfvZQvMst+4etn6pw99I+oHs1HUV1Vt/fCJeJlIF35Hk50oH37x30T92RceiPrZxnLU9/bGUV9V1d+6FfUXdv951J8+fTrq70h/cKSr7y0uRv2Vbz8W9cMHvhj1VVXTF3486q+98s5sA6Pwehfe92dn/yDqB/s/H/V7598d9Svv+m7UV1UN+tl9aunSB6O+G3VRfyfeNv23o75/ZT7qp3PZfWG8tx/1x37061FfVXX5xm7UZ2d/1TPPvyfqe4fZdfv4XHYv7168GfVvMK3RAAAEwklEQVSnJ2tR3x+H99mqGryYPX+9spjdm18r3jgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABoMTgAAAA3Du7Wh6XSaLTAYZP1kEq5/Luu//2LWV9Xxnf2oX3tuPeq7QTb3DvbGWX/zdtRXuD/7P3xf1HeH2f5XVfXC/vTeXtT/YPjZcAuhXvadznZ3j3T90/tmUb93cBD1VVULXbaNCo+L8akvRf3q0lLU7194f9T3wv0fXL4R9d1OeB5X1TT8jk4uHs/WPwt/4zsR3xPCvyOm96jwHji7g+9o9e1/GfW3vvdjUd+bz+6bXXj+n1zP7oFX6w+jvr/w4ai/9szjUV9VtZAeF3PZcdGlz1J3oDcKHw3Tzxz2g82NqN86nx3XVVVzp/88W2D+j7P+pQ9E+XQve37cOvGnUX9s5xejfu4uHHfdNLvmXb6R3QtHw9dm5PHGCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoGN6tDU2m06h/YfjZqH/g8FeivsL96SaTbP1Vtdf7X6J+/qmfjPr+6krU12iU9alhdjj1Fhez9Q8GWV9V3f5B1Pfm5qL+4dmvRn2qC4/T6mV/C3lx8fejfnRjPuoX09+4ql68+c+i/uGHH476jcXsvOn3s+/0+91nov7c9t+J+hpk+3Mn165LV69G/cpK9p1O0+P6DtzJ5470e1HejbP9SY+7OzE+9aWoPwy/08Wz2fVieze7R13b2or6k+lxN7qDR6T02aLrsvWn/Z2YHfE2wvV3B4fZ+u/gWSG95j36wANRv/wjz0T9MPwMJw7PRf23zv/PUX/uRvaM3buD5830mv3cxYtRf3xzM+pfjTdOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAEDD8G5taHx4GPVvuf3JqO+GB1k/GET9C8PPRn1V1bn+2ag/OP3lqJ/74RNR319djfoahodHv5f10+nR9neg18s+Q1fdEe3JnXl2+jtRf3z1TNQPw/NmEPZVVWfvvz9eJjGeTI50/WdOnoz6i1d/L+rP7vx81M8OsmtjVdXo1FzU7+zsxNs4cuNxlHezWbb+Ljv3u3F2D7y9txf1VVWz8DPMjUZH2qdu7+8f6fq7w+yY6A3z63sXHhfpfa27C/fB9F7ehc93lZ5r/fBv/HfwHS1uLmabCD/DJNynvfQ7DZ08fjzquxd3s/5OrhXhNXt3eivqR3PZfe3VeOMEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADT0uq671/sAAADwuuaNEwAAQIPBCQAAoMHgBAAA0GBwAgAAaDA4AQAANBicAAAAGgxOAAAADQYnAACABoMTAABAg8EJAACgweAEAADQYHACAABoMDgBAAA0GJwAAAAaDE4AAAANBicAAIAGgxMAAECDwQkAAKDB4AQAANBgcAIAAGgwOAEAADQYnAAAABr+T7j11JI0Drv8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 8 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "weighted_pca(d=64, missing_frac=0.1) # using weights with 10% missing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With 10% of the data missing (and assigned zero weight), the reconstructed quality is barely degraded. Note how this allows us to make a sensible guess at the missing data! (statisticians call this [imputation](https://en.wikipedia.org/wiki/Imputation_(statistics))."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Non-linear Dimensionality Reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The methods above find latent variables that are linear functions of the original features. \n",
    "\n",
    "There are also non-linear methods:\n",
    " - When they work, the results are spectacular.\n",
    " - However, they are generally [very sensitive to your choice of hyperparameters](https://github.com/scikit-learn/scikit-learn/issues/10530).\n",
    " - I recommend that you always start with a linear method and only prefer a non-linear model if it has clearly better performance and gives consistent and robust results.\n",
    " \n",
    "For an overview of non-linear methods, see [here](https://github.com/dkirkby/MachineLearningStatistics/blob/master/notebooks/Nonlinear.ipynb). One key idea is the \"kernel trick\", which is also central to the power of neural networks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Model-Driven Compression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most dimensionality reduction methods assume that variance is a good proxy for \"information\". However, when you have a good generative model of your data, you can do much better. In particular, there is an optimal compression algorithm for data generated by a model with $n$ parameters that will reduce your whole dataset down to $n$ numbers! See this [2018 paper](https://doi.org/10.1093/mnras/sty819) for details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Acknowledgement:\n",
    "**H2020-Astronomy ESFRI and Research Infrastructure Cluster (Grant Agreement number: 653477).**"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}