reading_assignments/4_Note-Unsupervised.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "$\\newcommand{\\xv}{\\mathbf{x}}\n",
    " \\newcommand{\\wv}{\\mathbf{w}}\n",
    " \\newcommand{\\yv}{\\mathbf{y}}\n",
    " \\newcommand{\\zv}{\\mathbf{z}}\n",
    " \\newcommand{\\uv}{\\mathbf{u}}\n",
    " \\newcommand{\\vv}{\\mathbf{v}}\n",
    " \\newcommand{\\Chi}{\\mathcal{X}}\n",
    " \\newcommand{\\R}{\\rm I\\!R}\n",
    " \\newcommand{\\sign}{\\text{sign}}\n",
    " \\newcommand{\\Tm}{\\mathbf{T}}\n",
    " \\newcommand{\\Xm}{\\mathbf{X}}\n",
    " \\newcommand{\\Zm}{\\mathbf{Z}}\n",
    " \\newcommand{\\I}{\\mathbf{I}}\n",
    " \\newcommand{\\Um}{\\mathbf{U}}\n",
    " \\newcommand{\\Vm}{\\mathbf{V}} \n",
    " \\newcommand{\\muv}{\\boldsymbol\\mu}\n",
    " \\newcommand{\\Sigmav}{\\boldsymbol\\Sigma}\n",
    " \\newcommand{\\Lambdav}{\\boldsymbol\\Lambda}\n",
    "$\n",
    "\n",
    "In this note, we discuss the two subtopics of unsupervised learing, clustering and dimensionality reduction.\n",
    "\n",
    "\n",
    "# Clustering\n",
    "\n",
    "\n",
    "When there is no information available for classification target, often we still want to find different groups of data. We call these as *clusters*, and we call this approach as *clustering*. Let us talk about two clustering techniques, k-means and Gaussian mixture models. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## K-means\n",
    "\n",
    "First and easy clustering approach is K-means algorithm, which is non-probabilistic. \n",
    "Without knowing about actual groups, we want to find K partitions of N observations in D dimensions. \n",
    "That is, the observation data $\\Xm = {\\xv_1, \\cdots, \\xv_N}$ and $\\xv_i \\in \\R^D$.\n",
    "When we devide data into K clusters, an intuitive way is to find groups whose inter-point distances inside a cluster are smaller than the out-side-cluter-point distances.  \n",
    "\n",
    "We can formulate this by selecting a center of each cluster, named $\\muv_k$:\n",
    "\n",
    "$$\n",
    "E = \\sum_{i=1}^{N} \\sum_{k=1}^{K} \\delta_{ik} \\Vert \\xv_i - \\muv_k \\Vert^2.\n",
    "$$\n",
    "\n",
    "where $\\delta_{ik} \\in {0, 1}$ is a binary indicator variable that has 1 if $\\xv_i$ is assigned to cluster $k$, otherwise 0. \n",
    "\n",
    "Thus, our goal is finding the $\\delta_{ik}$ and $\\muv_k$ that minimizes the $E$. \n",
    "\n",
    "For this goal, \n",
    "\n",
    "1. we first choose an initial $\\muv_k$ randomly,\n",
    "\n",
    "2. minimize $E$ w.r.t. the $\\delta_{ik}$.\n",
    "\n",
    "3. Now, fixing the $\\delta_{ik}$, update $\\muv_k$ that minimizes $E$. \n",
    "\n",
    "4. Repeating 2 and 3, we obtain the $K$ clusters after convergence. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import pairwise_distances_argmin\n",
    "# K-means: Sketch. \n",
    "\n",
    "def kmeans(X, K=2, maxiter=100):\n",
    "    N = X.shape[0]\n",
    "    \n",
    "    # select initial centers\n",
    "    cidx = np.random.choice(N, K, replace=False)\n",
    "    centers = X[cidx, :]\n",
    "    \n",
    "    E = []\n",
    "    classes = []\n",
    "    \n",
    "    # repeat until convergence or up to maxiter\n",
    "    for step in range(maxiter):\n",
    "        #### TODO: finish this for-loop for k-means!\n",
    "        # Assignment: find clusters that minimizes E with current centers\n",
    "        classes = pairwise_distances_argmin(X, centers)\n",
    "        new_centers = np.array([X[classes == i].mean(0) for i in range(K)])\n",
    "        # print(new_centers)\n",
    "        \n",
    "        # compute E and check convergence\n",
    "        E.append(np.linalg.norm(new_centers - centers))\n",
    "        if np.all(centers == new_centers):\n",
    "            break\n",
    "        \n",
    "        # Update: update cluster centers\n",
    "        centers = new_centers\n",
    "            \n",
    "    return centers, classes, E"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hint and Sample Data Practice\n",
    "\n",
    "Here follows some hints to fill in the blanks. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.70927282, 0.31832524],\n",
       "       [0.57779857, 0.56855147],\n",
       "       [0.87739271, 0.27913218],\n",
       "       [0.82521838, 0.43965491],\n",
       "       [0.12078108, 0.8725917 ],\n",
       "       [0.89902624, 0.71040118],\n",
       "       [0.90048783, 0.19736553],\n",
       "       [0.74468361, 0.9112552 ],\n",
       "       [0.30831214, 0.16429025],\n",
       "       [0.68711384, 0.39313944],\n",
       "       [0.38432047, 0.41342159],\n",
       "       [0.82943824, 0.30707913],\n",
       "       [0.51877973, 0.27752902],\n",
       "       [0.92174925, 0.26527966],\n",
       "       [0.30696058, 0.29709969],\n",
       "       [0.82008463, 0.29757764],\n",
       "       [0.30065385, 0.14609689],\n",
       "       [0.75158671, 0.2817105 ],\n",
       "       [0.68699054, 0.39486834],\n",
       "       [0.59210228, 0.9779764 ],\n",
       "       [0.8209551 , 0.31053766],\n",
       "       [0.48263587, 0.01051243],\n",
       "       [0.96480441, 0.18420852],\n",
       "       [0.63918616, 0.53724871],\n",
       "       [0.92473093, 0.30437004],\n",
       "       [0.28355544, 0.84481468],\n",
       "       [0.29232161, 0.26997459],\n",
       "       [0.81083568, 0.52777983],\n",
       "       [0.5577938 , 0.1367492 ],\n",
       "       [0.41037732, 0.26623261],\n",
       "       [0.02822828, 0.22624883],\n",
       "       [0.81226555, 0.71298094],\n",
       "       [0.39825516, 0.39390867],\n",
       "       [0.29258756, 0.57139359],\n",
       "       [0.3326682 , 0.71795073],\n",
       "       [0.41950556, 0.18040067],\n",
       "       [0.14319777, 0.40593188],\n",
       "       [0.1491473 , 0.06544729],\n",
       "       [0.76213957, 0.816357  ],\n",
       "       [0.24457425, 0.17326279],\n",
       "       [0.77890045, 0.88093008],\n",
       "       [0.17846856, 0.20041891],\n",
       "       [0.17744428, 0.05086838],\n",
       "       [0.18199995, 0.89104003],\n",
       "       [0.17601107, 0.26715891],\n",
       "       [0.95673683, 0.36321704],\n",
       "       [0.85309333, 0.31741938],\n",
       "       [0.47972642, 0.37368045],\n",
       "       [0.74928187, 0.05695936],\n",
       "       [0.62087824, 0.21432981],\n",
       "       [0.05515765, 0.71415629],\n",
       "       [0.43297869, 0.70129851],\n",
       "       [0.9655006 , 0.17183757],\n",
       "       [0.36044758, 0.93963938],\n",
       "       [0.29024029, 0.57118315],\n",
       "       [0.00288033, 0.39807743],\n",
       "       [0.35349155, 0.21997848],\n",
       "       [0.18231788, 0.43291974],\n",
       "       [0.42454848, 0.18211916],\n",
       "       [0.83593024, 0.9423098 ],\n",
       "       [0.19187005, 0.46868053],\n",
       "       [0.1811219 , 0.16063729],\n",
       "       [0.92350698, 0.86198317],\n",
       "       [0.93366782, 0.67317259],\n",
       "       [0.38082371, 0.9927587 ],\n",
       "       [0.46921014, 0.75929878],\n",
       "       [0.39308995, 0.71146706],\n",
       "       [0.73765296, 0.33042061],\n",
       "       [0.97894214, 0.14025818],\n",
       "       [0.48267114, 0.27099531],\n",
       "       [0.86943124, 0.63462268],\n",
       "       [0.55823324, 0.12061148],\n",
       "       [0.57910048, 0.71764507],\n",
       "       [0.40412442, 0.40345183],\n",
       "       [0.45984057, 0.94712086],\n",
       "       [0.79274054, 0.46372402],\n",
       "       [0.19184363, 0.40913747],\n",
       "       [0.54617717, 0.86309895],\n",
       "       [0.9521329 , 0.368326  ],\n",
       "       [0.39823397, 0.18927255],\n",
       "       [0.48540609, 0.23770222],\n",
       "       [0.14830469, 0.99364082],\n",
       "       [0.46427422, 0.72553205],\n",
       "       [0.608116  , 0.27848508],\n",
       "       [0.49543854, 0.63476373],\n",
       "       [0.78271708, 0.31046256],\n",
       "       [0.61989883, 0.56608542],\n",
       "       [0.98107033, 0.41782416],\n",
       "       [0.77794754, 0.95458621],\n",
       "       [0.22124813, 0.04728764],\n",
       "       [0.54737344, 0.5116328 ],\n",
       "       [0.13705478, 0.49404279],\n",
       "       [0.17777574, 0.4690663 ],\n",
       "       [0.88652592, 0.18850814],\n",
       "       [0.93606791, 0.52212998],\n",
       "       [0.26711852, 0.99079727],\n",
       "       [0.35350564, 0.95360747],\n",
       "       [0.80473274, 0.4788397 ],\n",
       "       [0.00872364, 0.23767149],\n",
       "       [0.81226516, 0.26552186]])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.random.rand(100,2)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.57779857, 0.56855147],\n",
       "       [0.12078108, 0.8725917 ]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c = X[[1,4], :]\n",
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[0.57779857, 0.56855147]],\n",
       "\n",
       "       [[0.12078108, 0.8725917 ]]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "C = c[:, np.newaxis, :]\n",
    "C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 1, 2)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "C.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[ 0.13147425, -0.25022623],\n",
       "        [ 0.        ,  0.        ],\n",
       "        [ 0.29959414, -0.28941929],\n",
       "        [ 0.24741981, -0.12889657],\n",
       "        [-0.45701749,  0.30404022],\n",
       "        [ 0.32122766,  0.1418497 ],\n",
       "        [ 0.32268925, -0.37118595],\n",
       "        [ 0.16688503,  0.34270372],\n",
       "        [-0.26948643, -0.40426122],\n",
       "        [ 0.10931527, -0.17541204],\n",
       "        [-0.1934781 , -0.15512989],\n",
       "        [ 0.25163966, -0.26147234],\n",
       "        [-0.05901884, -0.29102246],\n",
       "        [ 0.34395068, -0.30327182],\n",
       "        [-0.270838  , -0.27145179],\n",
       "        [ 0.24228606, -0.27097384],\n",
       "        [-0.27714473, -0.42245459],\n",
       "        [ 0.17378813, -0.28684097],\n",
       "        [ 0.10919197, -0.17368314],\n",
       "        [ 0.01430371,  0.40942493],\n",
       "        [ 0.24315653, -0.25801381],\n",
       "        [-0.09516271, -0.55803904],\n",
       "        [ 0.38700584, -0.38434295],\n",
       "        [ 0.06138759, -0.03130277],\n",
       "        [ 0.34693235, -0.26418144],\n",
       "        [-0.29424313,  0.27626321],\n",
       "        [-0.28547697, -0.29857689],\n",
       "        [ 0.2330371 , -0.04077164],\n",
       "        [-0.02000478, -0.43180227],\n",
       "        [-0.16742125, -0.30231886],\n",
       "        [-0.54957029, -0.34230264],\n",
       "        [ 0.23446698,  0.14442947],\n",
       "        [-0.17954341, -0.17464281],\n",
       "        [-0.28521101,  0.00284211],\n",
       "        [-0.24513038,  0.14939926],\n",
       "        [-0.15829301, -0.3881508 ],\n",
       "        [-0.4346008 , -0.16261959],\n",
       "        [-0.42865127, -0.50310419],\n",
       "        [ 0.184341  ,  0.24780553],\n",
       "        [-0.33322433, -0.39528868],\n",
       "        [ 0.20110188,  0.31237861],\n",
       "        [-0.39933002, -0.36813256],\n",
       "        [-0.40035429, -0.5176831 ],\n",
       "        [-0.39579862,  0.32248856],\n",
       "        [-0.40178751, -0.30139257],\n",
       "        [ 0.37893826, -0.20533444],\n",
       "        [ 0.27529475, -0.25113209],\n",
       "        [-0.09807215, -0.19487103],\n",
       "        [ 0.1714833 , -0.51159211],\n",
       "        [ 0.04307967, -0.35422166],\n",
       "        [-0.52264092,  0.14560482],\n",
       "        [-0.14481988,  0.13274704],\n",
       "        [ 0.38770203, -0.39671391],\n",
       "        [-0.217351  ,  0.3710879 ],\n",
       "        [-0.28755829,  0.00263168],\n",
       "        [-0.57491825, -0.17047405],\n",
       "        [-0.22430703, -0.34857299],\n",
       "        [-0.39548069, -0.13563173],\n",
       "        [-0.1532501 , -0.38643231],\n",
       "        [ 0.25813167,  0.37375833],\n",
       "        [-0.38592853, -0.09987094],\n",
       "        [-0.39667668, -0.40791418],\n",
       "        [ 0.3457084 ,  0.2934317 ],\n",
       "        [ 0.35586925,  0.10462112],\n",
       "        [-0.19697486,  0.42420722],\n",
       "        [-0.10858844,  0.19074731],\n",
       "        [-0.18470862,  0.14291559],\n",
       "        [ 0.15985439, -0.23813087],\n",
       "        [ 0.40114357, -0.4282933 ],\n",
       "        [-0.09512743, -0.29755616],\n",
       "        [ 0.29163267,  0.06607121],\n",
       "        [-0.01956534, -0.44794   ],\n",
       "        [ 0.00130191,  0.14909359],\n",
       "        [-0.17367416, -0.16509964],\n",
       "        [-0.117958  ,  0.37856939],\n",
       "        [ 0.21494197, -0.10482745],\n",
       "        [-0.38595495, -0.159414  ],\n",
       "        [-0.0316214 ,  0.29454748],\n",
       "        [ 0.37433433, -0.20022548],\n",
       "        [-0.17956461, -0.37927893],\n",
       "        [-0.09239248, -0.33084926],\n",
       "        [-0.42949388,  0.42508935],\n",
       "        [-0.11352436,  0.15698058],\n",
       "        [ 0.03031743, -0.2900664 ],\n",
       "        [-0.08236003,  0.06621226],\n",
       "        [ 0.2049185 , -0.25808892],\n",
       "        [ 0.04210025, -0.00246606],\n",
       "        [ 0.40327175, -0.15072731],\n",
       "        [ 0.20014897,  0.38603474],\n",
       "        [-0.35655044, -0.52126383],\n",
       "        [-0.03042514, -0.05691867],\n",
       "        [-0.4407438 , -0.07450869],\n",
       "        [-0.40002284, -0.09948518],\n",
       "        [ 0.30872735, -0.38004334],\n",
       "        [ 0.35826934, -0.04642149],\n",
       "        [-0.31068005,  0.4222458 ],\n",
       "        [-0.22429294,  0.385056  ],\n",
       "        [ 0.22693417, -0.08971177],\n",
       "        [-0.56907494, -0.33087998],\n",
       "        [ 0.23446658, -0.30302962]],\n",
       "\n",
       "       [[ 0.58849174, -0.55426645],\n",
       "        [ 0.45701749, -0.30404022],\n",
       "        [ 0.75661163, -0.59345951],\n",
       "        [ 0.7044373 , -0.43293679],\n",
       "        [ 0.        ,  0.        ],\n",
       "        [ 0.77824516, -0.16219052],\n",
       "        [ 0.77970675, -0.67522617],\n",
       "        [ 0.62390253,  0.0386635 ],\n",
       "        [ 0.18753106, -0.70830144],\n",
       "        [ 0.56633276, -0.47945226],\n",
       "        [ 0.26353939, -0.45917011],\n",
       "        [ 0.70865716, -0.56551257],\n",
       "        [ 0.39799865, -0.59506268],\n",
       "        [ 0.80096817, -0.60731204],\n",
       "        [ 0.1861795 , -0.57549201],\n",
       "        [ 0.69930355, -0.57501406],\n",
       "        [ 0.17987277, -0.72649481],\n",
       "        [ 0.63080563, -0.59088119],\n",
       "        [ 0.56620946, -0.47772336],\n",
       "        [ 0.4713212 ,  0.1053847 ],\n",
       "        [ 0.70017402, -0.56205404],\n",
       "        [ 0.36185479, -0.86207926],\n",
       "        [ 0.84402333, -0.68838317],\n",
       "        [ 0.51840508, -0.33534299],\n",
       "        [ 0.80394985, -0.56822166],\n",
       "        [ 0.16277436, -0.02777702],\n",
       "        [ 0.17154053, -0.60261711],\n",
       "        [ 0.6900546 , -0.34481186],\n",
       "        [ 0.43701272, -0.73584249],\n",
       "        [ 0.28959624, -0.60635908],\n",
       "        [-0.0925528 , -0.64634286],\n",
       "        [ 0.69148447, -0.15961076],\n",
       "        [ 0.27747408, -0.47868303],\n",
       "        [ 0.17180648, -0.30119811],\n",
       "        [ 0.21188712, -0.15464096],\n",
       "        [ 0.29872448, -0.69219102],\n",
       "        [ 0.02241669, -0.46665981],\n",
       "        [ 0.02836622, -0.80714441],\n",
       "        [ 0.64135849, -0.05623469],\n",
       "        [ 0.12379317, -0.69932891],\n",
       "        [ 0.65811937,  0.00833839],\n",
       "        [ 0.05768748, -0.67217278],\n",
       "        [ 0.0566632 , -0.82172332],\n",
       "        [ 0.06121887,  0.01844834],\n",
       "        [ 0.05522999, -0.60543279],\n",
       "        [ 0.83595575, -0.50937466],\n",
       "        [ 0.73231225, -0.55517231],\n",
       "        [ 0.35894534, -0.49891125],\n",
       "        [ 0.62850079, -0.81563233],\n",
       "        [ 0.50009716, -0.65826188],\n",
       "        [-0.06562343, -0.1584354 ],\n",
       "        [ 0.31219761, -0.17129318],\n",
       "        [ 0.84471952, -0.70075413],\n",
       "        [ 0.2396665 ,  0.06704768],\n",
       "        [ 0.16945921, -0.30140855],\n",
       "        [-0.11790075, -0.47451427],\n",
       "        [ 0.23271047, -0.65261322],\n",
       "        [ 0.0615368 , -0.43967195],\n",
       "        [ 0.3037674 , -0.69047253],\n",
       "        [ 0.71514916,  0.0697181 ],\n",
       "        [ 0.07108897, -0.40391116],\n",
       "        [ 0.06034082, -0.7119544 ],\n",
       "        [ 0.8027259 , -0.01060852],\n",
       "        [ 0.81288674, -0.1994191 ],\n",
       "        [ 0.26004263,  0.120167  ],\n",
       "        [ 0.34842906, -0.11329291],\n",
       "        [ 0.27230887, -0.16112463],\n",
       "        [ 0.61687188, -0.54217109],\n",
       "        [ 0.85816106, -0.73233352],\n",
       "        [ 0.36189006, -0.60159638],\n",
       "        [ 0.74865016, -0.23796901],\n",
       "        [ 0.43745216, -0.75198022],\n",
       "        [ 0.4583194 , -0.15494663],\n",
       "        [ 0.28334334, -0.46913986],\n",
       "        [ 0.33905949,  0.07452916],\n",
       "        [ 0.67195946, -0.40886767],\n",
       "        [ 0.07106255, -0.46345422],\n",
       "        [ 0.42539609, -0.00949274],\n",
       "        [ 0.83135182, -0.5042657 ],\n",
       "        [ 0.27745289, -0.68331915],\n",
       "        [ 0.36462501, -0.63488948],\n",
       "        [ 0.02752361,  0.12104913],\n",
       "        [ 0.34349314, -0.14705964],\n",
       "        [ 0.48733492, -0.59410662],\n",
       "        [ 0.37465746, -0.23782797],\n",
       "        [ 0.661936  , -0.56212914],\n",
       "        [ 0.49911775, -0.30650628],\n",
       "        [ 0.86028925, -0.45476754],\n",
       "        [ 0.65716646,  0.08199452],\n",
       "        [ 0.10046705, -0.82530405],\n",
       "        [ 0.42659236, -0.36095889],\n",
       "        [ 0.0162737 , -0.37854891],\n",
       "        [ 0.05699466, -0.4035254 ],\n",
       "        [ 0.76574484, -0.68408356],\n",
       "        [ 0.81528683, -0.35046171],\n",
       "        [ 0.14633744,  0.11820558],\n",
       "        [ 0.23272456,  0.08101577],\n",
       "        [ 0.68395166, -0.39375199],\n",
       "        [-0.11205744, -0.6349202 ],\n",
       "        [ 0.69148408, -0.60706984]]])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X - C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[1.72854777e-02, 6.26131674e-02],\n",
       "        [0.00000000e+00, 0.00000000e+00],\n",
       "        [8.97566469e-02, 8.37635261e-02],\n",
       "        [6.12165599e-02, 1.66143245e-02],\n",
       "        [2.08864989e-01, 9.24404561e-02],\n",
       "        [1.03187211e-01, 2.01213379e-02],\n",
       "        [1.04128354e-01, 1.37779006e-01],\n",
       "        [2.78506142e-02, 1.17445841e-01],\n",
       "        [7.26229362e-02, 1.63427134e-01],\n",
       "        [1.19498280e-02, 3.07693825e-02],\n",
       "        [3.74337766e-02, 2.40652817e-02],\n",
       "        [6.33225203e-02, 6.83677867e-02],\n",
       "        [3.48322402e-03, 8.46940697e-02],\n",
       "        [1.18302071e-01, 9.19737957e-02],\n",
       "        [7.33532204e-02, 7.36860721e-02],\n",
       "        [5.87025347e-02, 7.34268198e-02],\n",
       "        [7.68091997e-02, 1.78467877e-01],\n",
       "        [3.02023152e-02, 8.22777432e-02],\n",
       "        [1.19228855e-02, 3.01658316e-02],\n",
       "        [2.04596098e-04, 1.67628770e-01],\n",
       "        [5.91250961e-02, 6.65711282e-02],\n",
       "        [9.05594096e-03, 3.11407574e-01],\n",
       "        [1.49773519e-01, 1.47719503e-01],\n",
       "        [3.76843583e-03, 9.79863278e-04],\n",
       "        [1.20362057e-01, 6.97918327e-02],\n",
       "        [8.65790214e-02, 7.63213587e-02],\n",
       "        [8.14970988e-02, 8.91481580e-02],\n",
       "        [5.43062918e-02, 1.66232658e-03],\n",
       "        [4.00191063e-04, 1.86453201e-01],\n",
       "        [2.80298745e-02, 9.13966949e-02],\n",
       "        [3.02027503e-01, 1.17171099e-01],\n",
       "        [5.49747653e-02, 2.08598704e-02],\n",
       "        [3.22358358e-02, 3.05001097e-02],\n",
       "        [8.13453213e-02, 8.07761263e-06],\n",
       "        [6.00889010e-02, 2.23201378e-02],\n",
       "        [2.50566779e-02, 1.50661045e-01],\n",
       "        [1.88877858e-01, 2.64451322e-02],\n",
       "        [1.83741915e-01, 2.53113822e-01],\n",
       "        [3.39816026e-02, 6.14075789e-02],\n",
       "        [1.11038452e-01, 1.56253144e-01],\n",
       "        [4.04419660e-02, 9.75803960e-02],\n",
       "        [1.59464462e-01, 1.35521582e-01],\n",
       "        [1.60283560e-01, 2.67995789e-01],\n",
       "        [1.56656548e-01, 1.03998872e-01],\n",
       "        [1.61433200e-01, 9.08374795e-02],\n",
       "        [1.43594206e-01, 4.21622319e-02],\n",
       "        [7.57872017e-02, 6.30673282e-02],\n",
       "        [9.61814750e-03, 3.79747178e-02],\n",
       "        [2.94065225e-02, 2.61726489e-01],\n",
       "        [1.85585764e-03, 1.25472984e-01],\n",
       "        [2.73153536e-01, 2.12007629e-02],\n",
       "        [2.09727978e-02, 1.76217759e-02],\n",
       "        [1.50312865e-01, 1.57381925e-01],\n",
       "        [4.72414554e-02, 1.37706232e-01],\n",
       "        [8.26897679e-02, 6.92571336e-06],\n",
       "        [3.30530991e-01, 2.90614006e-02],\n",
       "        [5.03136426e-02, 1.21503133e-01],\n",
       "        [1.56404979e-01, 1.83959671e-02],\n",
       "        [2.34855920e-02, 1.49329930e-01],\n",
       "        [6.66319569e-02, 1.39695286e-01],\n",
       "        [1.48940827e-01, 9.97420540e-03],\n",
       "        [1.57352385e-01, 1.66393979e-01],\n",
       "        [1.19514300e-01, 8.61021615e-02],\n",
       "        [1.26642922e-01, 1.09455786e-02],\n",
       "        [3.87990951e-02, 1.79951768e-01],\n",
       "        [1.17914486e-02, 3.63845362e-02],\n",
       "        [3.41172749e-02, 2.04248658e-02],\n",
       "        [2.55534253e-02, 5.67063099e-02],\n",
       "        [1.60916162e-01, 1.83435149e-01],\n",
       "        [9.04922773e-03, 8.85396692e-02],\n",
       "        [8.50496122e-02, 4.36540468e-03],\n",
       "        [3.82802432e-04, 2.00650242e-01],\n",
       "        [1.69497357e-06, 2.22288995e-02],\n",
       "        [3.01627131e-02, 2.72578918e-02],\n",
       "        [1.39140894e-02, 1.43314779e-01],\n",
       "        [4.62000502e-02, 1.09887945e-02],\n",
       "        [1.48961221e-01, 2.54128233e-02],\n",
       "        [9.99912926e-04, 8.67582165e-02],\n",
       "        [1.40126190e-01, 4.00902423e-02],\n",
       "        [3.22434479e-02, 1.43852503e-01],\n",
       "        [8.53637097e-03, 1.09461231e-01],\n",
       "        [1.84464994e-01, 1.80700956e-01],\n",
       "        [1.28877794e-02, 2.46429023e-02],\n",
       "        [9.19146654e-04, 8.41385149e-02],\n",
       "        [6.78317475e-03, 4.38406283e-03],\n",
       "        [4.19915934e-02, 6.66098901e-02],\n",
       "        [1.77243131e-03, 6.08143948e-06],\n",
       "        [1.62628107e-01, 2.27187232e-02],\n",
       "        [4.00596091e-02, 1.49022819e-01],\n",
       "        [1.27128219e-01, 2.71715984e-01],\n",
       "        [9.25688857e-04, 3.23973506e-03],\n",
       "        [1.94255095e-01, 5.55154416e-03],\n",
       "        [1.60018269e-01, 9.89730034e-03],\n",
       "        [9.53125767e-02, 1.44432940e-01],\n",
       "        [1.28356920e-01, 2.15495473e-03],\n",
       "        [9.65220933e-02, 1.78291515e-01],\n",
       "        [5.03073218e-02, 1.48268120e-01],\n",
       "        [5.14991166e-02, 8.04820215e-03],\n",
       "        [3.23846282e-01, 1.09481563e-01],\n",
       "        [5.49745786e-02, 9.18269487e-02]],\n",
       "\n",
       "       [[3.46322528e-01, 3.07211302e-01],\n",
       "        [2.08864989e-01, 9.24404561e-02],\n",
       "        [5.72461158e-01, 3.52194193e-01],\n",
       "        [4.96231907e-01, 1.87434261e-01],\n",
       "        [0.00000000e+00, 0.00000000e+00],\n",
       "        [6.05665522e-01, 2.63057646e-02],\n",
       "        [6.07942610e-01, 4.55930376e-01],\n",
       "        [3.89254362e-01, 1.49486633e-03],\n",
       "        [3.51678994e-02, 5.01690931e-01],\n",
       "        [3.20732797e-01, 2.29874467e-01],\n",
       "        [6.94530096e-02, 2.10837188e-01],\n",
       "        [5.02194965e-01, 3.19804461e-01],\n",
       "        [1.58402924e-01, 3.54099589e-01],\n",
       "        [6.41550016e-01, 3.68827913e-01],\n",
       "        [3.46628049e-02, 3.31191050e-01],\n",
       "        [4.89025459e-01, 3.30641166e-01],\n",
       "        [3.23542119e-02, 5.27794704e-01],\n",
       "        [3.97915738e-01, 3.49140584e-01],\n",
       "        [3.20593151e-01, 2.28219606e-01],\n",
       "        [2.22143676e-01, 1.11059359e-02],\n",
       "        [4.90243657e-01, 3.15904738e-01],\n",
       "        [1.30938885e-01, 7.43180658e-01],\n",
       "        [7.12375383e-01, 4.73871391e-01],\n",
       "        [2.68743827e-01, 1.12454920e-01],\n",
       "        [6.46335354e-01, 3.22875855e-01],\n",
       "        [2.64954922e-02, 7.71562601e-04],\n",
       "        [2.94261520e-02, 3.63147380e-01],\n",
       "        [4.76175347e-01, 1.18895219e-01],\n",
       "        [1.90980115e-01, 5.41464174e-01],\n",
       "        [8.38659846e-02, 3.67671339e-01],\n",
       "        [8.56602017e-03, 4.17759097e-01],\n",
       "        [4.78150778e-01, 2.54755935e-02],\n",
       "        [7.69918671e-02, 2.29137440e-01],\n",
       "        [2.95174669e-02, 9.07202997e-02],\n",
       "        [4.48961505e-02, 2.39138280e-02],\n",
       "        [8.92363150e-02, 4.79128412e-01],\n",
       "        [5.02507988e-04, 2.17771383e-01],\n",
       "        [8.04642373e-04, 6.51482094e-01],\n",
       "        [4.11340710e-01, 3.16234090e-03],\n",
       "        [1.53247479e-02, 4.89060918e-01],\n",
       "        [4.33121109e-01, 6.95287285e-05],\n",
       "        [3.32784500e-03, 4.51816248e-01],\n",
       "        [3.21071825e-03, 6.75229212e-01],\n",
       "        [3.74775037e-03, 3.40341219e-04],\n",
       "        [3.05035142e-03, 3.66548861e-01],\n",
       "        [6.98822023e-01, 2.59462545e-01],\n",
       "        [5.36281227e-01, 3.08216299e-01],\n",
       "        [1.28841756e-01, 2.48912435e-01],\n",
       "        [3.95013248e-01, 6.65256103e-01],\n",
       "        [2.50097169e-01, 4.33308704e-01],\n",
       "        [4.30643475e-03, 2.51017771e-02],\n",
       "        [9.74673492e-02, 2.93413548e-02],\n",
       "        [7.13551075e-01, 4.91056349e-01],\n",
       "        [5.74400299e-02, 4.49539163e-03],\n",
       "        [2.87164228e-02, 9.08471117e-02],\n",
       "        [1.39005878e-02, 2.25163791e-01],\n",
       "        [5.41541607e-02, 4.25904009e-01],\n",
       "        [3.78677766e-03, 1.93311428e-01],\n",
       "        [9.22746313e-02, 4.76752316e-01],\n",
       "        [5.11438319e-01, 4.86061405e-03],\n",
       "        [5.05364123e-03, 1.63144229e-01],\n",
       "        [3.64101418e-03, 5.06879071e-01],\n",
       "        [6.44368864e-01, 1.12540759e-04],\n",
       "        [6.60784854e-01, 3.97679782e-02],\n",
       "        [6.76221714e-02, 1.44401083e-02],\n",
       "        [1.21402807e-01, 1.28352838e-02],\n",
       "        [7.41521215e-02, 2.59611469e-02],\n",
       "        [3.80530917e-01, 2.93949489e-01],\n",
       "        [7.36440406e-01, 5.36312384e-01],\n",
       "        [1.30964418e-01, 3.61918208e-01],\n",
       "        [5.60477061e-01, 5.66292507e-02],\n",
       "        [1.91364388e-01, 5.65474251e-01],\n",
       "        [2.10056676e-01, 2.40084576e-02],\n",
       "        [8.02834455e-02, 2.20092211e-01],\n",
       "        [1.14961341e-01, 5.55459629e-03],\n",
       "        [4.51529519e-01, 1.67172773e-01],\n",
       "        [5.04988551e-03, 2.14789815e-01],\n",
       "        [1.80961836e-01, 9.01121824e-05],\n",
       "        [6.91145852e-01, 2.54283896e-01],\n",
       "        [7.69801042e-02, 4.66925056e-01],\n",
       "        [1.32951398e-01, 4.03084650e-01],\n",
       "        [7.57549210e-04, 1.46528916e-02],\n",
       "        [1.17987535e-01, 2.16265382e-02],\n",
       "        [2.37495329e-01, 3.52962674e-01],\n",
       "        [1.40368214e-01, 5.65621410e-02],\n",
       "        [4.38159264e-01, 3.15989170e-01],\n",
       "        [2.49118524e-01, 9.39460988e-02],\n",
       "        [7.40097586e-01, 2.06813511e-01],\n",
       "        [4.31867757e-01, 6.72310084e-03],\n",
       "        [1.00936278e-02, 6.81126782e-01],\n",
       "        [1.81981040e-01, 1.30291322e-01],\n",
       "        [2.64833178e-04, 1.43299274e-01],\n",
       "        [3.24839095e-03, 1.62832747e-01],\n",
       "        [5.86365165e-01, 4.67970318e-01],\n",
       "        [6.64692620e-01, 1.22823411e-01],\n",
       "        [2.14146473e-02, 1.39725586e-02],\n",
       "        [5.41607187e-02, 6.56355571e-03],\n",
       "        [4.67789875e-01, 1.55040633e-01],\n",
       "        [1.25568703e-02, 4.03123666e-01],\n",
       "        [4.78150227e-01, 3.68533788e-01]]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(X - C)**2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.07989865, 0.        , 0.17352017, 0.07783088, 0.30130544,\n",
       "        0.12330855, 0.24190736, 0.14529646, 0.23605007, 0.04271921,\n",
       "        0.06149906, 0.13169031, 0.08817729, 0.21027587, 0.14703929,\n",
       "        0.13212935, 0.25527708, 0.11248006, 0.04208872, 0.16783337,\n",
       "        0.12569622, 0.32046351, 0.29749302, 0.0047483 , 0.19015389,\n",
       "        0.16290038, 0.17064526, 0.05596862, 0.18685339, 0.11942657,\n",
       "        0.4191986 , 0.07583464, 0.06273595, 0.0813534 , 0.08240904,\n",
       "        0.17571772, 0.21532299, 0.43685574, 0.09538918, 0.2672916 ,\n",
       "        0.13802236, 0.29498604, 0.42827935, 0.26065542, 0.25227068,\n",
       "        0.18575644, 0.13885453, 0.04759287, 0.29113301, 0.12732884,\n",
       "        0.2943543 , 0.03859457, 0.30769479, 0.18494769, 0.08269669,\n",
       "        0.35959239, 0.17181678, 0.17480095, 0.17281552, 0.20632724,\n",
       "        0.15891503, 0.32374636, 0.20561646, 0.1375885 , 0.21875086,\n",
       "        0.04817598, 0.05454214, 0.08225974, 0.34435131, 0.0975889 ,\n",
       "        0.08941502, 0.20103304, 0.02223059, 0.0574206 , 0.15722887,\n",
       "        0.05718884, 0.17437404, 0.08775813, 0.18021643, 0.17609595,\n",
       "        0.1179976 , 0.36516595, 0.03753068, 0.08505766, 0.01116724,\n",
       "        0.10860148, 0.00177851, 0.18534683, 0.18908243, 0.3988442 ,\n",
       "        0.00416542, 0.19980664, 0.16991557, 0.23974552, 0.13051187,\n",
       "        0.27481361, 0.19857544, 0.05954732, 0.43332784, 0.14680153],\n",
       "       [0.65353383, 0.30130544, 0.92465535, 0.68366617, 0.        ,\n",
       "        0.63197129, 1.06387299, 0.39074923, 0.53685883, 0.55060726,\n",
       "        0.2802902 , 0.82199943, 0.51250251, 1.01037793, 0.36585386,\n",
       "        0.81966662, 0.56014892, 0.74705632, 0.54881276, 0.23324961,\n",
       "        0.8061484 , 0.87411954, 1.18624677, 0.38119875, 0.96921121,\n",
       "        0.02726705, 0.39257353, 0.59507057, 0.73244429, 0.45153732,\n",
       "        0.42632512, 0.50362637, 0.30612931, 0.12023777, 0.06880998,\n",
       "        0.56836473, 0.21827389, 0.65228674, 0.41450305, 0.50438567,\n",
       "        0.43319064, 0.45514409, 0.67843993, 0.00408809, 0.36959921,\n",
       "        0.95828457, 0.84449753, 0.37775419, 1.06026935, 0.68340587,\n",
       "        0.02940821, 0.1268087 , 1.20460742, 0.06193542, 0.11956353,\n",
       "        0.23906438, 0.48005817, 0.19709821, 0.56902695, 0.51629893,\n",
       "        0.16819787, 0.51052009, 0.6444814 , 0.70055283, 0.08206228,\n",
       "        0.13423809, 0.10011327, 0.67448041, 1.27275279, 0.49288263,\n",
       "        0.61710631, 0.75683864, 0.23406513, 0.30037566, 0.12051594,\n",
       "        0.61870229, 0.2198397 , 0.18105195, 0.94542975, 0.54390516,\n",
       "        0.53603605, 0.01541044, 0.13961407, 0.590458  , 0.19693035,\n",
       "        0.75414843, 0.34306462, 0.9469111 , 0.43859086, 0.69122041,\n",
       "        0.31227236, 0.14356411, 0.16608114, 1.05433548, 0.78751603,\n",
       "        0.03538721, 0.06072427, 0.62283051, 0.41568054, 0.84668402]])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum((X - C)**2, axis=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1,\n",
       "       0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.argmin(np.sum((X - C)**2, axis=2), axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[0.80654892, 0.46345329],\n",
       "        [0.3133424 , 0.45397584]]),\n",
       " array([0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1,\n",
       "        0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1,\n",
       "        1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1,\n",
       "        1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0,\n",
       "        0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0]),\n",
       " [0.3619402443717892, 0.028146618176779926, 0.0])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kmeans(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x11f59e050>]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAiWklEQVR4nO3de5CU9Z3v8fe3ey4NzAwgDLe5NKi4EW+gLQybjUklJmKyC+YqEkdzKqesnKOVPeWeOms2W8k5pmp3k1TlZC+eSqzdrYoQRDSbhLgxrEk0ySZyaRRRQHRA7rfhzgBz/54/+hlohsbpYXrm6cvnVTU1Tz/P7+n+zgP9eS6/7t9j7o6IiBSvSNgFiIjI8FLQi4gUOQW9iEiRU9CLiBQ5Bb2ISJErC7uA/iZOnOjTp08PuwwRkYKyYcOGI+5em2lZ3gX99OnTSSaTYZchIlJQzGzX5Zbp0o2ISJHLKujNbIGZbTOzFjN77D3afdrM3MwSafO+Eqy3zczuykXRIiKSvQEv3ZhZFHgC+CiwF1hvZqvcfUu/dtXAnwNr0+bNAhYDNwDTgF+a2XXu3pO7P0FERN5LNkf0c4EWd9/h7p3ACmBRhnbfAL4JtKfNWwSscPcOd38XaAmeT0RERkg2QV8H7El7vDeYd56Z3Qo0uPu/D3ZdEREZXkPujDWzCPAd4C+G8BwPmVnSzJKtra1DLUlERNJkE/T7gIa0x/XBvD7VwI3Ay2a2E2gCVgUdsgOtC4C7P+nuCXdP1NZm/BioiIhcoWyCfj0w08xmmFkFqc7VVX0L3f2ku0909+nuPh1YAyx092TQbrGZVZrZDGAmsC7nfwVw8mwX33nxbd4+dHo4nl5EpGANGPTu3g08AqwGtgIr3X2zmT1uZgsHWHczsBLYAvwCeHi4PnHT6873frOdH/xh53A8vYhIwbJ8u/FIIpHwK/1m7F+sfJ0X3jzA2r/6CNWx8hxXJiKSv8xsg7snMi0rqm/GNs+Pc7azhx+/dkk3gIhIySqqoJ/dMI6b68fy1Cu7yLczFRGRsBRV0APc3xSn5XAba3YcC7sUEZG8UHRBv/CWaYwdVc7SNTvDLkVEJC8UXdDHyqN8LlHP6s2HOHSqfeAVRESKXNEFPaQu3/T0OsvX7g67FBGR0BVl0McnjOGD19Xy9LrddPX0hl2OiEioijLoAR6YH+fw6Q5e3HIo7FJEREJVtEH/oT+aRN24UTz1ys6wSxERCVXRBn00Yny+qZE1O47xjsa/EZESVrRBD3BvooGKaISlay57z1wRkaJX1EE/oaqST9w8lX97dR9tHd1hlyMiEoqiDnpIjX/T1tGt8W9EpGQVfdDPaRjHDdNqWKbxb0SkRBV90JsZD8yPs+3Qada9q/FvRKT0FH3QAyy8pY6aWJk6ZUWkJJVE0I+qiPLZRAO/ePMgh09r/BsRKS1ZBb2ZLTCzbWbWYmaPZVj+JTN7w8w2mtl/mtmsYP50MzsXzN9oZt/L9R+Qrc/Pa6S711mxbk9YJYiIhGLAoDezKPAEcDcwC7ivL8jTLHf3m9x9NvAt4Dtpy7a7++zg50s5qnvQrq6t4gMzJ7J87W66Nf6NiJSQbI7o5wIt7r7D3TuBFcCi9Abufirt4RggLz/e0twU5+Cpdn65VePfiEjpyCbo64D06x17g3kXMbOHzWw7qSP6L6ctmmFmr5nZb8zsA5lewMweMrOkmSVbW1sHUf7gfPh9k5g2NqZOWREpKTnrjHX3J9z9GuAvgb8OZh8AGt19DvAosNzMajKs+6S7J9w9UVtbm6uSLlEWjfD5pji/bzlKy+G2YXsdEZF8kk3Q7wMa0h7XB/MuZwVwD4C7d7j70WB6A7AduO6KKs2RzyUaKI8ay3RULyIlIpugXw/MNLMZZlYBLAZWpTcws5lpDz8BvBPMrw06czGzq4GZwI5cFH6laqsr+fhNU/nRhr2c7dT4NyJS/AYMenfvBh4BVgNbgZXuvtnMHjezhUGzR8xss5ltJHWJ5sFg/h3ApmD+c8CX3D30r6c2N8U53dHNT17bH3YpIiLDzvJt/JdEIuHJZHJYX8PdufvvfwfAC3/+AcxsWF9PRGS4mdkGd09kWlYS34ztLzX+zXTeOniaDbuOh12OiMiwKsmgB1g0exrVlRr/RkSKX8kG/ZjKMj59Wz0/f+MArac7wi5HRGTYlGzQA9zfFKerx1mZ1Pg3IlK8Sjror51UxfuvncAP1+zS+DciUrRKOugh9VHL/Sfb+fVbh8MuRURkWJR80N95/WSm1Gj8GxEpXiUf9GXRCEvmNfK7d46wo1Xj34hI8Sn5oAdYfHsDZRHjh2t3h12KiEjOKeiBSTUxFtw4hWeTezjX2RN2OSIiOaWgDzQ3xTnV3s2q199rYE4RkcKjoA/MnXEVfzS5mqde2UW+jf8jIjIUCvqAmXH//Dib95/itT0nwi5HRCRnFPRpPjmnjqrKMpa9oo9aikjxUNCnqaos41O31vH8pgMcbdP4NyJSHBT0/TQ3xens6WVlcm/YpYiI5ERWQW9mC8xsm5m1mNljGZZ/yczeMLONZvafZjYrbdlXgvW2mdlduSx+OMycXE3T1VexbM0uenrVKSsihW/AoA/u+foEcDcwC7gvPcgDy939JnefDXwL+E6w7ixS95i9AVgA/L++e8jms+am6ew7cY6Xt2n8GxEpfNkc0c8FWtx9h7t3AiuARekN3P1U2sMxQN+h8CJghbt3uPu7QEvwfHntYzdMZlJ1JU+pU1ZEikA2QV8HpA/YvjeYdxEze9jMtpM6ov/yINd9yMySZpZsbW3NtvZhUx6NcN/cRn7zdiu7jp4JuxwRkSHJWWesuz/h7tcAfwn89SDXfdLdE+6eqK2tzVVJQ7JkXiNRjX8jIkUgm6DfBzSkPa4P5l3OCuCeK1w3b0yuiXHXDZNZmdxDe5fGvxGRwpVN0K8HZprZDDOrINW5uiq9gZnNTHv4CeCdYHoVsNjMKs1sBjATWDf0skdGc9N0Tpzt4mev7w+7FBGRKzZg0Lt7N/AIsBrYCqx0981m9riZLQyaPWJmm81sI/Ao8GCw7mZgJbAF+AXwsLsXzOFx09VXMXNSlW5KIiIFzfJtAK9EIuHJZDLsMs576pWdfO2nm/npw+/nloZxYZcjIpKRmW1w90SmZfpm7AA+OaeO0RVRfdRSRAqWgn4A1bFyPjmnjp9t2s/xM51hlyMiMmgK+iw0z4/T2d3LyuSegRuLiOQZBX0W3jelhrnTr2LZ2l30avwbESkwCvosNc+Ps+fYOX7zTvjf3BURGQwFfZbuumEKE6sqWapOWREpMAr6LFWURVgyt4GXth1mz7GzYZcjIpI1Bf0g3DevkYgZy9bqqF5ECoeCfhCmjh3FR6+fzMr1Gv9GRAqHgn6QmufHOX62i3/fdCDsUkREsqKgH6Q/vmYCV9eO0fg3IlIwFPSDZGY0N8XZuOcEb+w9GXY5IiIDUtBfgU/dWs+o8ihL1+wMuxQRkQEp6K/A2FHl3DOnjp9u3M/Js11hlyMi8p4U9FeouSlOR3cvz27Q+Dcikt8U9Fdo1rQaEvHxLFuj8W9EJL9lFfRmtsDMtplZi5k9lmH5o2a2xcw2mdmvzCyetqzHzDYGP6v6r1vImufH2Xn0LL9rORJ2KSIilzVg0JtZFHgCuBuYBdxnZrP6NXsNSLj7zcBzwLfSlp1z99nBz0KKyIIbpzCxqkLj34hIXsvmiH4u0OLuO9y9E1gBLEpv4O4vuXvfADBrgPrclpmfKsui3Ht7A79+6xB7j2v8GxHJT9kEfR2Q3uO4N5h3OV8EXkh7HDOzpJmtMbN7Mq1gZg8FbZKtrYU1DPCSeamrVMvX7g65EhGRzHLaGWtm9wMJ4Ntps+PBDWuXAN81s2v6r+fuT7p7wt0TtbW1uSxp2NWNG8VHrp/MM+v30NGt8W9EJP9kE/T7gIa0x/XBvIuY2Z3AV4GF7t7RN9/d9wW/dwAvA3OGUG9eam6Kc/RMJy+8cTDsUkRELpFN0K8HZprZDDOrABYDF316xszmAN8nFfKH0+aPN7PKYHoi8H5gS66Kzxd/cu1EZkzU+Dcikp8GDHp37wYeAVYDW4GV7r7ZzB43s75P0XwbqAKe7fcxyuuBpJm9DrwE/J27F13QRyLG5+c1smHXcTbv1/g3IpJfzD2/vuyTSCQ8mUyGXcagnTzbxby//SWfnFPH337q5rDLEZESY2Ybgv7QS+ibsTkydnQ5i26p4yev7efkOY1/IyL5Q0GfQ83z45zr6uFHG/aGXYqIyHkK+hy6sW4scxrHsWzNLvLtkpiIlC4FfY41N8XZceQMv285GnYpIiKAgj7nPn7TVK4aU6GbkohI3lDQ51isPMrnEg28uOUQ+0+cC7scEREF/XD4/LxGHHh6nca/EZHwKeiHQcNVo/nwH03i6XV76OzuDbscESlxCvph0jw/zpG2Dn6xWePfiEi4FPTD5I6ZtTReNZqlr+wMuxQRKXEK+mESiRj3NzWyfudxth44FXY5IlLCFPTD6LO3NVBZFmGZRrUUkRAp6IfR+DEV/Nkt0/jxa/s41a7xb0QkHAr6YfbA/DhnO3v48auX3KtFRGREKOiH2c3147ilfixLNf6NiIREQT8CmudPp+VwG6/s0Pg3IjLysgp6M1tgZtvMrMXMHsuw/FEz22Jmm8zsV2YWT1v2oJm9E/w8mMviC8Wf3jyVcaPLWfqKOmVFZOQNGPRmFgWeAO4GZgH3mdmsfs1eAxLufjPwHPCtYN2rgK8D84C5wNfNbHzuyi8MsfIo9yYa+I8thzh4sj3sckSkxGRzRD8XaHH3He7eCawAFqU3cPeX3P1s8HANUB9M3wW86O7H3P048CKwIDelF5Yl8xrpddf4NyIy4rIJ+jpgT9rjvcG8y/ki8MJg1jWzh8wsaWbJ1tbWLEoqPPEJY/jgdbU8vW43XT0a/0ZERk5OO2PN7H4gAXx7MOu5+5PunnD3RG1tbS5LyisPzI9z+HQH/7H5UNiliEgJySbo9wENaY/rg3kXMbM7ga8CC929YzDrlooPXjeJ+vGjeErj34jICMom6NcDM81shplVAIuBVekNzGwO8H1SIX84bdFq4GNmNj7ohP1YMK8kRSPG/U1x1r57jLcPnQ67HBEpEQMGvbt3A4+QCuitwEp332xmj5vZwqDZt4Eq4Fkz22hmq4J1jwHfILWzWA88HswrWZ9LNFBRFtFHLUVkxFi+fVszkUh4MpkMu4xh9ejKjax+8yBrv3onVZVlYZcjIkXAzDa4eyLTMn0zNgTNTXHOdPbw41f3hl2KiJQABX0IZjeM46Y6jX8jIiNDQR8CM6O5Kc7bh9pY925Jd1mIyAhQ0Ifkz26ZxthR5Tylm5KIyDBT0IdkVEWUz95Wz+o3D3L4lMa/EZHho6AP0eeb4nT3Ok+v2zNwYxGRK6SgD9GMiWO447palq/bpfFvRGTYKOhD1twU59CpDn65RePfiMjwUNCH7MPvm0TduFEsVaesiAwTBX3IohFjybxG/rD9KC2HNf6NiOSegj4P3Ht7AxXRCMvW6KYkIpJ7Cvo8MLGqko/fNIUfbdjLmY7usMsRkSKjoM8TzfPjnO7o5icbS3a4fhEZJgr6PHFr43hmTa1h6Ssa/0ZEcktBnyfMjOb5cd46eJrkruNhlyMiRURBn0cWzZ5GdaxMNyURkZzKKujNbIGZbTOzFjN7LMPyO8zsVTPrNrPP9FvWE9x16vydpySz0RVlfOa2el548wCtpzsGXkFEJAsDBr2ZRYEngLuBWcB9ZjarX7PdwBeA5Rme4py7zw5+FmZYLmnub4rT1eM8s14ftRSR3MjmiH4u0OLuO9y9E1gBLEpv4O473X0ToAFbhuia2ir+5NqJLF+7m26NfyMiOZBN0NcB6cMr7g3mZStmZkkzW2Nm92RqYGYPBW2Sra2tg3jq4tQ8P87+k+386q3DYZciIkVgJDpj48ENa5cA3zWza/o3cPcn3T3h7ona2toRKCm/feR9k5g6NsYyjX8jIjmQTdDvAxrSHtcH87Li7vuC3zuAl4E5g6ivJJVFIyyZ28jv3jnCjta2sMsRkQKXTdCvB2aa2QwzqwAWA1l9esbMxptZZTA9EXg/sOVKiy0l985toDxqGv9GRIZswKB3927gEWA1sBVY6e6bzexxM1sIYGa3m9le4LPA981sc7D69UDSzF4HXgL+zt0V9FmYVB1jwY1TeXbDHs52avwbEblyZdk0cvefAz/vN+9radPrSV3S6b/eH4CbhlhjyXpgfpyfvb6fVRv3s3huY9jliEiB0jdj81giPp73TanmKY1/IyJDoKDPY33j32w5cIpXd58IuxwRKVAK+jx3z+w6qivL9FFLEbliCvo8N6ayjE/fVs+/bzrAkTaNfyMig6egLwD3NzXS2dPLyuSegRuLiPSjoC8A106qZv7VE/jhmt309KpTVkQGR0FfIB6YH2ffiXO8pPFvRGSQFPQF4s5Zk5lcU8lT6pQVkUFS0BeI8miEJXPj/PbtVnYeORN2OSJSQBT0BeS+uQ2URYwfrtVRvYhkT0FfQCbVxLjrximsTO7lXGdP2OWISIFQ0BeY5qY4J8918bNN+8MuRUQKhIK+wMybcRXXTa5iqca/EZEsKegLjJnR3BTnjX0neX3vybDLEZECoKAvQPfMqWNMRZSnXtkZdikiUgAU9AWoOlbOp26t5/lNBzh2pjPsckQkz2UV9Ga2wMy2mVmLmT2WYfkdZvaqmXWb2Wf6LXvQzN4Jfh7MVeGlrnl+nM7uXp7V+DciMoABg97MosATwN3ALOA+M5vVr9lu4AvA8n7rXgV8HZgHzAW+bmbjh162XDe5mnkzrmLZ2l0a/0ZE3lM2R/RzgRZ33+HuncAKYFF6A3ff6e6bgN5+694FvOjux9z9OPAisCAHdQupo/o9x87x27dbwy5FRPJYNkFfB6RfH9gbzMvGUNaVAdx1wxRqqyvVKSsi7ykvOmPN7CEzS5pZsrVVR6fZKo9GuG9uIy+/3cruo2fDLkdE8lQ2Qb8PaEh7XB/My0ZW67r7k+6ecPdEbW1tlk8tAEvmNhIxjX8jIpeXTdCvB2aa2QwzqwAWA6uyfP7VwMfMbHzQCfuxYJ7kyJSxMT42azLPJPfQ3qXxb0TkUgMGvbt3A4+QCuitwEp332xmj5vZQgAzu93M9gKfBb5vZpuDdY8B3yC1s1gPPB7Mkxxqbopz4mwXz286EHYpIpKHLN/GS0kkEp5MJsMuo6C4O3d+5zdUxcr56cPvD7scEQmBmW1w90SmZXnRGStD0zf+zet7TrBp74mwyxGRPKOgLxKfuq2e0RVRlr6iTlkRuZiCvkjUxMq5Z04dq17fz4mzGv9GRC5Q0BeR5qY4Hd29PJvcG3YpIpJHFPRF5PqpNdw+fTzL1u6iV+PfiEhAQV9k7m+Ks+voWX77jr5hLCIpCvoic/eNU5lYVcGyNeqUFZEUBX2RqSiLsPj2Rn711mH2HNP4NyKioC9KS+Y1YsDydbvDLkVE8oCCvghNGzeKO6+fzDPr99DRrfFvREqdgr5IPTB/OsfOdPLzNzT+jUipU9AXqT++ZgJXTxyjb8qKiIK+WEUixv1NcV7dfYI3950MuxwRCZGCvoh9+rZ6YuURfdRSpMQp6IvY2FHl3DO7jp9s3MfJs11hlyMiIVHQF7nm+XHau3p57lWNfyNSqhT0Re6GaWO5tXEcy9Zo/BuRUpVV0JvZAjPbZmYtZvZYhuWVZvZMsHytmU0P5k83s3NmtjH4+V6O65csPDB/Ou8eOcPvtx8JuxQRCcGAQW9mUeAJ4G5gFnCfmc3q1+yLwHF3vxb4v8A305Ztd/fZwc+XclS3DMLdN01hwpgKfdRSpERlc0Q/F2hx9x3u3gmsABb1a7MI+EEw/RzwETOz3JUpQ1FZFuXe2xv45dZD7DtxLuxyRGSEZRP0dcCetMd7g3kZ27h7N3ASmBAsm2Fmr5nZb8zsA5lewMweMrOkmSVbWzW87nBYMq8RgKfXavwbkVIz3J2xB4BGd58DPAosN7Oa/o3c/Ul3T7h7ora2dphLKk3140fz4fdNZsX63Rr/RqTEZBP0+4CGtMf1wbyMbcysDBgLHHX3Dnc/CuDuG4DtwHVDLVquTPP8OEfaOvnFmwfDLkVERlA2Qb8emGlmM8ysAlgMrOrXZhXwYDD9GeDX7u5mVht05mJmVwMzgR25KV0G6wPXTmT6hNH6pqxIiRkw6INr7o8Aq4GtwEp332xmj5vZwqDZvwATzKyF1CWavo9g3gFsMrONpDppv+Tux3L8N0iW+sa/Wb/zOFsPnAq7HBEZIeaeX1+iSSQSnkwmwy6jaJ0428m8v/kVn76tnr/55E1hlyMiOWJmG9w9kWmZvhlbYsaNrmDR7Gn85LV9nGrX+DcipUBBX4Kam6ZztrOHf9ug8W9ESoGCvgTdVD+W2Q3jWLpmF/l26U5Ecq8s7AIkHM1Ncf7i2df5xvNbmTYuRmV5lFHlUWLlEWJlUWJ908HvymDeqIoosbIIZVEdI4gUCgV9ifrEzVP5h1+/w7/+/t0rWr8sYv12An07hQs7jMryaLDTuLDDuHQn0u9xWvvK8kjwXFHKtWMRuWIK+hIVK4/y8v/8EF09Tnt3D+1dPXR09dLe1cO5rh7ag+n2rh7au3uD5Rfmn2/Tb9327tSy42c7OZc+P3ieniscKjkaMWJlF3YOled3GpHgLCNtfoYdzKjyaGrHUx696Hku2sFUXJgujxoarkmKhYK+hJkZFWVGRVmEmlj5iLxmV09f8KftANJ2GP13MB3B9MU7n1T79B3PyXNdnOtMPe7ovjC/+wp3LBHjws4gbccwfkw5k2tiTKmJMWVs7KLpiVWVRCPaOUj+UdDLiCqPRiiPRqiOjczrdff0nj8juWjHkv67u/+O5/LLjp7pZM32oxw+3XHJTiQaMWqrKpk8NsaUmkqm1MSC6dhF02Mq9baTkaX/cVLUyqIRqqIRqnIcrr29zpEzHRw62cHBU+0cPNXOoZPB71Pt7Gg9wx+2H+V0e/cl61ZXlp0P/ck1MaaMrUybTs2foLMDySEFvcgViESMSdUxJlXHuImxl213trObg2k7gIMnO4LfqXnbtx/h8OmOS/ouohFjUnXlpZeJxl48b3SF3sIyMP0vERlGoyvKuLq2iqtrqy7bpqfXOdoWnBmcDHYIaTuFltY2ft9yhNMdGc4OYmWX9BdcfLmokoljKono7KCkKehFQhaNGJNqYkyqiXFz/eXbnenovugS0cXTHbxz6AiHT7fTv/+5rO/s4KLLRZdOj6qIDu8fKqFR0IsUiDGVZVxTW8U1A5wdHGnr6He56ML024dO87t3jtCW4eygJlZ2ySeJ+k9PGFOhs4MCpKAXKSLRiDE5OFK/5T3anW7vOt9nkGmHsO3gaY60dVxydlAeTfVNTK6pPB/+UzPsEGLlOjvIJwp6kRJUHSunOlbOtZOqL9umu6eX1uDs4MKO4EJn8lsHTvPytlbOdl56a8pxo8uZUhNj/OgKxlSWUVUZTf2OlVFVURbMSz2+aHnwM6ayTN+GziEFvYhkVBaNMHXsKKaOHXXZNu7O6Y7uC30F/TqTT57rZN+Jc7R1dHGmo4e2jm46u3uzev2KsgjVQej37QyqKi/sJPrmX2gTvWhH0fe7OlZGZVmkpL/pnFXQm9kC4O+BKPDP7v53/ZZXAk8BtwFHgXvdfWew7CvAF4Ee4Mvuvjpn1YtIqMyMmlg5NbFyZk6+/NlBuq6eXs50dHO6vZsznd2c6eimraOHtva+6Qu/L0z30NbRxZG2TnYdPXt+/pkMZxOZRCPGmIpov7OIMsZUBGcZwY7i4h3HxTuN820qygqun2LAoA/u+foE8FFgL7DezFa5+5a0Zl8Ejrv7tWa2GPgmcK+ZzSJ1j9kbgGnAL83sOnfP7l9HRIpOeTTCuNEVjBtdMeTn6u31YGfR02/HcOnO4nybYAfT1tHNoVPttLUHbTp7sh6LaUxF9JIdwWAuUY2pSJ1pjNQlqmyO6OcCLe6+A8DMVgCLgPSgXwT872D6OeCfLHWetAhY4e4dwLvBPWXnAq/kpnwRKWWRiJ3vbxgqd6ejuzd1ppG+g+gMzj46ejKcaVz4ve/EuWCHknrcMYhLVH1nDLc0jOMf75sz5L+lv2yCvg7Yk/Z4LzDvcm3cvdvMTgITgvlr+q1b1/8FzOwh4CGAxsbGbGsXEckZMzs/eF1tdeWQn6/vElXb+bOJLtrSdxZ9O5TOC2ccU8cOzyBQedEZ6+5PAk9C6ubgIZcjIjJkubxENVTZXBzaBzSkPa4P5mVsY2ZlwFhSnbLZrCsiIsMom6BfD8w0sxlmVkGqc3VVvzargAeD6c8Av/bUzUhXAYvNrNLMZgAzgXW5KV1ERLIx4KWb4Jr7I8BqUh+v/Fd332xmjwNJd18F/AuwNOhsPUZqZ0DQbiWpjttu4GF94kZEZGRZ6sA7fyQSCU8mk2GXISJSUMxsg7snMi3Td4xFRIqcgl5EpMgp6EVEipyCXkSkyOVdZ6yZtQK7hvAUE4EjOSonl1TX4KiuwVFdg1OMdcXdvTbTgrwL+qEys+Tlep7DpLoGR3UNjuoanFKrS5duRESKnIJeRKTIFWPQPxl2AZehugZHdQ2O6hqckqqr6K7Ri4jIxYrxiF5ERNIo6EVEilxBBr2ZLTCzbWbWYmaPZVheaWbPBMvXmtn0PKnrC2bWamYbg5//OkJ1/auZHTazNy+z3MzsH4K6N5nZrXlS14fM7GTa9vraCNXVYGYvmdkWM9tsZn+eoc2Ib7Ms6xrxbWZmMTNbZ2avB3X9nwxtRvw9mWVdobwng9eOmtlrZvZ8hmW53V7uXlA/pIZK3g5cDVQArwOz+rX578D3gunFwDN5UtcXgH8KYZvdAdwKvHmZ5R8HXgAMaALW5kldHwKeD2F7TQVuDaargbcz/FuO+DbLsq4R32bBNqgKpsuBtUBTvzZhvCezqSuU92Tw2o8CyzP9e+V6exXiEf35m5W7eyfQd7PydIuAHwTTzwEfCW5WHnZdoXD335K6T8DlLAKe8pQ1wDgzm5oHdYXC3Q+4+6vB9GlgK5fe63jEt1mWdY24YBu0BQ/Lg5/+n/IY8fdklnWFwszqgU8A/3yZJjndXoUY9JluVt7/P/tFNysH+m5WHnZdAJ8OTvWfM7OGDMvDkG3tYZgfnHq/YGY3jPSLB6fMc0gdDaYLdZu9R10QwjYLLkNsBA4DL7r7ZbfXCL4ns6kLwnlPfhf4X0DvZZbndHsVYtAXsp8B0939ZuBFLuyxJbNXSY3fcQvwj8BPRvLFzawK+BHwP9z91Ei+9nsZoK5Qtpm797j7bFL3hZ5rZjeOxOsOJIu6Rvw9aWZ/Chx29w3D/Vp9CjHoh3Kz8lDrcvej7t4RPPxn4LZhrilbeXkTd3c/1Xfq7e4/B8rNbOJIvLaZlZMK0x+6+79laBLKNhuorjC3WfCaJ4CXgAX9FoXxnhywrpDek+8HFprZTlKXeD9sZsv6tcnp9irEoB/KzcpDravfNdyFpK6x5oNVwAPBJ0magJPufiDsosxsSt91STObS+r/67CHQ/Ca/wJsdffvXKbZiG+zbOoKY5uZWa2ZjQumRwEfBd7q12zE35PZ1BXGe9Ldv+Lu9e4+nVRO/Nrd7+/XLKfba8Cbg+cbH8LNyvOgri+b2UJSN0o/RqrHf9iZ2dOkPo0x0cz2Al8n1TGFu38P+DmpT5G0AGeB/5IndX0G+G9m1g2cAxaPwA4bUkdczcAbwfVdgL8CGtNqC2ObZVNXGNtsKvADM4uS2rGsdPfnw35PZllXKO/JTIZze2kIBBGRIleIl25ERGQQFPQiIkVOQS8iUuQU9CIiRU5BLyJS5BT0IiJFTkEvIlLk/j/3q/2ViKOIWQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "mus, classes, errs = kmeans(X)\n",
    "\n",
    "plt.plot(errs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x11f6298d0>]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAax0lEQVR4nO3db4wkdZ3H8fd3e4JmEsSDHRPD7s4ALhdXjztggojJqeefIA8gFz0PHM4zR9ywm/UeaC7BjDEGs4k+OC9nYFlHz6jsAnL3wMxFLhgVJFlcjiHIKhjM7Moui0YG4XhCFFm+96C62Z6e/lPVXX9+9avPK+nMdHVNz6+qq7/1rd+/MndHRETqb1PVBRARkXwooIuIREIBXUQkEgroIiKRUEAXEYnEVFX/ePPmzT43N1fVvxcRqaVHHnnkOXef6fdaZQF9bm6OlZWVqv69iEgtmdnxQa+pykVEJBIK6CIikVBAFxGJhAK6iEgkRgZ0M/ummT1rZr8Y8LqZ2VfNbNXMjpjZJfkXU0RERkmToX8LuHLI6x8CtrcfO4HbJi9WXA4ehLk52LQp+XnwYNUlCov2j0g+RgZ0d38AeH7IKtcA3/HEYeCNZvbmvApYdwcPws6dcPw4uCc/d+6sPmiFEkRD3T8idZRHHfq5wNNdz0+2l21gZjvNbMXMVtbW1nL41+FbXISXXlq/7KWXkuVVCSmIhrh/ROqq1EZRd19y93l3n5+Z6TvQKTonTmRbXoaQgmiI+6cqoVw1SX3lEdCfAbZ2Pd/SXibAtm3ZlpchpCAa4v6pQkhXTVJfeQT0ZeDj7d4ulwMvuvtvc3jfKOzdC9PT65dNTyfLqxJSEA1x/1QhpKsmqa803RbvBH4K/LmZnTSzG8zsRjO7sb3KPcAxYBX4OrC7sNLW0MICLC3B7CyYJT+XlpLlVQkpiIa4f6oQ0lWT1JdVdU/R+fl51+Rc1Tl4MMn+TpxIMvO9e5sXREMyN5dUs/SanYWnniq7NBIyM3vE3ef7vaaRog21sJAEildfTX4WFczV0JdOSFdNhdIBUajKps+V+HUa+jp1w52GPtDVQK/O/oj6qkkHROFU5SKFCaUaQdVLgQjlgKg5VblIJQY16B0/Xt7VtroDBiTPll9V3fSlgC6FGdYNsqzAqu6AAcmrv6zO0gMpoEth+jX0dSsjsKo7YEDyavnVWXogBXQpTHcf80GKDqwhDaJqvLwGHegsPZACuhSq0z1yUFAvOrA2pjtgXeTRX1Zn6YEU0KUUVQVWjUSNkM7SAymgSymqDKxlDaJqtDJ7negsPZD6oYvIZHoHDEGSMSvIFkL90EWkOOp1EgwFdBGZjHqdBEMBPSAa/Ca1pF4nwVBAD4QGv0ltqddJMBTQA6FqSKmtrL1OdClaGPVyCcSmTUlm3sss6W4nEgX1iJmYerkUKK9kQ9WQ0gi6FC2UAvoE8qz3VjVkHFSbMIJ6xBRKAX0CeSYbGvxWf2rYTqGul6I1OVOrDn0CqveWbrohTwp1rEMPrMyqQy9IXZMNKYZqE1Ko46Vojer9FdAnEFu9d02uKoOlE3xKdZstrUZnagX0CdQx2RhE9b+Ti+0EL201OlMroE+obsnGIDW6qgxWTCd46VKjM7UaRQVQA6/IUAcPJtnNiRNJZr53b2Vn6mGNolNlF0bCtG1b/x4aAV5VipRvYaEWl1qqchGgVleVIjKAAroAqv+VgqjrVKlU5SKvqclVpdRF74CcTtcp0IFWEGXoIlIMdZ0qXaqAbmZXmtmTZrZqZjf1eX2bmd1nZo+a2REzuyr/oopIrdRoQE4sRgZ0M2sBtwIfAnYA15nZjp7VPgfc7e4XA9cC+/IuqIjUTI0G5MQiTYZ+GbDq7sfc/WXgLuCannUceEP797OA3+RXRBGpJXWdKl2agH4u8HTX85PtZd2+AFxvZieBe4BP9XsjM9tpZitmtrK2tjZGcUWkNtR1qnR5NYpeB3zL3bcAVwG3m9mG93b3JXefd/f5mZmZnP61iAQrlrkxaiJNQH8G2Nr1fEt7WbcbgLsB3P2nwOuBzXkUUERE0kkT0B8GtpvZeWZ2Bkmj53LPOieA9wGY2VtJArrqVERESjQyoLv7K8Ae4F7glyS9WR43s5vN7Or2ap8BPmlmjwF3Ap/wqmb9EhFpqFR16O5+j7tf6O4XuPve9rLPu/ty+/cn3P1d7v6X7v5X7v6DIgstMi6NRJeYaei/NIZGokvsNPRfGkMj0SV2CujSGBqJLrFTQJfG0Eh0iZ0CujSGRqJL7BTQpTE0El1ip4AujRHQfX5FCqFui9II6rIoTdC4DF0DS4oR+n4Nscti6PtMasjdK3lceumlXrYDB9ynp93h9GN6Olku48t7vx444D47626W/Mzj8zFbX77Ow2zy9x6HjsUUijgQQvhfEwJWfEBcbVRAn53t/6WenS29KFHJc78WFehC++xDK09wyjzj1ezsqoDeFlqWFos89+s4gS5NchXad1bH4ghlnvFqdnYdFtAbVYeugSXFyHO/Zh3N2WnsPH48+RZ2Gjt766ND67KoY3GEMof1RjSEuFEBXQNLipHnfs0a6LI0doZ08xwdiyOUecaL6ew6KHUv+lFFlYt7rdo+aiWv/Zq1aqTOVRc6FodQHfpAqA5d6iRLoKtZ9adkoV4ufQ0L6Ja8Xr75+XlfWVmp5H9LPHoHDEFSdaEh/RIrM3vE3ef7vdaoOnSJT2iNndIAAY8I09B/qb2FBQVwKUngc0goQxcRSSvEOSS6KKCLiKQVeJ91BXQRkbQC77OugC4iklbgI8IU0EVE0gq8W5UCukiPgHulyTjy/kBDmkOih7otinQJvFeaZNWwD1QjRUW6zM0l3/les7NJMiY1E+EHqpGiIikF3itNsmrYB6qALtIl8F5pklXDPlAFdJEugfdKk6wa9oEqoIt0CbxXmmTVsA80VaOomV0J/DvQAr7h7l/qs85HgS8ADjzm7h8b9p5qFBURyW5Yo+jIbotm1gJuBT4AnAQeNrNld3+ia53twGeBd7n7C2b2pnyKLiIiaaWpcrkMWHX3Y+7+MnAXcE3POp8EbnX3FwDc/dl8iykiIqOkCejnAk93PT/ZXtbtQuBCMztkZofbVTQbmNlOM1sxs5W1tbXxSiwiIn3l1Sg6BWwH3gNcB3zdzN7Yu5K7L7n7vLvPz8zM5PSvRUQE0gX0Z4CtXc+3tJd1Owksu/uf3P3XwK9IAryIiJQkTUB/GNhuZueZ2RnAtcByzzrfI8nOMbPNJFUwx/IrppRp926Ymkp6eU1NJc9FJHwjA7q7vwLsAe4Ffgnc7e6Pm9nNZnZ1e7V7gd+b2RPAfcC/uPvviyq0FGf3brjtNjh1Knl+6lTyXEE9Z+PMAKhpIGUETc4l60xNnQ7m3VoteOWV8ssTpd4ZACEZvThswMs4fyNRGtYPXQFd1jEb/FpFh0p8xpkBMMJZA2U8mm1RUmu1si2XMYwzA2DDZg2U8Sigyzqduf/TLpcxjDMDYMNmDZTxKKDLOvv2wa5dpzPyVit5vm9fteWKyjgzADZs1kAZjwK6bLBvX9IA6p78VDDP2TgzAA77G/V+kTY1iorUmXq/NI4aRUVitbi4PphD8nxxsZry5ElXHpkpoIvUWay9XzpXHsePJ3V/x48nz0MJ6oGebBTQReos1t4vIV95BHyyUUAXyVmpyVusvV9CvvII+GSjgC6So9KTt1jvmRnylUfAJxsFdJEcVZK8LSwkw/9ffTX5WfdgDmFfeQR8slFAF8lRwMlbvYR85RHwyUYBvQCBNoBLCQJO3uon1CuPgE82Cug5C7gBXEoQcPImeQr0ZKOAnrOAG8Antv8nR3nw6HPrlj149Dn2/+RoRSUKT8DJmzSAAnrOYq5DvWjLWey549HXgvqDR59jzx2PctGWsyouWVgCTd7qQ3WWY5uqugCx2bat/30IYqhDveKCzdzysYvZc8ejXP+ObRx46AS3fOxirrhgc9VFk1j0zk3TqbMEnRlTUIaes3HrUHuTkt27w0xSrrhgM9e/Yxtf/fEq179jm4K55CvmOssSKKDnbJw61H4NqbfdFmbD6oNHn+PAQyf45795CwceOrGhTl0ks+5spt/lLcRRZ1kCTZ8bgEG3i+xV9e0jO3XmnWqW3ucimfWb/refqg/+gGj63MClTT6qTlKOnHxxXfDu1KkfOflitQWT+upXxdJL/T5TU6NoAAY1pPZbr0o3vvuCDcuuuGCzsnMZ37AsxSw56PfuVYNoSsrQA9CvIbWXkhSJ0qAsZXZW/T7HoIAegH4Nqbt2aXCKNECWbmHqnz6SqlwC0QnWi4vJVeg99+hKUxqg58Df/8F/4qK//QBXLPz9a6s8ePQ5jvz3/dy4qP7poyhDD4TmgJHG6hpae9GtX2LPs2dvHI18x9fUPz0FdVsMxKCui+qtJU3TCeLrRiNvf1OS6fQyS+raG0TdFmsg5jlgRLLoOxo5lnmJC24HUEAPRCzH67jU3iUdfUcjxzAvcRn1qu5eyePSSy/12B044D47626W/DxwYPi609PuySedPMySn6P+tu76bfv0dNzbLP0dWl3zi2/+gR9aXdv4PMsXKkSzs+sP8s5jdjbT2wArPiCupgq+wJXAk8AqcNOQ9T4MODA/6j1jD+jjBKnO8dodzJsQ4IYd53X/Dks2t92/+low7zi0uua33b9aUYly1Pul7s7cMpgooAMt4ChwPnAG8Biwo896ZwIPAIeLCuh1+nJPcjLO6UReG4OO886JrCknNolcCRl6mjr0y4BVdz/m7i8DdwHX9Fnvi8CXgT+MUfMzUt269U3SyNm0BtJh7QTqqVaw3bthairpLTI1lTyXYpTQDpAmoJ8LPN31/GR72WvM7BJgq7t/P7eS9ajbNMmTNHI2rYE0zdQH3WI9sZVu9+5knuZTp5Lnp04lzxXUi1HC/Qkn7uViZpuArwCfSbHuTjNbMbOVtbW1TP+nblnrJCfjGBr0s+g9zlut4evHemIr3dJStuUyuYLvT5gmoD8DbO16vqW9rONM4O3A/Wb2FHA5sGxmGzq+u/uSu8+7+/zMzEymgtYta53kZNzEGw13H+fDxonEfGIrXSczT7tcwjeocr3zIJnv5RhwHqcbRd82ZP37KaBRVF3bmmNQ21Grlf/nXaeG9ty1WoN3tASLSRpF3f0VYA9wL/BL4G53f9zMbjazq4s4yfTTxKy1qQZVOX372/l+3nVraM9dZ3KrtMtluBBGxw2K9EU/Yu+HLpMpI3NuWvfQvnbtOp2pt1rJc8lu0MjAAvYnQzJ0Tc4ljbVpk+Z7kpwMml3PDG6/PddLS03OJdJH3RraJWCDutu5l9q3WgFdGqtp3UOlQMOygBL7ViugS2OpoV1ys3dvchD1U+Iln25BJ422sKAALjlYWIBDh2D//vUNMyVf8ilDFxHJw759SQNohZd8CugiMr4Q+l6HpOCh/aOoykVExtMZmdWZNa8zMgtUj1URZegi0t+o7DvEKVAbfsWgDF1ENkqTfYc2BaquGJShi0gfabLvSUZmZc2kO+t3bsRhtvHvQrxiKJkCuohslCb7HndkVtZZ0brXh9PT+/b+XZFXDHWpyhk0yUvRD03OVS+Nnma2idLOXDbOgZF1VrRB6/f+XVGzrQU2dzeT3CS6qIcCen0EdjxLGYr80AfdFdys//rDgnn33xVV5sCm5RwW0FXlIiOparKBipwXIWvde9p7EhZV5tAaf4dQQJeRanQ8S57GGSSTpq45a937sFvi9f5dEQN7ajQtpwK6jFSj47lwpbeN1aUxDtI3dmbNpGdn+y9vtbJl4OPuyzpNyzmoLqboh+rQ60N16InS90PddnzIjZKTvkdAvQJQo2jYAjpWBpan6jIW+f/TvnfpbWOBNcaNlLWxM4tJD4C67cshFNADFloSFlp5ii5TlvcuMl71Vfo/nFDeQTPPs/gk+7LqbKaHAnrAQkscQitP0WXK8t7K0EfI88yb91l83H0ZYIajgB6w0JKw0MpTdJmyvLfq0FPIK5stItsfZ18GeFJVQA9YaMdLaOUpukxZ37v0q+/ALvdLU8RZfJx9GWCGo4AesNCSsNDKU3SZQtxe8XAyi1DK0SW6gB5b0hLa9oRWHvdiy7Rrl3urlXwbWq3kuVQslDNtKOXoElVAD3D/So3peApYKJlFKOVoGxbQLXm9fPPz876yspL57+bmTs+i2W12NhnpK5KFjiepGzN7xN3n+71Wu6H/w+YVqdMoaQmD5qmRmNQuoA+aP+Tss7PNmS/lCP0kq3lqJCa1C+iD5skBTfEamqw3pqlCneZdAsI/Q0q1BlWuF/3Iu5dLgN1FGy/AHl99BdbmNZhacMUjaxQdRI1b4dm0KYk6vcyS6aolIx3kQg6NomZ2pZk9aWarZnZTn9c/bWZPmNkRM/uRmc1OWuisanfp3CPGK2nVT+dMLbjFiuFLOCh17zyAFnAUOB84A3gM2NGzznuB6fbvu4DvjnrfIkaK1ubSuUesV9Kxbldl6lKHVUc1OliZZGAR8E7g3q7nnwU+O2T9i4FDo95XQ/9Pi/l7Ou5Jtq4n50LVKOgEr/cAO+ec2nwJJw3oHwG+0fX8H4Bbhqx/C/C5Aa/tBFaAlW3btpW0+eFTg+56aeJWYwN+Yzc8R/0OsEGPsicDS6G0gA5cDxwGXjfqfZWhnxZzhj6OUftj166NJ0ElqpLaoAMszy9hgVdTwwJ6mkbRZ4CtXc+3tJetY2bvBxaBq939jyneV9qKaNCtc/vOqNHA+/dv7D2jMQeSWtpG5Em+hIuL1QyMGRTpOw9gCjgGnMfpRtG39axzMUnD6fZR79d5KENfL8+rs7pXtQ7L0IclV02topKMBh1E55yT35dwWNY/ISadbRG4CvhVO2gvtpfdTJKNA/wQ+B3ws/ZjedR7KqAXp4gqnDKrboedkAa1NzS5ikoyKiPj6czH3PtotSZ+64kDehEPBfSN8gqaeTeyVpHxD9oXg05WZvW5ApEAFJ2hhJyhF/FQQF8vz6CZd4YeUqNtv/1kpptSSGAK/NIMC+i1m5wrVnm2oeTdyBrSAMWFBVhaSka7myU/b78d9u0rvywiA1U0dF0BPRB5Bs1+QW9pKVme1cGDSU+Zfqoawr+wkExd8uqryc9xtkukUHl+CTNQQA9E3vOe5BH0OtPfnjq18bU6zZMjUoneLyEU3pdYAT0QIU4u1q8aCKDVKiXZiEOdBwRIfkq6OUA00+fG4ODBJIieOJFk5nv3Vhs0Nf3thDpf4u6z4vS0zoZNlOPUx8Omz1VAl4E0/faEtAOlI8fsKKqbREt5QqwGqpWQugdJtUq6OYACugxUUUN9PM4+O9tyiVdJ2dFUru8m0VlYUAAXmVjnS1RwI5kCukhRnn8+23KJWwnZkapcRIqim6pKyRTQRYqiVmUpmQK6SFHUqiwlUx26SJHUqiwlUoYuIhIJBXQRkUgooIuIREIBvQSacE9EyqBG0YL1TrjXmTUT1FYmIvlShl6wPG8tJxI8XY5WSgG9YGkn3NP3QGqvpJs4yGAK6AVLM/p73O+BTgISFF2OVk4BvWBpRn+P8z1QMiTB0fzvlVNAL1ia0d/jfA+UDElwNBlZ5RTQS9B78+/e3i3jfA+UDElwNBlZ5RTQAzDO90DJkARHk5FVTgE9AON8D5QMSZBGXY5KoTSwKBBZJ+Ur6Y5WIlIjytBrTMlQPtT9U2KhDF0aTVMzSExSZehmdqWZPWlmq2Z2U5/XX2dm322//pCZzeVeUpECqPunxGRkQDezFnAr8CFgB3Cdme3oWe0G4AV3fwvwb8CX8y6oSBHU/VNikiZDvwxYdfdj7v4ycBdwTc861wDfbv/+X8D7zMzyK6ZIMdT9U2KSJqCfCzzd9fxke1nfddz9FeBF4JzeNzKznWa2YmYra2tr45VYJEfq/ikxKbWXi7svufu8u8/PzMyU+a9F+tJYGIlJml4uzwBbu55vaS/rt85JM5sCzgJ+n0sJRQqWdQyASKjSZOgPA9vN7DwzOwO4FljuWWcZ+Mf27x8Bfuzunl8xRURklJEZuru/YmZ7gHuBFvBNd3/czG4GVtx9GfgP4HYzWwWeJwn6IiJSolQDi9z9HuCenmWf7/r9D8Df5Vs0ERHJQkP/RUQioYAuIhIJq6rt0szWgONj/Olm4Lmci1MXTd12bXezNHW7Id22z7p7337flQX0cZnZirvPV12OKjR127XdzdLU7YbJt11VLiIikVBAFxGJRB0D+lLVBahQU7dd290sTd1umHDba1eHLiIi/dUxQxcRkT4U0EVEIhFsQG/qbe9SbPenzewJMztiZj8ys9kqylmEUdvetd6HzczNLIqubWm228w+2v7cHzezO8ouYxFSHOvbzOw+M3u0fbxfVUU582Zm3zSzZ83sFwNeNzP7anu/HDGzS1K/ubsH9yCZBOwocD5wBvAYsKNnnd3A/vbv1wLfrbrcJW33e4Hp9u+7YtjutNveXu9M4AHgMDBfdblL+sy3A48Cf9Z+/qaqy13Sdi8Bu9q/7wCeqrrcOW37XwOXAL8Y8PpVwP8ABlwOPJT2vUPN0Jt627uR2+3u97l757bGh0nmp49Bms8c4Isk96z9Q5mFK1Ca7f4kcKu7vwDg7s+WXMYipNluB97Q/v0s4Dcllq8w7v4Ayay0g1wDfMcTh4E3mtmb07x3qAE9t9ve1Uya7e52A8mZPAYjt7196bnV3b9fZsEKluYzvxC40MwOmdlhM7uytNIVJ812fwG43sxOksz2+qlyila5rHHgNammz5XwmNn1wDzw7qrLUgYz2wR8BfhExUWpwhRJtct7SK7IHjCzv3D3/6uyUCW4DviWu/+rmb2T5J4Lb3f3V6suWKhCzdCz3PaOiG57l2a7MbP3A4vA1e7+x5LKVrRR234m8HbgfjN7iqRucTmChtE0n/lJYNnd/+TuvwZ+RRLg6yzNdt8A3A3g7j8FXk8yeVXsUsWBfkIN6E297d3I7Tazi4GvkQTzGOpSO4Zuu7u/6O6b3X3O3edI2g+udveVaoqbmzTH+vdIsnPMbDNJFcyxEstYhDTbfQJ4H4CZvZUkoK+VWspqLAMfb/d2uRx40d1/m+ovq27xHdISfBVJJnIUWGwvu5nkSwzJh/ufwCrwv8D5VZe5pO3+IfA74Gftx3LVZS5r23vWvZ8Iermk/MyNpLrpCeDnwLVVl7mk7d4BHCLpAfMz4INVlzmn7b4T+C3wJ5KrrxuAG4Ebuz7vW9v75edZjnMN/RcRiUSoVS4iIpKRArqISCQU0EVEIqGALiISCQV0EZFIKKCLiERCAV1EJBL/D7JTTC8r/g1CAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(X[classes==0, 0], X[classes==0, 1], 'or')\n",
    "plt.plot(X[classes==1, 0], X[classes==1, 1], 'ob')\n",
    "plt.plot(mus[:, 0], mus[:, 1], 'x')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x11f6bcc10>]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkV0lEQVR4nO3de5Qc1X0n8O9veh6a0RM9nGCJGWEsQRRjgjUHsHMSJ4FNBHuOtHuc4wCDWR5GwVoZsvZmTXYc48VWFpaN92CDwCLBDySMiR17FUc+wsEClocEIwkJkJAYSSNpRkIzGr01mkd3//aP6rZarX5Ud1fVvVX1/ZzTZ6arq7t/U1P1q3tv3XtLVBVERBR9daYDICKiYDDhExHFBBM+EVFMMOETEcUEEz4RUUww4RMRxUTZhC8iT4lIv4i8U+R1EZFvi0i3iGwVkU94HyYREdXKTQn/+wAWlHj9egBzMo/FAB6vPSwiIvJa2YSvqi8DOFJilUUAfqiO9QCmiMiFXgVIRETeqPfgM2YC2J/zvDez7GD+iiKyGE4tAOPHj59/2WWXefD1wTtyBNi7F0inzy6rqwPa2oCpU83FlWVrfLbGFQbDo6MYTSZx4vhJzLrwt0yHQwZt3LjxsKrOqOa9XiR811R1BYAVANDe3q5dXV1Bfr1nZs8+N2kBzvN0GrDhT7I1PlvjCoPXd+5EU2MDnvzOD/H4399vOhwySET2VvteL3rp9AG4KOf5rMyyyNq3r7LlQbM1PlvjssGqVc4Jsa7O+blq1bmvNzbU42D/ESy+ZZGJ8CgivEj4qwHcmumtcw2A46p6XnNOlLS2VrY8aLbGZ2tcpq1aBSxe7DR3qTo/Fy8+P+knx5Jobh5nJkiKBDfdMn8E4HUAl4pIr4jcKSJ3i8jdmVXWANgNoBvAkwCW+BatJZYtA1pazl3W0uIst4Gt8dkal2mdncDQ0LnLhoac5UReKtuGr6o3lXldAfxnzyIKgY4O52dnp9Mc0drqJK3sctNsjc/WuEwr19S1/u3taJrUjGQyifHjxwcXGEWOmJoPP8wXbYm8NHu204yTr60N6OkBnt+4GdOnTcF3HnoS33v874IOjywjIhtVtb2a93JqBfJEuYuOVJybpq7RsSTGTh8PNrCgcOcJTKDdMimashcds+3Q2YuOAJtr3Ih1Uxd3nkCxhE81s/GiY9gKjR0dTvNNOu38zOa6dDqN8eObTYbmLxt3nghjwqeaFbvouHevmWTrtptjGPT096OleRwOHzmGz3X8R9PheM/rwRlhO9MHjAmfalaqH72JZBulQuNYMgUA2LThbfzZn11nOBofeDk4I0pnep8w4VPNCl10zBV0so3kiF4znen85+XgjCid6X3ChE816+gAVqxwuhEWE2SyjdKI3rFk0nQI/srdeUScnytWVHfBNpJnem8x4ZMnshcdiyX9IJNtlEb0nhgZhqriUO8B06H4p9gV60pF6UzvEyZ88pQNydbLQqNp45oa8cHAEdz+FzeYDsV+Nux8lmPCJ0/Zkmy9KjTaYHRkDBdcMNl0GPazZeezGBM+eS5KyZaqYLJrJHe+kjjSlshSG7fvRF1LI1KpFJqaQjItMkfOWo0lfCJLHRkagqpi7T+vxaxZM02H4w67RlqNCZ/IYiOjYxg+Pmg6DPfYNdJqTPgB4Ghvig12jbQaE77PONqbqiVhPDrZNdJqYdylQoVNmlSNIydPYtrUKUimUpgWpi6Z1XSNZBU4MLzjlc/q6pySfT4Rp+cYUSHb9u7HCFJ47fW3sPgz/x4NDQ2mQ/JHfq8ewKkRsP98Ubzjlce8LHCwSZNqMXCgP7rJHmAVOGBM+Hm8bnNnkyZRCezVEygm/DxeFzg42js+vKwZ9g4cBgCkDTW5BiYKVeAQXYPgSNs8fhQ4OjqY4KPO6wGm06ZPwekzwzh9qNe7IG20bFnhNvywVIFDNrKYJfw8UShwUPC8rhnWiWBg4CjuXXpX7cHZLOxV4JBdg2DCz8M2d6oGm6JrEOYJz0L2j2fCzxP2AkcpIWpqDB0va4bDIyO1BUPBCVmTABN+AWEucBTDEb/+8rJm+NLWdwAABw/0Y8aMGR5ER74JWZMAE35MhKypMXS8rBnWJRIYGR3DK6t/iebmZu+DJe+ErEmAvXRiImRNjaHkZW+stCqaGiM84CpKQtQNjyX8mAhZUyMR+YAJPyZC1tQYW8dPn8a0qZORSqUwcdJ40+FQxDDhx0TImhpja//AYYgI3n37fTz04P2mw/EWu4kZxzb8GAlRU2Ps9e3ri9YF25CNSI0qlvCJyH/sJmYFJnwiixw8fMR0CP5gNzEruEr4IrJARHaISLeI3Ffg9VYRWScim0Vkq4jc4H2oRNGmqpg6bQrODI/geN9+0+F4i93ErFA24YtIAsBjAK4HMA/ATSIyL2+1rwJ4TlWvBHAjgOVeB0oUB3V1goOHBvHfv3KP6VC8xW5iVnBTwr8KQLeq7lbVUQDPAliUt44CmJT5fTKAA96FSEShx25iVnDTS2cmgNz6ZS+Aq/PW+TqA50XkiwDGA7iu0AeJyGIAiwGglVU5onMko36TY3YTM86ri7Y3Afi+qs4CcAOAp0XkvM9W1RWq2q6q7ZwUiuhcv974FgBgcOAopkyZYjQWiiY3Cb8PwEU5z2dlluW6E8BzAKCqrwMYB2C6FwESxUWiIYGRsTGs++lqjBs3znQ4FEFuEv6bAOaIyMUi0gjnouzqvHX2AbgWAETkd+Ak/AEvAyWKg3QqjebmJtNhUESVTfiqmgSwFMBaANvh9MZ5V0QeEJGFmdW+DOAuEdkC4EcAblON+t2XiYjCxdXUCqq6BsCavGVfy/l9G4Df9zY0ovg4cXoIF1wwGUNnRjBhQoSmVCCrcKQtRV4Y5uzac6gfdSJ4b9suLPvmV02HQxHFhE+RFrZbO/Z092DixImmw6CIYsKnSOOcXURnMeFTpIVlzq7+o0dNh0AxwIRPkRaGObtUFVOnTsbw6CgG91t2JqJIYcKnSAvDnF0KoK6uDn19A/jG/f/VdDgUYUz4FGmcs4voLCZ8irRVq5wLtPv2Oc04y5bZl+w16pOmkTWY8CmywtIl8982vQUAOH7sBCZMmGA2GIo0JnxyLQwDmHLZ3CUzd1veuuhyrP7pOPzq2X+O1o3LyTqhTvhhS0Bh5ldp2c//oa1dMvO3Zf/BRvzPzg+hv/8PzQYWBiYO+iglGlU18pg/f77WYuVK1ZYWVeeQcR4tLc5y8l5b27nbOvtoa6v+M/3+H/oRs59xTZgwaDYw25k46C1MNAC6tMq8G9qEb+vBHFUihbe3SPWfWe3/cOVKZx0R52exY8/CY1VVi29LIG02MNuZOOgtTDS1JPzQNunYWl2PKj8GMFXzP6ykacnWLpnFttmsWeytU5KJgz5iiSa0CT8MIyijxI8BTNX8Dyu9ENvRAfT0AOm089N0sgcKb8v6hjE8+GDCTEBhYeKgj1iiCW3CD8MIyijxo7Rczf8wCgWuc7el4sKZSSxY9IIVJyOrmTjoo5Zoqm0LqvVRaxu+qvu2XLJXpf9DC5tUa7Lp/V26qWePfuWbj5gOJRxMHPSWJRrU0IYvzvuD197erl1dXUa+m8Ir24af26zT0mJH23ylVBWv7diB5nFN+IdHV2L5//5b0yFRCIjIRlVtr+a9oW3SoXiy9UJsNVQVLc3jcOjwUdy48I9Nh0Nesbjfvqt72hLZpKMjnAm+mFMnh3D1FfNMh0FeyK+CZruRAVbstCzhExF5xeb5PMCET2TM9j37AThNO3V1PBQjwfJuZNzLiAw5eOIYVBX/7/mXMWXKFNPhkBcs77fPhE9kiIhgaHgEyeNHTIdCXrG83z4TPhGRVyzvRsZeOkQGqCoaGxtMh0F+sLgbGUv4RAa833sAE8Y34/TQMP7gD64yHQ7FBBM+EYIfK3PyzDAA4MW1r+Dmm//C3y+jwiweIOUXNulQ7JkcK5NKJv39AirM8gFSfmEJn2LP8rEy5IeY/tOZ8Cn2gh4ro6oYSo5CVXH65Gl/voRKs3yAlF+Y8Cn2gh4rk1bFhPHNGBg8hoXXXu3Pl1Bplg+Q8gsTPsWeqbEyx46fwic/yR46Rlg+QMovTPgUe5aPlSE/xPSf7qqXjogsAPAIgASAf1DVBwus81kAXwegALao6s0exknkqyDHyuzqOxDMF1FpFg+Q8kvZEr6IJAA8BuB6APMA3CQi8/LWmQPgbwD8vqr+LoC/8j5UomjYd3gQAPDGixswadIkw9FQnLhp0rkKQLeq7lbVUQDPAliUt85dAB5T1aMAoKr93oZJFB11dYJTQ8MYOXrIdCgUM24S/kwA+3Oe92aW5ZoLYK6IvCoi6zNNQOcRkcUi0iUiXQMDA9VFTEREVfHqom09gDkA/gjATQCeFJEp+Sup6gpVbVfV9hkzZnj01UThUl/PAe5khpuE3wfgopznszLLcvUCWK2qY6q6B8BOOCcAIsrxft8BTJzQguHhEbTPv8J0OBQzbhL+mwDmiMjFItII4EYAq/PW+Tmc0j1EZDqcJp7d3oVJFA3HTjkja9etfQW33XaL4WgobsomfFVNAlgKYC2A7QCeU9V3ReQBEVmYWW0tgEER2QZgHYC/VtVBv4Km2qxaBUyf7nQ/FnF+j8FEgVaZ98arsZupkcxz1YavqmtUda6qXqKqyzLLvqaqqzO/q6p+SVXnqerlqvqsn0FT9VatAu64AxjMOR0PDgK3386cE5QLfv5zfPYXP3FmaFQ9O1Ojm39ADKf0Je9wpG3MdHYCo6PnLx8bi/xEgcZlJ0278OGH0Tg2du6LbmZqzE7pW82JgghM+LFTajLAiE8UaFxaFZMmjEfjgYOFVyj3D4jplL7kHSb8mCk1GWDEJwq0xtCMDxV+odw/IKZT+pJ3mPBjZtkyoLHx/OUNDZGfKNAaW+74PLS5+dyFbmZqjOmUvuQdJvyY6egAnnoKmDbt7LJp04DvfS9280gFbu8HzlQK+679dxh59NHKZ2qM6ZS+5B0m/Bjq6AAOH3au+6k6vzPZ+2/3IWeKqbc2bEHT7bcDPT1AOu38dPMPKDWlL3vvkAsc400UkLq6OpwaOoPhgYMQkeo+pNCUvjG9ITdVjiV8oqBVm+yLiVPvHdZkasKETxSQRMKnwy0uvXfCMg7B4pMSEz5RAHYd+ACTJo7HyMgYfnfeXG8/PC69d8JQk7H8pMSETxSAwRMnAAAv/uoV3HXXbd5+eFx674ShJmP5SYkJn8gjbmryp4+d8v6L43JD7jDUZCw/KTHhE3nAeE2+o6Pybp5hE4aajOUnJSZ8Ig+UqsmrKobGnBnrRscKzFxH7oShJmP5SYkJ3yWLL7yTBUrV5FPpNCZPmoDBoyfwySs8vmAbN7bXZCw/KXHglQsc10LltLY6+0Wh5Vn9/YP4888sPH8lipZCg+MswRK+C5ZfeCcLWF6TJwLAhO+K5RfePfXES7vw2q7D5yx7bddhPPHSLkMRhUOpmvwHR46aDo8IABO+K5ZfePfUx2dNxtJnNv8m6b+26zCWPrMZH5812XBk9ivWvPxeb5/zc+tONDU1GYsvtHgBzTNM+C7Eqbr+qUum49Gbr8TSZzbjW8/vwNJnNuPRm6/Epy6Zbjq00EokEjh5+gxOf7C/+knT4sp4f9doYcJ3oZYL7/mFkyVL7C+sfOqS6bjl6lZ8+9fduOXqViZ7TyjqhIdbxXgBzVPcA12qpjdYocLJ44/bX1h5bddhrNywD/f8yUexcsO+89r0iXyVW0oq1PUJiOYFtAAw4fuoUOEkn22FlWyb/aM3X4kv/emlv2neYdKvzsjYGCZOcG5nKHVszikrv5RUTBQvoAWACd9HbgshNhVWtvYeP6fNPtumv7X3uOHIwumt7j1IJBLYvasXf3XvX5oOx35uSklRvYAWACZ8H7kthNhUWLn705ec12b/qUum4+5PX2IoonBLp9MAgE2vbsSHP/xhw9GEQKnSj4UjV8OGCd9HhXr35GNhhShHsdJPW5u90ymECBO+jwr17vnCF6ydZoN8MMzJ0ipTaR9o9tGvCOfS8ZnF02qQz9LpNCZOnoCxZBLHBwdMhxMO2YOls9Np3mltdZJ9oYOIk1xVjCX8ALAQEk8pVSTq6rCn5wCefOxB0+GER14f6CdmXVN4uo+n17GPfoWY8H3GgYKUTCY5wrYGRaf7eO/Nwm+wqdubZZjwfcaBgkS1KTrdB4p0Fbap25tlmPB9FqeZNulc6za+BQAYHR5FXR0PtVoUnO4jqpNc+dgGzL3QZ3GaadOtuFzTaGxuwmgyiV/99BdIJBKmwwm1gtN9WH53qar43AbMhO+zQoUQEef/GOVkV0zcrmkMD4+ibRYnn6tFyek+bL/lYaV8bgNmwq9QpaXT3EII4CT77BQhUU92hZTan+NS8qfKxGq6D7/bgFW17APAAgA7AHQDuK/Eep8BoADay31mY+N8FVFta1NduVJDYeVK1ZYWVSdlO4+WFvfxt7Wd+97so63Nz6jtIlJ4G2S3ZbXb1jYjo6P6Rne3vvjOu3rf39xvOhwKCxdJAkCXusjbhR5lS/gikgDwGIDrAcwDcJOIzCuw3kQA9wLY4OZEMzoavip9rbUtXsAtfe0iSr2ZNnfvRn19Avv3H8Tdf3mH6XDKW7IEqK93qqD19c5zCp7PF6LdNOlcBaBbVXer6iiAZwEsKrDeNwA8BGC40iDCcmDXmrB5Adfd/EK5wnoyTGYmTXvthdfQ1mb5P3jJEudGDamU8zyVcp4z6QfP5wvRbhL+TAD7c573Zpb9hoh8AsBFqvqvpT5IRBaLSJeIdOW/FoYDu9aEHdVeZJXI35/LdV6J08nQmBUrKltO/vLxQnTNF21FpA7AtwB8udy6qrpCVdtVtT3/tTAc2LUm7Cj2IqtG7v6cKQgXFOaT4VgyaToE97Ile7fLKbTcJPw+ABflPJ+VWZY1EcDHALwoIj0ArgGwWkTOS+rFhOXA9iJhR60XWa2KnegTifCeDNPpNCZMbEEylcKpo0dNh1NesWoWxw5EjpuE/yaAOSJysYg0ArgRwOrsi6p6XFWnq+psVZ0NYD2Ahap6XrNNrsbGcJZymbC9VazW9IMf+Ldt/e7+mUqnkUgksKfnAP7x8Ye8/XA/ZGeYdLucQqtswlfVJIClANYC2A7gOVV9V0QeEJGF1X7x5ZczaVLwzVxBDvwaHRlFfX0IZiBfvty5UUO2RJ9IOM+XLzcbV9RYMNBEtNSNgn3U3t6uXV0lKwFEnps920ny+dranMKHF8aSSbzT14t3t+1Cx4I/4UyZdP7c/YBTwrn77opPrCKysdB1UDc40pZiJYixEC9vfhsAMDYaogu35K9Cg3hUgSeeCLSkz4RPsRLEWIj65kaMJZN44WdrWLonR7EShWqgg5CY8ClWghoLMTQ8go+0zvD2Qym8SpUoAhyExIRPscKxEGTEsmXODldIgIOQmPApdvzsWjs2lkRzc5N3H0jR0NHhXKDNT/oBD0Jiwify0Mb3d6Ghvh4HDxxGR8dnTYdDNlm+HHj6aaPVSyZ8Ig+NpZyeOS+ueRGXXjrXcDQBs6CfufUMj9wMwagQojAyM77FmPx+5tkRbQAvkFiEJXwiD6XTMUv0WT7fmo+8wYRP5JFUKoWWic1IplIYOhmx2++Va66x/e4+bG4CwIRP5JmxdBr1iQR69h7Ed78dgulf3XIzAVGtI9qqSchu3xPkBEq2q/beiLU+5s+fX9u9H4ksc2Z0VDf17NEf/OJ506F4y83NmGu54XM17y30nuwNk/NvlB2xm0nDz3vaElHMuWmuqWVEWzXt/8XmpgHOL8EH0dwUkiYjJnz6jZDss9Z6fes2AEAqVeI2XmHktrmm2i6H1STkcsk694Th9wRKIWoyYsInAKHaZ61V11SPsWQKv/6/vzQdirf8noComoQ8dWr5z82eFPyOP0Q9lJjwCUCo9ll7CXD6zDAunX2h6Ui85fcERH4l5OwJw+/4be+hlIMDrwhAqPZZMqGjw78BVNnP7ex0drjWVifZl/q+I0dKf2b+CcPP+FtbC99VJ8BJ0dxiCZ8ABDNPfJiVu76RSqfRPM6HSdPCfmHFbfyVtv+X2jGrKcHXsp2DmnPbC9V276n1wW6ZdqmlV13Uudk2r7/7nm7q2aM/fuFl3br17eC+2GZ+xu/lZ3vxWStXOt08Rc7vFuox1NAtkwk/pgrtnwHus8ZV8re66cb90pa3dVPPHr3rnr/1Lsiw9x/3O36vdtiQbedaEj6bdHxka228WI8cwOhEfufE5+d2q7RHUkXXN7ycSifsF1b8iD935+jsdJpNat1ha43T1gO9kGrPFLU+ol7Ct7k2bnOBJojtVunfX1EJ/4ss4f+G1/H7tXPUEqeBAx1s0rGPzcdqdgR6/kPEdGTBbLdK/343x/S/bd6im3r26J33fNW7QG0uNbjhdfx+7Ry1xGngQGfCt1Dck2q1gthu1fz9pZqLh0dGdFPPHn1h69v66KNPehdouS8OAy/j93PnqDZOAwc6E76FbE6qNhccg9huXv/9Q5mE/71/WetdkHQ+Gw+qkJXwedHWJzZ3zfV74GEtgthuNv/9VIKNB5WNMZVS7Zmi1sf8+fNDX1stJ+p/n1+C2m5efc/AsRMs4QfFxoMq4JhQQwlfnPcH7yMfaddDh7rOmb+lpYUlLQpG/i1Yger3vxc2b8HUqZOx5l/WoXPp7d4GSpRHRDaqans17zXWpNPXx8m6yBwvJ4traGrAiVNDGOnv8yY4Ip8YS/ijo4WXZ8c6hGksA4WP12OC0uk0xrc0Vx8QUQCMJfzGxsLLW1s5N3sYhe0E7eVkcVJbKESBMZbwZ84sfnGbc7OHSxhP0F51rth7aAATxrcgmUph2vRp3gVYTNjOrGQVYwl/6tTiXePCPoVI3ITxBO1V18yegx8AAF598Q3cccetPkSaI4xnVrKKsV467e3t2tXVVfC12bML30+grc2ZI4nsUlfn5J98Is68VlH20ltvY9IFE/Gdh5/EU4/63PeaBwYhgF46IrJARHaISLeI3Ffg9S+JyDYR2SoiL4hIWzXBZIVtLEMpcaiB8+YpAWHVl2pUNuGLSALAYwCuBzAPwE0iMi9vtc0A2lX14wB+AuB/1RJUVEZCxqUGHqUTdKWSGmAVhmdWcyJScnNTwr8KQLeq7lbVUQDPAliUu4KqrlPVbCvuegCzag2s0jue2SiMbdvViMoJulJnRkcxdepkHD95Gpe2fsj/L4zzmdWkCJXc3CT8mQD25zzvzSwr5k4Avyz0gogsFpEuEekaGBhwH2VIxakGXssJOqyFp3TmwsWeXb34yn+71/8vjOuZNWj5O+S990am5OZpLx0RuQVAO4CHC72uqitUtV1V22fMmOHlV1uJNfDy3BaewnpS8FwUqr42K7RDDg4WXrfWkpuBndpNwu8DcFHO81mZZecQkesAdAJYqKoj3oQXbqyBl+em2WvJEuBzn7OvRj2WTJoNgLxXaIcsppaSm6FmIjcJ/00Ac0TkYhFpBHAjgNW5K4jIlQC+CyfZ93sfZjj5WQOPSom3XLPXqlXAE0+c3+3Thhp1185uAMChg9zlI8Ntqb3WkpuhC3xlE76qJgEsBbAWwHYAz6nquyLygIgszKz2MIAJAP5JRN4SkdVFPi52/KiBR+gaUtlmr87Own38AfPXQpqaGnHy9BBO7NtlNhDyTrEdcto0b0tuhcZTlFruEVdt+Kq6RlXnquolqross+xrqro68/t1qvpbqvp7mcfC0p9ItfCrcGCi1lCu2atUUrfhWkgymcL0IKZUoGAU2yEfecTbklsiUdlyj/COVwHwOpH60fvHVK2hXLNXsaQuYsG1EM6aFj1B9YRKpSpb7hErp1aIEi9vtJHlxwh7W0ftF9p+IsDddwPLl5uLq+/wIPpPn8Tho8ext2sTPv953viEKlDDARfKG6DEhR/NL370/rF1zEChAtfTT5tN9gDwfu8BAMD6Vzbh9tt9njSNosdQFz4mfJ/5kUi9rnWuWuU0NxViQzu5zV3Pd72zAwmf210pggwNomPC95lfg6+8SoLZJpNCTYccM0Dko0IHsc89J5jwfWb74Kti40wSCY7aLyWZdnFxLSqDJSgYAfScYML3me3TnxRrWkqn7YnRNmdGRjBt2hScPD2E2b89tfBKURosQcEIYDAWe+nEnK29c2x26swZvN9/CJs3v4c7/sOCwitxw1KlXN5JiL10qGq2NznZTFGisGRrtyeyVwCzLTLhx5ztTU42SrmpFXOqVKpUAKUvJnyyutujjd54bycA4Ojho8VXuuGGypYTBVD6qvfsk4hiorm5CSeHzmBg53vFV1qzprLlRICT3H0scbGET1SFsdEkLmr9cPEV2IZPFmLCJ/ID2/DJQkz4RBX44MgxjG9pRlrTaG4eV3xFdn8iCzHhE1Vg+16nSabr9S249dYSba3s/kQWYsInqkC2Q+a2jVvR0NBQemV2fyLLMOETEcUEEz5RBVKaLr8SkaWY8IlcOjMygmlTJ+P0mWH89vSJpsMhqhgTPpFLY6kURATvbd+Dhx+833Q4RBVjwq8SpzqPL02bmWGWqFZM+FXgVOfxxDRfIZaKrMOEX4UA7lNAFnpj+w4AwIljJwxHEgIsFVmJCb8KbqdJYQEnWlrGN+PUmWH0vbPFdCj2Y6nISkz4VXAzTUqtBRyeLOw0MjKKyy77qOkw7MfJ46zEhF8FN9Ok1FLAYW2YQo+Tx1mJCb8KbqZJqaWAw9qwfdLpNBrqefsI1zh5nJWY8KtUbpqUWgo4rA3bZ8P2nWhoqMeJ46dw9dVV3T86Xjh5nJWY8H1SSwGHtWH7nBkdBQD87Ic/wxW/d4XhaEKCk8dZhwnfJ7UUcFgbtlc6nTIdAlHV2Cjpo2pvT5l9T2en04zT2uokexaQDFIOu6LwY8K3lM/3MqYKpNJpjJ/YglQqjXRyzHQ4RFVjkw7FUiXjHE4OD6OxsQH7DxzC//jqPUGFSOQ5JnyKnWrHOfR/MIi5c+cEEySRD1wlfBFZICI7RKRbRO4r8HqTiPw48/oGEZnteaREHuE4B4qrsglfRBIAHgNwPYB5AG4SkXl5q90J4KiqfhTA/wHwkNeBEnml0nEOO3r2+xcMUYDclPCvAtCtqrtVdRTAswAW5a2zCMAPMr//BMC1IiLehUnknUrHORw9fQpDwyPY9MoG/4IiCoCbXjozAeQWcXoBXF1sHVVNishxANMAHM5dSUQWA1iceToiIu9UE3QETUfetoqxALbF9KlAaxsgOQUeTe/du2+vyOEjpd654pFv+hvaubhfnMVtcdal1b4x0G6ZqroCwAoAEJEuVeUYdXBb5OK2OIvb4ixui7NEpKva97pp0ukDcFHO81mZZQXXEZF6AJMBDFYbFBERec9Nwn8TwBwRuVhEGgHcCGB13jqrAfynzO9/DuDXqhyaSERkk7JNOpk2+aUA1gJIAHhKVd8VkQcAdKnqagD/COBpEekGcATOSaGcFTXEHTXcFmdxW5zFbXEWt8VZVW8LYUGciCgeONKWiCgmmPCJiGLC94TPaRnOcrEtviQi20Rkq4i8ICJtJuIMQrltkbPeZ0RERSSyXfLcbAsR+Wxm33hXRJ4JOsaguDhGWkVknYhszhwnN5iI028i8pSI9BcbqySOb2e201YR+YSrD1ZV3x5wLvLuAvARAI0AtgCYl7fOEgBPZH6/EcCP/YzJ1MPltvhjAC2Z378Q522RWW8igJcBrAfQbjpug/vFHACbAVyQef4h03Eb3BYrAHwh8/s8AD2m4/ZpW/whgE8AeKfI6zcA+CUAAXANgA1uPtfvEj6nZTir7LZQ1XWqmp3Waz2cMQ9R5Ga/AIBvwJmXaTjI4ALmZlvcBeAxVT0KAKraH3CMQXGzLRTApMzvkwEcCDC+wKjqy3B6PBazCMAP1bEewBQRubDc5/qd8AtNyzCz2DqqmgSQnZYhatxsi1x3wjmDR1HZbZGpol6kqv8aZGAGuNkv5gKYKyKvish6EVkQWHTBcrMtvg7gFhHpBbAGwBeDCc06leYTALzjlZVE5BYA7QA+bToWE0SkDsC3ANxmOBRb1MNp1vkjOLW+l0XkclU9ZjIoQ24C8H1V/XsR+SSc8T8fU9W06cDCwO8SPqdlOMvNtoCIXAegE8BCVR0JKLagldsWEwF8DMCLItIDp41ydUQv3LrZL3oBrFbVMVXdA2AnnBNA1LjZFncCeA4AVPV1AOPgTKwWN67yST6/Ez6nZTir7LYQkSsBfBdOso9qOy1QZluo6nFVna6qs1V1NpzrGQtVtepJoyzm5hj5OZzSPURkOpwmnt0BxhgUN9tiH4BrAUBEfgdOwh8INEo7rAZwa6a3zjUAjqvqwXJv8rVJR/2bliF0XG6LhwFMAPBPmevW+1R1obGgfeJyW8SCy22xFsCfisg2ACkAf62qkasFu9wWXwbwpIj8FzgXcG+LYgFRRH4E5yQ/PXO94n4ADQCgqk/AuX5xA4BuAEMAbnf1uRHcVkREVABH2hIRxQQTPhFRTDDhExHFBBM+EVFMMOETEcUEEz4RUUww4RMRxcT/BxExovaUdYjmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "xs, ys = np.meshgrid(np.linspace(0, 1, 500), np.linspace(0, 1, 500))\n",
    "\n",
    "Xb = np.vstack((xs.flat, ys.flat)).T\n",
    "\n",
    "# find classes from the mus\n",
    "edists = np.sum((Xb - mus[:,np.newaxis,:])**2, axis=2)\n",
    "cs = np.argmin(edists, axis=0)\n",
    "\n",
    "# plot the boundary\n",
    "plt.clf()\n",
    "plt.contour(xs, ys, cs.reshape(xs.shape), cmap=plt.cm.bone)\n",
    "plt.plot(X[classes==0, 0], X[classes==0, 1], 'or')\n",
    "plt.plot(X[classes==1, 0], X[classes==1, 1], 'ob')\n",
    "plt.plot(mus[:, 0], mus[:, 1], 'x')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "# Gaussian Mixture\n",
    "\n",
    "Previously we learned K-means clustering algorithm with linear border for each classes.  \n",
    "We observed that near the border, there are vague assignment. \n",
    "How confident are you on the assignments, especially near the border in K-means? \n",
    "\n",
    "With probabilistic model, we can softly assign clusters with a certain probability. \n",
    "For this, we can assume each cluster is Gaussian distribution:\n",
    "\n",
    "$$p(\\xv_k) \\sim N(\\xv_k \\mid \\muv_k, \\Sigmav_k).$$\n",
    "\n",
    "For the entire data $\\xv \\in \\{\\xv_1, \\cdots, \\xv_K \\}$, \n",
    "the Gaussian mixture distribution can be written as \n",
    "\n",
    "\n",
    "$$p(\\xv) = \\sum_{k=1}^K \\pi_k N(\\xv \\mid \\muv_k, \\Sigmav_k).$$\n",
    "\n",
    "\n",
    "Here, we assume the latent indicator variable $\\zv$, which satisfys $z_k \\in \\{0, 1\\}$ and $\\sum_k z_k = 1$. \n",
    "In the above Gaussian mixture model, $\\pi_k$ is called as a *mixing coefficient*, which is the marginal distribution over $\\zv$, \n",
    "\n",
    "$$\n",
    "p(z_k = 1) = \\pi_k,\n",
    "$$\n",
    "\n",
    "such that \n",
    "\n",
    "\n",
    "$$\n",
    "0 \\le \\pi_k \\le 1, \n",
    "$$\n",
    "\n",
    "and \n",
    "\n",
    "$$\n",
    "\\sum_{k=1}^{K} \\pi_k = 1.\n",
    "$$\n",
    "\n",
    "Since $\\zv$ is a indicator variable, we can write\n",
    "\n",
    "$$\n",
    "p(\\zv) = \\prod_{k=1}^K \\pi_k^{z_k}.\n",
    "$$\n",
    "\n",
    "From our first assumption of Gaussian for each class, \n",
    "\n",
    "$$\n",
    "p(\\xv_k) = p(\\xv \\mid z_k = 1) = N(\\xv \\mid \\muv_k, \\Sigmav_k).\n",
    "$$\n",
    "\n",
    "Rewriting this with a vector notation,\n",
    "\n",
    "$$\n",
    "p(\\xv \\mid \\zv) = \\prod_{k=1}^K N(\\xv \\mid \\muv_k, \\Sigmav_k)^{z_k}.\n",
    "$$\n",
    "\n",
    "Marginalizing the joint distribution $p(\\xv, \\zv)$ over $\\zv$, \n",
    "\n",
    "$$\n",
    "p(\\xv) = \\sum_{\\zv} p(\\zv) p(\\xv \\mid \\zv) = \\sum_{k=1}^K \\pi_k N(\\xv \\mid \\muv_k, \\Sigmav_k). \n",
    "$$\n",
    "\n",
    "\n",
    "Remember that in logistic regression and discriminant analysis models, we are interested in the posterior probability\n",
    "$p(T=k \\mid \\xv)$. \n",
    "Similarly, we are interested in the probability of the classification from the observation $\\xv$, thus,\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "p(z_k = 1 \\mid \\xv) &= \\frac{p(\\xv \\mid z_k = 1) p(z_k= 1)}{p(\\xv)} \\\\\n",
    " \\\\\n",
    "                    &= \\frac{ \\pi_kN(\\xv \\mid \\muv_k, \\Sigmav_k)}{\\sum_{l=1}^K \\pi_l N(\\xv \\mid \\muv_l, \\Sigmav_l)}.\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "For concise notation, let us define $\\kappa(z_k) \\equiv p(z_k = 1 \\mid \\xv)$. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Likelihood\n",
    "\n",
    "Now, consider the batch data input $\\Xm$ with $N$ data samples and $D$ dimensional input for each. \n",
    "Here, the latent variables are now in matrix $\\Zm$ for $N$ samples and $K$ classes. \n",
    "\n",
    "From the assumption of i.i.d, we can write the joint distribution:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "p(\\Xm, \\zv \\mid \\pi, \\muv, \\Sigmav) &= \\prod_{n=1}^N p(\\xv_n, \\zv_n) \\\\\n",
    "                               &= \\prod_{n=1}^N p(\\xv_n \\mid \\zv_n) p(\\zv_n) \\\\\n",
    "                               &= \\prod_{n=1}^N \\prod_{k=1}^{K} \\Big[ \\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)\\Big]^{z_k} .\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "From the marginal distribution and the assumption of i.i.d, we can write the likelihood function:\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "p(\\Xm \\mid \\pi, \\muv, \\Sigmav) &= \\sum_\\zv p(\\Xm, \\zv \\mid \\pi, \\muv, \\Sigmav)\\\\\n",
    "                               &= \\prod_{n=1}^N \\sum_\\zv p(\\xv_n \\mid \\zv_n) p(\\zv_n) \\\\\n",
    "                               &= \\prod_{n=1}^N \\sum_\\zv \\prod_{k=1}^{K} \\Big[ \\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)\\Big]^{z_k} .\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "Applying the logarihtm, the log-likelihood is \n",
    "\n",
    "$$\n",
    "LL = \\ln p(\\Xm \\mid \\pi, \\muv, \\Sigmav) = \\sum_{n=1}^N \\ln \\Big[ \\sum_{k=1}^K\\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)  \\Big]. \n",
    "$$\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The parameters: $\\muv, \\Sigma, \\pi$\n",
    "\n",
    "Now, let us find the maximum of the log-likelihood w.r.t $\\muv_k$, $\\Sigmav_k$, and $\\pi_k$. \n",
    "Before this, let's review the derivation of multivariate Gaussian distribution.\n",
    "\n",
    "### Derivation of Gaussian\n",
    "\n",
    "#### - w.r.t. $\\muv$,\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\frac{d}{d\\muv} N(\\xv; \\muv, \\Sigmav) &= \\frac{d}{d\\muv} \\Bigg[  \\frac{1}{ (2\\pi)^{\\frac{d}{2}} \\vert \\Sigmav \\vert^{\\frac{1}{2}}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Bigg] \\\\\n",
    "\\\\\n",
    "  &= \\Bigg[  \\frac{1}{ (2\\pi)^{\\frac{d}{2}} \\vert \\Sigmav \\vert^{\\frac{1}{2}}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Bigg] \\frac{d}{d\\muv}\\Bigg( - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) \\Bigg) \\\\\n",
    "  \\\\\n",
    "  &= \\Bigg[  \\frac{1}{ (2\\pi)^{\\frac{d}{2}} \\vert \\Sigmav \\vert^{\\frac{1}{2}}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Bigg] \\frac{d}{d\\muv}\\Bigg( - \\frac{1}{2} [ \\xv^\\top \\Sigmav^{-1} \\xv - 2 \\muv^\\top \\Sigmav^{-1} \\xv + \\muv^\\top \\Sigmav^{-1} \\muv ] \\Bigg) \\\\\n",
    "  \\\\\n",
    "  &= \\Bigg[  \\frac{1}{ (2\\pi)^{\\frac{d}{2}} \\vert \\Sigmav \\vert^{\\frac{1}{2}}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Bigg] \\Bigg( \\Sigmav^{-1} \\xv - \\Sigmav^{-1} \\muv \\Bigg) \\\\\n",
    "  \\\\\n",
    "  &= N(\\xv; \\muv, \\Sigmav) \\Sigmav^{-1} (\\xv - \\muv) \n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "#### -  w.r.t. $\\Sigmav$, \n",
    "$$\n",
    "\\begin{align*}\n",
    "\\frac{d}{d\\Sigmav} N(\\xv; \\muv, \\Sigmav) &= \\frac{d}{d\\Sigmav} \\Bigg[  (2\\pi)^{-\\frac{d}{2}} \\vert \\Sigmav \\vert^{-\\frac{1}{2}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Bigg] \\\\\n",
    "\\\\\n",
    "&= (2\\pi)^{-\\frac{d}{2}} \\frac{d}{d\\Sigmav} \\Bigg[  \\vert \\Sigmav \\vert^{-\\frac{1}{2}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Bigg] \\\\\n",
    "\\\\\n",
    "&= (2\\pi)^{-\\frac{d}{2}} \\Bigg[  \\frac{d \\vert \\Sigmav \\vert^{-\\frac{1}{2}}}{d\\Sigmav} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } + \\vert \\Sigmav \\vert^{-\\frac{1}{2}} \\frac{d  e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } }{d\\Sigmav}\\Bigg] \\\\\n",
    "\\\\\n",
    "&= (2\\pi)^{-\\frac{d}{2}} \\Bigg[ -\\frac{1}{2} \\vert \\Sigmav \\vert^{-\\frac{3}{2}} \\frac{d \\vert \\Sigmav \\vert }{d\\Sigmav} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } + \\vert \\Sigmav \\vert^{-\\frac{1}{2}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\frac{d \\big( - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) \\big)}{d\\Sigmav} \\Bigg] \\\\\n",
    "\\\\\n",
    "&= (2\\pi)^{-\\frac{d}{2}} \\Bigg[  -\\frac{1}{2} \\vert \\Sigmav \\vert^{-\\frac{3}{2}} \\vert \\Sigmav \\vert \\Sigmav^{-1} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } -\\frac{1}{2} \n",
    "\\vert \\Sigmav \\vert^{-\\frac{1}{2}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } (- \\Sigmav^{-1} (\\xv - \\muv) (\\xv - \\muv)^\\top  \\Sigmav^{-1} ) \\Bigg] \\\\\n",
    "\\\\\n",
    "&= -\\frac{1}{2} (2\\pi)^{-\\frac{d}{2}} \\Bigg[ \\vert \\Sigmav \\vert^{-\\frac{1}{2}} \\Sigmav^{-1} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } - \\vert \\Sigmav \\vert^{-\\frac{1}{2}} e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Sigmav^{-1} (\\xv - \\muv) (\\xv - \\muv)^\\top  \\Sigmav^{-1} \\Bigg] \\\\\n",
    "\\\\\n",
    "&= -\\frac{1}{2} (2\\pi)^{-\\frac{d}{2}} \\vert \\Sigmav \\vert^{-\\frac{1}{2}}  e^{ - \\frac{1}{2} (\\xv - \\muv)^\\top \\Sigmav^{-1} (\\xv - \\muv) } \\Sigmav^{-1} \\Bigg[ \\I - (\\xv - \\muv) (\\xv - \\muv)^\\top  \\Sigmav^{-1} \\Bigg] \\\\\n",
    "\\\\\n",
    "&= -\\frac{1}{2} N(\\xv; \\muv, \\Sigmav) \\Sigmav^{-1} \\Bigg[ \\I - (\\xv - \\muv) (\\xv - \\muv)^\\top  \\Sigmav^{-1} \\Bigg]\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "<hr/>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, back to the derivation of the log-likelihood: \n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\frac{\\partial LL}{\\partial \\muv_k} &= \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)}  \\nabla_{\\muv_k}  \\Big[ \\sum_{k=1}^K\\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)  \\Big] \\\\\n",
    "\\\\\n",
    "                    &=  \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)} \\nabla_{\\muv_k}  \\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k) \\\\\n",
    "                    \\\\\n",
    "                     &=  \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)} \\pi_kN(\\xv_n; \\muv_k, \\Sigmav_k) \\Sigmav_k^{-1} (\\xv_n - \\muv_k)\n",
    "                     \\\\\n",
    "                     &=  \\sum_{n=1}^N \\frac{\\pi_k N(\\xv_n; \\muv_k, \\Sigmav_k)}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)}  \\Sigmav_k^{-1} (\\xv_n - \\muv_k) \\\\\n",
    "                     \\\\\n",
    "                     &= \\sum_{n=1}^N \\kappa(z_k) \\Sigmav_k^{-1} (\\xv_n - \\muv_k)\n",
    "\\end{align*}                 \n",
    "$$\n",
    "\n",
    "Setting this to zero, \n",
    "\n",
    "$$\n",
    "\\sum_{n=1}^N \\kappa(z_k) \\Sigmav_k^{-1} (\\xv_n - \\muv_k) = 0 \\\\ \n",
    "\\sum_{n=1}^N \\kappa(z_k) (\\xv_n - \\muv_k) = 0 \\\\\n",
    "\\sum_{n=1}^N \\kappa(z_k) \\xv_n =  \\sum_{n=1}^N \\kappa(z_k) \\muv_k \\\\\n",
    "\\sum_{n=1}^N \\kappa(z_k) \\xv_n =  N_k \\muv_k \\\\\n",
    "\\\\\n",
    "\\muv_k = \\frac{1}{N_k} \\sum_{n=1}^N \\kappa(z_k) \\xv_n,\n",
    "$$\n",
    "\n",
    "where the number of samples for class $k$, $N_k = \\sum_{n=1}^{N} \\kappa(z_k)$.\n",
    "\n",
    "Now, w.r.t $\\Sigmav_k$, \n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "\\frac{\\partial LL}{\\partial \\Sigmav_k} &= \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)}  \\nabla_{\\Sigmav_k}  \\Big[ \\sum_{k=1}^K\\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)  \\Big] \\\\\n",
    "\\\\\n",
    "                    &=  \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)} \\nabla_{\\Sigmav_k}  \\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k) \\\\\n",
    "                    \\\\\n",
    "                     &=  \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)} \\pi_kN(\\xv_n; \\muv_k, \\Sigmav_k) \\Sigmav_k^{-1} (\\I - (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top \\Sigmav_k^{-1} ) \\\\\n",
    "                     \\\\\n",
    "                     &=  \\sum_{n=1}^N \\frac{\\pi_k N(\\xv_n; \\muv_k, \\Sigmav_k)}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)}  \\Sigmav_k^{-1}  (\\I - (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top \\Sigmav_k^{-1} ) \\\\\n",
    "                     \\\\\n",
    "                     &= \\sum_{n=1}^N \\kappa(z_k) \\Sigmav_k^{-1}  (\\I - (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top \\Sigmav_k^{-1} )\n",
    "\\end{align*}                 \n",
    "$$\n",
    "\n",
    "Setting the last one to zero, and multiplying $\\Sigma$ on both side of equal, \n",
    "\n",
    "$$\n",
    "\\sum_{n=1}^N \\kappa(z_k) (\\I - (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top \\Sigmav_k^{-1} ) = 0 \\\\\n",
    "\\sum_{n=1}^N \\kappa(z_k) \\I  = \\sum_{n=1}^N \\kappa(z_k) (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top \\Sigmav_k^{-1} \\\\\n",
    "N_k \\Sigmav_k = \\sum_{n=1}^N \\kappa(z_k) (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top \\\\\n",
    "\\Sigmav_k = \\frac{1}{N_k} \\sum_{n=1}^N \\kappa(z_k) (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top\n",
    "$$\n",
    "\n",
    "\n",
    "w.r.t $\\pi_k$, \n",
    "$$\n",
    "\\begin{align*}\n",
    "\\frac{\\partial LL}{\\partial \\pi_k} &= \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)}  \\nabla_{\\pi_k}  \\Big[ \\sum_{k=1}^K\\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)  \\Big] \\\\\n",
    "\\\\\n",
    "                    &=  \\sum_{n=1}^N \\frac{1}{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)} \\nabla_{\\pi_k}  \\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k) \\\\\n",
    "                    \\\\\n",
    "                     &= \\frac{1}{\\pi_k} \\sum_{n=1}^N \\frac{\\pi_k N(\\xv_n; \\muv_k, \\Sigmav_k) }{\\sum_{l=1}^K\\pi_l N(\\xv_n \\mid \\muv_l, \\Sigmav_l)} \\\\\n",
    "                     \\\\\n",
    "                     &= \\frac{1}{\\pi_k} \\sum_{n=1}^N \\kappa(z_k) \n",
    "\\end{align*}                 \n",
    "$$\n",
    "\n",
    "Setting this to zero does not give any info but zero. What did we do wrong? \n",
    "\n",
    "Here, let us consider the constraint $\\sum_k \\pi_k = 1$. \n",
    "Adding this constraint with a Lagrange multiplier, \n",
    "\n",
    "$$\n",
    "LL + \\lambda (\\sum_k \\pi_k - 1) = \\sum_{n=1}^N \\ln \\Big[ \\sum_{k=1}^K\\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)  \\Big] + \\lambda (\\sum_k \\pi_k - 1)\n",
    "$$\n",
    "\n",
    "Computing the derivative and setting it to zero, \n",
    "\n",
    "$$\n",
    "\\frac{1}{\\pi_k} \\sum_{n=1}^N \\kappa(z_k) + \\lambda = 0 \\\\\n",
    "\\pi_k = -\\frac{N_k}{\\lambda} \n",
    "$$\n",
    "\n",
    "From the constraint, $\\sum_k \\pi_k = 1$, we can see that \n",
    "\n",
    "$$\n",
    "\\sum_k \\pi_k = - \\sum_k \\frac{N_k}{\\lambda} = -\\frac{1}{\\lambda} \\sum_k N_k = -\\frac{N}{\\lambda} = 1 \\\\\n",
    "\\\\\n",
    "\\Rightarrow \\lambda = -N\n",
    "$$\n",
    "\n",
    "Thus, \n",
    "\n",
    "$$\n",
    "\\pi_k = \\frac{N_k}{N} \n",
    "$$\n",
    "\n",
    "Collecting all the updates, \n",
    "\n",
    "- $\\muv_k = \\frac{1}{N_k} \\sum_{n=1}^N \\kappa(z_k) \\xv_n$\n",
    "<br/><br/>\n",
    "- $\\Sigmav_k = \\frac{1}{N_k} \\sum_{n=1}^N \\kappa(z_k) (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top$\n",
    "<br/><br/>\n",
    "- $\\pi_k = \\frac{N_k}{N} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Algorithm\n",
    "\n",
    "- Initialize $\\muv_k$, $\\Sigmav_k$, and $\\pi_k$. \n",
    "\n",
    "- **Expection Step**: evaluate the responsibilities $\\kappa(z_{nk})$, \n",
    "\n",
    "$$\n",
    "\\kappa(z_{nk}) = \\frac{ \\pi_kN(\\xv \\mid \\muv_k, \\Sigmav_k)}{\\sum_{l=1}^K \\pi_l N(\\xv \\mid \\muv_l, \\Sigmav_l)}. \n",
    "$$\n",
    "\n",
    "- **Maximization Step**: re-estimate $\\muv_k$, $\\Sigmav_k$, and $\\pi_k$,\n",
    "\n",
    "  - $\\muv_k = \\frac{1}{N_k} \\sum_{n=1}^N \\kappa(z_k) \\xv_n$,\n",
    "<br/><br/>\n",
    "  - $\\Sigmav_k = \\frac{1}{N_k} \\sum_{n=1}^N \\kappa(z_k) (\\xv_n - \\muv_k)(\\xv_n - \\muv_k)^\\top$,\n",
    "<br/><br/>\n",
    "  - $\\pi_k = \\frac{N_k}{N} $,\n",
    "  \n",
    "  with \n",
    "  \n",
    "  - $N_k = \\sum_{n=1}^{N} \\kappa(z_k)$. \n",
    "  \n",
    "- Compute the log-likelihood and check convergence: \n",
    "\n",
    "  $$\n",
    "LL = \\ln p(\\Xm \\mid \\pi, \\muv, \\Sigmav) = \\sum_{n=1}^N \\ln \\Big[ \\sum_{k=1}^K\\pi_k N(\\xv_n \\mid \\muv_k, \\Sigmav_k)  \\Big]. \n",
    "$$\n",
    "\n",
    "- If not converged, repeat EM steps. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr/>\n",
    "\n",
    "## Normal PDF\n",
    "\n",
    "Before starting to write the EM algorithm for GMM, let us review our multivariate normal pdf!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# multinomial pdf\n",
    "def normal_dist(X, mu, Sigma):\n",
    "    \"\"\" multinomial distribution probability density function\n",
    "    \n",
    "        parameters\n",
    "        -----------\n",
    "        X        ndarray (N x D)\n",
    "                 input data\n",
    "        mu       ndarray (D x 1)\n",
    "                 mean vector\n",
    "        Sigma    ndarray (D x D)\n",
    "                 covariance matrix\n",
    "        \n",
    "        return\n",
    "        ------\n",
    "        @pdf_evals  ndarray (N x 1)\n",
    "    \"\"\"\n",
    "    N, D = X.shape\n",
    "    SigmaDet = Sigma if D == 1 else np.linalg.det(Sigma)\n",
    "    try:\n",
    "        SigmaInv = 1. / Sigma if D == 1 else np.linalg.inv(Sigma)\n",
    "    except LinAlgError:\n",
    "        raise np.linalg.LinAlgError('normalD: failed to compute inverse of Sigma')\n",
    "    \n",
    "    scale = 1. / np.sqrt((2 * np.pi)**D * SigmaDet)\n",
    "    shiftedX = X - mu\n",
    "    return scale * np.exp(-0.5 * np.sum(shiftedX @ SigmaInv * shiftedX, axis=1, keepdims=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.04122038],\n",
       "       [0.03977096],\n",
       "       [0.10178436],\n",
       "       [0.02839527],\n",
       "       [0.03653966],\n",
       "       [0.11047418],\n",
       "       [0.08474013],\n",
       "       [0.15204393],\n",
       "       [0.13118574],\n",
       "       [0.01222199]])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mu = np.array([1, 1])\n",
    "cov = np.eye(2)\n",
    "\n",
    "X = np.random.multivariate_normal(mu, cov, 10)\n",
    "\n",
    "normal_dist(X, mu, cov)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PDF for Mixture Model\n",
    "\n",
    "Now, let's consider multiple means for mixture model. \n",
    "\n",
    "For this, we review the Python *map* function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[0, 10], [1, 11], [2, 12], [3, 13], [4, 14]]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "a = np.arange(5)\n",
    "b = np.arange(5) + 10\n",
    "\n",
    "list(map(list, zip(a,b)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 10\n",
      "1 11\n",
      "2 12\n",
      "3 13\n",
      "4 14\n"
     ]
    }
   ],
   "source": [
    "#list(map(lambda a,b: a+b, zip(a,b)))\n",
    "\n",
    "for i,j in zip(a,b):\n",
    "    print(i, j)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[10, 12, 14, 16, 18]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map(lambda t: t[0]+t[1], zip(a,b)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" Revised multinomial PDF function for mixture models\n",
    "\"\"\"\n",
    "\n",
    "from numpy.linalg import LinAlgError\n",
    "\n",
    "# multinomial pdf\n",
    "def mixnorm_dist(X, mus, Sigmas):\n",
    "    \"\"\" multinomial distribution probability density function\n",
    "    \n",
    "        parameters\n",
    "        -----------\n",
    "        X        ndarray (N x D)\n",
    "                 input data\n",
    "        mu       list of ndarray (D x 1)\n",
    "                 mean vectors\n",
    "        Sigma    list of ndarray (D x D)\n",
    "                 covariance matrices\n",
    "        \n",
    "        return\n",
    "        ------\n",
    "        @pdf_evals  ndarray (N x K)  for K = len(mus)\n",
    "    \"\"\"\n",
    "    N, D = X.shape\n",
    "    \n",
    "    SigmaDets = np.fromiter(map(lambda S: S if D == 1 else np.linalg.det(S), Sigmas), dtype=np.float)\n",
    "    try:\n",
    "        SigmaInvs = np.array(list(map(lambda S: 1. / S if D == 1 else np.linalg.inv(S), Sigmas)))\n",
    "    except LinAlgError:\n",
    "        raise LinAlgError('normalD: failed to compute inverse of Sigma')\n",
    "    \n",
    "    scale = 1. / np.sqrt((2 * np.pi)**D * SigmaDets)\n",
    "    shiftedX = X - np.asarray(mus)[:, np.newaxis, :]\n",
    "    \n",
    "    #quad = np.array(list(map(lambda V: np.sum(V[0] @ V[1] * V[1], axis=1, keepdims=True), zip(shiftedX, SigmaInvs))))\n",
    "    quad = np.array(list(map(lambda V: np.sum(V[0] @ V[1] * V[0], axis=1, keepdims=True), zip(shiftedX, SigmaInvs))))\n",
    "    quad = np.hstack(quad)\n",
    "    return scale * np.exp(-0.5 * quad)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.07835631e-01, 2.93791529e-02],\n",
       "       [7.68274825e-02, 4.91634722e-02],\n",
       "       [1.06453184e-01, 6.09266442e-02],\n",
       "       [8.33220649e-02, 8.53882996e-02],\n",
       "       [1.47285668e-01, 2.31599621e-02],\n",
       "       [3.51973946e-02, 1.45651397e-04],\n",
       "       [1.04178359e-01, 1.11761426e-03],\n",
       "       [1.50564530e-01, 5.90979996e-03],\n",
       "       [5.64125767e-02, 6.69194225e-02],\n",
       "       [7.14233576e-02, 1.25091483e-03],\n",
       "       [2.70116290e-03, 9.61648968e-02],\n",
       "       [8.77509252e-04, 9.73298159e-02],\n",
       "       [1.46341113e-02, 1.58839245e-01],\n",
       "       [1.21425938e-03, 6.68724467e-02],\n",
       "       [8.50389488e-04, 1.62034602e-02],\n",
       "       [3.09325433e-02, 7.28799892e-02],\n",
       "       [1.99299807e-02, 7.89141247e-02],\n",
       "       [3.61881033e-02, 1.16415601e-01],\n",
       "       [1.54148216e-01, 1.62336354e-02],\n",
       "       [5.60663166e-02, 1.12507130e-01]])"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mu1 = np.array([1, 1])\n",
    "mu2 = np.array([3, 2])\n",
    "cov = np.eye(2)\n",
    "\n",
    "X = np.vstack((np.random.multivariate_normal(mu1, cov, 10),\n",
    "               np.random.multivariate_normal(mu2, cov, 10)))\n",
    "\n",
    "mixnorm_dist(X, [mu1, mu2], [cov, cov])\n",
    "#X[:, :] - np.asarray([mu1, mu2])[:, np.newaxis, :]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.10783563]\n",
      " [0.07682748]\n",
      " [0.10645318]\n",
      " [0.08332206]\n",
      " [0.14728567]\n",
      " [0.03519739]\n",
      " [0.10417836]\n",
      " [0.15056453]\n",
      " [0.05641258]\n",
      " [0.07142336]\n",
      " [0.00270116]\n",
      " [0.00087751]\n",
      " [0.01463411]\n",
      " [0.00121426]\n",
      " [0.00085039]\n",
      " [0.03093254]\n",
      " [0.01992998]\n",
      " [0.0361881 ]\n",
      " [0.15414822]\n",
      " [0.05606632]]\n",
      "[[2.93791529e-02]\n",
      " [4.91634722e-02]\n",
      " [6.09266442e-02]\n",
      " [8.53882996e-02]\n",
      " [2.31599621e-02]\n",
      " [1.45651397e-04]\n",
      " [1.11761426e-03]\n",
      " [5.90979996e-03]\n",
      " [6.69194225e-02]\n",
      " [1.25091483e-03]\n",
      " [9.61648968e-02]\n",
      " [9.73298159e-02]\n",
      " [1.58839245e-01]\n",
      " [6.68724467e-02]\n",
      " [1.62034602e-02]\n",
      " [7.28799892e-02]\n",
      " [7.89141247e-02]\n",
      " [1.16415601e-01]\n",
      " [1.62336354e-02]\n",
      " [1.12507130e-01]]\n"
     ]
    }
   ],
   "source": [
    "# compare with the normalD\n",
    "print(normal_dist(X, mu1, cov))\n",
    "print(normal_dist(X, mu2, cov))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.7858893 , 0.2141107 ],\n",
       "       [0.6097857 , 0.3902143 ],\n",
       "       [0.63599769, 0.36400231],\n",
       "       [0.49387638, 0.50612362],\n",
       "       [0.86412112, 0.13587888],\n",
       "       [0.99587892, 0.00412108],\n",
       "       [0.98938597, 0.01061403],\n",
       "       [0.9622315 , 0.0377685 ],\n",
       "       [0.45740422, 0.54259578],\n",
       "       [0.98278738, 0.01721262],\n",
       "       [0.02732144, 0.97267856],\n",
       "       [0.00893527, 0.99106473],\n",
       "       [0.08435942, 0.91564058],\n",
       "       [0.01783402, 0.98216598],\n",
       "       [0.04986496, 0.95013504],\n",
       "       [0.29796541, 0.70203459],\n",
       "       [0.20163044, 0.79836956],\n",
       "       [0.23713778, 0.76286222],\n",
       "       [0.90472204, 0.09527796],\n",
       "       [0.33259281, 0.66740719]])"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "probs = mixnorm_dist(X, [mu1, mu2], [cov, cov])\n",
    "kappa = probs / np.sum(probs, axis=1, keepdims=True)\n",
    "kappa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classes = np.argmax(kappa, axis=1)\n",
    "classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.04461633, 0.77626776])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean1 = X[classes==0].mean(axis=0)\n",
    "mean1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.6522112 , 0.05388207],\n",
       "       [0.05388207, 0.39389779]])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cov1 = np.cov(X[classes==0].T)\n",
    "cov1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 9, 11])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.bincount(classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 1, 0, 3])"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.bincount([0,1,1,2,4,4,4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-68-e3836d91b5aa>, line 27)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-68-e3836d91b5aa>\"\u001b[0;36m, line \u001b[0;32m27\u001b[0m\n\u001b[0;31m    pi = # TODO- finish this\u001b[0m\n\u001b[0m                             ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "def GMM(X, K=2, maxiter=100):\n",
    "    N, D = X.shape\n",
    "    \n",
    "    # select initial centers - randomly for now, but YOU CAN USE K-MEANS for initial assignment\n",
    "    cidx = np.random.choice(N, K, replace=False)\n",
    "    mus = X[cidx, :]\n",
    "    Sigmas = [np.eye(D) for k in range(K)]  # uni-variance\n",
    "    pi = np.array([1/K] * K)\n",
    "    \n",
    "    LL = []  # LogLikelihood log - expected to monotonically increasing\n",
    "    b = []\n",
    "    \n",
    "    # repeat until convergence or up to maxiter\n",
    "    for step in range(maxiter):\n",
    "        \n",
    "        # Expectation\n",
    "        probs = mixnorm_dist(X, [mu1, mu2], [cov, cov]) # TODO- finish this \n",
    "        kappa = probs / np.sum(probs, axis=1, keepdims=True) # TODO- finish this \n",
    "        classes = np.argmax(kappa, axis=1)\n",
    "        \n",
    "        # Maximization\n",
    "        Nk = np.bincount(classes)\n",
    "        for i in range(K):\n",
    "            m_i = np.sum(self.X[:,c], axis=0)\n",
    "            mus[i, :] = (1/m_i)*np.sum(self.X*r_ic[:,c].reshape(len(self.X),1),axis=0) # TODO- finish this             \n",
    "            Sigmas[i][...] = np.dot((b[j].reshape(len(X),1) * (X - means[j])).T, (X - means[j])) / (np.sum(b[j])+eps) # TODO- finish this \n",
    "        pi = # TODO- finish this \n",
    "        \n",
    "        ll = np.sum(np.log(np.sum(probs * pi, axis=1)))\n",
    "        # convergence check: let us run w/o this for now.\n",
    "        #if len(LL) > 0 and np.abs(ll - LL[-1]) < np.finfo(float).eps:\n",
    "        #    break\n",
    "        LL.append(ll)  # sum all the errors\n",
    "            \n",
    "    return kappa, mus, Sigmas, pi, LL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us bring back our old example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-3, 7)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGedJREFUeJzt3X2MXFd5x/HfYzuBbMiLYq/SNsnaUUupooi3rBGICmUJ\npUmwCERtZLokgv7hYF4UJJBFulqjBlk0RkVEBVKsBFqy27pWsNOW8ha0K6H+AfUGUiBxgkKUOEag\nOEQoFKMmrp/+cWeys+N5uTNz5p5z7/1+pNFmJ7Mzz8yunznznOecY+4uAEB1rIsdAAAgLBI7AFQM\niR0AKobEDgAVQ2IHgIohsQNAxZDYAaBiSOwAUDEkdgComA0xHnTTpk2+ZcuWGA8NAKX1wAMPPOPu\nk/1uFyWxb9myRSsrKzEeGgBKy8yezHM7SjEAUDEkdgAD2btXWl5ee93ycnY90kBiB0SyGsTWrdIN\nN6y+XsvL2fdbt8aNC6tI7IBIVoOYmZEOHMhen927s68HDmTXIw0kdkAkq0HNzEg7d0qf+ET2ldcp\nLSR2oIFkld/ysnTnndL8fPa1vYyFuEjsQAPJKp9mmerAAem221Y/6fB6pYPEDohkNYjDh9eWqZpl\nrMOH48aFVRbjzNPp6WlngRJSsndvNlHaWn5ZXs6S1a5d8eICWpnZA+4+3fd2JHYAKIe8iZ1SDABU\nDIkdACqGxA4AFUNiB4CKIbEDQMWQ2AGgYoIkdjM738zuNbNHzOyImb0hxP0CAAYX6gSlOyR9w93/\nzMzOlDQR6H4BAAMaecRuZudJepOkuyXJ3Z9391+Ner9AbOzRjrIKUYq5VNJxSV8ysx+Y2V1mdnaA\n+wWiYo92lFWIxL5B0msl3enur5H0G0kfa7+Rme0wsxUzWzl+/HiAhwXGiz3aUVYhEvsxScfc/XuN\n7+9VlujXcPd97j7t7tOTk5MBHhYYP/ZoRxmNnNjd/ReSnjKzVzSuukrSw6PeL5AC9mhHGYXqivmQ\npMVGR8zjkt4b6H6BaFr3aJ+ZyS6UY1AGQRK7uz8oqe9WkkBZ7N0r/fSnpyfx66/P9mgnsSNlrDwF\nOti6VTp4cPX75uh9+3YO3kD6QpVigEpp7YjZuTOrr1OCQVkwYge6oCMGZUViB7qgIwZlRWIHOmjt\niLntttWyDMkdZUBiBzo4fHhtTb1Zcz98OG5cg2Cvm/oisQMd7Np1ek19ZqZcHTHsdVNfdMUAFUVn\nT30xYgcqjM6eeiKxAxVGZ089kdiBiqKzp75I7EBFVaGzJxVl6zAisQMVVYXOnlSUrcOIrhgA6KNs\nHUaM2AEghzJ1GJHYASCHMnUYkdgBoI+ydRiR2AGgj7J1GJm7F/6g09PTvrKyUvjjYjB792az/q21\nxOXl7I+ZzgqgeGb2gLv3PYaUETu6KluLV2hl610Gmkjs6Kq1xWv37tUaY8rdACHV/Y0N5UViR09l\navEKre5vbCgvEjt6KlOL1zjU+Y0N5UViR1dla/HqZdh6ed3f2FBOJHZ0VbYWr16GqZdX6Y0N9UK7\nI2qjmajz7vVBuydCCPl3RLsj0GbQejm7IyKEGN1VJHbUBvVyxBCju4rEjiSFXhxEvRwxFd1dRWJH\nkkJ/fK3SRDDKp/BPi+5e+OWKK65w1MPtt7svLa29bmkpu76fpSX3TZvc5+ezr+33A5RB8++4+ffb\n/v0gJK14jhzLiB1jNcrIm8VBqIIYnxZpd8TYDdpmOOrPlQ1tlciLdkckY5iRd50mO9lsDKGR2DF2\nw0wc1WGys9n509oOd9NN0rZt1f10goLkKcTnuUhaL+kHkr7a77ZMntZHyImjqml/LW680V3KvgKd\nKMLk6S2SjgS8P1RAqJF3FQ+9aB+pLyxIN94off3r1Sw5oThBEruZXSzpbZLuCnF/qI5Qy/JTrUOP\n+oYzMyNdc410zz3Su98tffnL1Z5PQDFCjdg/I2mXpFOB7g9jVMbRb6qHXoz6hrO8LB08uHakXsX5\nBBQsT72m10XSNkmfb/z3lepSY5e0Q9KKpJWpqakCqlHopsx17/n5rA49Px87klXDLqQq8+8BcShn\njT1EYv+kpGOSnpD0C0knJC30+hkmT+Mr46rOlGMe5g1nlFW5qKfCEvuaO+sxYm+9kNjTkOLot5uU\nR7cpv+GgWvImdvrYa6psW9im2tdep4VUKA+2FKih1mQ0MyPdfLO0f790332riZMl7fmwHQCKxJYC\n6Kp99Lt9u2SWJXcpnVbCMuCUJaRoQ+wAULz2pDMzIx06lCXzCy+s9oZbQB0wYocktsjtpYx9/1XA\n6z48EjsklW8ytUijLEIaJjmR0DKprjYuhTytM6EvtDumJeVWwlQUuQiJ38cqWknXUow+9rwXEnta\nWCiTz7B9/8MkJxLaqjKttxg3EjsQ0KiJdpjkRELjDa4diR0IZNTSCCP24VSlJBXyE3HexM7kKQpR\n5gnBUVa9DrMyldWsmVRXGw8qxiQwiR2F6PXHXeak388wyWncCa0sr3dVFn9F2XI6z7A+9IVSTD11\nKy+k/pE79fgGVbXnUxYh5kxEjR0p6vbHnXpNOfX4BlW155O6UK83iR3J6ffHnXoXSOrxDapqzydV\nIT8hkdiRlH5/3KmPIFOPb1B5ng/rG8KI0RVDYkchev1xp17zTT2+QeV9PlV73lWQN7GzHzuiS31P\n89TjG9Qgz6fZvbRzJ7t+piDvfuwkdgA97d6d7fo5P5/11SMeDtoAMDJ2/SwnEjtQUuNeaMQK2PIi\nsQMlNe6l6lVZ0l9H1NiRvKpNXobE5Ga9UGNHZXCSTnccaYhOSOxIXpRNlEqCyU10QmJHKTAyPR2T\nm+iGxI5SYGR6OiY30Q2Tp0hO+2Tp8rL0jndI27dLX/jC2pEqI3fUCZOnKK32ydL9+yWzLLFLjEyB\nfkjsSE77ZOnBg9KhQ2tH562j+VYpngQEFI3EjiTlmSylDRLojMSOJOWZLKUNEuiMxA5JaR1wPEgb\nX9XaIFP6PaC8SOyQlFZZY5A2vqq1Qab0e0CJ5TmNI/SFE5TSVLbj36p6wk/Zfg8ojnKeoMSIHS8q\nW1mjqgt0yvZ7QHpGTuxmdomZLZvZw2b2kJndEiIwFK9sZY1du05PejMz6e34OGjdvGy/B6QnxIj9\npKSPuPtlkl4v6QNmdlmA+0WB2HdkfAapm/N7QAgjJ3Z3/7m7f7/x37+WdETSRaPeb7IWF6UtW6R1\n67Kvi4uxIwqiqmWNFAzSlsnvASEE3SvGzLZI+o6ky939uW63K+1eMYuL0o4d0okTq9dNTEj79kmz\ns/HiQilwKDRGVfheMWb2MklfkfThTkndzHaY2YqZrRw/fjzUwxZrbm5tUpey7+fm4sSD0qBujiIF\nSexmdoaypL7o7gc73cbd97n7tLtPT05OhnjY4h09Otj1gKibo3ghumJM0t2Sjrj7p0cPKWFTU4Nd\nDyiNujkrWuslxIj9jZJulPRmM3uwcbk2wP2mZ8+erKbeamIiux7oIoW2TFa01suGUe/A3f9TkgWI\nJX3NCdK5uaz8MjWVJXUmTpG41s6cnTuzOj8bplUXK08HNTsrPfGEdOpU9pWkjpJoX9F6+DDlmaoq\nf2KvaF85EFp7Z86GDZRnqqrcib3ZV/7kk5J79nXHjuold968MKJOnTmf/KR0663sZ19F5U7sdegr\nr8ubF8aqW2fOyZNsOFZF5U7sdegrr8ObV0DDtPXVoRWwW2fO1q0snKqicif2OvSV1+HNK6Bh2vrK\n3go47BsTC6cqLM+m7aEvwQ7aWFhwn5hwz4oU2WViIru+KjZvXvv8mpfNm2NHlqxhDqoo8+EWwx44\ncvvtp99maSm7HmlSzoM2yp3Y3bMkvnmzu1n2tUpJ3b0eb15jMD+fvVTz8+P9mdiaybn1jem889x3\n7IgdGcahPom9Dqr+5hVYnUbsraPz5hvTWWeVJ34MhsSOWhqmLBHy7NQY5Y2lJfdzz80+yJ11VjZi\nJ7FXU97EXu7JU6DNMBtuhdykK9ZE7MmTWbPURz8qHTrEJGjdBT1oI6/SHrQB5NBM5kXtyXLzzdL+\n/dItt6w+npS9MaV2/itGU/hBGwAy7XuyjDOpLy9LBw9K9923tmVRIqnXGYkdCKzI05JS2Osd6aEU\nAwTUuuhnZub074FRUIoBImAEjRQwYgeAkmDEDgA1RWIHgIohsQNAxZDYi8IpSKVXh33bUQ0k9iJw\nClIllH3fdtQHXTFF2LIlS+btNm+Wnnii6GgwgqK3CwBa0RWTEk5BqowitwsYBuUiSCT2YtThCL+a\nKHK7gHZ5kjblIkgk9mLs2SNNTKy9bmIiu77myjTCjH1GaJ6k3VzpesMN0u7dbGdQW3k2bQ99qeVB\nG5yC1FHIQy7GLYUzQvOe9FTGY/7QnzhBqTpSSCjjjKesx9LF0i9p83pWF4m97FpG+EsXbvdN5/w2\nmVHtOEbZjDDz6Ze0y/QJCIMjsZfVwoL7xo3Zr6blsvSSq33TOb9NZhQWclTICDOfPEk7tU93CIvE\nXkYLC9mJxG1JvXmZP+8zSY1qQ4yyGWHmR9JG3sTOAqWUdFvIJGlZV+oGHdDO+ckkFsaEWqizd2/W\n1dH6s8vLnNcJdJJ3gRIj9pSYdRypL+lK36SnfenC7e4ef1TLKBuIQzlH7PSxp6TLgqXD2qoDL7lJ\nM3+7TVL8U3k4JQhIG6WYlDQ3CztxYu31GzdKd9whzc7GiQsDo8SEcSh0rxgzu9rMHjWzx8zsYyHu\ns5ZmZ6V9+7LNwcyyrwsL0jPPkNRLhqX9iGnkxG5m6yV9TtI1ki6T9C4zu2zU+62t2dlsx8dTp7Kv\nZUvo7DsviaX9iCvEiP11kh5z98fd/XlJ+yVdF+B+UTbsO79G6jtBorpCJPaLJD3V8v2xxnWom7m5\n0+cHTpzIrq+hmDtBot4K64oxsx1mtmJmK8ePHy/qYYtB+SHDvvMvir0TJOotRGL/maRLWr6/uHHd\nGu6+z92n3X16cnIywMMmgvLDKvadfxEtoYhp5HZHM9sg6SeSrlKW0A9L+gt3f6jbz1Sq3ZFj71Z1\natecmMg6fco2CQwkqLB2R3c/KemDkr4p6YikA72SeuVQfljVqV1z377s/1GqAgoTpMbu7l9z9z90\n999393DHApWhdp2z/FCmk4JG0t6uKVGqAgqW7pYCZald5zz2rrYLVuiUAQqX7pYCqdeuFxez5HT0\nqHTBBdl1zz6bjdT37OlYUw61I2KprFuXvTG3M8tG9QByy1tj31BEMENJuXbdPkn4y19mo/R77uk5\nSdi6YGV+vgZJXcre6Dq9QdewUwYoSrqlmJRb54YsL0RdsBJrviJnqQpAQHn29g19ybUfe6fThCYm\nsutj67Jvupt1/ZFce5i3nHPqmzeHe66xX8txPS+gZlT6/di7tc6l0A89xKeJvgtWxjlZzAQmUCvp\nTp6mbBwLccY5WRxzApNFS0Awhe7HXjvj+DTRa7J41Pp4zPkKPi0AhSOxDyv0vundkuwFF/Qu0eRJ\n+iEmMId9c0m5uwmoqjyF+NAXDrPuoNsE58aNnSdqm5OQeSdFR5nAHGXydfPm7vEDGIhyTp7WI7Gn\n3pXRjE9yX79+beLu1YFTVNIc5XFid+QAFUJib0o9sfSLr1dSHaLtciijPk7qb6xASeRN7NWvsac+\nedcvvl718UEmRdtr5O9/f/6a+aiTr2U/xxUomzzZP/Sl0BF7qFHtuEadeeLr9th5P410ul37pden\nmNQ/9QA1IUoxDSHq0ONMbKPGl+cNp9tjDPKYlFOA6PIm9uqXYkK0+oUs57SXRK69drT48pQ58rYW\ntt6uPU6JcgpQFnmyf+hL0BF7npHkqKPNkOWcTiP/nTvHOxoedMTe6xMKI3cgGtWiFBO6RLKwsLZv\nfOPGta2Io7YVxurpHrTG3i3OjRuptQMR1SOxh0yUCwvuZ5xx+n2deWY2og6R0LqN/KXxJ8f2kXav\nTwm94ozxxgTA3euS2EP2cfcqVzQT36gliF6P0fpGEbvckbd0M8rrDWBgeRN7uSdPQ25u1WuC8ejR\nML3YnSZym5qTscNs3xv6EI1uE84bN3a+fQqHnwBYlSf7h74kWWPvNUrduDFMvM2Ye418By0v7dx5\n+ieXEHXvTp8a6GcHolItSjHu4coW3WrszTr7KMmrPcZeG3sNUl7qtZfMuOresctEQI3lTewctNFq\ncVG66abOh08Me+BFp4MmzjwzS78vvLB6XfPwibm5/AdudDucQyrmEA0AheKgjWHMzmYJt5Nh9w/v\ntLjp+eelc8/tfFDHIAuqesVE3RuorQ2xA0jO1FTnUfCwibJb8n32WemZZ06/vjkpOzeX/ezUVJbU\nO03WdovVbLCVtQAqhRF7uxBbELQapnOntQNnz54syXfqeOkUq5n0vvex5B+oMRJ7u9DnmXZLvtde\n2/9n+7U+dor1nnukz39+uFgBVEOeGdbQl0J2d0ype2PYlsRRTy5K5fkDCEK17orp1InS7DqJUaLo\n1r3Sr9Nm3brOk7n9Ol5Se/4Agqh3V0xqpyZ1m0A9erT3qtFudfgLLuj9eKk9fwCFSjuxD7tUvlci\njaFXgu5VQ9+zJ+t5b/fcc71fi9SeP4BCpZvYh9kzpSnkHjIhdOu0kXqPrGdnpXPOOf3+Xnih9+g7\ntecPoFDpJvZRygmhWxZH1a3T5tlnO9++dWSd5zbtUnv+AAo1UmI3s0+Z2SNm9kMzO2Rm54cKbKRy\nQuiWxRA67Q6ZZ2Q9bB/8qM8/xI6RoXedBJBPntaZbhdJb5W0ofHft0u6Pc/P5Wp3jHXaUJHy7JYY\nY0fFEI/JTpBAcCp6d0dJ75S0mOe2uRJ71RND65F769evvmmN48zWQYV4U63DGzNQsLyJPVgfu5n9\nu6R/cfeFfrfN3ce+uJhvz5SySb3PfNj++dD3AWCNvH3sfRO7mX1b0u90+F9z7v6vjdvMSZqWdL13\nuUMz2yFphyRNTU1d8WS37WbrYNgFS0UJEV/qzxEooWALlNz9Le5+eYdLM6m/R9I2SbPdknrjfva5\n+7S7T09OTg7wVCoo9T7zEF01dOYA0YzaFXO1pF2S3u7uJ/rdHg2p95mH6KpJsTMJqIlR+9g/K+kc\nSfeb2YNm9vcBYqq+VEezre2Jc3NZPKMc3h3iAHAAAxvpoA13/4NQgdTKIIdpFKV9Qre50lciIQMl\nk+7K06pLbTQ76EpfFh8ByeJoPGQGmdBldA8kjRF7FYQYPQ8yocu2wEDSSOxlN8oumK0GmdBNvV0T\nqDkSe9mFGj0P0p6YersmUHMk9rILOXrOO6GbarsmAEkk9vKLMXpm8RGQNBJ72cUaPafWrgngRST2\nsmP0DKANfexVMDtLIgfwIkbsAFAxJHYAqBgSexmwLwuAAVBjTx37sgAYECP21LEvC4ABkdhTx74s\nAAZEYk8d+7IAGJD1OH96fA9q9mtJjxb+wL1tkvRM7CDabNoknZqSNlvLm7BLp45KTz4jPRsrLiX4\nWomY8iCm/FKM6xXufk6/G8WaPH3U3acjPXZHZraSYkzHE4tJSve1Iqb+iCm/FOMys5U8t6MUAwAV\nQ2IHgIqJldj3RXrcXogpvxTjIqZ8iCm/FOPKFVOUyVMAwPhQigGAiomW2M3sE2b2QzN70My+ZWa/\nFyuWlpg+ZWaPNOI6ZGbnJxDTn5vZQ2Z2ysyiztCb2dVm9qiZPWZmH4sZS5OZfdHMnjazH8eOpcnM\nLjGzZTN7uPG7uyWBmF5qZv9lZv/diOmvY8fUZGbrzewHZvbV2LFIkpk9YWY/auSmXF0o42Zm55vZ\nvY38dMTM3tDr9jFH7J9y91e6+6slfVXS7oixNN0v6XJ3f6Wkn0i6NXI8kvRjSddL+k7MIMxsvaTP\nSbpG0mWS3mVml8WMqeEfJF0dO4g2JyV9xN0vk/R6SR9I4LX6X0lvdvdXSXq1pKvN7PWRY2q6RdKR\n2EG0mXH3VyfU7niHpG+4+x9JepX6vF7REru7P9fy7dmSohf73f1b7n6y8e13JV0cMx5Jcvcj7p7C\nYq7XSXrM3R939+cl7Zd0XeSY5O7fUbyFWh25+8/d/fuN//61sn+EF0WOyd39fxrfntG4RP83Z2YX\nS3qbpLtix5IqMztP0psk3S1J7v68u/+q189ErbGb2R4ze0rSrNIYsbf6S0lfjx1EQi6S9FTL98cU\nOVmVgZltkfQaSd+LG8mLJY8HJT0t6X53jx6TpM9I2iXpVOxAWrikb5vZA2a2I3Ywki6VdFzSlxol\nq7vM7OxePzDWxG5m3zazH3e4XCdJ7j7n7pdIWpT0wXHGkjemxm3mlH2cLmTj8zwxoXzM7GWSviLp\nw22fUKNw9/9rlD4vlvQ6M7s8Zjxmtk3S0+7+QMw4Ovjjxut0jbIy2psix7NB0msl3enur5H0G0k9\n57jGuqWAu78l500XJX1N0sfHGI6k/jGZ2XskbZN0lRfUCzrA6xTTzyRd0vL9xY3r0IGZnaEsqS+6\n+8HY8bRy91+Z2bKyuYmYk85vlPR2M7tW0kslnWtmC+7+7ogxyd1/1vj6tJkdUlaGjDnHdUzSsZZP\nWPeqT2KP2RXz8pZvr5P0SKxYmszsamUfC9/u7if63b5mDkt6uZldamZnStou6d8ix5QkMzNl9dAj\n7v7p2PFIkplNNru8zOwsSX+iyP/m3P1Wd7/Y3bco+3taip3UzexsMzun+d+S3qq4b35y919IesrM\nXtG46ipJD/f6mZgnKP1NI9BTkp6U9L6IsTR9VtJLJN2f/dvUd909alxm9k5JfydpUtJ/mNmD7v6n\nRcfh7ifN7IOSvilpvaQvuvtDRcfRzsz+WdKVkjaZ2TFJH3f3u+NGpTdKulHSjxo1bUn6K3f/WsSY\nflfSPza6m9ZJOuDuSbQXJuZCSYca//43SPond/9G3JAkSR+StNgYVD0u6b29bszKUwCoGFaeAkDF\nkNgBoGJI7ABQMSR2AKgYEjsAVAyJHQAqhsQOABVDYgeAivl/w56bP0ZzLmQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112c7fa58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# simulated samples\n",
    "\n",
    "mu1 = [-1, -1]\n",
    "cov1 = np.eye(2)\n",
    "\n",
    "mu2 = [2,3]\n",
    "cov2 = np.eye(2) * 3\n",
    "\n",
    "C1 = np.random.multivariate_normal(mu1, cov1, 50)\n",
    "C2 = np.random.multivariate_normal(mu2, cov2, 50)\n",
    "\n",
    "plt.plot(C1[:, 0], C1[:, 1], 'or')\n",
    "plt.plot(C2[:, 0], C2[:, 1], 'xb')\n",
    "\n",
    "plt.xlim([-3, 6])\n",
    "plt.ylim([-3, 7])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from matplotlib.colors import BASE_COLORS\n",
    "COLORS = list(BASE_COLORS.keys())[:-1]\n",
    "\n",
    "# scatter plots for K classes\n",
    "def kscatter_plot(X, classes):\n",
    "    K = len(np.unique(classes))\n",
    "    Cs = [X[classes == k] for k in range(K)]\n",
    "    csm = [''.join([c,'.']) for c in COLORS]\n",
    "    mum = [''.join([c,'x']) for c in COLORS[::-1]]\n",
    "\n",
    "    for k in range(K):\n",
    "        plt.plot(Cs[k][:,0], Cs[k][:, 1], csm[k])\n",
    "        plt.plot(mus[k][0], mus[k][1], mum[k], markersize=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 0 0 0 0 0 0 1 1 1 0 0 1 0 1 0 1 0 1 1 1\n",
      " 0 1 0 0 0 1 0 0 1 0 1 1 1 1 0 0 0 0 0 0 0 1 0 0 1 1]\n",
      "----means-------------\n",
      "[[ 2.06069238  4.17384165]\n",
      " [-0.28087196 -0.40858763]]\n",
      "----Class k: Cov-Mat----\n",
      "[[ 2.58347819  0.09917755]\n",
      " [ 0.09917755  1.18408964]]\n",
      "----Class k: Cov-Mat----\n",
      "[[ 2.25513196  1.1274902 ]\n",
      " [ 1.1274902   2.16567898]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x11664f9e8>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXv0JVdV57+76nZ3IIG8ugmhO7Eb0oAtJCR2QniIAYx5\n4BhhcEgcRRAngxJlZJYYR8UnKgtBxiEQMxh5iGRpiJrBVhBGYFTA7vDIk0CbYLoD5EFCEkjo7lu1\n54+quvfUqXNOnVNV575++7NWVv9u3ap7d9Uvv7PPPvu79yFmhiAIgiAAQDJvAwRBEITFQZyCIAiC\nMEGcgiAIgjBBnIIgCIIwQZyCIAiCMEGcgiAIgjBBnIIgCIIwQZyCIAiCMEGcgiAIgjBhNG8DQtm4\ncSNv3bp13mYIgiAsFdddd929zLyp7bylcwpbt27Fnj175m2GIAjCUkFE/+5zniwfCYIgCBPEKQiC\nIAgTxCkIgiAIE8QpCIIgCBPEKQiCIAgTojkFIrqSiO4mohst7xMR/RER7SWi64notFi2CIIgCH7E\njBTeDeBcx/vnAdhe/ncxgHdGtEUQBEHwIFqdAjN/koi2Ok65AMB7udgP9NNEdBQRHc/MX4thz61f\nfwh/e/1XY3y0IOCZTzwWzzlpY+3Ynd98BF+66yE8/ymPm4kN//jFu/G5O+43vvfskzbizCcea732\n7ge/gy/sfwBn7ziudvyRgxn+7sav4cWnbgYRTY4zM66+bj8ueMZmrB/V55Z/sXsf9t//cPNLiPDD\npzwBJz3uiNrh6/d/EwTC07cc2XaLAIBvfOsAdn/lPpz7tOOt52Q544Of3Y+XnLoZo7Ru359/5g58\n/YFHWr/n/JOPx1Mf/1gvm0IZZzmu+dydeOlpW5AkZD3vL/bsw/77ps9y59Zj8Lwnt9af9WKexWub\nAexTXu8vjzWcAhFdjCKawIknntjpy/be/S38r3/c2+laQXDBDOy45W7seu331Y6/71P/jvd+6iu4\n+bdcAfNw/Pq1N+GO+x4GaWMMM/D/9t6Lv/rZ51ivvWr3Przto1/C3jeeXxukPnrLXXjdX3wBp554\nNLZtPHxy/KavPohfvPp6HHvEerzgqVNH8q0DY7z+g9cDgNGOe791AL/74qfXjv/urluQEOHP/8uZ\nXvd5zWfvxBt33YKbf+scPHq9eQj7/L778fqrr8eWox6FZyvO+r5vH8T/+KsbjPbptu67/xH84cue\n4WVTKP96+314/dXX40mbDsf3ftcxxnMePjjG66+uP8tXf/+TVtopeMPMVwC4AgB27tzJXT7jRScf\njxed/KJB7RIEAPjZ91+HL9/1rcbxA+MMB8f5zOw4OM7xsp0n4E0vPbl2/FXv3o27HvqO89oD4ww5\nA+OcsV5xCgdK+/X7sB0/VL7+jf+wA694zrbae8/+vY9N3tc/K3WN0AZbAeBQZh8KDhwqvudAVv++\nyt7fe8nTcdEZ9gnmC9/ycRzM4v3uqud3wPH/x6FxcX9v+KEd+KnnbrOeNzTzdAp3AjhBeb2lPCYI\nS0VChCxvDlBZzhjnDGauLb3EYpwz0rT5PUlCGDsG0OpaAI37yPK8fD/XjnPtOv1zUsOSSJLYnxMH\nPB6brcZzMt2+3GqfSppQ49oh8bsHP1uHZp6S1GsBvLxUIZ0J4IFY+QRBiMkoocbgCEz/8B1/94OS\n5eYZ98gyGNeuzapBvj742wav6rymE6mcQnNosT6njFvtM32Hbqv5HIt9LU46TRKjrUMxdbb278gc\nDjYm0SIFIvoAgLMAbCSi/QB+HcA6AGDmywHsAnA+gL0AHgbwyli2CEJM0iQxz4CVgTZN0uh2jHM2\nDiCph1OwRwruwVWPQKqBehRgR5YzQobfoEjBEsmMDBGVSuFI4y0f2SIZ0zmmZxmTmOqji1reZwCv\nifX9gjArbDPxjMtIYUZphSxn4wAySmhii42c3U4htxzXP7e6V5NzGtmcJzMQ8IxyD6dgs8939p0m\nhIirR1b7TOesTKQgCGuFNDUvi9SXOeJHCpklp5AmSY+cgjtSsC0rmWbiqWX5qMgp+I/APpHC1D5t\nOSzzm33HjhRsz890TltUMzTiFAShJ7YBxGfwGhJnpOCdUzAvt9iWYaxr9iY7UttzClMf2RyV/plA\nc3mriohMOQ+V1CM53wfb8zOd02br0EjvI0HoiX0G3J5MHApmLnMKzT9pWySj0jlSyMyJaVtOwfic\nMg56RrYkt9E+W06hLVJI2x1pH2yRjOmclckpCMJawTYTr2aas4gUqq+wRwrupRCbA5veg3nwt0cK\nZvWR8TnlHLS4Zkty+9nnK0lNMM6zAKvCGHvdw3wkqeIUBKEnNvmizzLHULgGENsMvX69efDPLMsw\nmWW23ilSCFUfeThba6QQlFOIGClk/tHOrCMFWT4ShJ64ZsCAW3Y4FK4BxCunYHFg1pyCJQfhmonb\n1EfjfPg6hS45DxUfR9qHsJyCRAqCsFRU+nu9ctln8BoK1wDiU4hlW84IVh85ZuLOSKFFMmuy1TnL\nnszELTmPOdcpBKmPJNEsCMtFNQD6Vv3GIHMMxiGRgm05aBD1kVWllXeMFLrkFALUR/OOFLL5RAri\nFAShJ1VtgLXqdyY5hXIASQ3qIyWSabve9x6sdQCOmbhN5ln1iPIlSH3UqLhekJxCiPpoxnUK4hQE\noSf2SGF26qO2nEKbHfbEsXsZJkh9ZJF5ds8pRFYfzb1OYe01xBOElaAaAOcbKTjUR5ZIpna9pSGe\nPVIonUXATNzUIyrPGczFdb55BZtSymR35zqF6JGC+fnVzxH1kSAsJdZIwaLxj0H/SMGdOPbPKbjU\nR811evW17xjcr07BU33kUfDXh0VWH4lTEISeVH+01ln2DCSpbeoj9RzX9b7dULMWZ+HbJVV97avS\n8qtTsCx7TRLy7qEvuvoooNZC1EeCsGQshPrIMYAERQqW5aCh1EfN/Rqmr32f0yDqo5bk7UKojyRS\nEITlZBIpeGr8Y+CSL9oimdr1Q6mPHDPx9kjBN6fgoT7qEMmoLIb6yL43RUzEKQhCTyrJoKiP3DPx\ntpyCb+X3zNRH844UpE5BEJaThVIfWeoDAN/EbKD6KCinkDQG/m6RQoD6aGHrFAJ6H0mdgiAsF+11\nCnNWH1kiGdP1/SuaHeojg6KnFinMMqfgs/NagEw2FMkpCMIK06o+mkmk0FN9ZG1w15JTCKpTMOQU\nlOu91Ucey3JW9ZGnosdnya0PtudnOkfUR4KwZNjrFJZQfdS4hwVWH0WuUzBdPxQSKQjCCjONFCyz\n7LnXKQygPrLWKfjPxNOEkHNRxax/vum77bb22HktYD+Ftu/og+351c4R9ZEgLCfVADhf9ZF9AAlS\nHzW21+za+8hhB5sdgXekYFnq8rMvBxGQeKiP2r6jDxIpCMIKszx1Ct2XW7zVR879FJrOsxYpeEZU\nXuojx7KXz8w7fk7Bv9ZCIgVBWDJM6h5m9hq8hsIlX7RFMqbrh1Af2WbiI4Nzmof6yGfm7bPk1geJ\nFARhhTENIOrf+izVRzbVD9AvMWuvaPafiVd2qIob1WHOSn3ko+aZmfqoxVGnCdV285sF4hQEoSem\nAaSLqqYP07V88z4GbXYMFynYZ+KjtOk8xzUHESFSMCzpBUUKkUQCvpHCrKMEQJyCIPTGtGbfRVXT\nB69IwTITZ+bWiMC795FjJp4anOfM1Ud57pdT8HCkffDtfTTrfAIgTkEQemNas++yVt6Htn0MXHbU\nB+lA9VHATHzwnELH5TC/SGFG6qOWe5BIQRCWEGOkoFbqzrBOwR0pmO2wDdKAIv9sDP72Bnr2nEKL\n+sg7UvCZZVsihWyJ1EeeSqmhEacgCD2ZDiBq0tQ++46Buz7ArT7KLM4McOQULJLPfpFC+3Oqtu/U\nr9VxRgoeDeaiq488ay1MOaLYRP1GIjqXiG4lor1EdKnh/SOJ6P8Q0ReI6CYiemVMewQhBqak5Mxz\nCi37GLjscEYKgbkG10w8NTjPzJJ0tuG73LQS6iPPqGZoojkFIkoBXAbgPAA7AFxERDu0014D4GZm\nPgXAWQDeQkTrY9kkCDEwJSXnpj6y7GNQnGOe9dqWc4B29ZExUrDMxIfIKfg6W1eDv7A6hbhOYa2p\nj84AsJeZb2PmgwCuAnCBdg4DeAwVQtwjANwHYBzRJkEYnKVRH1lm4mPLshegDl62DrDh6qM+EZWv\ns+2tPvIo+OuDX14kn/leCkBcp7AZwD7l9f7ymMrbAXw3gK8CuAHAa5k5/gKsIAzIwquPWuSVfuqj\ngEihpU7B1tpiyEjB2sgvNFKIJBJYq5GCD+cA+DyAJwB4BoC3E9Fj9ZOI6GIi2kNEe+65555Z2ygI\nTtojhfjznElLBEP1a2tOIbMPtG29jxo5BQ/1UZ+Iyjcx7Vr2WoQ6Bd9ai5XKKQC4E8AJyust5TGV\nVwK4hgv2ArgdwFP1D2LmK5h5JzPv3LRpUzSDBaELRvVRh0rdPmQ5I7H2HApQH4XmFDrUKdgjqnbn\n6dtAz5UgX4TeR761FqumPtoNYDsRbSuTxxcCuFY75w4ALwQAIjoOwFMA3BbRJkEYnNZIYUZ1Cq1r\n+YHqo5r807MDrE/vo7FNfRQcKbgGVIv6KFsM9ZFf/6b5RAqjWB/MzGMiugTAhwGkAK5k5puI6NXl\n+5cD+G0A7yaiGwAQgF9i5ntj2SQIMZgMIJatJWcVKbTP0D3UR5alpCHrFPrkXmz2Nc4bLFKIlFPw\nrlNYIacAAMy8C8Au7djlys9fBfCDMW0QhNgshPrIoz7AHinYiu7seRGr+sgxEx8iovJ1tvZlrxwb\n1rUPewujPlqxnIIgrAmIqLEp/TzUR231AbZN4m3qI9cAXN1fzs3tNe2RQtKwo1+dQvsmOwsbKfio\nj7K1qT4ShJUgTWju6qPukUL7IG1THwHQtte0a+uHiKj8cwo91UctS2598VYfrVidgiCsGUYJWZOm\n884pmCIZ/VrTz86cguW9fnUKgeojj5xCljOY/exTiVmnkOc82YRpzfU+EoS1QjNSCFPV9KWtp49u\nX+1aj+UcW/2C/t6iqY9MtnqpjyLWKWTsdw+rWKcgCGuGkZ5TKAfaDaNk7pHC1D63+ki3dWw5Xr23\nYdTMEXRRH61LCQn55hTyiU22WXw1E6/s0x3JvHMK+vNWIxmVtVrRLAgrQZokxlmsa/Aakra1cmek\noA60BnlqcQ9N9dF00K3P9kPVR2lCGGnPz4aPs61m4hOnpeV35t37SHW2ru8Q9ZEgLDGjhIyqmg3r\n0tmpj1ojBXdOobC1qT7S76HavnPDurR2ffVzq/rIsJzjynnYbLUl8NVzqu+YvOep6IkaKWR2+1Qk\nUhCEJcamPipm3zNQH7UMdnokU7vWEtXU78F8XL2++NnV+8gVKVDQfgquSME1Ex97Knpiqo/UyEy3\nT0VyCoKwxIxSs/poljkF12CnRzL6tYB/TsE16Lpm4tN6iXo0MkoIaWrPedhstTk5ddmr+g71+rlH\nCg6nqlI4elEfCcJSYlMfrR+lM1MfuQYQd06hOL5+lDaWgqrjYyUhqh5Xr69+ttYppOZIIUkIKdnt\n87G1fs702av2TuwL6X0UIR801p6fRAqCsIKkZK5oXj/LSKEl0WxXH1WDaNIY4KvjABra+vWTSKE+\nE08M7buBaVtvXaU1Stx1FD621s/RZuKe6qiarTOIFNYbIhmVcekwZ404BUEYAHdOYRaRQnuiua1O\nQV8myrS1b70K15xTCKusruUUAiKF7jkFP0UPkb9MNhRRHwnCGqDIKZjrFPIZOAWfSCG36OFzVh1Y\nc08IffDSE6XNmbh5WLHVKVQ5BZ/nZMt/mM5Zb3BavpFCYW9SKzQbCt2puu5D1EeCsKSkSX2Qmg60\ns8wptNQpWPdontqqliNkyj0A08Gr8hvV8Vyr0G3rfVSLRjisTiGr2dq2fNS0z7f3UWVvlIrm3Pxc\nTeet1H4KgrCWaFQ0T7Tyi5FT0CMZ/VqgsrXZHmLDOkuksC5sJm7qwZSVrbazsvahDfW5utbia/Zl\nlTMrNg3yVfT4ymRDcT2/+nls7XwbE3EKgjAARU7BMKDOtE7BpT5y1CmolcsO6aneebTLmr2ee6ki\nHGK/brIhy0fNZa/iX9/Oo74y2VBClo8kpyAIS4q999GsKppbIgWfimZdkqrcg3qeutwE1GfiOaM1\n4V2PRopW275LNep3t7ftSGuvq8/3zyn4Jb9DsT0/FWZulRnHQpyCIAyAqU6BCFifxhlYdMaOTXam\n9rUst/SMFKocRJdIwXcArgrfNowSMMOYnLZHCnmrfbqtcXMK9kihOiSRgiAsKaacQqG/T6IUQOn0\nixSma9ym4rVJTqHazSyrH2/OxO3Dim5HlofVKTTyBYZrbOeERwpx5MS251c7pzwm6iNBWFLSpNk3\nKE0Io5lFCh7qozZdf1pv5dy2DGNbVnJHCs1oZKI+8nCePsqdPvbVbZ2f+igLtHVIxCkIwgCYIwX/\n7p996ZtTSAhYl2rLQS3LMI1lpax9Jq73YMoCn1NzScswy270PtIjhQD1UZScgvn51c8Ji2qGRJyC\nIAxAmjbVR9O18tns0dyqPnLUKYySpNGbKDSnMFmzb81tGCKF1O85+azHN8/Ja/aHRQrzUR9VjlMi\nBUFYUpqRQj5ZK88tCdEh6RspVA6seq3+q++bMNaOh6zZ691kM+U5hUQKpmrl6TnTfSCAaeTgE8mo\nuAr++mB7fqZz0nT2Q7TUKQjCAJh6H9UGWmYkiDfrG2fu3kd6JFO/dpoUB7pECv4z8UakkKl1Cj45\nhcKJuHZG6xPJqLgK/vpge36mcyRSEIQlxVSnoA60sfMKfdVHaWqIFLL62rc++Ou9jzqrjwLrFFRn\n61QfWXMKvpFCJPWR5fnVz5mf+kgiBUEYANMezepAG1uB1NYSoU19VC3hFK/1wV9bhtFUSX3UR1UD\nPQJ71inUbTXJfYdSH7kcaR9sz69+zvwiBXEKgjAALvURYB68hiRuTkFbhmno7OvOImSv6MohZfCL\nptTEdPHapPG32RemPnIV/PXB9vxq58xRfSROQRAGoEhKGtRHjsFrKHxaIrTt0aw6sGrQClcfhecU\nqudE8O99NEoVZ2ucZbuXvUIihUPZnNRHE1ulzYUgLCUu9REQN6dQfXTvSCG1RAptvY+0CCKk95H6\nnHyiKa+cQma2zyeSUXEtufXB9vxq5wQqpYZEIgVBGIA0dauPYuYUfJKSeiRTv96tPlqf+qmPJrPb\nttyGofKbKEx95ErgWyOZwIE2Xk5B1EeCsPK4eh8BcSMFr/qANvWRMadQyj8nEUR9GeawdWYn4lQf\n6TvU5eG9j3zVR4ets6mjAtRHEesU9OdXP6d09HPYTyGqUyCic4noViLaS0SXWs45i4g+T0Q3EdEn\nYtojCLGo1uyrvkGzjRQ81vIdPZiqWgGT+sh0Dw31jDboBqmPyn0gQnZeqy/L2WfZoj7qRrTlIyJK\nAVwG4GwA+wHsJqJrmflm5ZyjALwDwLnMfAcRPS6WPYIQk+qPN2cgpapOIXEOXkPh3XPIkVMYGesU\n9AFYW1Ya6ZFCR/VRSsjyQPWRlhTXzwEGqFPwbL0Ril+dwvxyCjEjhTMA7GXm25j5IICrAFygnfNj\nAK5h5jsAgJnvjmiPIERDn2XPMlLw28egHsmoVMqlVLNV7WAKNJdh9GUfv0jBnnvxUh81nK1LfZTW\nXk8jBf+GeFEiBcvmRbVzVlR9tBnAPuX1/vKYypMBHE1EHyei64jo5aYPIqKLiWgPEe255557Ipkr\nCN3RZ9ljZUcxwDwbHArfSmJgqlTSrze1jpjIPzVVUnUv1eDczCl0VB+FRAqp3dnaIplFUx+5+zet\nZqTgwwjA9wJ4EYBzAPwaET1ZP4mZr2Dmncy8c9OmTbO2URBa0WfZNplnDHzrA4pzzRu61HIKmR4p\nNO8NwKRie6I+ytpnt+pAq27f6b3z2sTZtncYXZcSEjLkFHx7H0VUH9X/3zDlRcJ2iRuSmJLUOwGc\noLzeUh5T2Q/gG8z8bQDfJqJPAjgFwJci2iUIgzOJFJQB1STzjIFvTgGwL1WsSxNDnYJea9EsAusW\nKZSfryx7ZaDJ9pqJ4/oQ9dFk856uOYWIvY/ShJBSe63FqkUKuwFsJ6JtRLQewIUArtXO+RsAzyWi\nERE9GsAzAdwS0SZBiEJq0PKbZJ4x8N3HQLWvfn039VH1XiOn4LQjMTbQcy0Hqfiqj4rah/qy1HTZ\na96RQuFsk6QeydTPCYtqhiRapMDMYyK6BMCHAaQArmTmm4jo1eX7lzPzLUT09wCuB5ADeBcz3xjL\nJkGIRTOnoLWOiKk+8pyhA/YGciNjnYJbfZQS1WbToeojNeKozGobhH0jhcqGUUI96hTsBX99qNtn\njkZC5bND4uUUiOi1AP4UwEMA3gXgVACXMvNHXNcx8y4Au7Rjl2uv3wzgzQE2C8LC0aY+mntOIXUk\nNTNP9ZHiLBICkipSCKlTUOol1GUv4upechRzSDNZzli/Lm1VH1U2pMqmPgujPsqnzQttCfbQ5n1D\n4vuNP8XMDwL4QQBHA/gJAL8fzSpBWDKM6qPaksxiqI9sA5BRfWSRf1ZRUPGd3dVH6rKXb48om0zW\ndE71fZ3VR46Cvz6ozQvVSEY/p3p/1vguH1WWnQ/gfeUy0OytFYQFpaE+yuqDV8zW2YOoj9KmA7MV\nimXqoKvMxH209WltkFYiBe1ebEyS3w5Vl20mvjAVzRkbI5naOXPcZMc3UriOiD6Cwil8mIgegyIH\nIAgC0JhlV5W6s4kU/NbyVfvq1+s5hXzy7ygtEqJEivooqw+6XdVHdRWTX4+oceabU6hm4gb1kWfy\n1lXw1wdbJKOfU70/a3wjhVcBeAaA25j5YSI6FsAr45klCMuFrvGfaZ2Ch6omSH2k1SlUn60uw1QD\na019lLVr61OjRLTYec1mn0qmOVvjLDsbLlIApq1LhqJytrp99XMWX5LKAHYA+Pny9eEADotikSAs\nIYuvPnK3mh4ZHJhrGWZ6PGlGCo4R1B4p2NVRuq1FA70A9ZFhucqHWL87L/WRRyFgLHy/8R0AngXg\novL1Qyia3QmCgOlAOFf1kU+dgjUxa1cfAc1lGHXQDe19lJVLMrV6B88d6qZFgS3qo9pM3D/noRLr\ndxekPlrgOoVnMvNpRPQ5AGDm+8uCNEEQsKLqo1L+WXy2Hil0Vx9Vn92lTmHqbN19g1Jl0NW3F/Vd\nkYn1u2uoj5Y0p3CobIXNAEBEmyCJZkGYYO19NINNdvzqFBzqo0zrfWSMFKgRBVXHg9RHSuXy2Fin\n0CZJbd/mVJ2Jq5v6VMolX+Gkq+CvD81IYTnVR38E4K8API6I3gjgnwD8bjSrBGHJMKqPZhYpRFQf\nteYUwmbiqh1qKwdf9ZG+LOdaDivsS4xOzgdXwV8fbJGMfg6ASX+kWeIVKTDz+4noOgAvRFGz8CPM\nLD2KBKFEHfzznMGMWkI0i9AuoSKsTsEyiJrqFDJbu4i8Vqdw4FDdibhm4mqDwOlyk7/6qHJIukxW\npRYp1NRRHLQcEy+nkBsjmfo506rxWeMVKRDRkwDczsyXAbgRwNnlrmmCIAC1Wbaa+FWXS2IxmPpI\nWy6p5J9AVWTVdBahM3FjpJC4JaY1WzO/9fjaTFxr8OdLNPWR5fnVzlHyNrPG91s/CCAjopMA/DGK\nlth/Hs0qQVgyVHWPOkjPRH3kuY8B0HROzGxVH1Xyz+qz1eMjY51C+0xcHWjVBnqu5aDavaqOyqrc\nyc2RQl5sGuRLVPWR4fnp58wjnwD4O4WcmccAXgLg7cz8iwCOj2eWICwXqsZ/0tNn1uqjlvqA4tz6\nrLcyq1r2ceUOslpEYFYf9Y8U/HIKxXV2jX89UuiYU5iB+kiNZGrnBC51DYmvUzhERBcBeDmAD5XH\n1sUxSRCWD7WYqh4pLIj6yDIT19td672JbOqj+kxcOd4yE1ftUCWsvvspjC3JbxVrJKNc60Os350t\n51E/J59LjQLg7xReiaJ47Y3MfDsRbQPwvnhmCcJyoapn1EG6GoPmrj6ytNvQC870LqbmSCHvPBNX\n7VC37/RRH6nbd1a2mhv8aeojQ9sOH1wFf31o5jxsOYX5OAVf9dHNKFtcENHRAB7DzG+KaZggLBPm\nSCEBUX0f4xj4RAq2thB6wZk7UpgeX5dOk70hM3Gz+sivTkHdvrO6rrVOQc8pLJr6aFlzCkT0cSJ6\nLBEdA+CzAP43Eb01rmmCsDyo6hl9kLbNBofCR31km4lnmSlS8MkpKOqZgJm4vU6hXX2kV26rMlmV\nmn1pD/WRZ+uNUGzPTz9n0dVHR5ab7LwEwHuZ+ZkAfiCeWYKwXNQiBW3T9VFCM9pPob3NhTVSSJvF\nXnX5Z31v5a4z8Vb1kcN5NpytVeNvUR9lYQPtLHofLW2kAGBERMcD+E+YJpoFQShR1TPqjmLVe/OP\nFMwzcWNOQZn5m+SfY8VZqLuTxVYfNZ2tXeO/0OqjxvMz50UWXX30WwA+DGAvM+8moicC+HI8swRh\nuVC3h9QH6VGazKhOoUtOwVN9lHqojzxm4upAW1MfObbX1G312YvAHMnkzk6yOnNXHy14ovkvAfyl\n8vo2AP8xllGCsGyktTqFWecUchC5WyLYZuKNSCH1VB8ZoiC/SGE60KoN9DKaOiL7fTaX5Wwa/0F6\nH8WKFHI2Pr/aOdn8lo+8nAIRHYZi97XvgbK5DjP/VCS7BGGpsKmPqvdiq4/alhpsM3Gb+sgs/2zL\nKbTPxG11Cj7qI5OznY36aNjfna/6KCSqGRLf5aP3AXg8gHMAfALAFhQb7QiCACChpvqoquNKKH5O\noW1WWa3q2CKFiVOgYpCq5J9Vl86ENPVRdVyLFJKWrp5qxFI1CUwTqj0/131WtlTX2XIKierksp69\njyLUKfjcwzw6pAL+TuEkZv41AN9m5vcAeBGAZ8YzSxCWi3qkUA12ZaRgUckMhY980bYpjZ6PqAYp\nvXXGKLXXL3RXH6k5hQD1UZ/eRwHqI9/WG6Goz8kV7Sy6+uhQ+e83iehpAI4E8Lg4JgnC8jFt5czW\ngTYWPgP++s1LAAAeuElEQVRIm/pId2DNpZqkFilMB+bi+LSxXkf1kaXium6r5my96xS65RR8HFUX\n1JxCIfU1qY/yudUp+O68dkVZyfxrAK4FcASAN0SzShCWkJE+y1Zmq3nUSKG9kng6GNePN6uEE63W\nQm1Tbd55rTqW5YzD1nVTH1U5BbdTqN9LmtDEfpXclfPosHyUG76jD6p9aUIw3XKeuyXGMfFVH72r\n/PETAJ4YzxxBWF7ScvC3DbSx8MspmDel0fsmjaz3QKguVZdhJhEIc2lHyzJWOUPOy2R29R3VhN+9\nfKTbmuCRQ5nhPFXdk0zuJVTR4yOTDUVtVV58h61OIceGdb5z9mFxfisRvc71PjNLqwtBKKmKqXRF\nj01hMhS+bZZNm9KYl7pywwDsFyn4RizNSMFn+chvWU5XHzEXTihU0eOzpBVK9VGLnFNoc0WPKf9l\nFNtwqsT7v1wQlpDqD1zt/lkdjx4peAx2pgHItNR1KMuNA7Bap6AeL4755RRSrU4hTYp9HBKP7TjN\nzrY+y9Zn4qp9PpGMSoycgtnZmvMiC9kllZl/EwCI6D0AXsvM3yxfHw3gLfHNE4TloZpN+wxeQ+Lb\nPM3UFsKk6Hnk0DRZbqtTaEQKGXut2euRQvU5Pt1k1WK3yjZ9aUefiauRTOhA69OkL5Sms00mkYxa\nfLgM6qOTK4cAAMx8P4BT2y4ionOJ6FYi2ktElzrOO52IxkT0Uk97BGHhmEQKhoF2aK27iu8A4o4U\n6q2wm/eQ1HsiTdQ99VbYISoo3Ym0RVQNR2WQ+pradlTHQwfaOJFC8x5M37EMXVKTMjoAAJQttNvy\nESmAywCcB2AHgIuIaIflvDcB+Iiv0YKwiFQSycYSQfQ6BT9VjSmpaZKe1quNp1LVqtKZue5EgO45\nBXWQbusma3RUFidnjhS6qY+G/N3prcpd7UcWNadQ8RYAnyKiqv/RjwJ4Y8s1Z6BooHcbABDRVQAu\nAHCzdt7PAfgggNM9bRGEhaRq5WxaIhjnTZXMUPSLFJpr3OZ70OoX0uZMXO055LKhsll3Iq2Rgsd6\nfKNthxLJhEcK5oK/PuitykfK8wNS5bwwBzYkvpLU9xLRHgAvKA+9pNyNzcVmAPuU1/uhVUET0WYA\nLwbwfIhTEJacuamPPNfKTcVepj0KXOojU2IaCIkUpjJPNSEMtHeTdSW/J+cYNg2qru2eUxgwUjDc\ng+k7skVviAdMtuRscwShvA3ALzFzTo4+H0R0MYCLAeDEE08c2ARBGIZGTmGW6iOfSMGwjGUa5G2R\nQs7AwbKCzKo+amuIp+3RHBYpmJLf5uWw6aZBivrII5JRmcziB8wHmZxtZV/9vPk1xItZHXEngBOU\n11vKYyo7AVxVOoSNAM4nojEz/7V6EjNfAeAKANi5c6dIYYWFZG7qI88dxYzqI4t81nQPAHBw3Cwg\nA6pIobv6qHovVH3knVMoI5OQgdZW8NcH09Kielw9b+EjhQ7sBrCdiLahcAYXAvgx9QRm3lb9TETv\nBvAh3SEIwrIwrVNozqYXIlJwqY9SW6Qw3ZcAAA6Ms8l51WcC6nJQoPoo7RspxFMf2b6jDzZnu0jq\no2hOgZnHRHQJih3bUgBXMvNNRPTq8v3LY323IMyDagAZWwbaWPi2RPBWH+XmOgUAODCJFLqpj6p2\n0PZIwZVT0J1t0lArNSOF6Uy8S/LWVnHcFVO0A8B4H6sYKYCZdwHYpR0zOgNmfkVMWwQhNs2cQl3m\nGYuo6iNNZXTgkDYwp9OZuJ44NpEkhIQ6qo8MdQqt6qNq2SvLa5sG+WLbB7orpnsAYHDW81MfzSc+\nEYQVZNTQ+C+g+sgyiNaWujKD+iitIoWsfjwwUiiuSZRIIakdD6tTcOUU6pFMlQtZnEhhcesUxCkI\nwkBY1UeGGe2QDFPR3K4+AqbLR3oEcSjzrwNQn9Pg6iNtJp6mdbtD1Ee27+jDxNmmdadqzimIUxCE\npWaU1tVH1d/0QvU+stYp1B2YLSF6wKI+0qWqbjvImJguKr/D1Ed52TeoeY7Z7sWLFJrqI71qfNaI\nUxCEgZjOgIv1YGrZh3co4kYKmvrokFl9VB336tZaDv7BvY8sjkrdaEefiTfs66I+GrROoV19pFeN\nzxpxCoIwEKr6KERV0xfv3kepfblFVx8dysxFVjb1UchMfPKcsv7qo+K4K1KopLT1ZS9fTAV/fbCq\nj5Tfi+6oZ404BUEYiGmdgr5WHnnnNc+WCDb1EREmbZsruw9ZGrfpg79+3GfJo5ZT6FmnoB43ndO0\nb87qI9s9ZM1oR3IKgrDk1FU1s4wU/Cp1beojfQkHsKuMGsc1VVJv9ZErUrB1GM1MkYIeyfjbpzJ8\nTsHsVE3RjkQKgrDk1GfASeM4D7wBfEXfnILuwAClHqGlTkE/f6bqI4PG3xopHFoQ9ZG1TsGQUxCn\nIAjLjdr7yDTQxooW+u68pl5rW24ZNaSd9XX9TjkFY0TlVh+lWgK/Oj49R3Namt3zjxT0Wgt7XkTU\nR4Kw5KQJGbelTA2zwSGJEilMllssvY80nb2+rNRuh0V95FD62JxtbZZtbc/RQ30UIaegR1oSKQjC\nCjJK7TNgIGak0GfnNd2B1Wf+beojW1Gbi2rwN9cpuNVHegK/OG7IKWgz8UWLFPTnV1MfZZJTEISV\nwL5WPvwOXipBkUJL47VGTsGyNm/PKXgsY6X25+QagL0iBVvxWkDOo2brwH2r/CKFbvLZoRCnIAgD\n4VIfAbFzCr51ClpOwbDRDeCvPtLP97FD7cRaVx+5l2pMieniuEnjr0cy9WUvX2Kpj/TnJ+ojQVhB\n6nUKzeTtkCqWipCWCNacQtp0YG31CLbiMJ+BTK2a1gf59kih7kSq4+o5dfvMy16+mAr++tC0b7o9\nqe2cWSNOQRAGYh7qo5CWCL3UR5bBX2845xcpKM9Jc0iuAdi0fSdQH1BtM/HFyynUn5+ojwRhBan1\nPkrdg9dQhCw1+KmPpj2OTPLPae8jrTgsoLfQSH1OwZFCPTFd3YN6jmrXIL2PhlQfadufivpIEFYY\nq/7eMHgNRUhLBC/1kTKzNt3DJFLQi9omM3G/ZSxb7sWdU9CdbTOB35iJL2qkoD2/el6kW1J8KMQp\nCMJApEkCZuBQZpZOxlAfDR8pTBOzrgT0ZCZO4XUA9pyCe5Mdn2W55ky8Xl+xKL2PnOqjTCIFQVgJ\n1Nn0zHMK3pFCS++jlnvQcwrV9pphdQqJUqegJI5bNiOyqY/GJvVRj0imbquojwRB6Iiq5Z+V+igk\nKVlFMvqmNLY6BWOkcKi5DDNKkqA6gKHVR66cQv86hTjqoyrCGhmiSNlPQRBWBHXpZSEjBVPzNYt8\ntriHegfT6rh6XvVzUJ1CSjiU50aH5FQfWSMFu/ooSQhEPesUBhQIZDkjoWmrcnekIOojQVhq1KWK\ntsFrKEJaItgGIKP6yHIPpmWYUULBdQoHDYnfNGlur6nSzCmUbS4MGv9qJt7FvpqtA++vrUuAzfsp\nSE5BEFYCdd3dOHhFVB/5DsbqNdXPJvmsb04BKGb+oeojXcWkfkdmaTFukrAW91B3cupMvPF9FDbQ\nxlAfqc+uimREfSQIK4i6j/Gi1imo11Q/m3MK2j2k0+PqedXPoXUKps8xNbhTaWzfaalT0B1TkfNY\nEPVR1mxJogsAJFIQhBWhHinUVTXAAuQUTPJHyzq9K1Igx0zct/eRqe2EyT4V0/adxfl19ZE+8Nfs\nW4DeR2natE/UR4KwgrTnFGKqj/wGY/Wa6mdbkZrNWTRnuklwTsHkRCaRjCWi8lIfWWbi3fdoHl59\nZHp+5joFSTQLwlJjmsUCcdVH+v4BLuyRQnOgBfRZfFOhZHrtu5+C61rbIOyrPjLNxE334UPsnILp\nO/Rai1kjTkEQBsLkCNTjMdRH0z2J/RK8QHOj+9TqCJSEqDI+NdfszYO8DT0f0bAvVH3kWA7rYp9+\nbUz10fQ7mvtMS05BEJYc2+ATU32UBQwg5o3u89ZBGgCIqNFkznSel/qoJaKy5xT81EcN+wwKJ19M\nBX998IsURH0kCCuBbZYdN1LwH0CM6qNMjxTsA2j1nu24rx01J2RocOcfKVSRT32WbVIfhdhXu3bg\n/bVtkYzUKQjCCmJfj68G43iJ5l7qI8tMWh9AbZGC7XobaW2QDlQfqfYZBmzbTDzEPtO1Q0V5WZ4b\nI5k1oz4ionOJ6FYi2ktElxre/89EdD0R3UBE/0JEp8S0RxBiYku4xqxTGAcMID7qI1fS2B4phM3E\n23MKZudp2r6zuofJORFyCsXnDuPQ9VqL4juShqOujs+DaN9KRCmAywCcB2AHgIuIaId22u0Avp+Z\nnw7gtwFcEcseQYiNNacQsU4hC5Av+qmPzLN4ABil5b7HqX3Q9YsUuuYUPNVHlkghTaabBvkyfKTA\nRme7ViKFMwDsZebbmPkggKsAXKCewMz/wsz3ly8/DWBLRHsEISrzVR+F5BTsxV6upOw0UkiMx33t\naIsUbBFVc/tOg/rIOBM3L3v50OaoQtFrLarvqCX/V3g/hc0A9imv95fHbLwKwN+Z3iCii4loDxHt\nueeeewY0URCGw16nMAP1UUidQi2p6ac+Ut9zHfeZibctV9mek64+qn7Ucwq+y14+tCW/Q9GjneI7\nmuojvWp8lixEopmIno/CKfyS6X1mvoKZdzLzzk2bNs3WOEHwpLa2rgyO1c8LqT7KuTb4JAa79ff0\n42ngTNzmeJKW5zTOuWYfEZUDal19pNtXOeXFiBRy4/NztR6ZNaOIn30ngBOU11vKYzWI6GQA7wJw\nHjN/I6I9ghCVmiMwNJObt/rItIylD0Amdc/kvdQ8+E+cgud6vT2n4J6V22bZvuqjLgNt5TCH2lMh\nM0hmTTmFJDD3MSQxI4XdALYT0TYiWg/gQgDXqicQ0YkArgHwE8z8pYi2CEJ0bDmFoWebKt1yCsU1\nec5g1mbrSdHKGXDkFCyJZt9Bt619hk3po+cUqu/MGstheoK8cmbhw93g6iPD8papTmElIwVmHhPR\nJQA+DCAFcCUz30REry7fvxzAGwAcC+Ad5VrkmJl3xrJJEGJiKsQqfi6OD1UVqzKNFHzUR/WZeLVv\ngUnCeSggYTuJFDx79bSptGzjb25bj1f2X8hzu33dcgqlTZY9HkLJbRXN7I52ZknM5SMw8y4Au7Rj\nlys//zSAn45pgyDMCqv6KGpOITxSqK6xbfuYlk7BVo9g6vJpOt5mh36NK1JgZqtyp16nkGPDulHj\nHP17fYmhPjI9v0fK/R6AKlk+v3TvQiSaBWEVsCZQE0JCseoU/Pcx0NU9tnYK08SsufGdqw7Ahy7d\nZKtDJkflnVPo0HV06MJDm316nmeekYI4BUEYCFc7haF38KqYRAoBLaurmbhtf+e2Hke2Lqm+Fbit\nld+G52RTWTVzCuaZuOlaH4YuPLR1SdW345xnTkGcgiAMRNueA3PvkqrNxKuB1pY4Ds4pdJCk1n+2\nq49s9xlbfVRFS0M5dGukkEmkIAgrh6tvkK4wGYphcgqhkYJZqtpFfWRKyJsjBbOto7RZp2CLZPqo\nj4aLFAw71xka4kmkIAgrgK1LKoCyE2bMOoVw9ZE9p2BWE7XWKfSOFKoBuPmcMkvrh/iRwrCSVL1V\nefEdSaOpn0QKgrACuPoGDb2DV0X1mT5jiLf6yDLzn6iPGlFQ2Jq9rXjNldSd5k7a1Ue2SKaP+mjQ\nnIIpilQdW9aMdmaJOAVBGAhX36B4OYViEPTpOaRvStNdfWRuiOer7rFGCo6krj2nkDQG1DiRwux2\nXpNIQRBWhHmpj7xn6NqmNLZtH4NzCoFr9q2RQqj6SG/b0TOSMV07VJsLnz2aszzvJJ8dCnEKgjAQ\nc1EfGYrMbDTVRy05hUD1kbcdhvbX6s/R1Ed96hQkUhAEIRT179g8G5xzpKANcOPOdQp+zsJuh+I8\nvesULOqjJLL6aPA6BUPOw+DYRH0kCCsAETln07HUR74tERq9j6rZd+86hbA2F53URxaVla7xX3j1\nkdG+pFGAJ5GCIKwItqUKffAaipABpDptrC0fWRPHnoN/qLqnU07BEtXoGn/bTDzEPtO1w1Y0N++h\nGSmI+kgQVgLrLDuNqz7yoYpkqpm4bZ1+kpg1yD8Bfydiw9QsUD1uSup6q48sM/EQ++rXDpdTmLYq\nd++nIJGCIKwQrln2vHMKhR2kRAqB6qOqfsG63BSmPkqovuWkl/rItJ+Crj7qGcnUP3+47TjHjuW6\nhvpInIIgrAbV+n6bdHIoQpOSagM5a6RgGUQHUx9NZu714We6vWY39ZFrJh5in+naIRy6q61IztP9\nNsaGWotZIk5BEAbE1kk01WaDQ9EvUhhYfeQp+UwdM3ddYlrhoz5yzcSLa3v0Psr6/+4mDQgtz6/a\naCcz1FrMEnEKgjAgLuVOvDoF/z/jUTrtszPtJ+S3b8LQ6iPT+brEtMKpPurY4M8HveCvD3b7mqqw\nLg5sKMQpCMKAuGbZi5dTaBtE9QHYtvNaN/WRKbLoFilorcB7RjKmawfNKVjsU38vklMQhBVh5pFC\nYEsEo/rIsy3E0Ooje6RgyimYB/w0SSZy1baZ+OLkFMzPT831SE5BEFYEe51CMvc6hcKOHuojW04h\nNTsLlw2m760+I6hOoRYphLXt8CGK+sjy/MaT3IiojwRhZXDNshdGfaRXNHsut8RWH03sc9Up6M42\nNeUUhlMfVZcMEim0tBWp5xTEKQjCSmBdj09XQH3UWqcQmFPooD4yObCG+qhnJKOiF/z1oW37U8kp\nCMIK4tL4x4sUAtRHSp8dm6InvE5hQPWRZYc6VxQwiRQCG/z5MpRIwFt9lIn6SBBWhjWtPvJMeCcJ\ngWjISEFTH/WMZHRsS1qhBKmPpE5BEFaDuaiPQnIKyky8Ksiy9j6KpD6qzjUN0m3qI9Msu71Oobv6\nqPq8maiPFFWY5BQEYUWYee+jwJYIxkjB0NEV6FDRHLDkkSZknA1b1UdekUJY4twXteCvD/6RgqiP\nBGFlmIv6KLhOwVN95J1T6BIpJEYnYo8U7EtdWc5g5ijqo+r6YSIFtwR4nDHynJFz96WuIRCnIAgD\nkpbLIkTNP/zxAP1zdEJbIgTlFDwjiC5dSNOEjIO0NafQ0pIjy9mjTqHbcDeY+iiz2Kfs7lb1P5JI\nQRBWhNC18r6EyhdHSWKIFMwDre/g32XN3v2cHOojQ50CUDyHtpn4/CMFd86juAdztDNLxCkIwoBY\nZ8BpHPVRaFLSFCnol6eTJTCL+mgAdY8zUjAofdqigCzn1pl4L/XRkDkFy/NzRTuzJKpTIKJziehW\nItpLRJca3ici+qPy/euJ6LSY9ghCbEbprCOFQPVRovY+Kq7Vl7qqQct3GWZQ9ZFlhzqX+gjQZ9nL\nqT4a57m11mKWRHMKRJQCuAzAeQB2ALiIiHZop50HYHv538UA3hnLHkGYBWmSWGbAhaqGeVjH0ClS\nyKaRgq1WQP1XP25X9wTkNlJbpOBWH6W6AzPNsgevU0hmUqdQ3IO51mKWxIwUzgCwl5lvY+aDAK4C\ncIF2zgUA3ssFnwZwFBEdH9EmQYhKMQM2q2oAYOhgITinoMzEi70YzFGN+q9+fF7qo4Tq23eq3z3O\n83b1UceBdmbqI0e0M0tGET97M4B9yuv9AJ7pcc5mAF+LaJcgRKNQH5mPA8A5b/skhvxzf+CRQ42B\n0kWaJPjKN76Ns9/6Cdz90AHjtZWttgG4KVUtcxAUYof9Od1610M4+62fqB2/91sHrMtyAPCyP/40\nDhzKjHZU5yQB9tWuTwn/8m/3NmwK5aHvjAv7LM/vV665ARvWpcU5HW0dgphOYTCI6GIUy0s48cQT\n52yNINi58PQTcMbWYxrHz95xHL749YcGkTaqPPnxj8EPn/IE7/N/9Hu3TGzYftwROGXLUY1zzt5x\nHB58ZIxjD19fO3761mPwX5/3RDx985G141uOfhQuef5JOOupm7zt+LkXnISNR2xoHL/ojBOwzjCj\n337cEfjuxz+2cfy52zfiR57xBBws5b7PO2wdth93RO2cJ246Aj9z1pPwfdv97VN5xbO34qO33NXp\nWp2jHr0e2zYeXju2/bgjcOHpJ+DB7xwCAJyy5Ug8d/vGQb6vCzT0Gufkg4meBeA3mPmc8vUvAwAz\n/55yzh8D+Dgzf6B8fSuAs5jZGins3LmT9+zZE8VmQRCEVYWIrmPmnW3nxcwp7AawnYi2EdF6ABcC\nuFY751oALy9VSGcCeMDlEARBEIS4RFs+YuYxEV0C4MMAUgBXMvNNRPTq8v3LAewCcD6AvQAeBvDK\nWPYIgiAI7UTNKTDzLhQDv3rscuVnBvCamDYIgiAI/khFsyAIgjBBnIIgCIIwQZyCIAiCMEGcgiAI\ngjBBnIIgCIIwIVrxWiyI6B4A/97x8o0A7h3QnGVhLd73WrxnYG3e91q8ZyD8vr+LmVvLupfOKfSB\niPb4VPStGmvxvtfiPQNr877X4j0D8e5blo8EQRCECeIUBEEQhAlrzSlcMW8D5sRavO+1eM/A2rzv\ntXjPQKT7XlM5BUEQBMHNWosUBEEQBAdrxikQ0blEdCsR7SWiS+dtTwyI6AQi+kciupmIbiKi15bH\njyGifyCiL5f/Hj1vW4eGiFIi+hwRfah8vRbu+SgiupqIvkhEtxDRs9bIff9C+f/3jUT0ASI6bNXu\nm4iuJKK7iehG5Zj1Honol8ux7VYiOqfPd68Jp0BEKYDLAJwHYAeAi4hox3ytisIYwH9n5h0AzgTw\nmvI+LwXwMWbeDuBj5etV47UAblFer4V7/p8A/p6ZnwrgFBT3v9L3TUSbAfw8gJ3M/DQUbfkvxOrd\n97sBnKsdM95j+Td+IYDvKa95RznmdWJNOAUAZwDYy8y3MfNBAFcBuGDONg0OM3+NmT9b/vwQikFi\nM4p7fU952nsA/Mh8LIwDEW0B8CIA71IOr/o9HwngeQD+BACY+SAzfxMrft8lIwCPIqIRgEcD+CpW\n7L6Z+ZMA7tMO2+7xAgBXMfMBZr4dxf40Z3T97rXiFDYD2Ke83l8eW1mIaCuAUwF8BsBxyo52Xwdw\n3JzMisXbALwegLoB8qrf8zYA9wD403LZ7F1EdDhW/L6Z+U4AfwDgDgBfQ7Fb40ew4vddYrvHQce3\nteIU1hREdASADwL4b8z8oPpeubHRykjOiOiHANzNzNfZzlm1ey4ZATgNwDuZ+VQA34a2ZLKK912u\no1+Awik+AcDhRPTj6jmreN86Me9xrTiFOwGcoLzeUh5bOYhoHQqH8H5mvqY8fBcRHV++fzyAu+dl\nXwSeA+CHiegrKJYFX0BEf4bVvmegmA3uZ+bPlK+vRuEkVv2+fwDA7cx8DzMfAnANgGdj9e8bsN/j\noOPbWnEKuwFsJ6JtRLQeRVLm2jnbNDhERCjWmG9h5rcqb10L4CfLn38SwN/M2rZYMPMvM/MWZt6K\n4vf6f5n5x7HC9wwAzPx1APuI6CnloRcCuBkrft8olo3OJKJHl/+/vxBF7mzV7xuw3+O1AC4kog1E\ntA3AdgD/2vlbmHlN/AfgfABfAvBvAH5l3vZEusfnoggprwfw+fK/8wEci0Kt8GUAHwVwzLxtjXT/\nZwH4UPnzyt8zgGcA2FP+vv8awNFr5L5/E8AXAdwI4H0ANqzafQP4AIqcySEUUeGrXPcI4FfKse1W\nAOf1+W6paBYEQRAmrJXlI0EQBMEDcQqCIAjCBHEKgiAIwgRxCoIgCMIEcQqCIAjCBHEKgjADiOjj\nRLTm9hEWlg9xCoIgCMIEcQrCmoWIDieivyWiL5S9+V9GRG8got3l6yvKqtlqpv+HRLSn3LvgdCK6\npuxt/zvlOVvLvQ3eX55zNRE92vC9P0hEnyKizxLRX5a9qkBEv1/uhXE9Ef3BbJ+GIBSIUxDWMucC\n+Cozn8JFb/6/B/B2Zj69fP0oAD+knH+QmXcCuBxFi4HXAHgagFcQ0bHlOU8B8A5m/m4ADwL4WfUL\niWgjgF8F8APMfBqKiuTXlde/GMD3MPPJAH4nzi0LghtxCsJa5gYAZxPRm4jo+5j5AQDPJ6LPENEN\nAF6AYuOSimuV627iYv+KAwBuw7Qh2T5m/ufy5z9D0XpE5UwUGz39MxF9HkUPm+8C8ACA7wD4EyJ6\nCYCHB71TQfBkNG8DBGFeMPOXiOg0FP2hfoeIPoZi9r+TmfcR0W8AOEy55ED5b678XL2u/pb0vjH6\nawLwD8x8kW4PEZ2BosHbSwFcgsIpCcJMkUhBWLMQ0RMAPMzMfwbgzShaTwPAveU6/0s7fOyJRPSs\n8ucfA/BP2vufBvAcIjqptOFwInpy+X1HMvMuAL+AYntNQZg5EikIa5mnA3gzEeUoulH+DIotDm9E\nsbPV7g6feSuKvbGvRNHK+p3qm8x8DxG9AsAHiGhDefhXATwE4G+I6DAU0cTrOny3IPRGuqQKwkCU\nW6B+qExSC8JSIstHgiAIwgSJFARBEIQJEikIgiAIE8QpCIIgCBPEKQiCIAgTxCkIgiAIE8QpCIIg\nCBPEKQiCIAgT/j9YXHohEvJ/ogAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c8fda58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFkFJREFUeJzt3X9sXWd9x/HP1zfE0FGaLAlD0KSuNATKYFWpw2Z1A3vp\nUFc6+gcoYhOUH39ERClqEVOhVJkQQcooEqOiKJIpZVRUYlHLBEIZUBJb6lQXxSlljBQmVtI6dIgQ\nhVER4RD7uz+unTjOvb7H9z73nOd5zvslVe61T8597nHyud/zfZ5zrrm7AAD5GKh6AACAsAh2AMgM\nwQ4AmSHYASAzBDsAZIZgB4DMEOwAkBmCHQAyQ7ADQGbWVPGkGzdu9KGhoSqeGgCSdfTo0V+5+6ZO\n21US7ENDQ5qenq7iqQEgWWb2bJHtaMUAQGYIdgDIDMEOAJkh2AEgMwQ7AGSGYAeAzBDsALo2NSXt\n29f8inhUso4diNnUlDQ5KY2OSiMjVY8mXlNT0vbt0tmz0tq10qFDHK9YEOzAEoRVcZOTzeM0N9f8\nOjnJsYoFrRhgiVZhhdZGR5tvfo1G8+voaNUjwiIqdmCJxbBarNgJq/ZGRppnNLSt4kOwA0sQVqsz\nMsIxihHBDixDWCF19NgBIDMEOwBkhmAHgMwQ7ACQGYIdADJDsANAZoIEu5mtM7OHzezHZva0mbFY\nDAAqEmod+72SvuXu7zCztZIuC7RfAMAq9RzsZnaFpDdJeq8kuftZSWd73S8QG+76iFSEqNivlnRS\n0pfM7BpJRyXd7u6/DbBvIArc9REpCdFjXyPpDZL2u/u1kn4r6aPLNzKznWY2bWbTJ0+eDPC0QHm4\n6yNSEiLYT0g64e7fW3j8sJpBfxF3H3f3YXcf3rRpU4CnBcrDLWqRkp5bMe7+CzObMbPXuPtPJG2X\ndKz3oQHx4K6PSEmoVTEflPTQwoqYZyS9L9B+gSgwcYqUBAl2d39K0nCIfQGxYeIUqeHKU6ADJk6R\nGoId6ICJU6SGT1ACOmDiFKkh2IEC+Lg8pIRWDFADU1PSvn3Nr8gfFTuQOVb11A8VO5A5VvXUD8EO\nZI5VPfVDKwbIHKt66odgB2qAVT3hpHB7CYIdAApKZSKaHjsAFJTKRDTBDgAFpTIRTSsGAApKZSKa\nYAeAVUhhIppWDArjsnQgDVTsKCSV1QD9ksISN2ARwY5CWq0GqEvA1f1NDemhFYNCUlkN0A+pLHED\nFlGxo5BUVgMUtZrWyuKb2mLFXqc3NaSJYEdhKawGKGK1rZXc3tSQP4IdtdPNfEEub2qoVlmT8AQ7\naofWCqpQ5iQ8k6dIQsg19Iutlb17WeGC8pQ5CU/Fjuj1o9Lpd2vlnnvu0bZt2zQ2NtZ2m4mJCR05\nckR33nln/waCaJR5pkjFjlJ1U3mnuNxw27Zt2rFjhyYmJlr+fGJiQjt27NC2bdtKHhmqUuqZoruX\n/t91113nqJ/HH3d/yUvcG43m18cf7++fq9rhw4d948aNfvjw4ULfBzqRNO0FMpaKHaXptvJOtSc+\nNjamAwcOXFS5L1bqBw4caNmm4X48CIEeO0rTS48x1eWGS8N9165d2r9//4qhzq0LEEKwit3MGmb2\nfTP7Zqh9Ii+pVt69Ghsb065du7R3717t2rWrbah//OPS7GxacwmIU8iK/XZJT0t6WcB9IjOpVt69\nmJiY0P79+7Vnzx7t379fY2NjF4X7YqU+OyvNz0sDA6yvR2+CVOxmdqWkt0q6P8T+gE5i7UUvH9fS\nnvonPvGJS3ru0oW5h8VQv+GGep3RoA+KzLB2+k/Sw5KukzQq6ZttttkpaVrS9JYtW0qYP0auYl0l\ns3xcn/tcsVUxsb4exEdlrYoxs5sl/dLdj3Z4Axl392F3H960aVOvT4uAYq1+24l1XfvScc3OTugj\nH2m9+mX5apm6zj2gf0L02K+X9DYzu0nSiyW9zMy+4u7vCrBv9FmKKzFivdfL0nGZHdGnPtV69Yt0\nIdyPHDmisbGxWs49oH+sWd0H2pnZqKR/cPebV9pueHjYp6engz0vurdvn7RnT7PKbDSaVeNdd1U9\nqs5i/ai6WMeFPJjZUXcf7rQd69hrrlX1m0I4xVrhxjou1EvQYHf3SUmTIfeJ/lr+IRJSeq0ZABej\nYsdFVea+ffX90GogF9wrBhep84dWF5XaKqLUcbxXj4odF+HzPVfWyyqibuYuUpjv6KcUV23FgGDH\nJZgAbK+bz0uVugsoQq374113tGKAVei2VdXNRVWxXohVJlqD3aFiB1ah21ZVNxdVxXohVploDXYn\n6AVKRXGBEurYO6bHXm8hfpdFL1Ai2FG61HvHhC1WK9Tf+aLBTo8dpVupdxz70rbFf6B79jS/xjrO\n1Yj9mOeg7PkSeuwoXbvecQqVfG6rNFI45jkoe76EYEfp2k2IpRCauU1opnDMc1D2JDDBjkq0Wiuf\nQmjmtkqj6DFnXqF3ZV4fwuQpokKAlK/TMaddEw9u24skcdVr+Todc9o16WFVDIAVcfVneqjYgQz0\ns4WV27xCHRDsQOLK6IHTIksLrRgkhwtqLsbNwrAcFTuSwgqNS6WwTBTlItiRFFZoXIoeOJYj2JEU\nqtPW6IFjKYId0ZuamdLk8UmNDo1qZGSE6hTogGBH1KZmprT9we06O3dWaxtrdejWQxoZGSHQgRWw\nKgZRmzw+qbNzZzXnczo7d1aTxydbbjc1M6V9j+3T1AxLZQAqdkRtdGhUaxtrz1fso0Ojl2zTsqrf\nTEmP+iLYEbWRzSM6dOuhCz32FoHdqqrPIdgvmlvI4PWgPAQ7WoopVEY2j6w4hiJVfWo4C0EvCHZc\nIrVQKVLVpybXsxCUo+dgN7PNkh6U9EeSXNK4u9/b635RnRRDpVNVn5ocz0JQnhAV+zlJH3b3J83s\ncklHzexRdz8WYN+oAKHSX0XaXDmehaA8wT9Bycy+Luk+d3+03TZ8glL8Yuqx5yS1NhfiUsknKJnZ\nkKRrJX0v5H5jlmsA5tbaiEWKbS6kJ1iwm9lLJT0i6Q53/02Ln++UtFOStmzZEuppK0X1hdWizYUy\nBLny1MxepGaoP+TuX2u1jbuPu/uwuw9v2rQpxNNWruhVkcCixd753rG9FALomxCrYkzSFyU97e6f\n6X1I6aD6QjeqbnPl2j7EBSFaMddLerekH5rZUwvf+5i7Hwyw76ixcgGpoX1YDz0Hu7v/hyQLMJYk\nVV19AavB5G09cHdHoEYW24cNa1zUPuTumHnJ6pYCdekd1uV1IrxW7UPaM/nJJtjr8pezLq8T/bO8\nfUh7Jj/ZtGLqsvSwLq8zhG7bC3VrS7RrzyBd2VTsdVl6WJfX2atuz2xSPyPqpk3H6q78ZBPsdfnL\nWZfX2atu2wsptyV6eVNidVdesgl2qT5/OevyOnvR7ZlNymdEk8cnNTs3q3mf1+zcbFJvSggrq2AH\nFnV7ZpPyGdGGyzZo3uclSfM+rw2Xbah4RKgKwY5sdXtmE+qMqOxlqafOnNKABjSveQ1oQKfOnOr7\ncyJOBDvQB1VMwo4OjWpwzWCSbSSERbADfVDFJGzKbSSERbADfVDVJCwT65AIdqAvqJ5RJYId6BOq\nZ1Qlm1sKAACaCHYAyAzBDgCZIdgrULe7B+aK3yNixeRpyVK/eyCa+D0iZlTsJeN+6nlI4ffIGUV9\nUbGXLOW7B+KCqn+Pne5DwxlFvRHsJePClTxU+XssEtop31cevSPYK8CFK+2l9EHdVf0ei4R21WcU\nqBbBnqDYwi/UeGgfFFMktDkzrDeCPQFLg1NSVOEXMoxpHxRTNLQ5M6wvgj1y40fHtfvgbs3Pz2tw\nzaDec817ogq/kGFM+6A4QhsrIdgjNjUzpdsO3qZz8+ckSbPnZiUpqvALGca0D4AwCPaITR6f1JzP\nnX88MDCgW6+5Vbdec2s04Rc6jKlEgd4R7BEbHRrVYGNQs3OzalhD99103/nQiyn8CGMgLkGC3cxu\nlHSvpIak+939n0Lst+5oTQDoRs/BbmYNSZ+X9NeSTkg6YmbfcPdjve4bVMM5iG15KvIXomJ/o6Sf\nuvszkmRmX5V0iySCveYINNbmoxohgv1VkmaWPD4h6c8C7BcJI9CaWJuPKpR2d0cz22lm02Y2ffLk\nybKetnTcUa8phbsflmFxOWjDGlEsT0U9hKjYfy5p85LHVy587yLuPi5pXJKGh4c9wPNGhyr1Ai42\namICHFUIEexHJL3azK5WM9DfKenvA+w3Oamddj/33D26/PJtWr9+rO02p09P6IUXjmjLljtXtW8C\n7QImwFG2noPd3c+Z2W2Svq3mcscH3P1HPY8sQalVqZdfvk3Hju3Q1q0HWob76dMT53/ejVaBxoQq\n0H/mXn5XZHh42Kenp1f951IIhaJjjOW1LA3vpeHe7vu9oFUF9MbMjrr7cKftkrnyNJVQKHLaHdNr\nWb9+TFu3HrgoxPsR6lJ6rSogVcl85mnMqyxWuxImtteyNNx/9rN/7EuoS6wQAcqSTMUea/+6m+o7\nxteyfv2YXvnKXXr22b266qo9wUNdYkIVKEsywR5rKHTTXijyWvrZg2+179OnJ/T88/t11VV79Pzz\n+7Vu3Vjfwj2W3x2Qq2SCXYozFLqtvld6Lf3swbfa92tf+ruL2i/r1o31rR0DoP+S6bHHarH63ju2\nN1gA97MHv3zfT/7si5eE+NKe++nTE8GeW+LKXKAMSVXssQp9JrHSWUCvLZql+x7+w4b+RI9o69av\nXVKZt1ot06uYVgMBOSPYI9SuB98pGIuE/tJ9/8UVJ/S6ze9oG9qL4f7CC0cu2qbbNxeWOwLlINgj\n1eosYKVgXE01vJozjPXrxy4J9W6r7hhXAwE5qmWPPeY+70pjW2kdeFlr43t5nn7MRwC4VO0q9pj7\nvJ3GttIyydVUw8tbKatprfRadce4sgnITe2CPUSft19rzIuMrV0wFl3nv/zN47M3flZ3fOuOwm90\nsV5PAOCC2gV7rxVnPyv+Mqrh5W8ejxx7pKsLrAh0IF61C/ZeK86QKzuWV/5lVMPL3zzevvXteuy5\nx1Z8M4nlTpQAisku2Isu+SsaUMv3F2plR7vKv9/VcKs3j9e//PVtj9lKZygEPhCnrII9dJtk/Oi4\nbjt4m+Z8ToONwfP7C1FVL6/8H/zBg6WF5PI3j5XeTNqdocQ8CQ3UXVbBHrpNsvvgbp2bPydJmp2b\nPb+/EFX10sp/zcAaPfDUA5qbn4uuKm53hsLFRkC8sgr2kBfATB6f1Pz8/PnHDWsEvaBmaeX/3P89\npy88+YUgVXHoN4J2ZyhcbATEK6tgDzn5ODo0qsE1g/rdud/JzPShkQ8Fr0gXK/+pmSl9+Qdf7rkq\nHj86rt0Hd2t+fl6DawaDtUdanaGw7BGIV1KfeVq2dj32XrSrqFt9fzUV+9TMlN78L2/W7+d/L0ka\n0IA++Vef1F1/eVdP4wUQj+w+87QKp86c0rzPa97ng/SRVwrqXqviyeOTmvO5848HBgZojwA1RbCv\nIHQfudtPWyryZjI6NKrBxqBm52bVsIbuu+k+2iNATRHsKwjdRw5x1Wu7sdDzBrCIHnvJFsN5w2Ub\ndOrMqcIhzLpxAPTYl6l6PfiixedebUj3sm48ltcOoBy1CPbYqt1uQrrbNk5srx1A/yUV7Ll8JFu7\nkC7aQ99w2YbzH3DRz0ofQJqSCfacPpKt1URnkdfXTRsnttcOoP+SCfZeKs8YV4wsX8ZY9PWt9jjE\n+NoB9FdPwW5mn5b0t5LOSvofSe9z91+HGNhyuX8kW9HX181x6PW1h5h8ZQIXKE9Pyx3N7C2SDrv7\nOTP7lCS5+0c6/blulzvmHg5FX1+ZxyHE5CsTuEAYpSx3dPfvLHn4hKR39LK/TmKvuru1NKiL3Nul\nzOMQYvKVCVygXCF77O+X9K/tfmhmOyXtlKQtW7YEfNq0xV7Nhph8ZQIXKFfHYDez70p6RYsf3e3u\nX1/Y5m5J5yQ91G4/7j4uaVxqtmK6Gm2GYq9mQ0y+MoELlKtjsLv7DSv93MzeK+lmSdu9ivsTJC6F\najZE6yfXNhoQo15Xxdwo6U5Jb3b3M2GGVC9UswBC67XHfp+kQUmPmpkkPeHuH+h5VDUTYzWb+wok\nIGe9ror541ADQTxWO6HLmwAQl2SuPEV5VjOhG/uqHqCOBqoeAMKampnSvsf2aWpmqut9LE7oNqzR\ncUK31ZsAgGpRsWckVPW8mgndFFb1AHVDsGck5Jr4ohO6rOoB4kOwZ6Sq6jnGVT1AnRHsGaF6BiAR\n7NmhegbAqhgAyAzBDgCZIdgTFGKtOoB80WNPDFd6AuiEij0xXOkJoBOCPTGrudwfQD3RikkMa9UB\ndEKwJ4i16gBWQisGADJDsANAZgh2AMgMwQ4AmSHYASAzBHuGuOUAUG8sd8wMtxwAQMWeGW45AIBg\nzwy3HABAKyYz3HIAAMGeoV5uOTA1M8WbApA4gh3nMfEK5IEeO85j4hXIQ5BgN7MPm5mb2cYQ+0M1\nmHgF8tBzK8bMNkt6i6Tneh8OqsTEK5CHED32f5Z0p6SvB9gXKsa93oH09dSKMbNbJP3c3X8QaDwA\ngB51rNjN7LuSXtHiR3dL+piabZiOzGynpJ2StGXLllUMEQCwGubu3f1Bs9dLOiTpzMK3rpT0vKQ3\nuvsvVvqzw8PDPj093dXzAkBdmdlRdx/utF3XPXZ3/6Gkly95wuOSht39V93uEwDQO9axA0Bmgl15\n6u5DofYFAOhe1z32np7U7KSkZ0t/4uI2SqKl1BnHqTOOUTEcp2Je4+6Xd9qoknvFuPumKp63KDOb\nLjJBUXccp844RsVwnIoxs0KrTuixA0BmCHYAyAzB3tp41QNIBMepM45RMRynYgodp0omTwEA/UPF\nDgCZIdhbMLNPm9mPzew/zezfzGxd1WOKiZndaGY/MbOfmtlHqx5PjMxss5lNmNkxM/uRmd1e9Zhi\nZWYNM/u+mX2z6rHEzMzWmdnDC9n0tJm1vQ0rwd7ao5Je5+5/Kum/Jd1V8XiiYWYNSZ+X9DeStkr6\nOzPbWu2oonRO0ofdfaukP5e0m+PU1u2Snq56EAm4V9K33P21kq7RCseMYG/B3b/j7ucWHj6h5g3O\n0PRGST9192fc/aykr0q6peIxRcfd/9fdn1z4/xfU/Ef4qmpHFR8zu1LSWyXdX/VYYmZmV0h6k6Qv\nSpK7n3X3X7fbnmDv7P2S/r3qQUTkVZJmljw+IQJrRWY2JOlaSd+rdiRR+qyaH9QzX/VAIne1pJOS\nvrTQtrrfzP6g3ca1DXYz+66Z/VeL/25Zss3dap5SP1TdSJEyM3uppEck3eHuv6l6PDExs5sl/dLd\nj1Y9lgSskfQGSfvd/VpJv5XUdn6rklsKxMDdb1jp52b2Xkk3S9rurAld6ueSNi95fOXC97CMmb1I\nzVB/yN2/VvV4InS9pLeZ2U2SXizpZWb2FXd/V8XjitEJSSfcffGs72GtEOy1rdhXYmY3qnl6+DZ3\nP9Np+5o5IunVZna1ma2V9E5J36h4TNExM1OzH/q0u3+m6vHEyN3vcvcrF+4M+05Jhwn11hY+vGjG\nzF6z8K3tko612762FXsH90kalPRo89+nnnD3D1Q7pDi4+zkzu03StyU1JD3g7j+qeFgxul7SuyX9\n0MyeWvjex9z9YIVjQto+KOmhhYLqGUnva7chV54CQGZoxQBAZgh2AMgMwQ4AmSHYASAzBDsAZIZg\nB4DMEOwAkBmCHQAy8//TbHrmM3LBLgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116450eb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAD8CAYAAAC2PJlnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGRpJREFUeJzt3X/wXXV95/HniwQodUFkCUIJ3yZsYWcBK8oXTKuwxVJl\nXWywdpQOAt26RNAy7dZZB2W1FnVGGZGOu7N0QksFF2SoILgUlhoVutsOsAkGE1C2/FKIUYKoMOoE\nKu/9457A9cv3nnNzk5v7zTfPx8yde8/nnPM9n89X/L7y+XHuSVUhSdIodpt0BSRJOy9DRJI0MkNE\nkjQyQ0SSNDJDRJI0MkNEkjQyQ0SSNDJDRJI0somGSJL3Jqkk+zfbS5L8NMna5vUXfccek2RdkgeS\nfDpJJldzSRLAwkldOMkhwBuAb8/Y9WBVHT3LKZcCZwN3AjcDJwO3dF1n//33ryVLlmxbZSVpF7Nm\nzZonqmpR13ETCxHgEuB9wI1dByY5CNinqu5otq8ETmWIEFmyZAmrV6/exqpK0q4lybeGOW4iw1lJ\nlgMbquqeWXYvbYaybk9yfFN2MPBY3zGPNWWSpAkaW08kySrgwFl2XQB8gN5Q1kwbgamq+n6SY4Ab\nkhw5wrVXACsApqamtvZ0SdKQxhYiVXXSbOVJXgEsBe5p5sYXA3cnOa6qvgtsbs5fk+RB4HBgQ3Pc\nFoubskHXXgmsBJienvZriiVpTHb4cFZVrauqA6pqSVUtoTc09eqq+m6SRUkWACQ5FDgMeKiqNgJP\nJVnWrMo6kyHmUiRJ4zXJifXZnABcmORZ4DngnKp6stn3buAzwF70JtQ7J9UlSeM18RBpeiNbPl8H\nXDfguNXAUTuoWpKkIXjHuiRpZIbIABt++FO+dN/3Jl0NSZrTDJEB3nPV3Zx95Wp++JNnJl0VSZqz\nDJEBFuzW+2quux5+suNISdp1TXxifa565eJ9WfOtH3DhTffx1fsfn3R1JGmrXbj8KHZfMN6+giEy\nwO4Lez2RZ3/2HF/+hiEiaefzp28udl8w3msYIi32XLgbd35g1hvvJUk4JzKYX5YiSZ0MkRY+9kqS\n2hkiA9gRkaRuhkiLYFdEktoYIpKkkRkiA1Q5oCVJXQyRFk6sS1I7Q2QAOyKS1M0QaWFHRJLaTTRE\nkrw3SSXZv9k+PcnavtdzSY5u9t2W5P6+fQdMsu6SpAl+7UmSQ4A3AN/eUlZVVwFXNftfAdxQVWv7\nTju9ecLh2DmaJUndJtkTuQR4H4P/Xv8ecM2Oq86LxZl1SWo1kRBJshzYUFX3tBz2duBzM8quaIay\nPhj/wkvSxI1tOCvJKuDAWXZdAHyA3lDWoHNfA/ykqtb3FZ9eVRuS7A1cB5wBXDng/BXACoCpqamR\n6u/qLEnqNrYQqapZv0O9metYCtzTdCYWA3cnOa6qvtscdhozeiFVtaF5fzrJ1cBxDAiRqloJrASY\nnp4eOQ7s6khSux0+sV5V64DnV1YleQSYrqonmu3dgLcBx/cdsxDYt6qeSLI7cAqwaqz1dGpdkjrN\nxYdSnQA8WlUP9ZXtCdzaBMgCegFy2dhrYldEklpNPESqasmM7duAZTPKfgwcs+NqJUkahnesD+DE\nuiR1M0RaOJolSe0MEUnSyAyRFt7PKEntDBFJ0sgMkQF8sqEkdTNEWjiaJUntDJEB7IdIUjdDpIUd\nEUlqZ4hIkkZmiAzgvLokdTNEWnifiCS1M0QG8KvgJambIdLCfogktTNEJEkjM0QGcGJdkroZIi2c\nV5ekdhMJkSQfTrIhydrm9aa+fe9P8kCS+5O8sa/8mCTrmn2fzpiXTtkRkaRuk+yJXFJVRzevmwGS\nHAGcBhwJnAz89yQLmuMvBc4GDmteJ4+/inZFJKnNXBvOWg5cU1Wbq+ph4AHguCQHAftU1R3V+3rd\nK4FTJ1lRSdJkQ+S8JF9PcnmSlzVlBwOP9h3zWFN2cPN5ZvmskqxIsjrJ6k2bNo1UOSfWJanb2EIk\nyaok62d5Lac3NHUocDSwEbh4e167qlZW1XRVTS9atGjkn+PEuiS1WziuH1xVJw1zXJLLgJuazQ3A\nIX27FzdlG5rPM8slSRM0qdVZB/VtvgVY33z+InBakj2TLKU3gX5XVW0EnkqyrFmVdSZw43hr6XiW\nJHUZW0+kw0VJjqb3l/oR4F0AVXVvkmuB+4B/Bt5TVT9rznk38BlgL+CW5jVWjmZJUruJhEhVndGy\n72PAx2YpXw0cNc56/fz1dtSVJGnnNdeW+M4pTqxLUjtDRJI0MkNkAIezJKmbIdIiTq1LUitDZACf\nbChJ3QyRFk6sS1I7Q0SSNDJDZAAn1iWpmyHSwtEsSWpniAxgR0SSuhkiLcb8BF5J2ukZIpKkkRki\nAzixLkndDBFJ0sgMkQG8Y12SuhkiLZxXl6R2k3o87oeTbEiytnm9qSn/rSRrkqxr3l/fd85tSe7v\nO+eASdRdkvSCST0eF+CSqvrkjLIngDdX1XeSHAXcChzct//05gmH4+doliR1mmSIvEhVfa1v815g\nryR7VtXmSdTH4SxJajfJOZHzknw9yeVJXjbL/rcCd88IkCuaoawPpuVOwCQrkqxOsnrTpk0jVc6O\niCR1G1uIJFmVZP0sr+XApcChwNHARuDiGeceCXwCeFdf8elVdSRwfPM6Y9C1q2plVU1X1fSiRYtG\nb4PfniVJrVqHs5Kso+Uf5VX1qy37ThqmAkkuA27q214MfAE4s6oe7Pt5G5r3p5NcDRwHXDnMNSRJ\n49E1J3JK8/6e5v2zzfvp23LRJAdV1cZm8y3A+qZ8X+BvgfOr6h/6jl8I7FtVTyTZvanXqm2pQ5fy\nlnVJ6tQaIlX1Legtva2qV/XtOj/J3cD5I173oiRH0+vlPMILw1Z/CPwK8KEkH2rK3gD8GLi1CZAF\n9ALkshGvPTQn1iWp3bCrs5LktVt6B0l+nW2YT6mqWeczquqjwEcHnHbMqNeTJI3HsCHyTuDyJC+l\n96ymHwB/MLZazQEOZklSt6FCpKrWAK9sQoSq+tFYazVHOJolSe2GGpJK8tIknwK+DHw5ycVbAmW+\ncl5dkroNO69xOfA08Lbm9RTw1+Oq1Fzhkw0lqd2wcyL/qqre2rf9Z0nWjqNCkqSdx7A9kZ8med2W\njSSvBX46nirNDY5mSVK3YXsi59L73qotq7OeBM4aW63mCAezJKndsKuz1tJbnbVPs/3UWGs1B3jH\nuiR129rVWV8BvrIrrM4C7IpIUgdXZ0mSRubqrAEczJKkbq7OauFoliS1G7Yncg5w5YzVWb8/rkrN\nCXZFJKnTsKuz7mEXW50F3rEuSV2GCpEke9J75vkSYOGWP65VdeHYaiZJmvOGHc66EfgRsAbYPL7q\nzB3leJYkdRo2RBZX1cnb66JJPgycDWxqij5QVTcnWQJ8A7i/Kb+jqs5pzjkG+AywF3Az8Ec15jsC\nHcySpHbDhsg/JnlFVa3bjte+pKo+OUv5g1V19Czll9ILnjvphcjJwC3bsT4/xxvWJalba4gkWUdv\nndJC4D8keYjecFaAqqpfHX8VIclBwD5VdUezfSVwKmMMkd51xvnTJWnn19UTOWWM1z4vyZnAauC9\nVfWDpnxpcyPjj4D/UlX/GzgYeKzv3MeaMknSBHWFyA+q6qkk+23tD06yCjhwll0X0Bua+gi9Xs5H\ngIvpPbN9IzBVVd9v5kBuSHLkCNdeAawAmJqa2trTAYezJGkYXSFyNb3eyBp6f/D7B3gKOHTQiVV1\n0jAVSHIZcFNzzmaa1V9VtSbJg8DhwAZgcd9pi5uyQddeCawEmJ6eHjkO4tS6JLVqDZGqOqV5X7o9\nL5rkoKra2Gy+BVjflC8CnqyqnyU5FDgMeKiqnkzyVJJl9CbWzwT+6/as00wu8ZWkbl0T669u219V\nd4943YuSHE2vN/MI8K6m/ATgwiTPAs8B51TVk82+d/PCEt9bGPOkOjixLklduoazLm7ZV8DrR7lo\nVZ0xoPw64LoB+1YDR41yPUnSeHQNZ524oyoy1zixLknduoazXl9VX0nyO7Ptr6rrx1MtSdLOoGs4\n69/SeyTum2fZV8C8DRE7IpLUrWs460+bj+c0y2+fN8q9I5Kk+WXYJxten+T5wElyIPCl8VRp7vB5\nIpLUbtgQuQH4myQLmm/a/Tvg/eOq1FzgxLokdRv2yYaXJdmDXpgsAd5VVf84zorNBfZDJKld1+qs\nP+nfBKaAtcCyJMuq6lPjrJwkaW7r6onsPWP7+gHl85DjWZLUpWt11p/tqIrMRc6rS1K7ruGsP6+q\nP07yP5nln+ZV9dtjq9mEObEuSd26hrM+27zP9hjbec+eiCS16xrOWtO8375jqiNJ2pkM+4z1We2o\nZ6xPgqNZktRtks9Yn/N8sqEktesazvrWzLIkp1TVTeOr0txQzqxLUqdhv/ak34XbetEkH06yIcna\n5vWmpvz0vrK1SZ5rnoBIktuS3N+374BtrUd3Pcd9BUnauQ31tSczbK8/rZdU1c+t+qqqq4CrAJK8\nArihqtb2HXJ684RDSdIcMEpP5F3dh2wXvwdcs4Ou9SIOZklSt6F6IjOfbJhkMfAjYF1VPT7itc9L\nciawGnhvVf1gxv63A8tnlF2R5Fl6z2H/aI154sLRLElqN+xw1juBXwO+2mz/BrAGWJrkwqr67MwT\nkqwCDpzlZ10AXAp8hN4/+D8CXAz8Qd+5rwF+UlXr+847vao2JNmbXoicAVw5W2WTrABWAExNTQ3Z\nxJ/nvLokdRs2RBYC/6aqvgeQ5OX0/oC/Bvh7Xriz/XlVddIwPzjJZcDM1V6nAZ+b8fM2NO9PJ7ka\nOI4BIVJVK4GVANPT06PHgTPrktRq2DmRQ7YESOPxpuxJ4NmtvWiSg/o23wKs79u3G/A2+uZDkixM\nsn/zeXd696/091IkSRMwbE/ktiQ3AX/TbP9uU/YS4IcjXPeiZuluAY/w85P1JwCPVtVDfWV7Arc2\nAbIAWAVcNsJ1h+ZoliR1GzZE3gP8DvC6ZvsK4LpmYvvErb1oVZ3Rsu82YNmMsh8Dx2ztdbaVg1mS\n1G7Yx+NWkv8DPEPvH+l3jXtl1KTN8+ZJ0nYx1JxIkrcBd9EbxnobcGeS3x1nxeYC59Ulqd2ww1kX\nAMduuSckySJ68xKfH1fFJElz37Crs3abcVPh97fiXEnSPDVsT+R/JbmVF+7deDtw83iqNHc4miVJ\n7YadWP/PSd4KvLYpWllVXxhftSbPeXVJ6jb0t/hW1XX0vm5klxFn1iWpVdfjcZ9m9vvuQm/l7z5j\nqZUkaafQ9WTDvXdUReaa8p51SerkCqsWDmZJUjtDRJI0MkNkAFdnSVI3Q6SFi7MkqZ0hMoA9EUnq\nZoi0iFPrktTKEJEkjWxiIZLkvCTfTHJvkov6yt+f5IEk9yd5Y1/5MUnWNfs+nTHfTu59IpLUbeiv\nPdmekpwILAdeWVWbkxzQlB8BnAYcCfwSsCrJ4VX1M+BS4GzgTnpf/ngycMt4KzrWny5JO71J9UTO\nBT5eVZsB+r5mfjlwTVVtrqqHgQeA45IcBOxTVXc0T1S8Ejh1nBV0Yl2Suk0qRA4Hjk9yZ5Lbkxzb\nlB8MPNp33GNN2cHN55nlY2VHRJLajW04K8kq4MBZdl3QXHc/YBlwLHBtkkO347VXACsApqamtteP\nlSTNMLYQqaqTBu1Lci5wfTM0dVeS54D9gQ3AIX2HLm7KNjSfZ5YPuvZKYCXA9PT0SANThT0RSeoy\nqeGsG4ATAZIcDuwBPAF8ETgtyZ5JlgKHAXdV1UbgqSTLmlVZZwI3jruS3rEuSe0msjoLuBy4PMl6\n4BngrKZXcm+Sa4H7gH8G3tOszAJ4N/AZYC96q7LGuzLLrogkdZpIiFTVM8A7Buz7GPCxWcpXA0eN\nuWo/xzvWJamdd6xLkkZmiAzgHeuS1M0QaeHEuiS1M0QG8I51SepmiLSwJyJJ7QwRSdLIDJEBHM2S\npG6GSAvvE5GkdobIAOXMuiR1MkRaOLEuSe0MEUnSyAyRARzMkqRuhogkaWSGiCRpZIbIAC7OkqRu\nhkiLuDxLklpNLESSnJfkm0nuTXJRU/ZbSdYkWde8v77v+NuS3J9kbfM6YJz1syMiSd0m8mTDJCcC\ny4FXVtXmvkB4AnhzVX0nyVHArcDBfaee3jzhcMfUc0ddSJJ2UpN6xvq5wMerajNAVT3evH+t75h7\ngb2S7LnlOEnS3DKp4azDgeOT3Jnk9iTHznLMW4G7ZwTIFc1Q1gcz7gkLZ9YlqdPYeiJJVgEHzrLr\ngua6+wHLgGOBa5McWs0XViU5EvgE8Ia+806vqg1J9gauA84Arhxw7RXACoCpqaltaMPIp0rSLmFs\nIVJVJw3al+Rc4PomNO5K8hywP7ApyWLgC8CZVfVg38/b0Lw/neRq4DgGhEhVrQRWAkxPT4/UpbAf\nIkndJjWcdQNwIkCSw4E9gCeS7Av8LXB+Vf3DloOTLEyyf/N5d+AUYP24K2lHRJLaTSpELgcOTbIe\nuAY4q+mV/CHwK8CHZizl3RO4NcnXgbXABuCyCdVdktSYyOqsqnoGeMcs5R8FPjrgtGPGWqkX1WVH\nXk2Sdk7esd7CO9YlqZ0hMkA5tS5JnQyRFvZDJKmdISJJGpkhMoAT65LUzRBp4by6JLUzRAawJyJJ\n3QyRVnZFJKmNISJJGpkhMoCjWZLUzRBp4cS6JLUzRAYoZ9YlqZMh0sKOiCS1M0QkSSMzRCRJIzNE\nWjixLkntJhYiSc5L8s0k9ya5qClbkuSnfU81/Iu+449Jsi7JA0k+HR/2IUkTN5EnGyY5EVgOvLKq\nNjePwN3iwao6epbTLgXOBu4EbgZOBm4ZVx1dnCVJ3SbVEzkX+HhVbQaoqsfbDk5yELBPVd3RPIv9\nSuDUcVcyrs+SpFaTCpHDgeOT3Jnk9iTH9u1b2gxl3Z7k+KbsYOCxvmMea8rGxicbSlK3sQ1nJVkF\nHDjLrgua6+4HLAOOBa5NciiwEZiqqu8nOQa4IcmRI1x7BbACYGpqasQWOLEuSV3GFiJVddKgfUnO\nBa5vhqbuSvIcsH9VbQK2DHGtSfIgvV7LBmBx349Y3JQNuvZKYCXA9PS0XQpJGpNJDWfdAJwIkORw\nYA/giSSLkixoyg8FDgMeqqqNwFNJljWrss4EbhxnBZ1Yl6RuE1mdBVwOXJ5kPfAMcFZVVZITgAuT\nPAs8B5xTVU8257wb+AywF71VWWNbmbWFw1mS1G4iIVJVzwDvmKX8OuC6AeesBo4ac9Wed8Lhizjo\npb+woy4nSTulSfVE5rwPnnLEpKsgSXOeX3siSRqZISJJGpkhIkkamSEiSRqZISJJGpkhIkkamSEi\nSRqZISJJGllqnn9JVJJNwLdGPH1/4IntWJ2dgW3eNdjmXcO2tPmXq2pR10HzPkS2RZLVVTU96Xrs\nSLZ512Cbdw07os0OZ0mSRmaISJJGZoi0WznpCkyAbd412OZdw9jb7JyIJGlk9kQkSSMzRGaR5OQk\n9yd5IMn5k67P9pLkkCRfTXJfknuT/FFTvl+SLyX5p+b9ZX3nvL/5Pdyf5I2Tq/22SbIgydeS3NRs\nz+s2J9k3yeeTfDPJN5L82i7Q5v/U/He9PsnnkvzCfGtzksuTPN48FXZL2Va3MckxSdY1+z7dPHZ8\nNFXlq+8FLAAeBA6l9+z3e4AjJl2v7dS2g4BXN5/3Bv4fcARwEXB+U34+8Inm8xFN+/cElja/lwWT\nbseIbf8T4GrgpmZ7XrcZuAL4j83nPYB953ObgYOBh4G9mu1rgd+fb20GTgBeDazvK9vqNgJ3AcuA\n0HvU+L8btU72RF7sOOCBqnqoeo/xvQZYPuE6bRdVtbGq7m4+Pw18g97/+ZbT+6ND835q83k5cE1V\nba6qh4EH6P1+dipJFgP/HvjLvuJ52+YkL6X3x+avoPc46qr6IfO4zY2FwF5JFgK/CHyHedbmqvp7\n4MkZxVvVxiQHAftU1R3VS5Qr+87ZaobIix0MPNq3/VhTNq8kWQK8CrgTeHlVbWx2fRd4efN5vvwu\n/hx4H/BcX9l8bvNSYBPw180Q3l8meQnzuM1VtQH4JPBtYCPwo6r6O+Zxm/tsbRsPbj7PLB+JIbIL\nSvIvgOuAP66qp/r3Nf8ymTdL9pKcAjxeVWsGHTPf2kzvX+SvBi6tqlcBP6Y3zPG8+dbmZh5gOb0A\n/SXgJUne0X/MfGvzbCbRRkPkxTYAh/RtL27K5oUku9MLkKuq6vqm+HtNF5fm/fGmfD78Ll4L/HaS\nR+gNTb4+yf9gfrf5MeCxqrqz2f48vVCZz20+CXi4qjZV1bPA9cCvM7/bvMXWtnFD83lm+UgMkRf7\nv8BhSZYm2QM4DfjihOu0XTQrMP4K+EZVfapv1xeBs5rPZwE39pWflmTPJEuBw+hNyO00qur9VbW4\nqpbQ+9/yK1X1DuZ3m78LPJrkXzdFvwncxzxuM71hrGVJfrH57/w36c35zec2b7FVbWyGvp5Ksqz5\nXZ3Zd87Wm/Rqg7n4At5Eb+XSg8AFk67PdmzX6+h1db8OrG1ebwL+JfBl4J+AVcB+fedc0Pwe7mcb\nVnDMhRfwG7ywOmtetxk4Gljd/G99A/CyXaDNfwZ8E1gPfJbeqqR51Wbgc/TmfJ6l1+N85yhtBKab\n39ODwH+jufF8lJd3rEuSRuZwliRpZIaIJGlkhogkaWSGiCRpZIaIJGlkhogkaWSGiCRpZIaIJGlk\n/x/q81SN4OWvAQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1164b3390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# applying gmm on this data\n",
    "X = np.vstack((C1, C2))\n",
    "K = 2\n",
    "\n",
    "ks, mus, sigmas, pi, ll = GMM(X, 2, 1000)\n",
    "\n",
    "classes = np.argmax(ks, axis=1)\n",
    "print(classes)\n",
    "\n",
    "print(\"----means-------------\")\n",
    "print(mus)\n",
    "for k in range(K):\n",
    "    print(\"----Class k: Cov-Mat----\")\n",
    "    print(sigmas[k])\n",
    "\n",
    "\n",
    "plt.plot(classes)\n",
    "plt.ylabel('classes')\n",
    "plt.xlabel('samples')\n",
    "\n",
    "plt.figure()\n",
    "kscatter_plot(X, classes)\n",
    "# Cs = [X[classes == k] for k in range(K)]\n",
    "# csm = ['m.', 'c.']\n",
    "# mum = ['rx', 'bx']\n",
    "\n",
    "# for k in range(K):\n",
    "#     plt.plot(Cs[k][:,0], Cs[k][:, 1], csm[k])\n",
    "#     plt.plot(mus[k][0], mus[k][1], mum[k])\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(ll)\n",
    "plt.ylabel(\"log-likilihood\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## K-means vs GMM with data with different variances "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1166eb470>]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X10VfWZL/Dvw0uABJQIiCIkoOP0FumIGh17p9Xmqh1l\nbKneNV5apG+rKyW+LOdOe7tkWHjnlsVydGo77R0XlnHomgr3qvPC6G2ljlxDUu/UahCiGKCFaAIk\nxWjQQtM2RZ77xz477L2zX8/Zrznfz1pn5Zx99tn7yVnJ79m/1y2qCiIiItOErAMgIqJ8YWIgIiIb\nJgYiIrJhYiAiIhsmBiIismFiICIiGyYGIiKyYWIgIiIbJgYiIrKZlHUA5Zg9e7YuXLgw6zCIiApl\n165db6vqnKD9CpkYFi5ciM7OzqzDICIqFBHpDbMfm5KIiMiGiYGIiGyYGIiIyIaJgYiIbJgYiIjI\nhomBiCrS1/cgjh9vs207frwNfX0PZhQRVYqJgYgqMmPGlejuvm00ORw/3obu7tswY8aVGUdG5Srk\nPAYiyo/6+mYsXvwkurtvw7x5rejv34jFi59EfX1z1qFRmVhjIKKK1dc3Y968VvT2rse8ea1MCgXH\nxEBEFTt+vA39/RvR2LgO/f0bx/Q5ULEwMRBRRcw+hcWLn8SiRV8fbVZiciguJgYiqsiJEy/b+hTM\nPocTJ17OODIqV+KdzyLyJoATAN4HcEpVmxzvC4BvA1gGYBjA51X1laTjIqJ4NDR8bcy2+vpm9jMU\nWFqjkppV9W2P924CcHHp8YcANpZ+EhFRBvLQlLQcwPfV8CKAmSJyftZBERFVqzQSgwLYISK7RKTF\n5f0LABy2vD5S2kZERBlIIzF8RFWXwmgyulNErinnICLSIiKdItI5ODgYb4REIXH5B6oGiScGVT1a\n+vkWgG0ArnLschTAAsvr+aVtzuNsUtUmVW2aMyfwznREieDyD1QNEk0MIlInIjPM5wA+DmCvY7en\nAXxWDFcDeE9VB5KMi6hc1uUf3njjvtHx+xyBQ+NJ0qOS5gLYZoxIxSQA/0tVfyQiqwFAVR8B8AyM\noaoHYQxX/ULCMRFVxLr8Q2PjOiYFGncSTQyq2gPgUpftj1ieK4A7k4yDKE7O5R9mzuSYfRpf8jBc\nlagwuPwDVQMmBqIIuPwDVQPej4EoAi7/QNWANQYiIrJhYiAiIhsmBsrO1q3AwoXAhAnGz61bx+c5\niQqGiYGS5VUQb90KtLQAvb2AqvGzpSXZgjqLcxIVEBMDJcevIF67Fhgetu8/PGxsT+qq3u+clHtc\npyo9TAyUHL+CuK/P/TNm8kjiqt7rnF7bKVe4TlV6mBgoOX4FcUOD+3sTJ7onk3vuqTwer3N6ba9y\nebtC5zpV6WFioOT4FcQbNgC1tfbttbXA+++7f+addyqvNbidEwBOnmQ/g4s8XqFb16maN6+VSSEh\nTAyUHK/Cf8MGYOVKYNMmoLEREDF+mq+9VNoXYJ5z1iz79nfeAVatMuLgSKVR9fXNmD37Vuzde4vt\nCh1AZrUG5zpVXIokIapauMcVV1yhVBBbtqg2NqqKGD+3bAne3+hdGPsQiSemxkbvcwCqtbXBcVaJ\noaHntb19mra1QXt61unQ0PP6wguzdWjoee3tfUCHhp4fs39v7wOJxWKe2+01BQPQqSHKWNYYyC7u\nEUErVwJvvgmcPm38XLkyeP+6Ovf3zjmnslhMQZ3NfiOVqnAehEgNJkyYhsOHH8LevZ8abddPu6mJ\n61SlKEz2yNuDNYaEbNliXC1nffU8a5b7lfysWf6fC1s7CaoxeNVO8vL9xCDM1b71irynZ522tUHb\n22ttnzP36elZV4ir97RrOXkD1hgosryM8x8airYdiDZ5zasT2sqt4zwv308Mwlztm1foAEbb9UUm\n4a23Hh/dp76+GXV1l47pDA4zeimLUU957FDPpTDZI28P1hgSIhL+6jlJXlf0Zi3ArVbg9xmnLVu8\nayV+tYC8fD8xCXO1H9SuPzT0vHZ0nKXt7bXa0XGWDg09H7rtP6s+g6LVcuKEkDWGzAv5ch5MDAmJ\nUriaonYuh+HVZNPa6t2UE7bQdjt2TY2RKMptgvL7fnLObCLavfs61yaWrq6bPJterAW5kSDO1vb2\nadrRcXbowjarQtr8vXt61qVyvrzIRWIAsABAG4BuAK8DuMdln48BeA/AntLjvqDjMjEkJGobepJt\n7s6r+lmzvK/yGxvDF9qVJD8z0aTUx5B0e7i1UO7oOMtWoIe5enfGZ00yUaRdSLPGkH1iOB/A5aXn\nMwD8DMBixz4fA/CDKMdlYkhQlBpAklfQbknH7+GWNNwK7ajNQW5xmMeIq4bkIc6mFmchbjYB7d/f\nYnvd0XG27t593Zir/qCEFFTYeiW5/ftbUi2kq33Iay4Sw5iTAU8BuMGxjYmhqJJscw8zcshZUDsT\nhVuh7VfrsLLWEjJsPqrk6tZaGJvH6et7SHt7H9D9+1tcC//du68rjT6aFrrwdM5t6Ot7aEzhayYA\ne9/E2aP9EmHOEweOSspZYgCwEEAfgLMc2z8GYAjAqwC2A7jE4/MtADoBdDY0NCT0tVEkSdYYvJJO\nmKTgFcOWLUZ/gnPfyZPtSSRMbSXFDudym1qcBW1f30Pa1iba3b3KtQA2JrPVaXf3qtFC22xmMmsW\nbvwSkLMfwprk9u9vqepCOgu5SgwApgPYBeBWl/fOAjC99HwZgJ8HHY81hpwop0+i0maqujr7MfwK\nb3M/v/4Js3YR5twFqzG4fb67e5VrkrEW6ObP9vbaMXMXosx/cIu5Wjt98yI3iQHAZADPAvjzkPu/\nCWC23z5MDDGIazSR13Gc2/1GFHkdd/LksQVyTY39M16FeNgah9vVf9Bny+xwjtqMEVd7uFkYmzWF\nnp512t5ep319D9lis17ld3ScrTt31ujOnZN9m3u8YrKe0/r7dHScpbt3X1dV7fp5kovEAEAAfB/A\n3/jscx4AKT2/qtTcJH7HZWKoUNIzeP06bL2uvFtbVSdONLZNnGi8DtMfELWTOszVf1DfQpnfk9lU\nYxbIzmYXpzjaw81zGDUFGT232azkjMVs8jHXR9q9+7oxfQZGbaLOsxZjfL5W9+y5YfQc5radO6dG\nmutA8cpLYvgIAC31H5jDUZcBWA1gdWmfu0pDWbsAvAjgPwYdl4mhQkmPx4/acdzaWl5h7lY78Wsy\n8rv6dx7DWVuJKXE62/mdhW5YUZe0cOsUNpuLnAW82TFt9C8YfQ3OPoOwTVIHDrRqW5voT35ysS0R\nucVLyctFYkjqwcRQoSRHE/mtjupVwJs1hagPt8I6SlIyk0slk97KYBaqr7zy0bKvmo3C+yxboe8c\nfuo3Oc3kbPN3ayqydkRbC323GoNbR/RLL31oTLMSZSNsYjCbcAqlqalJOzs7sw6juBYuNNYScpo1\nC3j77fKPa65X5FxPyCRiFLum2lrj/gi33+59zJoaYGTE+/3GRmPVVtOECfZzuDHPa6706vV9OI8d\nA3NtntraD+K9936MuXNX4YMf/L7rvn19D2LGjCttN6M5frwNJ068jBkzrsTevbcAUMyevRzHjj2G\nCRPq8KEP/R8ACLy7WV/fgxCZhL6++zFvXiv6+zeioWENjh/fgQUL/tuYc/b2bsC77/5fzJ27CkND\n20ePbf4+Xufat++zOHbsMZx99kcxPLyPd1zLmIjsUtWmoP24iF412rDBKHCdfvnLypaRdltkzlRb\nC6xePfbGPCtXGrfz9KI69sY6Vr299pi97honcub5tGn298q4F3SoBeAcS3Qf/8e16O6+DQ0NazA8\nvA9z567CsWNbcPjwN13P4bXg269/fQgAsGTJNqiewrFjjwGYDNVTePdd/4LaJDIJhw59FQ0Na7Bo\n0dfR0LAGhw59FfX117t+7le/6kJj4zoMDv4zGhrWhFr6+vDhb+LYsS2YO3cVhof3oaFhje33oRwL\nU63I24NNSTEIO9ErCr/RPH5NMUF9DH5LXlj7KVS9m4X8+gzK6HMJHJ3jEkfvqsnat+0zru38QZPH\nrM025raurpt1584abWuDtrWh1Nl7ZkkKv34Ia5+DtYnI2eZf7sioqB3tlA6wj4F8JdHPUEmndlBy\nCOpUFvEeKhuUBMscpeU7x8Dju+hdPTPySCO3sf9GJ7aREHbunKI7d04dfe61yqlboR40ryDKyCjr\nvubsamsiMGdAMzFkh4mB/CUxMslt7oFzVnE5MYWdk+AVe5gkWOa8Ds+CtYLE69aB2929Stvb62wj\nh3bunGJJDEbNoavrZs/Zxs65C2bHclzzCqwd4qpnRmB1dd1s+104RDU7YRMD+xiqldvNamprje2V\nsLblu72OGpOzw9qP2d/gvPWmV7+DdXvUW5Ai4Mb0Yc7pwexbOHz4m6N9EkND27Fo0ddH2+hPntwD\nkUlobFwH4DRUR1BffwOA923t/vX1zZg3rxW9vesxZ86t6Ou7H8ePt+HAgS/jtdc+AUDR2LgWs2ff\nitdeuxkHDnzZ9vtFuWnOueeuACDYu/dTeOON+/Dmm/8DEybU4r33fow33rgvVN8H5USY7JG3B2sM\nMYn7Xgpx1EKcMYWpKTj7E5xNQlFnXYdQTh9DlHOaE8Kc6xqZzTHW+QUdHWeN1h6sNQJrXM6hpi+9\ndJnrhLdKr+7NuK1LaVhrVWGapqp9obskgTUGClTGVbKvMkb3eMb02GPlxeAc2jo8bIx+Gh4+M/rJ\nOiKqTIE3pl+50jiHcxQWMLZG46K+vhkLFnwFx449ZrtlZn19M6ZNuwhLlmzDwoX/HYcOfRWzZy/H\nhAlTcM45fzJaIwCMK/7XXvvE6MijxYufRF/f/Zg+/TL86le7MXfu7ejrux9vvHEf+vrux0UXfQMn\nTrwYw9W9jv48eXK3rVYlMinw1pq8/WYOhMkeeXuwxpBTcfVbBC1z4baGUthHgjfWKev38ognzOJ5\nvb0PjJmBbL2ydpvtbMxEPrNukvPzlSxyZ66FZM6abm+vc112w2+CXJTfn6IDO58pdXGtwRTUhDRr\nVnnNTHF0sFciZOIMO0Q0bOEZtF6Sc3mOcgtjo0N86ujx9+9v0fb2Ou3qutk2Mqm394FQCYgrscaP\niYGyEUe/RdAoJHNoqjMJTZ7sfr+FKCODkriHddDv5Ygn6jpIbq+dnKudOucV+C2qF8SM11pDsR7X\nOprKemzWGNLHxED551UIh7nPgtfnrdu81mDyqjEkvepsjEOEo3TQuhWyzs9bl90OOp7X8Z3DVN1u\nCBQmoZU7qY6CMTFQvvkVwlu2+F9dhy2ooxb0Sa86m3TicZFWIetMPl6rr3JUUraYGCjfggrh1lb3\n5DB9erSCNErTUJKrzpYTTwyiFrKVFMpuNwTilX6+MDFQvoWdjey2nEVSV9lJ1xhywq/wr2RtJLcO\nbjYD5UvYxMB5DJSNsLORp08fu8/wsLGSa9zcZl5PngycPBk476BI/OYJmPMxurtvCz2fwbr0dl3d\nElx00TdG51P4rb5KORYme+TtwRrDOBC2vT2N5h1nXNY7ubnNpM5qHkSMgkb9uN2z2fycs1mJfQLF\ngbzUGETkRhE5ICIHReRel/dFRL5Tev9VEbk86ZgoB7xmBjtnI1ew5lDZcZmzwadPd59JnURtJWXW\nNZSsM6uBM2tAOe8X4TUDuaHha2NqFPX1zWho+FryvwglYlKSBxeRiQAeBnADgCMAXhaRp1W127Lb\nTQAuLj3+EMDG0k8a71auDF6WYsOGsXeFi2OxvzDiWOIjp5wLAM6c2ex6R7bp05fi0KGv4uTJPbY7\nt9H4lnSN4SoAB1W1R1VHADwOYLljn+UAvl+q6bwIYKaInJ9wXFQUYWsWSUi7tpISa+FvrqFk9jk4\n14BasODPMXfu7WPWbKLxLenEcAGAw5bXR0rbou5D1Szuxf7CSmpp8oz5LQDobBY6frwNQ0Pb3ZcW\np3Er0aakOIlIC4AWAGgo+BUbFYSZgNauNZqPGhqMpJBWYkqIW9t/fX3zmNqAs1lp5sxm3lOhSiRd\nYzgKYIHl9fzStqj7QFU3qWqTqjbNmTMn9kCJXGVVW8mBwKXFadxKusbwMoCLRWQRjMJ+BYDPOPZ5\nGsBdIvI4jE7n91R1IOG4iChA2JoFjT+JJgZVPSUidwF4FsBEAJtV9XURWV16/xEAzwBYBuAggGEA\nX0gyJiIi8pd4H4OqPgOj8Ldue8TyXAHcmXQcREQUDpfEICIiGyYGIiKyYWIgIiIbJgYiIrJhYiAi\nIhsmBiIismFiICIiGyYGIiKyYWIgIiIbJgYiIrJhYiAiIhsmBiIismFiICIiGyYGIiKyYWIgIiIb\nJgYiIrJhYiAiIhsmBiIiskns1p4i8tcAPgFgBMAhAF9Q1Xdd9nsTwAkA7wM4papNScVERETBkqwx\nPAdgiar+AYCfAVjjs2+zqi5lUiAiyl5iiUFV/01VT5VevghgflLnIiKi+KTVx/BFANs93lMAO0Rk\nl4i0eB1ARFpEpFNEOgcHBxMJkoiIKuxjEJEdAM5zeWutqj5V2mctgFMAtnoc5iOqelREzgXwnIjs\nV9UO506qugnAJgBoamrSSuImIiJvFSUGVb3e730R+TyAmwFcp6quhbmqHi39fEtEtgG4CsCYxECU\npoEBYMUK4IkngPPcLn2IxrHEmpJE5EYAXwPwSVUd9tinTkRmmM8BfBzA3qRiIgpr/XrghReMn0TV\nJsk+hr8FMANG89AeEXkEAERknog8U9pnLoAXRKQLwEsAfqiqP0owJqJAAwPA974HnD5t/PzFL7KO\niChdic1jUNXf89jeD2BZ6XkPgEuTioGoHOvXG0kBAN5/33j98MPZxkSUJs58JrIwawsjI8brkRHW\nGqj6MDEQWVhrCyaz1kBULZgYiCx+8pMztQXTyAjw7/+eTTxEWUisj4GoiHbvzjoCouyxxkBERDZM\nDEQpGhgArr2WndmUb0wMRCnixDkqAiYGopRw4hwVBRMDUUrcJs4R5RETA1EKOHGOioSJgSgFnDhH\nRcLEQJQCTpyjIuEEN6IUcOIcFQlrDEQp41wGyjsmBqKUcS5D+ZhU08HEQJQizmWoDJNqOpgYKBCv\n0uLDuQzlY1JNT5L3fP5LETlauq3nHhFZ5rHfjSJyQEQOisi9ScVD5eNVWjw4l6EyTKrpSbrG8C1V\nXVp6PON8U0QmAngYwE0AFgP4tIgsTjgmioBXafHhXIbyMammK+umpKsAHFTVHlUdAfA4gOUZx0QW\nvEqLD+cylI9JNV1JJ4a7ReRVEdksIvUu718A4LDl9ZHSNsoBXqXFa/duQHXsg3McgjGppquixCAi\nO0Rkr8tjOYCNAC4EsBTAAICHKjxXi4h0ikjn4OBgJYeikHiVRnnBpJquimY+q+r1YfYTkb8D8AOX\nt44CWGB5Pb+0ze1cmwBsAoCmpiaNFimVg1dpRNUpsSUxROR8VR0ovbwFwF6X3V4GcLGILIKREFYA\n+ExSMVE0vBojqk5JrpX0oIgsBaAA3gTwZQAQkXkAHlXVZap6SkTuAvAsgIkANqvq6wnGREREARJL\nDKq6ymN7P4BlltfPABgzlJWI4jUwAKxYATzxBHDeeVlHQ3mW9XBVIkoJJypSWEwMRFWAExUpCiYG\nCsS1kpKR5vfKiYoUBRMDBWITRDKifK+VJBFOVKSomBjIF5sgkhH1e60kOXOiIkXFxEC+2ASRjCjf\na6XJmRMVKSomBvLEJohkRP1eK03OXE6ComJiIE9sgkhGlO+VyZmywMRAntgEkYwo3yuTM2WBiaEK\nhR3hwiaIZET5XpmcKQtMDFWIw0+jyXIeB5NzflTTfB4mhipT9OGnWfxzFiGRVlOhlZUi/B3EhYmh\nyhR9+Gna/5xFSaTVVGhloSh/B3FhYqgiRR/hksU/ZxESabUVWlkowt9BnJgYqkjRR7ik/c9ZlERa\nbYVW2orydxAnJoYqUuQRLnH/c4Zpky9CIq3GQittRfg7iBsTQxUp8giXuP85168Hfvxj4PLLvQvR\nIiTSaiy00laEv4O4JXnP5ycAfKD0ciaAd1V1qct+bwI4AeB9AKdUtSmpmKi44vznNK+yVY3na9YY\nr52KkDCrsdBKWxH+DuKW5K09/4v5XEQeAvCez+7Nqvp2UrFQ8cX5z+m8yn7sMeD++4t5u8tqLLQo\neYk3JYmIALgNwP9O+lxEQZxt8oDR9LJmTXYxJY1zHCiqNPoYPgrgmKr+3ON9BbBDRHaJSEsK8VAV\nc2uTB4xaw3gtODnHgaKqKDGIyA4R2evyWG7Z7dPwry18pNT3cBOAO0XkGo9ztYhIp4h0Dg4OVhI2\nVTG3NnnAqDX4dUQXFec4UDkqSgyqer2qLnF5PAUAIjIJwK0AnvA5xtHSz7cAbANwlcd+m1S1SVWb\n5syZU0nYVMV27waWjhkCYRgYGH9X1ZzjQOVIuinpegD7VfWI25siUiciM8znAD4OYG/CMVGVcw7b\n7e8Hpk413htPV9VpznFgP8b4knRiWAFHM5KIzBORZ0ov5wJ4QUS6ALwE4Ieq+qOEY6KYFb1QGK9X\n1WnOcWA/xvgiqpp1DJE1NTVpZ2dn1mFQyR13AN/9LrB6NfDww1lHE87AALBiBfCd7wBXXw385jdn\n3ps2DejpKebwVavLLgP27Bm7fenSeIe5DgwAF15ofIfj5bsbr0RkV5i5Ypz5TBUpauemeYW7cuX4\nnTmc1kz38VrjyqO0audMDFSRIhYK1mTW3c2Zw5XgWk3pSqvJrqoSQ9HbwvOmqIWCNZlNnmw0hRVx\n/ag8SKsfg/+76dbOqyoxsIMsXkVbwG1gAPjwh4HNm4uXzPIqrbWa+L+bbu28ajqf2UEWv7Q6N+Ny\nxx3Axo3AhAn2hFZTA3zpS8XpOK82/N+1fwemcr4Ldj47FLEtPO/yvIy3s+nBrIYDY2s57FPIN/7v\npl87r4rEUNS2cCqfs+nB+o9VUzO2XyGtZMa28mj4v2tIe3n1qkgMRWsLJ0O5haizk66rKz+FS5i2\nciaPM/i/a0i7dl4ViYE3MymmcjscnU0PUecqJFUwhx1Vwo7WM/i/mxFVLdzjiiuuUBrf+vtVp0wx\nroumTlUdGAj/ualT7ddVIm7XWqpLl7ofo7VVdcIE1TvuiO/3MY9bU2Ocu6bG/fjW+KdNC/97x6W/\nX/Waa9I/L6UDQKeGKGOrosZAxbN+PfC73xnPR0bCXz27NT1EmauQ1Fhxt7byzZuN4bPWc2Td0cra\nCgFV0pRExTIwYBSaZgF5+rTx2jrCyKupx6vpob093LmTKpjdEtbICPDii2fOkXVHa1GXN6H4MTFQ\n7lhrCyZrrcHvqtbZSdfaasxbuPba4PMmWTB3dIxNWGaiMM+RdUdrlKTIDvLxjYmBcqejY2wBefq0\ncdUf5ao26hVwkgXzNdcYCcps0mptNYbNWs+RZEdrUEEeNSmyyWl8Y2Kg3LnmmjOFpqmmxijYolzV\nRm0WSqpgDjt8dvv25IYkBhXkUZIim5yqQJge6rw9OCppfDJHxFxyifsoomnTxo448hq54zY6qdJR\nPuWO2HGORrrkkjOvzYfXKKU4WL+LqVNVr7567O+wdGn4kVthRldRPoGjkqhozKvaa68dW0S1tgK/\n/vXYK3qvq9okmoXKaT5xa6JJe6lv63fh7PA2hZ1AlXUHOaWDiYFywa95opx1juJuFiq3+cQtQU2a\nlN5S386C3IzFOsoriqw7yCkdFSUGEflTEXldRE6LSJPjvTUiclBEDojIH3t8/hwReU5Efl76WV9J\nPJRvfh2gfv0B5axzFPcSAuUOY3VLUL/7XXozd90KciDa3BArzkSuEmHam7weAD4I4AMAdgJosmxf\nDKALwBQAiwAcAjDR5fMPAri39PxeAA+EOS/7GIrJa0axX39AEn0FUcURQ1Yzmr36DrKaWU3ZQhp9\nDKq6T1UPuLy1HMDjqvpbVX0DwEEAV3ns9w+l5/8A4FOVxEP55dcU49c8kYemizhiyGpGs1lzsg6P\nNbEJiLwk1cdwAYDDltdHStuc5qrqQOn5LwDMTSgeyphfwejXPJF204Vbc1elMeShw5ZNQBRFYGIQ\nkR0istflsTzOQErVHPWJo0VEOkWkc3BwMM5TU8KCCka//oC0lxt2G3lUaQx5qPXk+aZKlD+BiUFV\nr1fVJS6Pp3w+dhTAAsvr+aVtTsdE5HwAKP18yyeOTarapKpNc+bMCQqbciQPBWMYSU3cyupqnctW\nULmSakp6GsAKEZkiIosAXAzgJY/9Pld6/jkAfsmGCqoozRhJ9QNkdbXOZSuoXGK04JT5YZFbAPxP\nAHMAvAtgj6r+cem9tQC+COAUgD9T1e2l7Y8CeERVO0VkFoAnATQA6AVwm6oOBZ23qalJOzs7y46b\nyCmum63nhfX3KfLvQfESkV2q2hS0X6Wjkrap6nxVnaKqc82kUHpvg6pepKofMJNCafuXVLWz9Pwd\nVb1OVS8uNVkFJgWiJBSluSusrO/rQMXGmc9EKE5zVxh5GAVFxcbEQISx/QD9/cYqr9u3B382b8Zb\n7YfSx8RA5KLIHbfjqfZD2WBiIHJI434DSQ4l5ZwFqhQTA5FDJR23YQv8ItdIaPxjYiCyqLTjNkyB\nzzugUd4xMRBZVNJxG7bA51BSyjsmBiKLSjpuwxT4HEpKRcDEQGRRbsdt2AKfQ0mpCJgYiGIQtsDn\nUFIqgklZB0A0HoQt8DlklIqAiYEoBizwaTxhUxIREdkwMRARkQ0TAxER2TAxEBGRDRMDERHZVJQY\nRORPReR1ETktIk2W7TeIyC4Rea308z95fP4vReSoiOwpPZZVEg8REVWu0uGqewHcCuC7ju1vA/iE\nqvaLyBIAzwK4wOMY31LVb1QYBxERxaSixKCq+wBARJzbraO6XwcwTUSmqOpvKzkfERElL40+hv8M\n4BWfpHC3iLwqIptFpD6FeIiIyEdgYhCRHSKy1+WxPMRnLwHwAIAve+yyEcCFAJYCGADwkM+xWkSk\nU0Q6BwcHg05NRERlCmxKUtXryzmwiMwHsA3AZ1X1kMexj1n2/zsAP/CJYxOATQDQ1NSk5cRERETB\nEmlKEpGEMSGIAAAEqklEQVSZAH4I4F5V/X8++51veXkLjM5sGueSvN8xEVWu0uGqt4jIEQAfBvBD\nEXm29NZdAH4PwH2Woajnlj7zqGVo64OlIa2vAmgG8F8riYeKgfc7Jso3US1eq0xTU5N2dnZmHQaV\nYWAAuPBC4De/AaZNA3p6gPPOyzoqouogIrtUtSloP858plTxfsdE+cfEQKnh/Y6JioGJgVLD+x0T\nFQMTA6WG9zsmKgbe2pNSw9tfEhUDawxERGTDxEBERDZMDEREZMPEQERENkwMRERkU8glMURkEEBv\n1nEEmA3jTnZFUKRYgWLFW6RYgWLFW6RYgXzE26iqc4J2KmRiKAIR6QyzJkkeFClWoFjxFilWoFjx\nFilWoFjxsimJiIhsmBiIiMiGiSE5m7IOIIIixQoUK94ixQoUK94ixQoUKF72MRARkQ1rDEREZMPE\nkDARuVtE9ovI6yLyYNbxBBGRr4iIisjsrGPxIyJ/XfpeXxWRbaX7jOeKiNwoIgdE5KCI3Jt1PF5E\nZIGItIlId+nv9J6sYwoiIhNFZLeI/CDrWIKIyEwR+afS3+s+Eflw1jEFYWJIkIg0A1gO4FJVvQTA\nNzIOyZeILADwcQB9WccSwnMAlqjqHwD4GYA1GcdjIyITATwM4CYAiwF8WkQWZxuVp1MAvqKqiwFc\nDeDOHMdqugfAvqyDCOnbAH6kqv8BwKUoQNxMDMlqBfBXqvpbAFDVtzKOJ8i3AHwNQO47nlT131T1\nVOnliwDmZxmPi6sAHFTVHlUdAfA4jIuE3FHVAVV9pfT8BIyC64Jso/ImIvMB/AmAR7OOJYiInA3g\nGgB/DwCqOqKq72YbVTAmhmT9PoCPishPRaRdRK7MOiAvIrIcwFFV7co6ljJ8EcD2rINwuADAYcvr\nI8hxYWsSkYUALgPw02wj8fU3MC5gTgftmAOLAAwC+F6p6etREanLOqggvFFPhURkB4DzXN5aC+P7\nPQdG9fxKAE+KyIWa0VCwgFj/AkYzUm74xauqT5X2WQujKWRrmrGNRyIyHcA/A/gzVf1l1vG4EZGb\nAbylqrtE5GNZxxPCJACXA7hbVX8qIt8GcC+AddmG5Y+JoUKqer3XeyLSCuBfSongJRE5DWO9lMG0\n4rPyilVEPgTjyqZLRACjWeYVEblKVX+RYog2ft8tAIjI5wHcDOC6rJKtj6MAFlhezy9tyyURmQwj\nKWxV1X/JOh4ffwTgkyKyDMBUAGeJyBZVvT3juLwcAXBEVc0a2D/BSAy5xqakZP0rgGYAEJHfB1CD\n7BfRGkNVX1PVc1V1oaouhPHHfHmWSSGIiNwIoznhk6o6nHU8Ll4GcLGILBKRGgArADydcUyuxLga\n+HsA+1T1m1nH40dV16jq/NLf6QoAz+c4KaD0P3RYRD5Q2nQdgO4MQwqFNYZkbQawWUT2AhgB8Lkc\nXtkW1d8CmALguVIt50VVXZ1tSGeo6ikRuQvAswAmAtisqq9nHJaXPwKwCsBrIrKntO0vVPWZDGMa\nT+4GsLV0gdAD4AsZxxOIM5+JiMiGTUlERGTDxEBERDZMDEREZMPEQERENkwMRERkw8RAREQ2TAxE\nRGTDxEBERDb/H5DEbcf9w5+/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11682c898>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mu1 = [-2, 2]\n",
    "cov1 = np.eye(2)\n",
    "\n",
    "mu2 = [2,2]\n",
    "cov2 = np.eye(2)\n",
    "\n",
    "mu3 = [0,-8]\n",
    "cov3 = np.eye(2) * 5\n",
    "\n",
    "C1 = np.random.multivariate_normal(mu1, cov1, 50)\n",
    "C2 = np.random.multivariate_normal(mu2, cov2, 50)\n",
    "C3 = np.random.multivariate_normal(mu3, cov3, 50)\n",
    "\n",
    "plt.plot(C1[:, 0], C1[:, 1], 'or')\n",
    "plt.plot(C2[:, 0], C2[:, 1], 'xy')\n",
    "plt.plot(C3[:, 0], C3[:, 1], '^b')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+QHPV55/H3oxGSHUNOazA/pUWkguOSLUNgJWcT4+ye\nMME6x2C7rCN1iWwcZy0KXCHnOw6Fkg9bFeRT7IBjiCwZcFl1znGqAAexiTHg3QMfi60VwYCEfyhY\n6BcxQpYdVVFISPvcHz2Dpkc9PT0/erp75vOq2pqdmd6eZ3ul79P9fH+0uTsiIiIVM7IOQERE8kWJ\nQUREQpQYREQkRIlBRERClBhERCREiUFEREKUGEREJESJQUREQpQYREQkZGbWAbTilFNO8fnz52cd\nhohIoWzZsuVld39Lo+0KmRjmz5/P1NRU1mGIiBSKmb2QZDuVkkREJESJQUREQpQYREQkRIlBRERC\nlBhERCREiUFEREKUGESkMyYnYc2a4FEKrZDzGEQkZyYnYckSOHwYZs2CRx6B4eGso5IW6YpBRNo3\nMREkhaNHg8eJiawjkjYoMYhI+0ZGgiuFUil4HBnJOiJpg0pJItK+4eGgfDQxESQFlZEKTYlBRDpj\neFgJoUekXkoysx1m9oyZPWVmx618Z4G/NbPtZva0mV2QdkwiIlJft64YRt395TrvvQ84t/z1LmBd\n+VFERDKQh87ny4CNHngCmGNmZ2QdlIhIv+pGYnDgYTPbYmZjEe+fBeyqer67/JqIiGSgG6Wkd7v7\nHjM7FXjIzH7k7o82u5NyUhkDGBwc7HSMIiJSlvoVg7vvKT++BNwLLK7ZZA8wr+r53PJrtfvZ4O5D\n7j70lrc0vDOdSLq0/IP0sFSvGMzsTcAMdz9Y/v4S4HM1m90PXGNmdxF0Ov/K3V9MMy6Rtmj5B+lx\naZeSTgPuNbPKZ/29u3/bzFYAuPtXgAeApcB24BXgypRjEmlP1PIPSgzSQ1JNDO7+PHBexOtfqfre\ngavTjEOkoyrLP1SuGLT8g/QYzXwWaZaWf5Aep8Qg0got/yA9LA8T3EREJEeUGEREJESJQUREQpQY\nREQkRIlBspXFDGLNWhaJpVFJkr7JyeihnVnMINasZZGGdMUg6ao0xKtWBY/VZ+lxN5BP66xeN60v\nPl3xpU5XDJKuuOUj6s0gTvOsXrOWi01XfF2hxCDpimuI680grk4mr74KGzd27j+/Zi03r14pMAta\np6orLFiqqFiGhoZ8auq420dLXjXbsExOwugoHDoUPJ81K50GIE8NXl7l7Qw9b/EUjJltcfehRtvp\nikHS1+zyEcPDcOWVsH49uAdnh51ODLUNzC23wP79ShK10rx6a4Wu+LpCiUHyaflyuPNOeO01KJU6\n3xdQ3eAdOgTXXAPT0zoLrTUyAjNnBsfJPfibLF9+/OiybjbUWqcqdRqVJMfLy6iP4D4exx47qdL3\nUSrBjBlBw5d0pFJejk83VK7eKn+DytVbRdyoMyksXTFIWF5quBMTcORIcJZ65EjyUlLSs9fqksTJ\nJ8O11yYbqZSX49MpSY7X8uXw9a9HH5+idgarfymWEoOE5eU/eivDSptttKtLEgsXJmso8nJ8OiHp\n8Yqr61f/nUol2Lkz2G8zx6TbjXSvJfcUKDFIWF7G+cc1RvUakmYa7dp9JK1b5+X4dEIzx6ve8an8\nnTZuhK99Db761eDqImljm0Uj3UvJPSWpJgYzmwdsJLj3swMb3P1LNduMAPcBPyu/dI+7fy7NuCRG\nK6M+0jrji2qM4hqSpI12O41RL42KiTperfwth4ePlf6abWyzaKR7KbmnJO0rhiPAp939STM7Cdhi\nZg+5+7aa7R5z9/enHIsk1cyoj7TP+GobqriGJGmj3UpjFHWF0S1pJt7q4wWt/y1bbWyzaKR7Kbmn\nJNXE4O4vAi+Wvz9oZs8BZwG1iUGKKs0zvqikE7eMRuU/+sqV8ftttjHKsibd6c+OK6GtWRP+W27c\nmLzxTNrYRn1+Fo20hrzG6lofg5nNB34b+H7E279rZk8De4D/4u5bI35+DBgDGBwcTC9QaU6aZ3xR\nSWflyuMbklYaz49+NHisHZNfrdKI7dzZVPLbuXYnJy06iYHRgbrbHBg/wMHNBxm8rsG/5U4k3srv\nUTv6qvY4Vf8tZ84M5iwcPZr8mFY3tlFXOfX+Tmqk88fdU/8CTgS2AB+KeO/XgRPL3y8Fftpofxde\neKFLjjz+uPtNNwWPnd7vG9/oXioFj/X2f9NNwTYQPN50U/v7rN5u1iz32bMb/0zZL777C//eKd/z\nX3z3Fy2931K8SX5+5kz3GTPij1Plb3n55e5myY5p0pib+TtJKoApT9Bmp37FYGYnAHcD33D3eyIS\n079Vff+Amf2dmZ3i7i+nHZt0SLN9Es2WJzZujN8uSXkJmjv7rz5TB/izP4PBwURxD4wOsGDTArYt\n28aCTQtCVw4Hxg9Evl5Xu6WW6t9jxoxgSKlZ/au7yv4/+9lgDglEzzyP+zvWu8pRp29xJMkerX4B\nRjAq6ZaYbU7n2GJ+i4Gdlef1vnTF0CFpnenX238rZ7/NnOHX+6zqM/6kZ//tnqn78VcGrz//8vfS\nPe7Van+P9euPP061sVSf2Zu5r1gRv8/a3yPuM9P+NyexSHjFkHZieDfBMNWngafKX0uBFcCK8jbX\nAFuBHwJPAL/baL9KDB3QgYav6f03KiWsX+9+ySXBY0Wr5YcVK46VQszCZZEVK5I1Th1oxCrJ4PlV\nzx9LCm98Y1DSOeGE8O+alnq/R71/A1GvV+8jyd+kst111x0rYaXx70yakjQxpD0q6Xvlq4a4bW4F\nbk0zDomQ9vjxqP3HlRI2bIBPfjL4/jvfCR7Hxo7/mZNPDkbPNBr58rWvhUshpVIwzn7WrPodzq1O\neosxMDrAmVedyQurX+DsVWczcPCuYNG+6eng6+qrg1nXrX5OktJcvd+j3r+BRsNYb7mlcUmo8nm/\n//vBcYfg99ZkskLQzOd+lWa9d3IyqOXPLP/zquw/rl5+993hfdx9d5AY4tY0qjdSpjLZCoJ6+ic+\nESSDuAY0pSGpB8YPsHfdXs5edTZ71+1lzn9/DwOlUpAUIHhstbGcnAx+n9degxNOgC9/ubmlwxvd\nRKneMNb9+5PPF6n00UDQx6F+hWJIclmRty+Vkjrk8ceDssqKFZ27xK+t7Sfd9/r1QWmi8nX55cf/\nXNKyUitlshRGzNTtY/jPG5srr9QrBa1YET5mpVLzpcGo8l3U57dSdqz8XDfLZhKLPPQxpPWlxNAh\nafQztNPArl/vvnhx0GhGxdRM0olqTOP6DNpp/CL2Wa+juekO6Li4Lr88nBiaHV7azO/can+LOptz\nRYlBGktjXHm7yaZRTJWrnNmzmzsTTRJXs41YnX2GGv+495PMY6h3PB5/PDgG1UkhLqFG/V6aV9B3\nkiYG3ainn1XfrKZT/QyVPoHVq1ur09eLqXJzHAjmE7z2WlCff+21oAO30Q1iojpao2JfuTJ5zHX2\neXDzwWCewsFHI9+vzHM4uPlg48+odzyq+1EqSqVgvkX1cY+7kU4af//KZ1bfyKifbmzUI9T53M/y\nuJhYVExR92dutgM3jc72Ovt8fZmLN9T/zIHRgcYT3CojjqLuR1357FdfPVZMOnIkSJqNliKvvD4y\n0vm/f1SHeJIBA5IrSgz9rtPr1HRidE9tTLWN2/79cOutwZVCZUbvySeHY6ht7NJIgo322c5nNjqO\nlX2vXQv/+I9BYohKeFHDfWuTbO3ntnOMNm4M9g3B4x136N4HBaTEIJ2VxvyIqDPzyj6vuSb4rGuv\nDeYDQP0GNY3F2hrts9XPTHocH3wwSAqlUtDIN0pO1fs9dCg4ftPTx5JEp8/uzzwTnnkmfrmSpCux\nStcoMUhndbJkU70qaNRqqPv3H5soVl0miSud5KGBSdLgJTmOlUZ+ejqYr7F/f/S+Kp9ROZaV/ZoF\nx6ly/O6+u/2kvnx5MLmwEvd11wVfcaXBqASk229mSolBOqtTJZtKw1CpoZvBG94QNDwV9RrPuNJJ\n1g1MJ+6zXNHqHesqfRa1EwY//GF47LH2kvrwMIyPR5fyKpJcDaU9M19iKTFI53WiZDMxEZQ6vLys\nhfvxSyrUazzrlU7y0MA0E0+SMlUrd6zbv//YzYwWLgz/fO3zVlRfoVQ/r0iS0LQSa6aUGCSfRkbC\nI48gevnnqMaz9rVmG5g0a9udbvCSJOGkS18k3V89SW8IlCSh5XHEXB+pLHddKENDQz41NZV1GNIJ\ncY3whg3HRh6VSnDbbcH6SZ38jKht0y49ZdGpmvZnVh83s2N9P6VSMKel0e1WpSvMbIu7DzXaTlcM\nkp1GjfDYWOdKG+1MWuv2SKY0pP2Z1cctyQ2BJNc081myU2/yVbXKbGTozuzZtGYDF02zs5Wrj9vs\n2cE8k1Znv0vmdMUg2Wl1VE2ajU0/1bbrlZdaOd79dNz6gBKDZKfVUTVpjyyqLbv04kSruMa/1eOd\nRYlMUpF6KcnMLjWzH5vZdjO7PuJ9M7O/Lb//tJldkHZMkiNJFq7LsrwTtwhdkcWV8WqPd+Wueb3y\nu0tDqV4xmFkJuA14L7Ab2Gxm97v7tqrN3gecW/56F7Cu/CgSyLJMkbd5EJ3SaAhrs3fNk56Sdilp\nMbDd3Z8HMLO7gMuA6sRwGbCxvFb4E2Y2x8zOcPcXU45NiiSrMkWvTrRKsgDg8PDxt/XslcQosdJO\nDGcBu6qe7+b4q4Gobc4ClBgke73cqdru5DjpWYXpfDazMWAMYHBwMONopK/0c6dqLydGqSvtxLAH\nmFf1fG75tWa3wd03ABsgmPnc2TBFpK5+Tox9Ku1RSZuBc83sHDObBVwB3F+zzf3A8vLopN8BfqX+\nBRGR7KR6xeDuR8zsGuBBoATc6e5bzWxF+f2vAA8AS4HtwCvAlWnGJCIi8VLvY3D3Bwga/+rXvlL1\nvQNXpx2HiIgko7WSREQkRIlBRERClBhERCREiUFEREKUGEREJESJQUREQpQYREQkRIlBRERClBhE\nRCREiUFEREKUGEREJESJQUREQpQYREQkRIlBRERClBhERCREiUFEREKUGEREJESJQUREQlK7taeZ\n/TXwh8Bh4F+AK939lxHb7QAOAkeBI+4+lFZMIiLSWJpXDA8B73D3dwI/AVbGbDvq7ucrKYiIZC+1\nxODu33H3I+WnTwBz0/osERHpnG71MXwc+Kc67znwsJltMbOxejswszEzmzKzqX379qUSpIiItNnH\nYGYPA6dHvHWDu99X3uYG4AjwjTq7ebe77zGzU4GHzOxH7v5o7UbuvgHYADA0NOTtxC0iIvW1lRjc\n/eK4983sY8D7gSXuHtmYu/ue8uNLZnYvsBg4LjGIiEh3pFZKMrNLgeuAD7j7K3W2eZOZnVT5HrgE\neDatmESSmpyENWuCR5F+k9pwVeBWYDZBeQjgCXdfYWZnAre7+1LgNODe8vszgb9392+nGJNIQ5OT\nsGQJHD4Ms2bBI4/A8HDWUYl0T2qJwd1/s87re4Gl5e+fB85LKwaRVkxMBEnh6NHgcWJCiUH6i2Y+\ni9QYGQmuFEql4HFkJOuIRLorzVKSSCENDwflo4mJICnoakH6jRKDSIThYSUE6V8qJYmISIgSg4iI\nhCgxiIhIiBKDSBdp4pwUgTqfRbpEE+ekKHTFINIlURPnRPJIiUGkSzRxTopCpSSRLtHEOSkKJQaR\nLtLEOSkClZJERCREiUFEREKUGES6THMZJO/UxyDSRZrL0J7JSXXed4MSg0gX6SZArVNS7Z407/l8\no5ntMbOnyl9L62x3qZn92My2m9n1acUjrZvcNcmax9YwuUu1j3ZpLkPrNEGwe9K+YrjZ3b9Q700z\nKwG3Ae8FdgObzex+d9+WclyS0OSuSZZsXMLho4eZVZrFI8sfYXieTtNapbkMrask1coVg5JqerIu\nJS0Gtpfv/YyZ3QVcBigx5MTEjgkOHz3MUT/K4aOHmdgxocTQJs1laI2SaveknRg+ZWbLgSng0+5+\noOb9s4BdVc93A+9KOSZpwsj8EWaVZr1+xTAyfyTrkKSPKal2R1uJwcweBk6PeOsGYB2wGvDy4xeB\nj7fxWWPAGMDg4GCru5EmDc8b5pHljzCxY4KR+SO6WhDpA20lBne/OMl2ZvZV4JsRb+0B5lU9n1t+\nLeqzNgAbAIaGhry5SKUdw/OGlRBE+kiao5LOqHr6QeDZiM02A+ea2TlmNgu4Arg/rZhERKSxNPsY\n1prZ+QSlpB3AJwHM7Ezgdndf6u5HzOwa4EGgBNzp7ltTjElERBpILTG4+5/UeX0vsLTq+QPAA2nF\nISIizdFaSSJ9Qms0SVJZz2MQkS7QchLSDF0xiPQBLSchzVBikIa0VlI6ulna0RpN0gyVkiSW1kpK\nR7OlnXaXm9ZyEtIMJQaJpbWS0tHM8tud6h/QchKSlEpJEquyVlLJSlorqYOaKe10o39g7dq1jI+P\nx24zPj7O2rVrO//hkjtKDBKrslbS6tHVKiN1UKW0s3p14yuAbvQPLFq0iGXLltVNDuPj4yxbtoxF\nixZ1/sMld8y9eMsODQ0N+dTUVNZhiHRNN25pWWn8N23axOjoaMPXpXjMbIu7DzXcTolBRCpqk4CS\nQm9JmhhUSupDGn4q9YyOjrJp0yaWLVvGZz7zGSWFPqVRSX1Gw0+bN7lrsq/uRzE6OspVV13F6tWr\nWbVqlZJCWTfKeXmhxNBnij78tNuNdFESaScbrfHxcdatW8eqVatYt24do6OjfZ8c+m1JESWGPlPk\nW3Vm0UgXIZF2stGq7VMYHR1VOYnm5p30AvUx9JkiDz+NaqTTVoR5HJ2a5xDV0Vzd59BonkMv67cl\nRXTF0IeKeqvOLK52inDP60qjVbliaKXRiht9VJ0c+vXKod+WFNFwVSmUTvUx9FqHcrt9DGvXrmXR\nokWxjf74+DibN2/muuuuazlOyVbm8xjM7H8Dv1V+Ogf4pbufH7HdDuAgcBQ4kiRoJQZpR6Wv4tDR\nQ5SsxK1Lb2XswrGswxJJXdLEkOatPf9jVTBfBH4Vs/mou7+cViwi1SZ2THDo6CGmfZppn+bqB65m\n4akLe+LKQaQTUu98NjMDlgH/K+3PEkliZP4IJSu9/nx6erorHdkiRdGNUUkXAT9395/Wed+Bh81s\ni5npel5SNzxvmFuX3srMGTOZwQxmz5ydy9FGnaJ7PUuz2iolmdnDwOkRb93g7veVv/8j4q8W3u3u\ne8zsVOAhM/uRuz8a8VljwBjA4OBgO2GLMHbhGAtPXdhTHdBR+m1ilnRGW4nB3S+Oe9/MZgIfAi6M\n2cee8uNLZnYvsBg4LjG4+wZgAwSdz22ELQKEh+322iilin6bmCWdkfY8houBH7n77qg3zexNwAx3\nP1j+/hLgcynHJBJSlGUvWtGJOQ7Sf9LuY7iCmjKSmZ1pZg+Un54GfM/Mfgj8APiWu3875ZhEQrKY\nUd0tzdwQqB3qx+gtqV4xuPvHIl7bCywtf/88cF6aMUj6il6GKfL6UUmkfa9n9WP0Hi2JIW0pahmm\nNpnlfdmLPFM/Ru9RYpC2FGH10Vr1klne484r9WN0T7fuCaHEIG0pYhmmiMksz/ptgbmsdLNk11eJ\noei18DwqYhmmiMks77rRj9HviaebJbu+SQxFrYUXQVHKMNUnBkVLZv1MnduBbpbs+uZGPb08JFGi\n7dy5lgMHgpvLVE4MVo2vYsnGJQCsvGglbzvxVXbuXJtlmNJAp25EVHTdGnoMfXTFoPJBf5ncNcmT\nO37C21nDeQvvYWLHE8edGLztxFfZtm0ZCxZsyjpciaHO7WPSLtlV9E1iKGItXFpTXTYcenOJz/Mh\n3nP66tCJwXtOO+n1pDAw0L07kqmfq3nq3O6+vkkMUJxauARabUSry4ZTv4CtfJjz932Whz6ylkd/\nfpD3nHYS7PtsJkkhST+XksfxunWmLIG+SgxSHO0MFqgtG15wzp/ythP/E9u2LeOKuVexd+/fxCaF\ntBrmJMNkNUhC8kCJQXKp+i5rh44eamquQb2y4ZlnXsULL6zm7LNXxSaFtBrmJP1cWc+x0LBQASUG\nyamTf+1kpn0agGmf5uRfO7mpn68tGx44MM7eves4++xV7N27jjlzRiOTQ5oNc23CAljz2JpQ8spy\nkISGhUqFEoPk0v5X9jODGUwzzQxmsP+V/a+/10ypZ3LXJE/+7A7ezt2ct/AeBgZGmTNntG7Hc9oN\ncyVhxS3LkdUgCa15JBVKDJJLI/NHmD1z9nENdDOlnsldk/zFfSNc/9bDXP+TWdx8zhsYHoCBgVEW\nLNgUmRzSaphrk1nclUlWgySaGRaqklNvU2KQXKrXQDdT6nnyZ3dw/VsP89nn4JlfHQ1t2yg5dLJh\njkpm3S4ZJbnKSjosVCWn3qfEILkV1UA306C+c85s/uujs3jmV0cjt60kh4MHN6c6bDUqma28aGXX\nSkbNXGUlGRaqklPvU2KQ3Ig7q21lnaOL3nkbNw/8cey2AwPRndDNxhenXjLrVsmoOjEdOnqIGydu\n5MaRG1v+bM1E7n3m7lnH0LShoSGfmprKOgzpoLiz2jyM7W83hiwnrVVirwz/ncEMZs+c3dZxVB9D\nMZnZFncfarRdW4vomdlHzGyrmU2b2VDNeyvNbLuZ/djM/qDOz7/ZzB4ys5+WHwfaiUeKK26Rwzws\ngNhuDMPzhhmZP8LEjgkmd3X3xsiV/pqLz7n49ZFe7R7H4WFYuVJJoVe1u7rqs8CHgEerXzSzBcAV\nwNuBS4G/M7NSxM9fDzzi7ucCj5SfS4+a3DXJmsfWRDaMlXJLyUrH9QfEvdct7cZQu7prFsnhxpEb\nmT1zdqbHUYqhrT4Gd38OwMxq37oMuMvdDwE/M7PtwGKg9n/DZcBI+fuvAxPAf2snJsmnRqWYuGGi\neVgAsd0Ysp7RDPk4jlIMaXU+nwU8UfV8d/m1Wqe5+4vl7/8VOC2leCRjSRrGuM7YPCyA2E4MeVn2\nPQ/HUfKvYWIws4eB0yPeusHd7+tUIO7uZla3J9zMxoAxgMHBwU59rHRJXhrGRtLqJNbZuhRJw8Tg\n7he3sN89wLyq53PLr9X6uZmd4e4vmtkZwEsxcWwANkAwKqmFmCRDRWgY0x79pLN1KYq0bu15P3CF\nmc02s3OAc4Ef1Nnuo+XvPwp07ApE8md43jArL1qZ28YxD6OfOimus18kTlt9DGb2QeDLwFuAb5nZ\nU+7+B+6+1cw2AduAI8DV7n60/DO3A19x9yng88AmM/tT4AVgWTvxiLSjKOWuJPIw90OKq91RSfcC\n99Z576+Av4p4/RNV3+8HlrQTg0inFKHclVQeRkFJcWlJDJEqvdIP0EtXP9J9SgwiPaiXrn6k+5QY\nRCJkubZRp/TK1Y90nxKDSA113Eq/S2u4qkhhdWPYqoaSSp7pikGkRjsdt0lKULoikbxTYhCp0WrH\nbdIGX0NJJe+UGEQitNJxm7TB11BSyTslBpEOSdrgayip5J1u7SnSQb0wzFV6V9Jbe+qKQaSDNHdA\neoGGq4qISIgSg4iIhCgxiIhIiBKDiIiEKDGIiEiIEoOIiIS0lRjM7CNmttXMps1sqOr195rZFjN7\npvz47+v8/I1mtsfMnip/LW0nHhERaV+78xieBT4ErK95/WXgD919r5m9A3gQOKvOPm529y+0GYeI\niHRIu/d8fg7AzGpf/+eqp1uBN5rZbHc/1M7niYhI+rrRx/Bh4MmYpPApM3vazO40s4EuxCMiIjEa\nJgYze9jMno34uizBz74d+B/AJ+tssg74DeB84EXgizH7GjOzKTOb2rdvX6OPFhGRFjUsJbn7xa3s\n2MzmAvcCy939X+rs++dV238V+GZMHBuADRAsotdKTCIi0lgqpSQzmwN8C7je3f9fzHZnVD39IEFn\ntoiIZKjd4aofNLPdwDDwLTN7sPzWNcBvAp+pGop6avlnbq8a2rq2PKT1aWAU+It24pFi0P2ORfJN\n92OQrtL9jkWyk/R+DJr5LF0VdftLEckXJQbpqsrtL0tW0v2ORXJKd3CTrtL9jkXyT4lBuk63vxTJ\nN5WSREQkRIlBRERClBhERCREiUFEREKUGEREJESJQUREQgq5JIaZ7QNeyDqOBk4huJNdERQpVihW\nvEWKFYoVb5FihXzEe7a7v6XRRoVMDEVgZlNJ1iTJgyLFCsWKt0ixQrHiLVKsUKx4VUoSEZEQJQYR\nEQlRYkjPhqwDaEKRYoVixVukWKFY8RYpVihQvOpjEBGREF0xiIhIiBJDyszsU2b2IzPbamZrs46n\nETP7tJm5mZ2SdSxxzOyvy8f1aTO7t3yf8Vwxs0vN7Mdmtt3Mrs86nnrMbJ6ZjZvZtvK/0z/POqZG\nzKxkZv9sZt/MOpZGzGyOmf1D+d/rc2aW+6WFlRhSZGajwGXAee7+duALGYcUy8zmAZcAO7OOJYGH\ngHe4+zuBnwArM44nxMxKwG3A+4AFwB+Z2YJso6rrCPBpd18A/A5wdY5jrfhz4Lmsg0joS8C33f1t\nwHkUIG4lhnRdBXze3Q8BuPtLGcfTyM3AdUDuO57c/TvufqT89AlgbpbxRFgMbHf35939MHAXwUlC\n7rj7i+7+ZPn7gwQN11nZRlWfmc0F/gNwe9axNGJm/w54D3AHgLsfdvdfZhtVY0oM6XorcJGZfd/M\n/q+ZLco6oHrM7DJgj7v/MOtYWvBx4J+yDqLGWcCuque7yXFjW2Fm84HfBr6fbSSxbiE4gZnOOpAE\nzgH2AV8rl75uN7M3ZR1UI7qDW5vM7GHg9Ii3biA4vm8muDxfBGwys9/wjIaCNYj1LwnKSLkRF6+7\n31fe5gaCUsg3uhlbLzKzE4G7gWvd/d+yjieKmb0feMndt5jZSNbxJDATuAD4lLt/38y+BFwPrMo2\nrHhKDG1y94vrvWdmVwH3lBPBD8xsmmC9lH3diq9avVjNbCHBmc0PzQyCssyTZrbY3f+1iyGGxB1b\nADP7GPB+YElWyTbGHmBe1fO55ddyycxOIEgK33D3e7KOJ8bvAR8ws6XAG4BfN7P/6e5/nHFc9ewG\ndrt75QrX6sTMAAABAUlEQVTsHwgSQ66plJSu/wOMApjZW4FZZL+I1nHc/Rl3P9Xd57v7fIJ/zBdk\nmRQaMbNLCcoJH3D3V7KOJ8Jm4FwzO8fMZgFXAPdnHFMkC84G7gCec/e/yTqeOO6+0t3nlv+dXgF8\nN8dJgfL/oV1m9lvll5YA2zIMKRFdMaTrTuBOM3sWOAx8NIdntkV1KzAbeKh8lfOEu6/INqRj3P2I\nmV0DPAiUgDvdfWvGYdXze8CfAM+Y2VPl1/7S3R/IMKZe8ingG+UThOeBKzOOpyHNfBYRkRCVkkRE\nJESJQUREQpQYREQkRIlBRERClBhERCREiUFEREKUGEREJESJQUREQv4/8INyapzA8FMAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116462780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "Xvs = np.vstack((C1, C2, C3))\n",
    "\n",
    "mus, classes, errs = kmeans(Xvs, K=3)\n",
    "\n",
    "kscatter_plot(Xvs, classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2QHPV95/H3d0fsYju6kyLxJKRFpELikk1wYJGzibna\nPWGCdbbB2NGRuoswxtmIQlRI+UqHQimhrApKiH3GQQQkYxyrQkJUAQXKxiaAtGeIFlsSwYAEThSF\nB0nEgCQ7ulCskPZ7f/QMmh7N9PQ89HT3zOdVtTVPPT3f7Z39ffv32ObuiIiIlPSlHYCIiGSLEoOI\niIQoMYiISIgSg4iIhCgxiIhIiBKDiIiEKDGIiEiIEoOIiIQoMYiISMi0tANoxuzZs33+/PlphyEi\nkis7dux4091PqbddLhPD/Pnz2b59e9phiIjkipm9HGc7NSWJiEiIEoOIiIQoMYiISIgSg4iIhCgx\niIhIiBKDiIiEKDGISFtMTMCaNcGt5Fsu5zGISLZMTMCiRXDkCPT3w+OPw/Bw2lFJs1RjEJGWjY8H\nSeHYseB2fDztiKQVSgwi0rKRkaCmUCgEtyMjaUckrVBTkoi0bHg4aD4aHw+SgpqR8k2JQUTaYnhY\nCaFbJN6UZGYvmdlzZvaMmZ2w8p0F/szMdpvZs2Z2ftIxiYhIbZ2qMYy6+5s1XvsYcE7x58PAncVb\nERFJQRY6ny8DNnjgKWCGmZ2RdlAiIr2qE4nBgcfMbIeZjVV5/Uzg1bLHe4vPiYhICjrRlPQRd99n\nZqcCj5rZi+7+vUZ3UkwqYwCDg4PtjlFERIoSrzG4+77i7evAJmBhxSb7gHllj+cWn6vcz3p3H3L3\noVNOqXtlOpFkaf0H6WKJ1hjM7H1An7sfLt6/BPhixWYPAcvN7D6CTuefuvtrScYl0hKt/yBdLumm\npNOATWZW+qy/cvfvmtkyAHe/C3gYWAzsBt4Crk44JpHWVFv/QYlBukiiicHd9wDnVXn+rrL7DlyX\nZBwibVVa/6FUY9D6D9JlNPNZpFFa/0G6nBKDSDO0/oN0sSxMcBMRkQxRYhARkRAlBhERCVFiEBGR\nECUGSVcaM4g1a1kkkkYlSfImJqoP7UxjBrFmLYvUpRqDJKtUEK9aFdyWn6VHXUE+qbN6XbU+91Th\nS55qDJKsqOUjas0gTvKsXrOWc00Vvs5QYpBkRRXEtWYQlyeTt9+GDRva99+vWcsNq9USmAYtU9UZ\nFixVlC9DQ0O+ffsJl4+WrGq0ZJmYgNFRmJwMHvf3J1MCZKnEy6isnaFnLZ68MbMd7j5UbzvVGCR5\njS4fMTwMV18N69aBe3B62O7EUFnC3HYbHDigJFEhycpbM1Th6wwlBsmmpUvhnnvgnXegUGh/X0B5\niTc5CcuXw9SUTkMrjIzAtGnBYXIP/iRLl544uKyTBbWWqUqeRiXJibIy7CO4jsfx23Yq9X0UCtDX\nF5R8cUcqZeX4dECp8lb6E5QqbyVRg84kv1RjkLCsNOKOj8PRo8Fp6tGj8ZuS4p6+lrdJzJoFN9wQ\nb6RSVo5Pm8Q5XEuXwje/Wf3w5LUzWN1L0ZQYJCwr/+nNDCtttNAub5M499x4JUVWjk8bxD1cUe36\n5X+mQgFeeSXYbyOHpNOFdJfl9kQoMUhYVsb5R5VGtUqSRgrtyn3EbbjOyvFpg0YOV63DU/ozbdgA\n3/gGfO1rQe0ibmGbRiHdRbk9MYkmBjObB2wguPazA+vd/asV24wADwL/WnzqAXf/YpJxSYRmhn0k\ndcpXrTSKKkniFtqtlEZdNCym2uFq5k85PHy85a/RwjaNQrqLcntikq4xHAW+4O5Pm9l0YIeZPeru\nuyq2e8LdP55wLBJXI8M+kj7lqyypokqSuIV2M6VRtRpGhySZd8sPFzT/p2y2sE2jkO6i3J6YRBOD\nu78GvFa8f9jMXgDOBCoTg+RVkqd81ZJO1DIapf/0lSuj99toaZRio3S7PzqqBW3NmvCfcsOG+IVn\n3MK22uenUUhryGu0jvUxmNl84JeB71d5+VfN7FlgH/C/3H1nlfePAWMAg4ODyQUqjUnylK9a0lm5\nkleufoTpk88y85rzg//uKqXnobffz+FthxlcUeO7ctVVwW3loPxypVLslVdSa5RuR94t/RqVg68q\nk0z5n3LatGDOwrFj8RNSeWFbrZZTK8mpkM6ejiQGM/sZ4H7gBnf/94qXnwYG3f3/mdli4O+Acyr3\n4e7rgfUQLImRcMgSV5KnfDWSzvTPfJBdS/pY8D/ez0w4ofQ89PWn2fXgFAs2Ljhxn5Wl09Kl1T+7\nfLtCISgpoeON0q3m3fJfwyyYwzc1VT3JBHMWbmVy8kIOHBjlwQeD0cKV227ZsoVt27axYsWKup9Z\nngDU6ZsfiScGMzuJICnc6+4PVL5enijc/WEz+3Mzm+3ubyYdm7RJo30SjbZPbNgQenrm6EwWbFzA\nriW7WLBxATPLSs9DhSF23f8BFjywgJknvwhrxo+Xpo2c/ZeXYgC//dswONjxRulW8275r9HXF+Q4\ns9pJ5jOfuZArrljCf/zHRtxHgfDE8y1btrBkyRL+8A83smZN9ZhqJQB1+uaIuyf2AxjBqKTbIrY5\nneOL+S0EXik9rvVzwQUXuLTB1q3ut9wS3HZi/1u3ur/nPe6FQnAb53Mj3nNw80F/cvaTfnDzQfet\nW/3gNWv9yRlb3n387vv6+90HBk68HxVDM7HGlfRxr/io8l9j3boT/ySVoVxzzWaH2Q6b3cx92bLg\n+c2bN/vs2bP99ts3Rx6aqM/s4K8uVQDbPU7ZHWejZn+AjxAMU30WeKb4sxhYBiwrbrMc2An8EHgK\n+NV6+1ViaIMkC75a+7/lluAxBLe33BJ+z7p17pdcEtyW1HlPKTnsWbXneJJwD0ozs+B9ZsfvFwrB\na3FKpyRKsdJx6etzP+mk8O+akFq/Rq2vwNat7v39QXLo79/sW7e63377Zn/ve4OkUO/PWP6ZK1a4\nT5sW/LpJfM2kMZlIDEn9KDG0QZz/7nbvPyoZrVsXbFv6KRWY9U553X3Pqj2+hS2+Z9We4+8ZGDi+\nr2nT4tcSkj6dveWWoJQsj62Fz2sl5KivwNatQc1hxozZfvXVqxxme1/f5nf/BHHOKbZuDXJf6Vft\n62v/10waEzcxaOZzr0qywXdiImjLr+ywjWowv//+8D7uvx/GxqLXNCqOPtp/537OWnUW++/cz4zR\nGcx8ajyYbQVBg/rnPx90Mkc11HdqSOrISNBoPzUVPJ6aaroXdmIi2N0778BJJ8Httze2cni9aygN\nD48yZ861rF69GrNVTE2NcuRI8Blxp4uUumgg6ONQv0JOxMkeWftRjaFNtm4NmlWWLWvfWXJl237c\nfVfWGC6//MT3VZziHrxmbaj56N0+h9ufbLyZLOkaVLl16xpqX6lVK1i2LHzICoXGWwartd6VlPoU\nKmsMcfedQquZ1IGakqSuJPoZWilg161zX7gwKDSrxVQW78FpF/qTJ38nSAJlQsmhsjSNandp9ljU\na8uJauCP0QYUFdbll4cTQ3k3SpzDHrXvUlLYvHmzu4f7GBqhzuZsUWKQ+pI4S2412dSLaetWP/iJ\nP/An2eQH7fyqp6Kh0UqNxNVoKVZvn21IvLUOR2U3ill0Pq32a9Xad2VSKKn1vOSHEoPUl9TIpFZO\nE6OGyhT3+fKlfxEkhYgO3IObD/rLf/Ly8SeSSIL19tmGz6x1OMp3XUoMAwMnttxF/YmrvVav8I+T\nHKqNUlatIRviJgZ1PveyLK4mVi2mio7hwdtug8efg3eK76nSgTtzdCYzR2ce328Sne319tniZ5bm\nAla7HHVp12+/fbwx6ejRYA5evZXIS8+PjJx4qG+9dRsbN25kdHS0akyjo6Ns3LiRbdu2Vd2mWod4\n1DIcklFxskfWflRjyLBO9VuUOnBLbSjlzUkttus3pNk+hhi7jdPydfnlwTa1+rGrjfZtdMJbIyo7\nxBcu7FyfvtSHagySiiQWxKl25l3a5/LlwWfdcENwFTaoPew0idXa6u2zyc+MexgfeSQogguFoGZR\nuU1lBax8v5OTweGbmgoO1W23tf/sfs4ceO656MVw467EKp2jxCDt1c4mm/JlQauthnrgwImrwkF0\n20kWSpgYJV6cw1gq5KemgukaBw5U/7jSR5QOZWm/ZsFhKh2+++9vPacvXRpcya0U94oVwU9Ey2DV\nBKTLb6ZLiUHaq139FqWSodSIbgYnnxxeDbVW6Vn+3KxZ2SphYpZ4cQ5jsxesK/VZVM4X/PSn4Ykn\nWsvpw8OwZcuJcdfr94i7EJ90hhKDtF87mmzGx4O2Di+usO4ePC4vIWqVnrXaTrJQwjQQT5xWqmYu\nWHfgwPFrGZ17bvj9lY+bUV5DKX9cEiehaSXWdCkxSDZVLh0B4fWfS6qVnpXPNVrCJNm43eYSL04O\nrr/0RWP7qyXuBYHiJLQsDpjrJaXlrnNlaGjIt2/fnnYY0g5RhfD69XDddcGpbqEAd9wRrJ/Uzs+o\ntm3STU8p9Kom/ZG1LghUKMDq1fWvtiqdYWY73H2o3naqMUh66hXCY2Pta9uI+95OND2lcC3LpD+y\n0QsCSbb1pR2A9LBas6/KDQ8fP91csyZIJkkqtbsUCj1dqk1MNHa4yw/bwACsXRvUFNLu65fmqMYg\n6Wl2WE2SpU0PNW7Xal5q5nD30GHrCUoMkp5mh9UkPbKost2lC2daRRX+zR7uFFrIJCGJNyWZ2aVm\n9iMz221mN1Z53czsz4qvP2tm5ycdk2RIqakoqkRJs3mnVIKuWhXcJt2U1SFRrXiVh3vWrM604kl2\nJFpjMLMCcAfwUWAvsM3MHnL3XWWbfQw4p/jzYeDO4q1IIM12iqzNg2iTekNYIy6a1w2/vtSRdFPS\nQmC3u+8BMLP7gMuA8sRwGbChuMDTU2Y2w8zOcPfXEo5N8iStdoounWlVL9eWDveaNV2ZF6WOpBPD\nmcCrZY/3cmJtoNo2ZwJKDJK+Lu5VbXVynHSv3HQ+m9kYMAYwODiYcjTSU3q4V7WL86JESDox7APm\nlT2eW3yu0W1w9/XAeghmPrc3TBGppYfzYs9KelTSNuAcMzvbzPqBK4GHKrZ5CFhaHJ30K8BP1b8g\nIpKeRGsM7n7UzJYDjwAF4B5332lmy4qv3wU8DCwGdgNvAVcnGZOIiERLvI/B3R8mKPzLn7ur7L4D\n1yUdh4iIxKO1kkREJESJQUREQpQYREQkRIlBRERClBhERCREiUFEREKUGEREJESJQUREQpQYREQk\nRIlBRERClBhERCREiUFEREKUGEREJESJQUREQpQYREQkRIlBRERClBhERCREiUFEREISu7Snmf0p\n8AngCPAvwNXu/pMq270EHAaOAUfdfSipmEREpL4kawyPAh90918C/glYGbHtqLt/SElBRCR9iSUG\nd/97dz9afPgUMDepzxIRkfbpVB/D54Dv1HjNgcfMbIeZjdXagZmNmdl2M9v+xhtvJBKkiIi02Mdg\nZo8Bp1d56SZ3f7C4zU3AUeDeGrv5iLvvM7NTgUfN7EV3/17lRu6+HlgPMDQ05K3ELSIitbWUGNz9\n4qjXzeyzwMeBRe5etTB3933F29fNbBOwEDghMYiISGck1pRkZpcCK4BPuvtbNbZ5n5lNL90HLgGe\nTyomkbgmXp1gzRNrmHh1Iu1QRDouseGqwFpggKB5COApd19mZnOAu919MXAasKn4+jTgr9z9uwnG\nJFLXxKsTLNqwiCPHjtBf6OfxpY8zPG847bBEOiaxxODuP1/j+f3A4uL9PcB5ScUg0ozxl8Y5cuwI\nx/wYR44dYfylcSUG6Sma+SxSYWT+CP2FfgpWoL/Qz8j8kbRDEumoJJuSRHJpeN4wjy99nPGXxhmZ\nP6LagvQcJQaRKobnDSshSM9SU5KIiIQoMYiISIgSg4iIhCgxiHSQJs5JHqjzWaRDNHFO8kI1BpEO\nqTZxTiSLlBhEOkQT5yQv1JQk0iGaOCd5ocQg0kGaOCd5oKYkEREJUWIQEZEQJQaRDtNcBsk69TGI\ndJDmMrRm4tUJdd53gBKDSAfpIkDNU1LtnCSv+Xyzme0zs2eKP4trbHepmf3IzHab2Y1JxSPNU9NH\n+2guQ/M0QbBzkq4xfMXdv1TrRTMrAHcAHwX2AtvM7CF335VwXBKTztLaS3MZmldKqqXvopJqctJu\nSloI7C5e+xkzuw+4DFBiyAg1fbSf5jI0R0m1c5JODNeb2VJgO/AFdz9U8fqZwKtlj/cCH044JmmA\nztIkS5RUO6OlxGBmjwGnV3npJuBOYDXgxdsvA59r4bPGgDGAwcHBZncjDdJZmkjvaSkxuPvFcbYz\ns68B36ry0j5gXtnjucXnqn3WemA9wNDQkDcWqbRCZ2kivSXJUUlnlD38FPB8lc22AeeY2dlm1g9c\nCTyUVEwiIlJfkn0Mt5rZhwiakl4CfgfAzOYAd7v7Ync/ambLgUeAAnCPu+9MMCYREakjscTg7r9V\n4/n9wOKyxw8DDycVh4iINEZrJYn0CE1UlLjSnscgIh2giYrSCNUYRHqAlpOQRigxSF1qgkhGJ4+r\n1miSRqgpSSKpCSIZjR7XVpeb1kRFaYQSg0TSWknJaOS4tis5a6KixKWmJImkJohkNHJc1T8gnaYa\ng0RSE0QyGjmuWshQOs3c87fs0NDQkG/fvj3tMEQ6Rpe0lHYwsx3uPlRvO9UYRHJA/QPSSepj6EEa\nfioiUVRj6DEafto4NeMI9Nb3QImhx+R9+Gmn/znzkkh7qdBKQ16+B+2ixNBj8jzCJY1/zjwk0l4r\ntNKQh+9BO6mPoceUhkmuHl2duwIkjfH8eZjHoXkOycvD96CdVGPoQXkd4ZJGbScP8zjyXAvMizx8\nD9pJ8xgkV9rVlt5tbfLd9vtIMuLOY0gsMZjZ3wC/WHw4A/iJu3+oynYvAYeBY8DROEErMUgrSm3y\nk8cmKViBtYvXMnbBWNphiSQu9Qlu7v7fy4L5MvDTiM1H3f3NpGIRKTf+0jiTxyaZ8immfIrrHr6O\nc089V2faIkWJdz6bmQFLgL9O+rNE4hiZP0LBCu8+npqaUoetSJlOjEq6CPixu/9zjdcdeMzMdpiZ\n6vOSuOF5w6xdvJZpfdPoo4+BaQNd3WGrme7SqJaakszsMeD0Ki/d5O4PFu//JtG1hY+4+z4zOxV4\n1MxedPfvVfmsMWAMYHBwsJWwRRi7YIxzTz236ztsNcdBmtFSYnD3i6NeN7NpwBXABRH72Fe8fd3M\nNgELgRMSg7uvB9ZD0PncQtgiQHjYbreO6um1iVnSHknPY7gYeNHd91Z70czeB/S5++Hi/UuALyYc\nk0hIo2fVr7xyK9OnX8jMmaM1tzl0aAuHD29jcHBFEiHHpjkO0oyk+xiupKIZyczmmNnDxYenAU+a\n2Q+BHwDfdvfvJhyTSEijM4enT7+QXbuWcOjQlqqvHzq0hV27ljB9+oUJRNuYTs10Vz9Gd0m0xuDu\nn63y3H5gcfH+HuC8JGOQ5OW9GabRs+qZM0dZsGAju3YtYcGCjaGaQykpVD6fpqRnuqsfo/toSQxp\nSV4Lhcpk1uhyB9WSQxaTQieoH6P7KDFIS/JYKNRKZo3GXZ4c5sy5lv377+y5pADqx+ikTtXOlRik\nJXksFNqZzGbOHGXOnGt5+eXVnHXWqp5LCtB7C8ylpZO1855KDHlvC8+iPBYK7Uxmhw5tYf/+Oznr\nrFXs338nM2aM9mxySLofI0/fsSR0snbeM4khr23heZCXZbzLC5d2JLPKPoUZM0Z7so8hafrfDXSy\ndt4ziSGPbeHSPtUKl5UXrWx6f9U6mqNGK0nz9L8b6GTtvGeu4NZrV2DqdZXj6tt5lbOo0UflyaHW\nPAdpjP53jxueN8zKi1Ymnhh7psaQx7ZwaU612kE7q+GHD2+LrBGUksPhw9tO2EZt5Y3T/27n6Qpu\nklnNFqJrnljDqi2rOObHKFiB1aOrWXnRytQL5bht5WnHKd0r9Qv1iLSilQ7HWrWDuJ3kSRXMcdrK\n1dEqWaDEIJlUfpW1yWOTDXU4ttL0kGTBHKc5K+2OVtVWBJQYJKNmvXcWUz4FwJRPMeu9sxp6f7ND\naJMsmCsTFgTNXuWFcJoTBlVbkRIlBsmkA28doI8+ppiijz4OvHXg3dcaOatt9Aw46YK5lLCiluVI\nq6M17dqKZIcSg2TSyPwRBqYNnFBAN3JW28wZcFIFc2WCiiqE05ow2EhSVJNTd1NikEyqVUA3clbb\n7BlwuwvmpIfPxo2hXkEeNymqyan7KTFIZlUroBspULOywF+1BLXyopUdazJqpCCPkxTV5NT9lBgk\nM6LOaptZ56jdzULNNp+0Ony2VeUF+eSxSW4ev5mbR25u+rOzknAlOZrgJpkQdVabhaaLVmNIs02+\nFHtp+G8ffQxMG2jpOKqPIZ/iTnBraa0kM/sNM9tpZlNmNlTx2koz221mPzKzX6/x/p81s0fN7J+L\ntzNbiUfyK2oto3auc5REfHEMzxtmZP4I4y+Nd/y6yKWa08VnX/zuSK9Wj2On1uyRdLS6iN7zwBXA\n98qfNLMFwJXAB4BLgT83s0KV998IPO7u5wCPFx9Ll4q6YHzUQmlZWESt1RhKZ+2rtqxi0YZFqSSH\nm0duZmDagBajk7pa6mNw9xcAzKzypcuA+9x9EvhXM9sNLAQq/xsuA0aK978JjAP/u5WYJJvqNcVE\n9QdkYRG1VmPIQodtFo6j5ENSnc9nAk+VPd5bfK7Sae7+WvH+vwGnJRSPpCxOwRjVGZuFiwG1EkNW\nOmyzcBwl++omBjN7DDi9yks3ufuD7QrE3d3MavaEm9kYMAYwODjYro+VDslKwVhPUp2qOluXPKmb\nGNz94ib2uw+YV/Z4bvG5Sj82szPc/TUzOwN4PSKO9cB6CEYlNRGTpCgPBWPSo590ti55kdQV3B4C\nrjSzATM7GzgH+EGN7a4q3r8KaFsNRLIn6yNZsjD6qZ2iOvtForTUx2BmnwJuB04Bvm1mz7j7r7v7\nTjPbCOwCjgLXufux4nvuBu5y9+3AHwMbzewa4GVgSSvxiLQiL81dcWRh7ofkV6ujkjYBm2q89kfA\nH1V5/vNl9w8Ai1qJQaRd8tDcFVcWRkFJfmlJDJEy3dIP0E21H+k8JQaRLtRNtR/pPCUGkSq6YS2g\nbqn9SOcpMYhUUMet9LqkhquK5FYnhq1qKKlkmWoMIhVa6biN0wSlGolknRKDSIVmO27jFvgaSipZ\np8QgUkUzHbdxC3wNJZWsU2IQaZO4Bb6GkkrW6dKeIm3UDcNcpXvFvbSnagwibaS5A9INNFxVRERC\nlBhERCREiUFEREKUGEREJESJQUREQpQYREQkpKXEYGa/YWY7zWzKzIbKnv+ome0ws+eKt/+1xvtv\nNrN9ZvZM8WdxK/GIiEjrWp3H8DxwBbCu4vk3gU+4+34z+yDwCHBmjX18xd2/1GIcIiLSJq1e8/kF\nADOrfP4fyx7uBN5jZgPuPtnK54mISPI60cfwaeDpiKRwvZk9a2b3mNnMDsQjIiIR6iYGM3vMzJ6v\n8nNZjPd+APgT4HdqbHIn8HPAh4DXgC9H7GvMzLab2fY33nij3keLiEiT6jYlufvFzezYzOYCm4Cl\n7v4vNfb947LtvwZ8KyKO9cB6CBbRayYmERGpL5GmJDObAXwbuNHd/yFiuzPKHn6KoDNbRERS1Opw\n1U+Z2V5gGPi2mT1SfGk58PPAH5QNRT21+J67y4a23loc0vosMAr8XivxSD7oesci2abrMUhH6XrH\nIumJez0GzXyWjqp2+UsRyRYlBumo0uUvC1bQ9Y5FMkpXcJOO0vWORbJPiUE6Tpe/FMk2NSWJiEiI\nEoOIiIQoMYiISIgSg4iIhCgxiIhIiBKDiIiE5HJJDDN7A3g57TjqmE1wJbs8yFOskK948xQr5Cve\nPMUK2Yj3LHc/pd5GuUwMeWBm2+OsSZIFeYoV8hVvnmKFfMWbp1ghX/GqKUlEREKUGEREJESJITnr\n0w6gAXmKFfIVb55ihXzFm6dYIUfxqo9BRERCVGMQEZEQJYaEmdn1Zvaime00s1vTjqceM/uCmbmZ\nzU47lihm9qfF4/qsmW0qXmc8U8zsUjP7kZntNrMb046nFjObZ2ZbzGxX8Xv6u2nHVI+ZFczsH83s\nW2nHUo+ZzTCzvy1+X18ws8wvLazEkCAzGwUuA85z9w8AX0o5pEhmNg+4BHgl7VhieBT4oLv/EvBP\nwMqU4wkxswJwB/AxYAHwm2a2IN2oajoKfMHdFwC/AlyX4VhLfhd4Ie0gYvoq8F13fz9wHjmIW4kh\nWdcCf+zukwDu/nrK8dTzFWAFkPmOJ3f/e3c/Wnz4FDA3zXiqWAjsdvc97n4EuI/gJCFz3P01d3+6\neP8wQcF1ZrpR1WZmc4H/Btyddiz1mNl/Bv4L8HUAdz/i7j9JN6r6lBiS9QvARWb2fTP7v2Z2YdoB\n1WJmlwH73P2HacfShM8B30k7iApnAq+WPd5LhgvbEjObD/wy8P10I4l0G8EJzFTagcRwNvAG8I1i\n09fdZva+tIOqR1dwa5GZPQacXuWlmwiO788SVM8vBDaa2c95SkPB6sT6+wTNSJkRFa+7P1jc5iaC\nppB7OxlbNzKznwHuB25w939PO55qzOzjwOvuvsPMRtKOJ4ZpwPnA9e7+fTP7KnAjsCrdsKIpMbTI\n3S+u9ZqZXQs8UEwEPzCzKYL1Ut7oVHzlasVqZucSnNn80MwgaJZ52swWuvu/dTDEkKhjC2BmnwU+\nDixKK9lG2AfMK3s8t/hcJpnZSQRJ4V53fyDteCL8GvBJM1sMnAz8JzP7S3f/nynHVcteYK+7l2pg\nf0uQGDJNTUnJ+jtgFMDMfgHoJ/1FtE7g7s+5+6nuPt/d5xN8mc9PMynUY2aXEjQnfNLd30o7niq2\nAeeY2dlm1g9cCTyUckxVWXA28HXgBXf/P2nHE8XdV7r73OL39Epgc4aTAsX/oVfN7BeLTy0CdqUY\nUiyqMSTrHuAeM3seOAJclcEz27xaCwwAjxZrOU+5+7J0QzrO3Y+a2XLgEaAA3OPuO1MOq5ZfA34L\neM7Mnil4lDsiAAAAT0lEQVQ+9/vu/nCKMXWT64F7iycIe4CrU46nLs18FhGREDUliYhIiBKDiIiE\nKDGIiEiIEoOIiIQoMYiISIgSg4iIhCgxiIhIiBKDiIiE/H8xGknX9+VxLQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116b0b9e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ks, mus, sigmas, pi, ll = GMM(Xvs, 3, 1000)\n",
    "\n",
    "classes = np.argmax(ks, axis=1)\n",
    "\n",
    "kscatter_plot(Xvs, classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to choose the initial centers for GMM? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def GMM(X, K=2, maxiter=100, init=None):\n",
    "    \"\"\" Gaussian Mixture Model\n",
    "    \n",
    "        Parameters\n",
    "        ----------\n",
    "        X         ndarray (N x D)\n",
    "                  input data to cluster\n",
    "        K         int\n",
    "                  the number of clusters\n",
    "        maxiter   int\n",
    "                  the maximum number of iteration\n",
    "        init      string\n",
    "                  kmeans init or random\n",
    "        \n",
    "        Returns\n",
    "        -------\n",
    "        @kappa    ndarray (N x K)\n",
    "                  responsibilities for each class\n",
    "        @mus      ndarray (K x D)\n",
    "                  centers for clusters\n",
    "        @Sigmas   list of ndarray (D x D)\n",
    "                  list of covariance matrices\n",
    "        @pi       vector (K,)\n",
    "                  mixing coefficient\n",
    "        @LL       list\n",
    "                  log-likelihood log\n",
    "    \"\"\"\n",
    "    N, D = X.shape\n",
    "    \n",
    "    if init is None:\n",
    "        # select initial centers - randomly for now, but YOU CAN USE K-MEANS for initial assignment\n",
    "        cidx = np.random.choice(N, K, replace=False)\n",
    "        mus = X[cidx, :]\n",
    "        Sigmas = [np.eye(D) for k in range(K)]  # uni-variance\n",
    "        pi = np.array([1/K] * K)\n",
    "    else:\n",
    "        # init with kmeans\n",
    "        mus, classes, errs = kmeans(Xvs, K=3)\n",
    "        Nk = np.bincount(classes)\n",
    "        pi = Nk / N\n",
    "        mus = np.asarray(mus)\n",
    "        Sigmas = [np.cov(X[classes==k].T) for k in range(K)]\n",
    "    \n",
    "    LL = []  # LogLikelihood log - expected to monotonically increasing\n",
    "    \n",
    "    # TODO: Finish GMM with your previous codes\n",
    "    \n",
    "    \n",
    "    \n",
    "            \n",
    "    return kappa, mus, Sigmas, pi, LL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAD8CAYAAABzTgP2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+U3HV97/HneydsAMs1aSJCIEvoKbYnGkGzYLeK3W2Q\nIlqpesul99QgVtfkJpzS472UyMkVTEskasErFFghHnKut5rTQAFNQUiyBW82kg0FgaAWkfxEwYA0\nPVx22Z33/eM7k8x3dn7PfOf7/e68HufsmZ2Z737nzbL5vL+f9+fH19wdERGRvK64AxARkWRRYhAR\nkRAlBhERCVFiEBGRECUGEREJUWIQEZEQJQYREQlRYhARkRAlBhERCZkRdwCNmDt3ri9YsCDuMERE\nUmXXrl2/cve3VDsulYlhwYIFjI6Oxh2GiEiqmNmeWo5TKUlEREKUGEREJESJQUREQpQYREQkRIlB\nRERClBhERCREiUFEWmJkBNauDR4l3VK5jkFEkmVkBJYsgfFx6O6GLVugry/uqKRR6jGISNOGh4Ok\nMDkZPA4Pxx2RNEOJQUSa1t8f9BQymeCxvz/uiKQZKiWJSNP6+oLy0fBwkBRURko3JQYRaYm+PiWE\n6SLyUpKZPW9mT5rZ42Y2Zec7C/wvM3vWzH5kZu+OOiYRESmvXT2GAXf/VZn3Pgickft6D3BL7lFE\nRGKQhMHni4ANHtgBzDKzk+MOSkSkU7UjMTjwkJntMrPBEu+fAuwreL4/95qIiMSgHaWk97n7ATM7\nEXjQzH7s7g/Xe5JcUhkE6OnpaXWMIiKSE3mPwd0P5B5fBO4Gzik65AAwv+D5qbnXis8z5O697t77\nlrdUvTOdSKRG9o2w9pG1jOzT/g8y/UTaYzCzNwFd7n449/35wBeLDrsXWGlm3yYYdH7V3V+IMi6R\nZozsG2HJhiWMT47Tnelmy9It9M3XPE2ZPqLuMbwV+IGZPQE8CnzP3e83s2Vmtix3zGbgOeBZ4BvA\nf4s4JpGmDD8/zPjkOJM+yfjkOMPPD8cdkkhLRdpjcPfngDNLvH5rwfcOrIgyDpFW6l/QT3em+0iP\noX9Bf9whibSUVj6L1Klvfh9blm5h+Plh+hf0q4wk044Sg0gD+ub3KSHItJWEBW4iIpIgSgwiIhKi\nxCAiIiFKDCIiEqLEILGKYwWxVi2LVKZZSRK5kX0jJad2xrGCWKuWRapTj0EilW+IV29bzZINS0JX\n6ZVWEEd1Va9Vy+k3MgJr1waPEg31GCRSpRri/BV6uRXEUV7Va9Vyuo2MwJIlMD4O3d3BfaZ1O9HW\nU2KQSFVqiMutIC5MJq9PvM6GJza0LDFo1XL9RkZgeBj6++NvhIeHg6QwORk8Dg/HH9N0ZMFWRenS\n29vro6NTbh8tCVVujKHS8QN3DjA2OQZAd6ab4UuHW96I1xtXJ0raFXrS4kkbM9vl7r3VjlOPQSJX\n7/YRffP7uOysy7ht1204zmR2MlSCaoXictWNF9zIodcOKUkUKbxCf/112LAh3oa4ry9IBknpwUxX\nSgySSEvPXMr6x9fzxuQbZLoyLR8LKCxXjU2MsXLzSrKe1UylIv39MGNGkBjcYf16WLo03CC3u9TU\n16eEEDXNSpIpkjLP37DQYyvlxz4ylqGrq4tJn6x5plJSfj/t0NcHl10GlvtfMDkZJIG8fGln9erg\nUTOFpgf1GCQkKfP8h58fZiI7geNMZCdqLiXVOm5QOAg95/g5XHH/FTXNVErK76dVarnaX7oU7rzz\naF2/v//oe2kdDE7SgHoSKTFISKXppe3UyLTSehvtwrGPRScuqimhJOX30wq1DuRWquv39wc/Oz4O\nmQzs3Ruct57Gtt2NtAawq1NikJCkzPOvNK20XK+gnka7+By1DpAn5ffTCvVc7Zer6+eTxoYN8M1v\nwje+EfQuam1s42ik09rLaadIE4OZzQc2ENz72YEhd/9a0TH9wD3Az3Mv3eXuX4wyLimvkXn+UU37\nLNVYV+oV1NpoN1MOmk7rIAqv9vMlokau3vv6gp+ZmKi/sY2jkS713y1hUfcYJoDPuftjZnYCsMvM\nHnT33UXHPeLuH444FqlRPdNLo665FyedSr2CWhvtRspBpXoY7RJVqaW4RASNX7032tjG0Uhrymt1\nkSYGd38BeCH3/WEzewY4BShODJJSUdbcC5POf+0xsr9/Q8VtNB77+R28c9ZMVp17c8Xz1lsOinPA\nudWlluIkU1giWrs2fPW+YUPtjWetjW2pz4+jkdaU18raNsZgZguAdwE/LPH275vZj4ADwH9396dL\n/PwgMAjQ09MTXaBSlyhr7oVJ5+lXu3jt4F/Te+Z3p/QKRvaN8Ff39HPV28b5Hw93c8PsP6/acF96\n5qVAsF6i2jjE3lf3xjbg3IpSS74xnjMHrriifJIpvHqfMSNYszA5WXtCKmxsS/VyyiU5NdLJ05bE\nYGa/AWwCrnD3fy96+zGgx93/w8wuBP4JOKP4HO4+BAxBsCVGxCFLjaKsuRcmnWf+o5vj513P7t0X\ns3DhRvrOXXXkuMd+fgdXvW2ca5+BJ1+tvEq6+Op/6ZlLqx6X6cowo2sGZGn7gHOzpZbCxtgMstng\nqzDJrFu3jrPPPpuBgYEjV++PPgr33BMsahsfhzvu2MYjj+zkyiuvrOszCxOABn3TI/LEYGbHECSF\nb7n7XcXvFyYKd99sZn9vZnPd/VdRxyatUe+YRK1JJJ90NjyxAYCu4xbzuws3HkkOs2cP8Mor23g7\nm7jqp908+erklPJS/rOAuq7+C3srZOEz7/4MPW/uafuAc7OllsLGuKsrmFJqFk4yZ599NhdffDEb\nN25kYGAAgGuvDZICgNk2Nm26mLvu2njkvJXGPcolAA36pkfUs5IMuAN4xt3/rswxJwG/dHc3s3MI\nVmMfijIuCUS9iVzx+Rut1d/5xJ2MT45z5xN3smXpFhbmksO8ecs5ePAWzlx0FzecfmzZz8p0ZTCM\niexEzVf/xSWySiWnun8vdQ4mN1NqKW6Mb7wRDh06+tkjI7BjxwBf+MLGI8lhx44BJibyZ9jGjBlB\nUsgnjWrjHsWfOWdOMH7R369B37SIusfwXuATwJNm9njutc8DPQDufivwn4HlZjYB/D/gEk/jlq8p\n047ZRMXnrzZQPbRriE27N/HxhR9ncPEgUGZw+9xVzJu3nD171nDaaauZPXuAvtmEzrXhiQ28PvE6\njpOdzALgeM1X/1GVyPKN6thYcPV+000wONiSU5dUqccRbuAHWLcuSA5f+MJGursHGBvbRjZ7MV/+\n8kaOPXbgSONerSRU+Jm//jWsWBGUr2bODF5ftQpJuKhnJf0AKm904+43ATdFGYdMFfUK3lLnrzRQ\nPbRriM9+97MAfP+57wMwuHhwys/MOX4ONz/8ad7OJk47bTUHD97CrFkDzJ49cORcI/tG+Obj3wwS\nAZDpypCxDBPZiYpX/40ueqvr9zIcJIV8rX/FCli0qPGr51p6H+V6HMUN/OHDA2zcGCSHSy5Zzne+\ncwvXX7+RxYsHQj2EG2+sXhLKf94f/AFHeh9jYxpXSAutfO5QUc4mGtk3wt5X904p2VS6Ct+0e1Po\nHJt2b2Jw8eCUPY3Wj1zOVW8b56qfdnPD6R9k4cKB0JgDHN1nCYIN+D79rk+z9MylFa/+2zUltb8/\n6Clkg04M2WzjjeXISHC+N96AY46Br389XCaqJZbiBr6vb4Dly5ezZs0aVq9ezcqVA1OmsR46VFtJ\naHg4+Jm8ri6NK6SGu6fua/HixS7N2753uy+7b5kvu2+Zb9+7vWXnPO5vjvPMtRnvXtNd87lvG73N\nuYYjX3/yD38y5edu+pe/8LsfwM+6Ec9cm/HrHr7O3d1ffnmr/+AHc/3ll7dOieG4vzmups+/7uHr\nPHNtxrkmfO4o3Hab+4wZ7l1d7scd5769Snjbt7tfd93U45Ytcw+GiIOvTCb4quWchbGcf37w6O6+\ndetWnzt3rq9evdrnzp3rW7du9e3bg3PWe+78z3V1uR9zzNHPkPgAo15DG6seQ4crHtht9iq5eDZP\nz5t7ajpnfkzhjsfu4LFfPMZ9P72PB372wJGYimcfZboy7H11LyP7RuibP3BkQHrhwo30zR8o2TOp\nNNjeaA+q2gB+qfcHB4PyUS2DsJUGen/xi/Cx2ezR6aW19EJGRo6ua3jkERgf38a11x6dnTQwMHBk\nQHrLloGGtsrQYHNK1ZI9kvalHkNrRHGV3MjVei0x7dlzvb/88tYjvZyZa2Z617VdfswXj/HbRoNL\n0Zdf3up79lzfcFzb92736x6+ruaYq52z2d+Fe9BTyGSO9giuy/0v2r7dfebMo70Fs6AXUuqqvlyP\no/DcXV1b/fjjgx5CoXwPovh1SSdq7DHoRj0drPBmNa0aZ8iPCawZWNNQD6RcTAfsXG59agcQ9ELe\nyL5B1rO8kX2DFZtXMLJvhNmzB+jpKb0Aq9RgeKnYV527quaYq52zls+sJj8OkMmEB3rzm9YVymTg\nM58J9yoq3Ugnf+6urmD20fXXH52SmjcwcHRAetu2bTXFPDISTE/Nf1bxc0k+lZI6WBJ3Ci0VU6n7\nM2csQ9aDEdxsNlt1VlUUg+3VztnsZ+ZnHBWvPYCjjfrrrx8dZZiYgJ6ecMmm1NTS/Ov5dQVf/OJO\nPvShjaxcGU4KefnksHPnzimJo1TMxQPilbbhkGRSYuhwrZ6S2YrZPcUxFV95H3rtEDddeBMrNq9g\nMjtJV1cXc46fE4qhONlFkQSrnbOZz6y2iCxfv1+3Du67L0gMpaaOllpsVjz19P3vv5LFi49+bqkx\ngfyYQzUbNgTnhvxWGtoGI42UGKSlolgfUerKO3/OlZtXMumTXHH/FSw6cRFA2cQUxbqEauds9DNr\n3VfogQeCpJDJBI188THFA8CF5x0bg5Urg0HrfJJo9dX9vHnw5JNT1zzUsv5Ct9+MjxKDtFQrSzb5\nK/85x88puRvqodcOkfUsWc+Gavil6vpJKpfV0uDVsq9QvpHPZoP9jw6V2Ugm/xnDw0GPIX9esyBB\n5DfV27Sp+av7pUuDO7nl477yyuCr8L+3lq3EdfvNeCkxSEu1qmSTL0nlt7UwjGNnHBvaDbVcEipe\nKR3XvRRKacV9lvNq3ZSu+DPzYxbF23B//OP5aauNb3LX1wfbtk2Nu9q4R60b8Ul7KDFIy7WiZDP8\n/DBjk2NHtrVwnLHJsZru2Fb4WtRbf9T93zXc/H2WC9+vdQVy8crl/H5Fxespal1fUUlhD6XweV4t\nCU07scZLiUESqX9Bf2jmEUDGMlNKU6WSUPFr9Za2otx1ttUNXi07r1b6zOKfb2Yn11pvCFRLQtPi\nuHiZp3Aj097eXh8dHY07DGmBSo3w0K6hIzOPMl0Zbr7w5iMrpFv1GaWOjbr0FMegatSfWe6GQJkM\nrFmjHVWTwsx2uXtvtePUY5DYVGuEBxcPsujERU1fvddT2mpH6SmOW1lG/ZmF5apyNwSS9NDKZ4lN\nPauRAdY+spaRfdEun41iNXga1btauXCF9syZwX0m1qzRbKK0Uo9BYlPr1NZ2bYkNyVwNHpVy5aVG\npopqTGB6UWKQ2NTaCLd7ZlFx6SnqW6DGoVLj3+hU0ThKZBKNyBODmV0AfA3IALe7+5eK3rfc+xcC\nrwGfdPfHoo5LkqGW+n+UNxWqpp29lXaq1PhXumezGv7OEGliMLMMcDPwAWA/sNPM7nX33QWHfRA4\nI/f1HuCW3KMIEG95J2nrIFql2hTWfFmo2tRTmZ6i7jGcAzzr7s8BmNm3gYuAwsRwEbAht1f4DjOb\nZWYnu/sLEccmKRLFPke1iLO3EqVqYwL5slDxbT21ArkzRJ0YTgH2FTzfz9TeQKljTgGUGCR203kw\nutnFcTJ9pWbw2cwGgUGAnp6emKORThJXbyUJNNuoM0WdGA4A8wuen5p7rd5jcPchYAiClc+tDVNE\nytFso84T9QK3ncAZZna6mXUDlwD3Fh1zL7DUAr8HvKrxBRGR+ETaY3D3CTNbCTxAMF11vbs/bWbL\ncu/fCmwmmKr6LMF01cuijElERCqLfIzB3TcTNP6Fr91a8L0DK6KOQ0REaqO9kkREJESJQUREQpQY\nREQkRIlBRERClBhERCREiUFEREKUGEREJESJQUREQpQYREQkRIlBRERClBhERCREiUFEREKUGERE\nJESJQUREQpQYREQkRIlBRERClBhERCREiUFEREIiu7WnmX0Z+GNgHPgZcJm7/7rEcc8Dh4FJYMLd\ne6OKSUREqouyx/Ag8A53fyfwU2BVhWMH3P0sJQURkfhFlhjc/fvuPpF7ugM4NarPEhGR1mnXGMOn\ngH8u854DD5nZLjMbLHcCMxs0s1EzG33ppZciCVJERJocYzCzh4CTSrx1tbvfkzvmamAC+FaZ07zP\n3Q+Y2YnAg2b2Y3d/uPggdx8ChgB6e3u9mbhFRKS8phKDu59X6X0z+yTwYWCJu5dszN39QO7xRTO7\nGzgHmJIYRESkPSIrJZnZBcCVwEfc/bUyx7zJzE7Ifw+cDzwVVUwiNRsZgbVrg0eRDhPZdFXgJmAm\nQXkIYIe7LzOzecDt7n4h8Fbg7tz7M4D/4+73RxiTSHUjI7BkCYyPQ3c3bNkCfX1xRyXSNpElBnf/\n7TKvHwQuzH3/HHBmVDGINGR4OEgKk5PB4/CwEoN0FK18FinW3x/0FDKZ4LG/P+6IRNoqylKSSDr1\n9QXlo+HhICmotyAdRolBpJS+PiUE6VgqJYmISIgSg4iIhCgxiIhIiBKDSDtp4ZykgAafRdpFC+ck\nJdRjEGmXUgvnRBJIiUGkXbRwTlJCpSSRdtHCOUkJJQaRdtLCOUkBlZJERCREiUFEREKUGETaTWsZ\nJOE0xiDSTlrL0JyREQ3et4ESg0g76SZAjVNSbZso7/l8jZkdMLPHc18XljnuAjP7iZk9a2ZXRRWP\nNEGlj9bRWobGaYFg20TdY7jB3b9S7k0zywA3Ax8A9gM7zexed98dcVxSK12ltZbWMjQun1Tzf4tK\nqpGJu5R0DvBs7t7PmNm3gYsAJYakUOmj9bSWoTFKqm0TdWK43MyWAqPA59z9laL3TwH2FTzfD7wn\n4pikHrpKkyRRUm2LphKDmT0EnFTirauBW4A1gOcevwp8qonPGgQGAXp6eho9jdRLV2kiHaepxODu\n59VynJl9A/huibcOAPMLnp+ae63UZw0BQwC9vb1eX6TSFF2liXSUKGclnVzw9KPAUyUO2wmcYWan\nm1k3cAlwb1QxiYhIdVGOMawzs7MISknPA58FMLN5wO3ufqG7T5jZSuABIAOsd/enI4xJRESqiCwx\nuPsnyrx+ELiw4PlmYHNUcYiISH20V5JIp9BCRalR3OsYRKQdtFBR6qAeg0gn0HYSUgclBqlOJYho\ntPP3qj2apA4qJUllKkFEo97fa7PbTWuhotRBiUEq015J0ajn99qq5KyFilIjlZKkMpUgolHP71Xj\nA9Jm6jFIZSpBRKOe36s2MpQ2M/f0bTvU29vro6OjcYch0j66paW0gJntcvfeasepxyCSBhofkDbS\nGEMn0vRTEalAPYZOo+mn9VMZR6Cj/g6UGDpN2qeftvsfZ1oSaQc1WrFIy99BiygxdJo0z3CJ4x9n\nGhJphzVasUjD30ELaYyh0+SnSa5Zk74GJI75/GlYx6F1DtFLw99BC6nH0InSOsMljt5OGtZxpLkX\nmBZp+DtoIa1jkHRpVS19utXkp9t/j0Si1nUMkSUGM/sO8Du5p7OAX7v7WSWOex44DEwCE7UErcQg\nTcnX5MfGgtLATTfB4GDcUYlELvYFbu7+XwqC+SrwaoXDB9z9V1HFIhIyPBwkhWw2+FqxAhYt0pW2\nSE7kg89mZsDFwD9E/VkiNenvD3oKedmsBmxFCrRjVtK5wC/d/d/KvO/AQ2a2y8zUn5fo9fUF5aMZ\nM6CrC2bOnN4DtlrpLnVqqpRkZg8BJ5V462p3vyf3/Z9RubfwPnc/YGYnAg+a2Y/d/eESnzUIDAL0\n9PQ0E7ZIMKawaNH0H7DVGgdpQFOJwd3Pq/S+mc0APgYsrnCOA7nHF83sbuAcYEpicPchYAiCwecm\nwhYJFE7bna6zejpsYZa0RtSlpPOAH7v7/lJvmtmbzOyE/PfA+cBTEcckEpa/ql69OnisUnLZu24v\nr2x7peIxr2x7hb3r9rYyysZ02MIsaY2oE8MlFJWRzGyemW3OPX0r8AMzewJ4FPieu98fcUwiYXWu\nHD7h7BPYffHussnhlW2vsPvi3Zxw9gmtj7Ve7VrprnGMaUUL3KR5aS/DNFCHzzf+CzcuZPbA7Kqv\nT2sax0iN2NcxSIdIa6NQnMzq3O5g9sBsFm5cGEoCHZkUQOMY05ASgzQnjY1CuWRWZ9yFyWHe8nkc\nvOVg5yUF0F5N7dSm3rkSgzQnjY1CC5PZ7IHZzFs+jz1r9nDa6tM6LylAx20wF5s29s47KzGkvRae\nRGlsFFqYzF7Z9goHbznIaatP4+AtB5k1MKtzk0OU/+/1b7etvfPOSQxprYWnQVq28S5sXFqQzIrH\nFGYNzOrMMYao6d9uoI29885JDGmshUvrlGpcVq1q+HSlBppLDUhLC+jfbqCNvfPOuYObFvp0luJ5\n9S28y1ml2UeFyaHaIjipkf7tHtXXF1zQRJwYO6fHkMZauDSmVO+ghd3wwzsPV+wR5JPD4Z2Hpx6j\nWnn99G+37TonMUB6auESaLQRLdU7WLWqZY1Lz5XVN3GcPTC7dFKopVau5DGV/u22VWclBkmPZgYc\ny/UOam1comqYa6mVa6BVEkCJQZKp8C5rY2P1DTg2U3qIsmGupZwV90CreiuCEoMk1Zw5QVKA4HHO\nnPp+vtHSQ5QNc3HCgmCAvLARjnPBoHorkqPEIMl06FBwd7VsNng8dOjoe/Vc1dZ7BRx1w5xPWJW2\n5YhroDXu3ookhhKDJFN/f3DLzeIGup6r2kaugKNqmIsTVKVGOK6B1nqSokpO05oSgyRTuQa6nqva\nRq+AW90wRzx9tuYYqjXktSZFlZymPSUGSa5SDXQ9DWpSNviLePpsVfU05LUkRZWcpj0lBkmOSle1\njexz1OqyUKPlk2anzzarsCEfG4Nrrgm+Gv3spCRciYzu4CbJUOmqNgmli2ZjiLMmn489P/23qysY\nv2nm96gxhlSq9Q5uTe2VZGZ/amZPm1nWzHqL3ltlZs+a2U/M7I/K/PxvmtmDZvZvuUftOtapKu1l\n1MJ9jiKJrxZ9fUEjOjzc/vsi53tO5513dKZXs7/HNu3ZI/FodhO9p4CPAQ8XvmhmC4FLgLcDFwB/\nb2aZEj9/FbDF3c8AtuSey3RV6YbxlTZKS8Imas3GkL9qX706eIwjOVxzTdBT0GZ0UkVTYwzu/gyA\nmRW/dRHwbXcfA35uZs8C5wDF/xouAvpz398JDAN/3UxMklDVSjGVxgOSsIlaszEkYcA2Cb9HSYWo\nBp9PAXYUPN+fe63YW939hdz3vwDeGlE8ErdaGsZKg7FJ2EStmRiSMmCbhN+jJF7VxGBmDwEnlXjr\nane/p1WBuLubWdmRcDMbBAYBenqq724pCZOUhrGaqAZVdbUuKVI1Mbj7eQ2c9wAwv+D5qbnXiv3S\nzE529xfM7GTgxQpxDAFDEMxKaiAmiVMaGsaoZz/pal1SIqo7uN0LXGJmM83sdOAM4NEyx12a+/5S\noGU9EEmgpM9kScLsp1aqNNgvUkFTYwxm9lHg68BbgO+Z2ePu/kfu/rSZbQR2AxPACnefzP3M7cCt\n7j4KfAnYaGZ/AewBLm4mHpGmpKXcVYskrP2Q1Gp2VtLdwN1l3vtb4G9LvP7pgu8PAUuaiUGkZdJQ\n7qpVEmZBSWppSwyRQtNlHGA69X6k7ZQYRKaj6dT7kbZTYhApZTrsBTRdej/SdkoMIsU0cCsdLqrp\nqiLp1Y5pq5pKKgmmHoNIsWYGbmspQalHIgmnxCBSrNGB21obfE0llYRTYhAppZGB21obfE0llYRT\nYhBplVobfE0llYRTYhBplXoafE0llQRTYhBpJTX4Mg1ouqqIiIQoMYiISIgSg4iIhCgxiIhIiBKD\niIiEKDGIiEhIU4nBzP7UzJ42s6yZ9Ra8/gEz22VmT+Ye/7DMz19jZgfM7PHc14XNxCMiIs1rdh3D\nU8DHgNuKXv8V8MfuftDM3gE8AJxS5hw3uPtXmoxDRERapNl7Pj8DYGbFr/9rwdOngePMbKa7jzXz\neSIiEr12jDF8HHisQlK43Mx+ZGbrzWx2G+IREZEKqiYGM3vIzJ4q8XVRDT/7duB64LNlDrkF+C3g\nLOAF4KsVzjVoZqNmNvrSSy9V+2gREWlQ1VKSu5/XyInN7FTgbmCpu/+szLl/WXD8N4DvVohjCBgC\n6O3t9UZiEhGR6iIpJZnZLOB7wFXu/n8rHHdywdOPEgxmi4hIjJqdrvpRM9sP9AHfM7MHcm+tBH4b\n+J8FU1FPzP3M7QVTW9flprT+CBgA/qqZeCQldL9jkUQz9/RVZXp7e310dDTuMKQRut+xSGzMbJe7\n91Y7Tiufpb1K3f5SRBJFiUHaK3/7y0xG9zsWSSjdwU3aS/c7Fkk8JQZpP93+UiTRVEoSEZEQJQYR\nEQlRYhARkRAlBhERCVFiEBGRECUGEREJSeWWGGb2ErAn7jiqmEtwJ7s0SFOskK540xQrpCveNMUK\nyYj3NHd/S7WDUpkY0sDMRmvZkyQJ0hQrpCveNMUK6Yo3TbFCuuJVKUlEREKUGEREJESJITpDcQdQ\nhzTFCumKN02xQrriTVOskKJ4NcYgIiIh6jGIiEiIEkPEzOxyM/uxmT1tZuvijqcaM/ucmbmZzY07\nlkrM7Mu53+uPzOzu3H3GE8XMLjCzn5jZs2Z2VdzxlGNm881sm5ntzv2d/mXcMVVjZhkz+1cz+27c\nsVRjZrPM7B9zf6/PmFnitxZWYoiQmQ0AFwFnuvvbga/EHFJFZjYfOB/YG3csNXgQeIe7vxP4KbAq\n5nhCzCwD3Ax8EFgI/JmZLYw3qrImgM+5+0Lg94AVCY417y+BZ+IOokZfA+53998FziQFcSsxRGs5\n8CV3HwP5Hma6AAAChklEQVRw9xdjjqeaG4ArgcQPPLn79919Ivd0B3BqnPGUcA7wrLs/5+7jwLcJ\nLhISx91fcPfHct8fJmi4Tok3qvLM7FTgQ8DtccdSjZm9GXg/cAeAu4+7+6/jjao6JYZovQ0418x+\naGb/YmZnxx1QOWZ2EXDA3Z+IO5YGfAr457iDKHIKsK/g+X4S3NjmmdkC4F3AD+ONpKIbCS5gsnEH\nUoPTgZeAb+ZKX7eb2ZviDqoa3cGtSWb2EHBSibeuJvj9/iZB9/xsYKOZ/ZbHNBWsSqyfJygjJUal\neN39ntwxVxOUQr7VztimIzP7DWATcIW7/3vc8ZRiZh8GXnT3XWbWH3c8NZgBvBu43N1/aGZfA64C\nVscbVmVKDE1y9/PKvWdmy4G7congUTPLEuyX8lK74itULlYzW0RwZfOEmUFQlnnMzM5x91+0McSQ\nSr9bADP7JPBhYElcybaCA8D8guen5l5LJDM7hiApfMvd74o7ngreC3zEzC4EjgX+k5n9b3f/85jj\nKmc/sN/d8z2wfyRIDImmUlK0/gkYADCztwHdxL+J1hTu/qS7n+juC9x9AcEf87vjTArVmNkFBOWE\nj7j7a3HHU8JO4AwzO93MuoFLgHtjjqkkC64G7gCecfe/izueStx9lbufmvs7vQTYmuCkQO7f0D4z\n+53cS0uA3TGGVBP1GKK1HlhvZk8B48ClCbyyTaubgJnAg7lezg53XxZvSEe5+4SZrQQeADLAend/\nOuawynkv8AngSTN7PPfa5919c4wxTSeXA9/KXSA8B1wWczxVaeWziIiEqJQkIiIhSgwiIhKixCAi\nIiFKDCIiEqLEICIiIUoMIiISosQgIiIhSgwiIhLy/wEVcTkp6pncgAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116573c50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ks, mus, sigmas, pi, ll = GMM(Xvs, 3, 1000, init='kmeans')\n",
    "\n",
    "classes = np.argmax(ks, axis=1)\n",
    "\n",
    "kscatter_plot(Xvs, classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr/>\n",
    "\n",
    "# Dimensionality Reduction\n",
    "<hr/>\n",
    "\n",
    "# Principal Component Analysis (PCA)\n",
    "\n",
    "Principal Component Analysis (PCA) is one of the dimensionality reduction tools. \n",
    "PCA is also used for feature extraction, data compression, and data visualization. \n",
    "It is also known as Karhunen-Loeve (KL) transform.\n",
    "\n",
    "PCA reduces the dimension of the data by finding the data samples that vary the most. \n",
    "For this reason, we first look at the variance of the data. \n",
    "\n",
    "$$\n",
    "var(\\zv) = \\frac{1}{N} \\sum_{n=1}^{N} \\zv_n^2 \n",
    "$$\n",
    "\n",
    "From the variance, PCA finds the orthogonal projection of the data onto the *principal subspace*, which is a lower dimensional linear space. \n",
    "Now, let us look for a direction $\\vv$ that maximazes the variances.\n",
    "Here, $\\vv$ is a unit vector, so the dot product represents a projection onto it. \n",
    "The projection of data $\\xv_n$ onto $\\vv$ is \n",
    "\n",
    "$$\n",
    "\\zv = \\xv_n^\\top \\vv.\n",
    "$$\n",
    "\n",
    "The variance of this projected data $\\zv$ is \n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "var(\\zv) &= \\frac{1}{N} \\sum_{n=1}^{N} \\vv^\\top \\xv_n \\xv_n^\\top \\vv \\\\\n",
    "         &= \\vv^\\top \\Big( \\frac{1}{N} \\sum_{n=1}^{N} \\xv_n \\xv_n^\\top \\Big) \\vv \\\\\n",
    "         &= \\vv^\\top \\Sigmav \\vv\n",
    "\\end{align*}\n",
    "$$\n",
    "\n",
    "where $\\Sigmav$ is a covariance matrix of $\\xv_n$.\n",
    "\n",
    "## Optimization Problem\n",
    "\n",
    "$$\n",
    "\\begin{equation*}\n",
    "\\begin{aligned}\n",
    "& \\underset{\\vv}{\\text{maximize}}\n",
    "& & \\vv^\\top \\Sigmav \\vv \\\\\n",
    "& \\text{subject to}\n",
    "& & \\vv^\\top \\vv = 1.\n",
    "\\end{aligned}\n",
    "\\end{equation*}$$\n",
    "\n",
    "Using a Lagrange multiplier that is denoted by $\\lambda$, we can make an unconstrained maximization of\n",
    "\n",
    "$$\n",
    "q(\\vv) = \\vv^\\top \\Sigmav \\vv + \\lambda ( 1 - \\vv^\\top \\vv). \n",
    "$$\n",
    "\n",
    "As usual, setting the derivative w.r.t. $\\vv$, we can get\n",
    "\n",
    "$$\n",
    "0 = 2 \\Sigmav \\vv - 2 \\lambda \\vv, \n",
    "$$\n",
    "\n",
    "$$\n",
    "\\lambda \\vv = \\Sigmav \\vv.\n",
    "$$\n",
    "\n",
    "We can see that the direction vector $\\vv$ is an eigenvector!\n",
    "\n",
    "Also, since $\\vv$ is a unit vector, we can apply $\\vv^\\top \\vv = 1$ by left-multiplying $\\vv^\\top$,\n",
    "\n",
    "$$\n",
    "\\lambda = \\vv^\\top \\Sigmav \\vv.\n",
    "$$\n",
    "\n",
    "So, we can obtain the maximum variance when $\\vv$ is the eigenvector having the largest eigenvalue $\\lambda$. \n",
    "This eigenvector is called as the *first principal component*. \n",
    "\n",
    "Other principal components, or other directions that are orthogonal to the first principal component, are found by the eigendecomposition of $\\Sigmav$, or the singular value decomposition of data sample matrix $\\Xm$ <u>**with zero means**</u>. \n",
    "\n",
    "$$\n",
    "\\Xm = \\Um \\Lambdav^{\\frac{1}{2}} \\Vm^\\top,\n",
    "$$\n",
    "\n",
    "where the $\\Lambdav$ is a diagonal matrix with eigenvalue elements. \n",
    "\n",
    "\n",
    "For implementation, we need to keep track of shapes of each matrix. \n",
    "\n",
    "- $\\Xm$: N x D\n",
    "\n",
    "- $\\Um$: N x D\n",
    "\n",
    "- $\\Lambdav$: D x D\n",
    "\n",
    "- $\\Vm$: D x D\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# PCA \n",
    "\n",
    "def pca(X):\n",
    "    \"\"\" principal component analysis\n",
    "    \n",
    "        parameter\n",
    "        ---------\n",
    "        X        ndarray (N x D)\n",
    "                 input data\n",
    "                 \n",
    "        return\n",
    "        ------\n",
    "        @U       ndarray (N x D)\n",
    "                 left singular vectors\n",
    "        @S       ndarray (D x D)\n",
    "                 diagonal matrix with square root of eigenvalues\n",
    "        @V       ndarray (D x D)\n",
    "                 right singular vectors\n",
    "        @mu      ndarray (D,)\n",
    "                 1d vector of column means \n",
    "    \"\"\"\n",
    "    \n",
    "    ### TODO: implement PCA using np.linalg.svd\n",
    "\n",
    "    \n",
    "    return U, S, V.T, means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-8, 8)"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAFpCAYAAACI3gMrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2MZFd55/HfM9Xd2D3jF6Y9iJXH7nG0IcghweDGwmGT\nhdhiiYOwtIqQo56RDdGOpkmcgSCxQGs3u39MNhvQhpESTEZkHHC3gozjZKPICa9JpEXgpG0M2B6S\nWGRmbAOi3QiMPV7GM/PsH6fudnX1fau6t+pWnfp+pJK7qqtunW7bv7r9nHOea+4uAEA8djQ9AABA\nvQh2AIgMwQ4AkSHYASAyBDsARIZgB4DI1BLsZvYeM3vMzB41sz81s4vqOC4AoHeVg93MrpT0m5IW\n3P1VklqSbqt6XABAf+oqxUxJutjMpiTNSvp2TccFAPSocrC7+9OSPizptKTvSPqhu3+26nEBAP2Z\nqnoAM3uppFslXSPpB5I+bWb73X2l63kHJR2UpJ07d17/yle+supbA8BEeeihh55x9z1Fz6sc7JJu\nlvSv7r4uSWZ2v6Sfk7Ql2N39mKRjkrSwsOBra2s1vDUATA4zO1XmeXXU2E9Ler2ZzZqZSbpJ0oka\njgsA6EMdNfYHJd0n6WFJ32gf81jV4wIA+lNHKUbu/tuSfruOYwEAqmHnKQBEhmAHgMgQ7AAQGYId\nACJDsANAZAh2AIgMwQ4AkSHYASAyBDsARIZgB4DIEOwAEBmCHQAiQ7ADQGQIdgCIDMEOAJEh2AEg\nMgQ7AESGYAeAyBDsABAZgh0AIkOwA0BkCHYAiAzBDgCRIdgBIDIEOwBEhmAHgMgQ7AAQGYIdACJD\nsANAZAh2AIgMwQ4AkSHYASAytQS7mV1uZveZ2TfN7ISZ3VjHcQEAvZuq6ThHJf2Nu/+Kmc1Imq3p\nuACAHlUOdjO7TNIvSLpDktz9rKSzVY8LAOhPHaWYayStS7rbzL5qZh83s53dTzKzg2a2ZmZr6+vr\nNbwtACBNHcE+Jem1ku5y99dIel7S+7uf5O7H3H3B3Rf27NlTw9sCANLUEexPSXrK3R9s379PIegB\nAA2oHOzu/l1JT5rZT7UfuknS41WPCwDoT12rYu6UtNpeEfMtSe+o6bgAgB7VEuzu/oikhTqOBQCo\nhp2nABAZgh0AIkOwA0BkCHYAiAzBDgCRIdgBIDIEOwBEhmAHgMgQ7AAQGYIdACJDsANAZAh2AIgM\nwQ4AkSHYASAyBDsARIZgB4DIEOwAEBmCHQAiQ7ADQGQIdgCIDMEOAJEh2AEgMgQ7AESGYAeAyBDs\nABAZgh0AIkOwA0BkCHYAiAzBDgCRIdgBIDIEOwBEhmAHgMjUFuxm1jKzr5rZX9V1TABA7+o8Yz8s\n6USNxwMA9KGWYDezvZJ+WdLH6zgeAKB/dZ2xf0TS+yRdyHqCmR00szUzW1tfX6/pbQEA3SoHu5m9\nVdL33P2hvOe5+zF3X3D3hT179lR9WwBAhjrO2N8g6W1mdlLSpyT9opmt1HBcAEAfKge7u3/A3fe6\n+z5Jt0n6orvvrzwyAEBfWMcOAJGZqvNg7v53kv6uzmMCAHrDGTsARIZgB4DIEOwAEBmCHQAiQ7AD\nQGQIdgCIDMEOAJEh2AEgMgQ7AESGYAeAyBDsGHurq9K+fdKOHeGfq6tNjwhoVq29YoBhW12VDh6U\nzpwJ90+dCvclaXGxuXEBTeKMHWNteXkz1BNnzoTHgUlFsGOsnT7d2+PAJCDYMdauvrq3x4FJQLBj\nLCUTpqdOSWZbvzc7Kx050siwgJFAsGPsJBOmp06F++6b4T4/Lx07xsQpJhurYjB20iZM3UOonzzZ\nyJCAkcIZO8YOE6ZAPoIdY4cJUyAfwY6xc+RImCDtxIQpsIlgx9hZXAwTpPPzYdKUCVNgKyZPMZYW\nFwlyIAtn7AAQGYIdACJDsGNs0J4XKIcaO8YC7XmB8jhjx1igPS9QHsGOsTDo3aZ1lHkoFWFUUIrB\nWLj66s2mX92PV1VHmYdSEUYJZ+wYqn7Page527SOMk+ZY3BGj2Eh2DE0ne123TfPassE3OKidPvt\nUqsV7rda4X4dZ8NVyjydfeHzjlHlZwd6VTnYzewqM/tbM3vczB4zs8N1DAzxKTqrzTujXV2VPvEJ\n6fz5cP/8+XC/jmDst6nY6qr0jndkh3rnMZj8xVC5e6WbpH8j6bXtry+R9M+Srs17zfXXX++YPGbu\n4Xx1683MfWXFfXZ26+Ozs+Fxd/f5+fTXzs9XH1fRe2eZm0sfU/fPVfSzA2VJWvMSuVz5jN3dv+Pu\nD7e//pGkE5KurHpcxCfvzLjojHaQq2L6bSq2sZH/fffNY9BqGMNUa43dzPZJeo2kB1O+d9DM1sxs\nbX19vc63xZjImwAtCu6qwVg0cbm4GK6+dOFC+Gcdtfv5+c2vaTWMoSpzWl/mJmmXpIck/cei51KK\nmVwrK6F8Yhb+WbbU0m+5pJ/XZo2xW14pJu34Wcct+36ASpZi6gr1aUmfkfRbZZ5PsKNbmfDtNwB7\nqc/38iGwsuI+M7P9uHNz5cdW5QMLk2dowS7JJH1S0kfKvoZgR5okuCX3VmszfKuGXC8Tl1kfAnNz\n5c62l5Z6+/AZ5KQw4jPMYP93klzS1yU90r7dkvcagn1y9HqWPYgz2F7CM+tDoGyppdexs1oGvRhq\nKabXG8E+GfoJukGcwfZS5ikT6lljKjP27g+6rDo9Z+xIQ7Cjcf2E9KDOYPP+ckgL/jK37jEVjT3t\nfaant9fpqbEjS9lgp6UABqafteeDWu/9pS+F93UPO0UPHdpc8pi2hr6M7jEVjT3tfV58Mdzm5sL9\nVmtz/T7tBtAvgh0D009IHzkiTU9vfWx6ur/13snadTPprrtCqCeee066447wnH42OaWtQS9aq571\nPu7Ss89KMzObLRPoJYMqCHYMTL+bcsy23//Sl3rrjNjZdCvLuXPhzHj37vxjdcvamVq0gzXvA+3F\nF6WzZ7c+Ri8Z9K1MvabuGzX2ydHrqpisunx3/brfSdi02/T09semprY/3sumo6zfRa+1fFbHoJOo\nsWMU9LpVP69c0anobLZseaXVCmfL3S67TLr77s3atyRdfPHW55RpxdvZymB5WbrxxnLjSpSZW6DP\nO7Ypk/513zhjR5ZezrTzzmbLHGdqKv/YWcskk01IRcsgezlD73d1DDtXJ4s4Y8c4SqvLd9fcE0WT\nsN3H6bRrl/Qnf7K1UVf3sbM6Tn7sY/m1++SvhbKrbVqt8NfB8eO9d5ikzzvSEOwYuF5KBWkTkIcO\n9T4Jm3acpaXN+0mJpZ+Ok91loW7JB07ZctCFC2G8/XSYHPRFvjGmypzW132jFDM56ioVFE1Slvl+\n2uagZOdnWm+aXneidv9sZV8/N9fjL7UDvWYmi9h5ilEwjOBJC+1kFU0S1GVCNgnlzud3r8bJ6yWT\ntqN10MFOjX2ylA12C88droWFBV9bWxv6+2L4duxIL12YhZJDHfIuJt2ruTnphRe21q3Nws8wPy/d\ncku41mrn92dns+vhV1xRfKWlqr+L1dVQUz99OpSBjhyp50IhGD1m9pC7LxQ9jxo7BmoYl4Srs568\nsbF9MjIJ9ZMnpY9+tLfL6B09mj+JK1X/XQzi6k8YbwQ7Bqqf3ae9TLaurobnDVry4bG6Kh0+vLl2\n/bnn8l/XOYkrpa/wee451p6jZmXqNXXfqLFPlqq7M/OuYNRPV8Z+bsm4066YtGNHb1dMSmvVS10c\nZYjJU4yyfq992qmfVSv93JLQzXu/zgnQtJ+t87FkBQ4rWdCrssE+1fRfDJg8yVb8pJadbMWXeluX\nPai12ska92TSM2klkDdBu7ERSkK7d4dOjUmbglOnpHe+M0R38ljSwbEba89RF2rsGLq83ZK9TLbW\nOQHb6brrpO9/f/P+xkYI5yLu4bndvWfOnk3vR9NtUD8PJg/BjqHLOyvvZbK1qG1AP3bulL7whe1L\nNLtb6tatTDtjoCyCHUOXd1Ze1NM8rVtiXatiZmbqOU5ZrVZvfWGAsgh2DF3RWXmyLvuee8L9AwdC\nmL/rXdvb5H7hC/VtdLrkEun55+s5VhGzsNGJtecYBIIdQ1d0Vi6l9zq/667+rk1aVtEO0SxZ3Sfz\nuHNdUwwOLQUwkupsE1BWqyVdfnl2wO/aFc7oO/+XmZ2Vbr89fDClrXZJ2hFkyWtHAHSjpQDGTmf9\nfNihLoVgzjtrf/75UB5KdpG2WuEviAceCH9ddF+EO3lO2uOJYfZO50pLk4Ngx0joLr2MomRyN5kj\nSM7QT50K9fKXvGT7a86dky69NPuCHsnrBx22ZS7jh3hQisFIKNMFMc3OneG1nWf4rZb0xjdKTzxR\n35l/UjKRQukla5NRmqR7Y5nyUlLaeeCBers1Zr130twM44FSDMbG6mp+qCdXPOouaczOSn/0R9tX\n2Zw/L335y+HxzotR9yspuRw+HDYq9RLq0ubyziNH8ssy0tZL79V5Zs2VliYLwY7G5dWY5+fD2e4z\nz4TrgqatpMnaybp/f/8rXaTwodBZctnY6H2jklno4b5vXxhPmR2o3X9E11GHH0b7ZIwOgh1DkzV5\nl3fW2LkbM6vv+CDOOpPNSlWXV7qH+nvVklDV1/fTPhnji2BH7dICPG/yLuuscW4uu7bc+R6D6Md+\nySVb+8X0a8eOetbet1rVXl9m7wAiUqYFZN032vbGK6ufeloPcim0sF1aCheWTmuF23Qf9mHeitoQ\nl+1pj3ipZNveWs51zOwtZvZPZvaEmb2/jmNiPGXVu7Nq3efPhx2labXnjY3tE4erq2HVyCB3oDbB\nLJSXspZFmrFUEeVVDnYza0n6Q0m/JOlaSb9qZtdWPS7GU9317s6Jw6Sc0+uqlHHQuXKmuxaetnt1\nmBubMH7qOGO/QdIT7v4tdz8r6VOSbq3huBhDefXyflvsJh8WaX8NjAqz0HIgzdxc8bLLW24J/0yr\nhWdtNWGpIrLUEexXSnqy4/5T7ccwgbJWXxw9GgKrn4nO5MNiVIOs1ZIOHQrrz9N+9re/XXrhhfxj\nPPDA5oTwgQPhsXvuyS/PsFQRWYa2KsbMDprZmpmtra+vD+ttMWR5qy8WF6WXvrT3Y546FcKz6ibp\nVktaWaln01Kn8+fDkkZp688+Nxcuq1emK2VyCb20Onrah+XMjPTcc/R9QbrKLQXM7EZJ/83d/0P7\n/gckyd3/R9ZraCkwuXbsaLYXzPz8YBuMzc+Hssq991bbHNV5vJMnQ3AvL4e/WrqvqyrRJXJSDLOl\nwD9K+kkzu8bMZiTdJukvazguxlzaevamyweD7hqZ9I2vI9SlzfJT5+asXbu2ryJKm0ylm+Pkqhzs\n7n5O0m9I+oykE5LudffHqh4X4y1rQ9Itt/R3YYpJlfZBWKbvC90cJxvdHTEQed0EB3nWPDdX39ly\n07LKK2U6NdLNMU50d0Sj8s4q83qTl5W2xT5ZfRODVitsxFpe3l5KKdP3hW6Ok41gx0Bk1dJ37w6r\nObqVLc8kq026l02ahSBcXMxeTz5Ozp/Pbt9bpu8L3RwnG8GOgchaovfss9tLJb2slMmaPHQPa8Gl\n9CsZjaO83aZZnS4TdHOcbAQ7BiLtrPKSS9J7wly4UO6MfceOcMZaVGbotStjsr69jhJRP3qZTO6l\nlHLxxZtfz82xHHKSEOwYmO6zyrzALXPGfuFCKEfs3p3+/aTM0Eu5wSwcs/NapsPmXn7TVJmfLVkR\n0/mXUdHOV8SFYMfQlAmlopYDyQ7OvDJDLwHtHmrZZuHSd00txXzhhXLhnldKSdat79+f3mGTpmGT\ng2DH0BQF7vy89MlPFofyxkaYKM2aPOwsA5WR/LWwsSE9/3y51/Rjair7e1kfWJ2KLjySrFvPwoqY\nCVKmaXvdNy60MblWVtIvujE7u3nxiJWV4otOdD6/6P3MRuMiGisr4Vb03B07ev95i35fyRgw3lTy\nQhsEOxqRhHfeFYGKrpSUvK7oOE2Hequ1dVxlQnh6OnwAlr1iUtGHV9kPQoy2ssFOKQaNKFqul+hc\n2dEtWdtdtG2+qdUuifPnt44r6b2e58UXw7LOot9PIm/+guubTh6CHSMpbWVHt1ar3CThKKzdPnMm\nTGru2xc6P5bRS008a936ykq5DwbEhWBH41ZXpSuuCBOhZuHrw4fze5jPzmZfIu/06a2dDZeXq13d\nqIxkNU/RqppTp8r3sull2WaZ3aiYHDQBQ6NWV8MFJs6eLf+a+flwhrq8nL4KxEyant56zOnpUNbo\n/DCYnpbuvjt8ffjwZuDu2BGe2yT6qyNN2SZgOQuwgMFbXu491Du7Ex48uP3M3n37MdN2vL74ovQ7\nvyOdOLF1g9RFF4XllJ/4xPCusTozE3bmfv/74Uz9yBFCHf2jFIOB677gw7vetXm/lxa+3b1OkvJD\nWqfHsh5/PL0ny7339rYWvh9J2WZ+Xjp+XHrmmfKTpUAeztgxUMkkaHLmm1xhqFdJ+aU78BYXNy/+\nXKekLHPyZHZv86rcQ32f/uioG2fsGKjl5erljKT8knYWu7pa3IagX8nqmkHu2NzY4KpGqB/BjoGq\nGop5rWaTvwbSVsdMT1fvy56MvY4e5nmrZejhgroR7BiosqE4Px9KE0n73DJL9rL+Gmi1wmqXqssY\nk+WXVcsw8/PSoUPZ36eHC+pGsGOgymwO6jwrL7sjVcoO3OQMvigw5+akpaXsD4CsJY87d4ZVLGUk\nP9tHP5r9PlzVCHUj2DFQi4v5Z879bqRZXc0vb+T1bU/s2hUC9+jRrWMsqtm7S7/2a+XG2bkT9ujR\n8bmqUfdKJuYBxkyZhjJ132gCNlnSmnlVbUpVppHW3Fx+EzGz4kZjeQ3IWq1yzzXb+rtIxp68vkyT\nr2EaxL8v1EN0d8QoKdOFsZfjlA3UlZXsAJ6fL3+stGP38iHQ/TOMcnBm/U5o+9u8ssFOSwGMje41\n8UWSZZJpr0u27B84sH2DUtljS8UTq2mtAbLWxXfvqm1K1sXFzZpvtTDpyrYUoMaOsdHrmvikPW5e\ng6yiOnyapC6e1lFxZibU67vfp7NmnfVhMCqrY7Imc5nkHSNlTuvrvlGKQZq8ck2ZKw+lXayiqLyR\ndjWn7tJD3riWlrZe8Wjnzu3vWbaOPyqljlEvFU0yUWPHOMkLk6JgzJvELArLvFp5UZCtrIQPj+7X\nzcz0fsWkUQvOuuZEUK+ywU6NHSMhr+4sZZcvZmfzyzNFdeGs952bC025Vlc32wO3WmGNfFHbYCk8\n98KFUL7Iq8Ob0c0R5ZWtsRPsGAl5E3ZS9gTnykp+wBZNSOZNrErZk7VFHyidzNLHPyqTpRgfTJ5i\npHVvgMmaxLz66uxJu/n5cJZ75Ej6TtDp6eLNP3kTq3mTtWfOlG8X7L59M9WobkxCHAh2DF1yltx5\nEeof/SgEcae81SfdbQiOH9+6e3RuLvSLKVPeSNoY3HNPuH/gQLlWvefPbx9zFncuW4chKlOIr/vG\n5OlkyJqAy5pMnJvLXxXTz2Te0tLm5GqrFe5njbV7grZoE1Iyjp07iydHk+cyIYkqNIxVMZI+JOmb\nkr4u6c8lXV7mdQR7/PJWuWQFZufW+zosLaW/T1q497oDteyKneS5S0ssIUR1ZYO90uSpmb1Z0hfd\n/ZyZ/c/2XwD/ueh1TJ7Gr59VLnVPJk5Npfdqb7Wkc+e2PpY1eSttTn52r4pZXMz+OTsviJ2UiJKr\nMnViAhW9GMrkqbt/1t2T/0W+ImlvleMhHlm7KE+fLq6Z1yUt1JPHuzsW5u2q9HZ9/Ny58HVnO+Gs\nn7NzieXGRnqo570eqKLOydN3SvrrrG+a2UEzWzOztfX19RrfFqMob1t63kqUOuWtWjl1KkzgJuGe\n9mHTKSuAq26zZ5s+BqEw2M3s82b2aMrt1o7nLEs6Jymza7O7H3P3BXdf2LNnTz2jx8gqs5Kl7AU1\n+nXwYP73O3ulJx82WR8GWQFc9IGQhyWPGJgyhfi8m6Q7JH1Z0mzZ1zB5OhlGYRVI56qYMhO2/fRJ\n6f45s/rP5K36AcrQkFbFvEXS45L29PI6gh39qPJB0UuP8aofSDTRwqCUDfaqNfY/kHSJpM+Z2SNm\n9rGKxwNSpW1q6qyRF+llwraOMtHFF29+PTfHhiQMV9VVMf/W3a9y9+vat5xrsQP9S9ve31kjL1LH\nhG2Z64AmH0Cdq2BeeKH8ewB1oAkYxkLTV/XJaxY2TldHwnijCRii0tRVfZKz9P37y/3FkLd+HxgW\ngh1jYVibmjp11vWzdAc2l5XDKCDYMRaGtampU5lrrHYHdhMfQEC3qaYHAJS1uDjclSVF5ZO0wE7G\nt7wcXs/VkdAEgh3IkHdZu85GYN2G/QEEdKMUA2TIKqusrAyuDQJQB4IdyNBEXR+oA6UYIAdlFYwj\nztgBIDIEOwBEhmAHgMgQ7AAQGYIdACJDsANAZAh2AIgMwQ4AkSHYASAyBDsARIZgB4DIEOwAEBmC\nHQAiQ7ADQGQIdgCIDMEOAJEh2AEgMgQ7AESGYAeAyBDsABAZgh0AIkOwA0BkCHYAiEwtwW5m7zUz\nN7Mr6jgeAKB/lYPdzK6S9GZJp6sPBwBQVR1n7L8v6X2SvIZjAQAqqhTsZnarpKfd/WslnnvQzNbM\nbG19fb3K2wIAckwVPcHMPi/p5SnfWpb0QYUyTCF3PybpmCQtLCxwdg8AA1IY7O5+c9rjZvYzkq6R\n9DUzk6S9kh42sxvc/bu1jhIAUFphsGdx929Ielly38xOSlpw92dqGBcAoE+sYweAyPR9xt7N3ffV\ndSwAQP84YweAyBDsABAZgh0AIkOwA0BkCHYAiAzBDgCRIdgBIDIEOwBEhmAHgMgQ7AAQGYIdACJD\nsANAZAh2AIgMwQ4AkSHYASAyBDsARIZgB4DIEOwAEBmCHQAiQ7ADQGQIdgCIDMEOAJEh2AEgMgQ7\nAESGYAeAyBDsABAZgh0AIkOwA0BkCHYAiAzBDgCRIdgBIDIEOwBEpnKwm9mdZvZNM3vMzH6vjkEB\nAPo3VeXFZvYmSbdKerW7/9jMXlbPsAAA/ap6xr4k6Xfd/ceS5O7fqz4kAEAVVYP9FZJ+3sweNLO/\nN7PXZT3RzA6a2ZqZra2vr1d8WwBAlsJSjJl9XtLLU7613H79bkmvl/Q6Sfea2U+4u3c/2d2PSTom\nSQsLC9u+DwCoR2Gwu/vNWd8zsyVJ97eD/B/M7IKkKyRxSg4ADalaivkLSW+SJDN7haQZSc9UHRQA\noH+VVsVIOi7puJk9KumspNvTyjAAgOGpFOzuflbS/prGAgCoATtPASAyBDsARIZgB4DIEOwAEBmC\nHQAiQ7ADQGQIdgCIDMEOAJEh2AEgMgQ7AESGYAeAyBDsABAZgh0AIkOwA0BkCHYAiAzBDgCRIdgB\nIDIEOwBEhmAHgMgQ7AAQGYIdACJDsANAZAh2AIgMwQ4AkSHYASAyBDsARIZgB4DIEOwAEBmCHQAi\nQ7ADQGQIdgCITKVgN7PrzOwrZvaIma2Z2Q11DQwA0J+qZ+y/J+m/u/t1kv5r+z4AoEFVg90lXdr+\n+jJJ3654PABARVMVX/9uSZ8xsw8rfEj8XPUhAQCqKAx2M/u8pJenfGtZ0k2S3uPuf2Zmb5f0x5Ju\nzjjOQUkH23d/bGaP9jfkobpC0jNND6IExlmfcRijxDjrNi7j/KkyTzJ37/sdzOyHki53dzczk/RD\nd7+0xOvW3H2h7zceEsZZr3EY5ziMUWKcdYttnFVr7N+W9O/bX/+ipH+peDwAQEVVa+z/SdJRM5uS\n9H+1WWoBADSkUrC7+/+RdH0fLz1W5X2HiHHWaxzGOQ5jlBhn3aIaZ6UaOwBg9NBSAAAi01iwj1M7\nAjO708y+aWaPmdnI7q41s/eamZvZFU2PJY2Zfaj9e/y6mf25mV3e9Jg6mdlbzOyfzOwJM3t/0+NJ\nY2ZXmdnfmtnj7f8eDzc9pixm1jKzr5rZXzU9lixmdrmZ3df+7/KEmd3Y9JjSmNl72v++HzWzPzWz\ni/Ke3+QZ+1i0IzCzN0m6VdKr3f2nJX244SGlMrOrJL1Z0ummx5Ljc5Je5e4/K+mfJX2g4fH8f2bW\nkvSHkn5J0rWSftXMrm12VKnOSXqvu18r6fWSfn1ExylJhyWdaHoQBY5K+ht3f6WkV2sEx2tmV0r6\nTUkL7v4qSS1Jt+W9pslgH5d2BEuSftfdfyxJ7v69hseT5fclvU/h9zqS3P2z7n6uffcrkvY2OZ4u\nN0h6wt2/5e5nJX1K4QN9pLj7d9z94fbXP1IIoiubHdV2ZrZX0i9L+njTY8liZpdJ+gWFjZVy97Pu\n/oNmR5VpStLF7RWIsyrIyyaD/d2SPmRmTyqcBY/M2VuXV0j6eTN70Mz+3sxe1/SAupnZrZKedvev\nNT2WHrxT0l83PYgOV0p6suP+UxrBwOxkZvskvUbSg82OJNVHFE40LjQ9kBzXSFqXdHe7ZPRxM9vZ\n9KC6ufvTChl5WtJ3FDaCfjbvNVXXseeqqx3BoBWMc0rSboU/e18n6V4z+wkf8nKigjF+UKEM07i8\ncbr7/24/Z1mhpLA6zLHFxMx2SfozSe9292ebHk8nM3urpO+5+0Nm9samx5NjStJrJd3p7g+a2VFJ\n75f0X5od1lZm9lKFvx6vkfQDSZ82s/3uvpL1moEGu7tnBrWZfVKhBidJn1aDf7IVjHNJ0v3tIP8H\nM7ug0FdifVjjk7LHaGY/o/Av/Guhq4P2SnrYzG5w9+8OcYiS8n+XkmRmd0h6q6Sbhv3hWOBpSVd1\n3N/bfmzkmNm0Qqivuvv9TY8nxRskvc3MbpF0kaRLzWzF3fc3PK5uT0l6yt2Tv3juUwj2UXOzpH91\n93VJMrP7FRouZgZ7k6WYcWlH8BeS3iRJZvYKSTMaoWZB7v4Nd3+Zu+9z930K/7G+tolQL2Jmb1H4\n8/xt7n6m6fF0+UdJP2lm15jZjMLk1F82PKZt2j2Z/ljSCXf/X02PJ427f8Dd97b/e7xN0hdHMNTV\n/n/kSTNLGmvdJOnxBoeU5bSk15vZbPvf/00qmOQd6Bl7gXFpR3Bc0vF2N8qzkm4fsTPNcfIHkl4i\n6XPtvy5ge8LSAAAAe0lEQVS+4u6Hmh1S4O7nzOw3JH1GYdXBcXd/rOFhpXmDpAOSvmFmj7Qf+6C7\nP9DgmMbZnZJW2x/m35L0jobHs027THSfpIcVSphfVcEOVHaeAkBk2HkKAJEh2AEgMgQ7AESGYAeA\nyBDsABAZgh0AIkOwA0BkCHYAiMz/A7pUTZJOv7R9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111055b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mu1 = np.array([0, 0])\n",
    "S1 = np.eye(2)\n",
    "\n",
    "X1 = np.random.multivariate_normal(mu1, S1, 300)\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "plt.plot(X1[:, 0], X1[:, 1], 'ob')\n",
    "plt.xlim([-8, 8])\n",
    "plt.ylim([-8, 8])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-8, 8)"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAFpCAYAAACI3gMrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2MnWd55/HfNWMP8dgJ1MdmkZJ4nK4KKEtLIQOCsu2W\nhlasi8g/K8Tq2DVJtVadlhrECgGj3Wr/sFQV1HakbYIsSORmRkUhpF2E3PLW3WpbLSkTXgokaRfR\n2CSAmEwEqbETv137xz3Pzpkzz8v9vJy3Z74faRTPmXOeczuQ33nmuu/7us3dBQBoj6lRDwAA0CyC\nHQBahmAHgJYh2AGgZQh2AGgZgh0AWqaRYDez95rZt8zsm2b2Z2Z2XRPXBQCUVzvYzexGSb8rad7d\nXyVpWtI7614XAFBNU6WYHZJ2mdkOSbOSvtfQdQEAJdUOdnd/WtJHJJ2T9H1JP3b3z9W9LgCgmh11\nL2BmPyXpDkm3SPqRpE+a2WF3X+p73jFJxyRp9+7dt73yla+s+9YAsK08+uijz7j7/qLn1Q52SW+R\n9M/uvipJZvawpF+QtCnY3f2UpFOSND8/7ysrKw28NQBsH2Z2NuZ5TdTYz0l6g5nNmplJul3S4w1c\nFwBQQRM19kckPSTpK5K+sX7NU3WvCwCopolSjNz99yT9XhPXAgDUw85TAGgZgh0AWoZgB4CWIdgB\noGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaBmCHQBahmAHgJYh2AGgZQh2AGgZgh0AWoZgB4CWIdgB\noGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaBmCHQBahmAHgJYh2AGgZQh2AGgZgh0AWoZgB4CWIdgB\noGUIdgBoGYIdAFqGYAeAliHYAaBlGgl2M3uJmT1kZk+Y2eNm9sYmrgsAKG9HQ9dZlPRX7v4fzGxG\n0mxD1wUAlFQ72M3sxZJ+SdK7JMndL0m6VPe6AIBqmijF3CJpVdL9ZvZVM/uYme3uf5KZHTOzFTNb\nWV1dbeBtAQBpmgj2HZJeK+led3+NpJ9I+kD/k9z9lLvPu/v8/v37G3hbAECaJoL9KUlPufsj698/\npBD0AIARqB3s7v4DSd81s1esP3S7pMfqXhcAUE1Tq2LeLWl5fUXMdyTd2dB1AQAlNRLs7v41SfNN\nXAsAUA87TwGgZQh2AGgZgh0AWoZgB4CWIdgBoGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaBmCHQBa\nhmAHgJYh2AGgZQh2AGgZgh0AWoZgB4CWIdgBoGUIdgBoGYIdmCDLy9LBg9LUVPjn8vKoR4Rx1NRh\n1gAGbHlZOnZMunAhfH/2bPhekrrd0Y0L44c7dmBCLCxshHriwoXwONCLYAcmxLlz5R7H9kWwAxPi\nwIH0x6emytXaqdO3H8EOTIiTJ6XZ2a2PX70aau0xAZ3U6c+eldw36vSDDnc+TIbL3H3obzo/P+8r\nKytDf19g0i0vS0ePhjDvNzcnPflk/usPHgxhXuW1VfVP+krhA+rUKSZ9yzKzR919vuh53LEDE6Tb\nla5dS/9ZTK29iTp92btvJn2Hj2AHJkxWrT3r8aZeK1Ur5TDpO3wEOzBh0mrts7Ph8SKHDpV7vF+V\nu++6HyYoj2AHJky3G+rTc3OSWfhnbL36zJlyj/ercvdd54MI1RDswJiJqWF3u2Gy89q18M/YSci6\nZZEqd991PohQDcEOjJG8GnYTSwbrlkWq3n1X/SBCNQQ7MEayatgnTjSz/rxuWYS778nQWLCb2bSZ\nfdXMPtPUNYHtJqsksrbWzJLB/mDudKRdu6QjR/J/C+j9bWFhIXwQcPc9vpq8Yz8h6fEGrwdsO2VX\nilRZMpiURR54QLp4MXxo5P0WMKrdqqiukWA3s5sk/bqkjzVxPWC7yiqVdDrpz9+7N/taRTX52KWL\nbDCaPE3dsf+xpPdLytgTJ5nZMTNbMbOV1dXVht4WaJesGvbiojQzs/X5zz2Xfuccc5cdu0KGDUaT\np3avGDN7m6RD7n63mf2ypP/s7m/Lew29YoDy9u0LZZN+aX1eYnrCxPaNGUV/GaQbZq+YN0l6u5k9\nKekTkn7FzJYauC6AHs8+m/74uXNbyy5pQSyFx5OyTOwKGTYYTZ7awe7uH3T3m9z9oKR3Svprdz9c\ne2QANsmaWDWTDh/eXHYxy75O75F6MUsXWeI4eRpt20spBhictPa3dVBKmTwjadvr7v+rKNQBVLdr\n18afp2r+18vkZ3ux8xQYgbLtAZK79d7J06y+7LGmpjjRqK0IdmDIqmz4SVtLXtfVqxvvf+RIqJ8T\n8u1AsANDltcPJkuVskmnk35GaloJJ5lqy9t9ypmlk4NgB4Ysrx9MVmBmrYjZvTv98Z07w6am/tUs\nS0sbIZ6lf1dp2m8Y3OGPN4IdGLK8fjC/8RshMM3ChqQkNA8dSl/CeN110vHjm1sOdDrS/feHPy8s\nhA+SAwfCuvNuN64fTe+HT9pvGEV3+Bgtgh0YsryNPb0Tomtr0p13SnffLZ0+nX6nvbYWfra4GH7u\nLj3zTPhZVh0/bcNRP/eNu/GszU4J+saMn0bXscdiHTu2u6z2AGmmp8NEZ57YNgCdTgj+5eUQxslm\nproxYFZ/lQ6KjWQdO4A4i4vFd82JolCXttbts+6ykzp+0rrXPbTvnZuLG0sWDqYeLwQ7MAL92/Sn\np7Ofm/ezRO8kalG9u+mySW/fGFbPjAeCHRiR3nNAT59Ob8u7c2eojRfd3Z8/vxGiecsmpc2NwHpX\nvJQxPb21bwwHcowPauzAmFheDqGc1N47nVCySUIzqYlnSerseQ3Aes3OhhYFsbX+xMyMdN99W5uA\n0d538GJr7DuGMRgA6ZLATpYkJkHeL3ns6NHsmnvZTUwXLlTbzZp1L8iBHOODUgwwImVKF8lz8yZS\nkwnMrGP0yuh0sidUL19Or9NnTaAysTp8BDswImXOEi3qFdM7gZl1jF6smZlwjbz19sldeO9k6fnz\nYU4ga1wYHoIdGICY1SFZJYqzZ7c+P6+c0X/wRbcr/eZvbqymmZpKD/qsWvz114dr5K2eOXBg628c\na2vhmp0OB3KMGjV2oGH9B2L0nljUG3IHDmRPhvY/P+u5aROTy8thlU1Strl2Tbp0afNzOp3sSdPk\nCL68D5OTJ9N/i7h0SdqzZ2P3K0aDO3agYbEllryt/f3PL3PuaEyL3z17smvoSU08qzbe6YQPHCZL\nxxfBDjQsNvC63bDKJcvZs6FPzMGDoZvirl1xZY6YYD13rvjDIuvni4vhz0yWji+CHWhYmcB78MH8\na9177+Ya9sWLoQXAk09m165jgvXAgeJDqtN+fvRo+I2AydIx5+5D/7rtttscaKulJffZ2aTXYvia\nnQ2P9+t9TuzX3Fz5948ZS5W/18yMe6fjbhbGVeW6iCdpxSMylslToGHJHW9aL/QmFJVa+t9/797w\n/bPP1hsLk6WTg1IMUFPa0sbePjB5ZZMqm4mSpYZ5yyl73/+ZZ8JX0ViK/m5ZK3iYLB0/BDtQQ93G\nV8lEZKzZ2XCaUv973nVX6PFuJu3YUe7YuqwPif6/WxYmS8dQTL2m6S9q7GiLublqdfBex4+HGnVs\njX1qKv65WfX0paXssc/M5P+8iXo9qlFkjZ3ujkANU1Ppd7NlTxS6+27pox+tf5JRmv5NTP0bqNJ0\nOqEmnzUes+bnDlCM7o7AEGTtCC1bnjhzZjChLoXxTU2FMR06FJYwFp3KtLYWPhBowzuZqLEDNZTZ\nEZpn0BOQSS3+3nvjjtqTmvu7YfgIdqCGok0+WfonLJMlieMiaRtQ9u/G0XjjgRo7MGRpNe6Zma2N\nuqqano6/K0+zc6d0//3la+dpf6/ZWTo8Nim2xs4dOzBkeRt96pidlZaWpCtXQuklq8lXkRtuqPa6\nMv3lMVgEOzBkWfX0n/wkBHOepMf63Jx0/Hh+mSSve2SetbVya/GT8kuTG5go6dQUsyay6S/WsWM7\nK1r7XnbteLLmPOnXcvz4xvedjvvu3fHr3suuxS/qS1N2TX/WNVkvH2hY69jN7GZJfyrpX0lySafc\nPXc/HTV2bFfLy9KJE1sPuTDbKJ9k3fn2SpYcxqxJT65dVsxa/Lw7dalajT3rmiyzHG6N/Yqk97n7\nrZLeIOm3zezWBq4LtEoSwmknFyXBGxPq0kZ5I+ZQjar3bjFr8csc2ReLAzzqq71Byd2/L+n763/+\nFzN7XNKNkh6re22gTWJCOFYSuoMKu9j16mWO7IvV1Kav7azRyVMzOyjpNZIeSfnZMTNbMbOV1dXV\nJt8WmAhNhXBv6A4i7MrcaQ9iExMbo+prLNjNbI+kT0l6j7s/1/9zdz/l7vPuPr9///6m3haYGE2E\n8PR0OMUoCd2mwy4J0KxQ71+tIlXboJWn6qYvbGhkg5KZ7ZT0GUmfdfc/LHo+k6fYjmImOmP0T0ju\n25det68qq4zCBqTRG9rkqZmZpI9Lejwm1IHtKrkTrat/08/iYrX16lnOnUtfR84GpMnRxHLHfyvp\nf0v6hqRkcdSH3P1M1mu4Y8d2VrREMNbc3MbRe7FdG2N0OuHQ7P4786zfNMq2KEZ1Q2vb6+5/K8nq\nXgfYLk6eDCce1ekNY7bx4XD27EYv96pr1hPJnX/anXlWDxpWq4wfWgoAA5K1Lb7bla6/Pv46/WWW\ntPBOvk/CPdb0dLhD752kfPbZ9OdevcpqlUlBsAMDUHQWalZ49kvCtneFSNEdeZlwv3o1NB/rPeg6\n6w48bSxMnI4ngh0YgKKJxpjyxc6doXa+sLC5lh4T2mXKMf3r6/PWkXe74QOg94MA44dgBwagaFv8\nyZOhB3ueF71IOn16813/IM5FTT5kktLRkSPSrl1bSzSE+OTgzFNgALK2xU9NbZw/unNn/gTq+fNb\nH4sJ9dnZEMwxa9uTO/H+Nepra+FnDzxAoE8i7tiBAcjqhX716sbd909+0vz7JnfXi4vFvxH03omz\nRr1duGMHBiC5y03q41NTzawxz5PWDqC/RXCnE0K//y6cjortwpmnwBBMTTVfG0/T6YR/JmGeFeT9\n6IE+GTjzFBgjw9rEs7a2+Q59bU26887so+V6j7XrX23DGvXJRbADQ3DyZJgsLSs547SOy5fTa+W9\na+2lzevfWQkz2Qh2YAi6XemGG8q9ZnY2BG/aztOy0mrlJ05snTBNjudjjfpkI9iBIYndbdq7dvye\ne6Q3vnHzz6dS/qs1CztIs0xNbS7HLC9nL4dkwnTyEezAAKT1idm7t/h1c3Obd3Xefbf0xS9ufk7a\n6hr3sKEpa4nj1aubWxoULWPMqsljMhDsQMOy+sQ8/3z+69ImK8v0b3/2Wem++zZWxvS7cEE6fLi4\nbbD75g8BTB6CHWhY1mafvA1JWZOVZda+790bXr+4mP+8mF7wbE6abAQ7UEJWK95eVWvUf/d3W69d\nZlXMc89tnHTUBGrtk4tgByIVteJNVFmzfvasdO+9m6995Ij0ilfEX+Py5VBqaeJ0Jin8PWI+yDB+\nCHYgUmw/lay2t1m17yzu0mOPSbfeWn6sTTh0KO6DDOOHYAcixfZTSQ6t7j+Qouqh0088Uf41MTqd\nMLYsDz5IY7BJRRMwIFJWK9600ku3m73BZ2GhXLmk7kHRU1PhA6Z3Inbnzo1J1iNH0vvYsM59cnHH\nDkTKO1koVnIC0dJStbv3frt3Fz/HPRzY0fsbxP33b3z4lG1OxuHV449gByJllViqbL1Pu1YZc3Ph\nw+H8+eIPiQMH8o+0K/PeO3fSGGwSEOxACXkBGbuCpPcIOimcUlQmLPt7uXS70tGj6c+dmSm+dswx\nfYkbbqCHzCSgHzvQgP6j5aRwF53c0Sfry5P2uL3/2c3MhO8vX457r+PHpTe9afMh1+fPp9fEOx3p\nmWeKr7lvX9xRemb1a/6oLrYfO8EONCDvoIqTJ7eGfh2djnTxYtz1YoM49iCQTic0G0s+UPpPbMJg\ncdAGMERZK0XOnk1vj1vH2lr89bImOvvLRjENymZmwu5W1rWPP4IdaEDeSpGYEscgZK3YSdtB++Mf\np19jz56Nyd3rr99aLmJd+3gi2IEGpC2FHISdO7N3sO7Zs9FbZno6TKj2l0mWl8Pj/Xf8V66kX7PT\n2Zgozuonz7r28UOwAw1Ili8Ompn0jnds/RCZmZFeeGFjE9LVq6H3zL59G6WS5E69TMfI3tDO+q2E\nde3jh2AHGtLtll+PXtalS2Gr/65dG491OullEimUgZI6eFqvmyK9od3EBi0MB8EONGgYJZm1tc11\n+4sX8+v4SR08r2QyM7P1sO3+0G5ygxYGq5FgN7O3mtk/mtm3zewDTVwTmFS9d9PDcOFCcd/2ZKI0\nzfR0OHnp/vuLQztvgxbGR+0mYGY2LelPJP2qpKckfdnMPu3uj9W9NjBJlpelu+4K5ZJhK1M379W7\niUoiqNuiiTv210v6trt/x90vSfqEpDsauC4wEZI14YcPlw/1665rZgx79lR73fPPh5Ob0C5NBPuN\nkr7b8/1T648Brde7JryK55+P69BYJO881TzXroXVM3ffXX8MGB9Dmzw1s2NmtmJmK6urq8N6W2Cg\nqqw06Vc1lHvV7QwyjKWaGJ4mgv1pSTf3fH/T+mObuPspd5939/n9+/c38LbA6MVszul0yh+LN2xV\na/QYT00E+5cl/YyZ3WJmM5LeKenTDVwXGHsxm3MWF8NX/3LCYZoq+C+9aFUNJkvtYHf3K5J+R9Jn\nJT0u6UF3/1bd6wKToGjdeqezcVJR/3LCrLv4ublQWllaauZOP+YD5dix+u+D8dFIjd3dz7j7y939\nX7s7+9CwbSSbdtICeHZ241zR5Lm9a8DTDrfu3RTU7VZf7ZIwK27de/y4dM899d4H44Wdp0BN3W44\nzGJpKX2DT7Ic0kzasSP88+DB8NqinZx1G2yZ5S/BnJsj1NuIYAcakrYrs385ZDJJmfQyl/J3csY2\n2JqbC3fe/b855N2pVz2/NPYIQIwOwQ4MUN5yyJhe5ocOFb9HcgbqPfeUK92YxT83kdbLncM2xg/B\nDgxQUSmlaGPTgw/m/9xsc/iXKd1culT+kIy0DyoO2xg/BDsQoWr5oaiUknbX3FuTLzp9yV06fXpj\nPGV7o5et4Wc9n8M2xgvBjm2hTl24Tvnh5MnQEjeL++brVGlR0HvHXLZtcNkPAg7bmBDuPvSv2267\nzYFhWVpyn511DzEavmZnw+Mx5uY2vzb5mpuLe++dO9Nfn3adrPcq+jLb/J4xr5mZif930HvtOv8u\nUY+kFY/IWIIdrVcnmN1DaBaFqXsIt7m58Pjc3Mb3MaEc+9yYD4e8v3PvV6dT7d9n2t8TwxEb7Bae\nO1zz8/O+srIy9PfF9jQ1ld4kq2jjTuLgwfTSyPR0eP2BA2EC8/TpzROLs7NxDcI6nXAKUp1mYv2b\njJKSTt41Y//+GB9m9qi7zxc9jxo7Wq9uXTirbn316kbN/aMfTV8tUiS5bt0OkWfObP6+9xi7LNTF\n24tgR+vVPYS5/6zPtIZZVX7xTXaaPvts+df2S1al9E4SLyyEv+PSUv7fnw1H7UOwo/XSDmE+ejQE\nX2yY9e4qbap8cf58+GeVJZH9DhzIXr0jZbcuYMNRS8UU4pv+YvIUozSoVTJVvmZn3Y8f3zqe/p8n\nk5V79qQ/J2/yNW+SuO7EMoZLkZOn3LFj26m7e7LsWvE8Fy6E+vipU+klnuTnTz4pPfDA1t8WzMJv\nH91utc1DbDhqJ4Id207dMIupuZcdT7ebXeJJxpX2geS+MXFaZZKYDUftRLBj22kizGJr7jH18amp\nja+8cRV9IFWZJK47sYzxRLBj26kaZll91ffuTX/+3FwonyRLDrNCPlk2mXbuaO+4Yj6Qdu3a+HOn\ns7W/e7+0ieWi12D8EezYdqqEWV5f9eee29oPpv+DwiwEcFrP9H7T0+H5nU4I6iNHwgfIoUPZH0jJ\n+Hqbhl28mP8+ibQ+8phs7DwFImTtPk10OqEX+rlzIcCTUL/zTuny5Y3nTU+HkkvvY/3Mwp1+/87R\n2dkwUXrmzOb36Xazx5f0akc7xO48JdiBCFltCRJp2/P37Stuu5smKd2UCeq6bRMwGWgpAJSUtwOz\naGI17edVQj0prZRducPqFvQi2AEV78DMW7ve1CqS3lp/2aBmdQt6EeyAijctdbuhvt2/sqV3g1C/\noknSxOxs6OfSO3GZddZp1uOsbkEvgh1QXOnjzJmtdWz3cC5pWglncXHrapmZmbAypiiA+7s1Fj0u\nsboFG3aMegDAODhwIH2ysrf0kRX+a2sb9fTexltJsC4sbF3FUoSt/qiDO3ZAcTXq2InICxdCeWZ5\nufpdNJOhqINgBxRXoy7T/Ovq1Xrtb4cxGUof9vYi2IF1RXfXaeGfN0FapmNk2ljSPmikZsKYPuzt\nxgYloIais0Wb3CCU9l6zs9VWv7BTdTKxQQkYguTOOqt1b52aeH+p5MSJen3kezE5224EO7aVQdSV\nu13p9Olma+JppZKsnaxVwpjJ2XYj2LFtDLKu3PQGobQNU1nSwrjoA4ydqi0Xc35e1pekD0t6QtI/\nSPpzSS+JeR1nnmIUJul8T7P4M1P7z2qNPdM1OSfVLPwz9sxXjI6GdObp5yW9yt1/TtI/SfpgzesB\nA5NVsjh7dvyW/WWVRDqd4t8KYs90Zadqe9UKdnf/nLtfWf/2S5Juqj8kYDCywtJs/Jb9ZZVKFheL\nw5iJUTRZY79L0l9m/dDMjpnZipmtrK6uNvi2QJy0sDTb2v+lzvrzptSp2TMxisJgN7MvmNk3U77u\n6HnOgqQrkjLvc9z9lLvPu/v8/v37mxk9UEJaWGZt4xjV3W3vpOfCQvgwKlsqYWIUtSZPQy1f75L0\nfyTNxr6GyVOMi0FMqFadlIyd9Ix5DyZG20mRk6d1Q/2tkh6TtL/M6wh2jIsyYTro68V+yDQ9ZkyO\n2GCv1VLAzL4t6UWSkq0TX3L33yp6HS0FME6Wl6u11k1TZ6t+7LmltAPYvjjMGhiS3g+GrP+cYnrG\nxAY2B1dvX/SKAQYomeQ0k44c2VgumSVmRUrspCerXlCEYAdK6m1NIOUHuhS/IiV2iSOrXlCEYAdK\niu3jkhfOWb1cYnaDJh8Avb3gd+2q9ndBO3HmKVBSzBr3vInM/r7qaeekxrh4cePPa2vVroF24o4d\nKKmolp1XFlleDueh1u2rHtsPBtsTwQ6UlNWaQMrf+p/cqV+9mn7dMrtd6QeDPAQ7UFLaJOcDD4RJ\n1Lyt/2knIPUqs6qFlTHIQ7ADFXS74c79wIFwl7ywkN8Rcnk5+wQkqfyqFlbGIA/BDlRQ9jSmvNr3\n9HT505aaPrEJ7cLOU6CCstv6s3aLStLSEoGMOOw8BQao7ORl3olIhDqaRrADFWQF9d696RuP8k5E\nKqvooOpBvRYTJKYFZNNftO3FpEtrnTsz475zZ3Y73ePH3aenw+PT0+H7Jt43tmUv7X4nn4Z0mDWw\nLaVNXl5/vXT58ubnJZuGlpel06c31rBfvRq+L7pj7r/DTlsyGbsxiU1N2weTp0BD8trpHjhQvod6\nf+uBPDEte2n3O/mYPAWGLG/TUJWdorHNxvLeO+Y5bGpqH4IdaEjepqEqoRrbHiB2YxKbmrYPgh1o\nSN6moSqhmrdEssrGJDY1bR+07QUalIRkclReMjHZ/3jM2aonT26tsSdLJKuGcbdLkG8HBDvQoKJe\n62XbBkjNHbSN7YNVMUCDyrYaAMpgVQwwAmVWv7ALFINCsAMNil39UrY7JFAGwQ40KHb1C7tAMUgE\nO9Cg2CWFHG2HQSLYgR5N1L273TBReu1a9lF57ALFIBHswLph1r3ZBYpBItiBdcOse7MLFINEsAPr\niureTS9PjCnZAFUQ7MC6vLo3yxMxSQh2YF1e3btMmYaNRxg1gh1Yl1f3jl2eyJ09xkEjvWLM7H2S\nPiJpv7s/U/R8esVg0sT2gKFXDAZpaL1izOxmSb8mia0VaK3Y5YlsPMI4aKIU80eS3i9p+G0igSGJ\nXZ7IxiOMg1rBbmZ3SHra3b8e8dxjZrZiZiurq6t13hYYiZjliWw8wjgoPGjDzL4g6WUpP1qQ9CGF\nMkwhdz8l6ZQUauwlxghMDA7HwDioPHlqZj8r6YuSkkVgN0n6nqTXu/sP8l7L5CkAlBc7eVr5aDx3\n/4akl/a84ZOS5mNWxQAABod17ADQMo0dZu3uB5u6FgCgOu7YAaBlCHYAaBmCHQBahmAHgJYh2AGg\nZQh2AGgZgh0AWoZgB4CWIdgBoGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaBmCHQBahmAHgJYh2AGg\nZQh2AGgZgh0AWoZgB4CWIdgBoGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaBmCHQBahmAHgJYh2AGg\nZQh2AGgZgh0AWoZgB4CWqR3sZvZuM3vCzL5lZn/QxKAAANXtqPNiM3uzpDskvdrdXzCzlzYzLABA\nVXXv2I9L+n13f0GS3P2H9YcEAKijbrC/XNIvmtkjZvY3Zva6rCea2TEzWzGzldXV1ZpvCwDIUliK\nMbMvSHpZyo8W1l+/V9IbJL1O0oNm9tPu7v1PdvdTkk5J0vz8/JafAwCaURjs7v6WrJ+Z2XFJD68H\n+d+b2TVJ+yRxSw4AI1K3FPMXkt4sSWb2ckkzkp6pOygAQHW1VsVIuk/SfWb2TUmXJB1NK8MAAIan\nVrC7+yVJhxsaCwCgAew8BYCWIdgBoGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaBmCHQBahmAHgJYh\n2AGgZQh2AGgZgh0AWoZgB4CWIdgBoGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaBmCHQBahmAHgJYh\n2AGgZQh2AGgZgh0AWoZgB4CWIdgBoGUIdgBoGYIdAFqGYAeAliHYAaBlCHYAaJlawW5mP29mXzKz\nr5nZipm9vqmBAQCqqXvH/geS/pu7/7yk/7r+PQBghOoGu0u6Yf3PL5b0vZrXAwDUtKPm698j6bNm\n9hGFD4lfqD8kAEAdhcFuZl+Q9LKUHy1Iul3Se939U2b2Dkkfl/SWjOsck3Rs/dsXzOyb1YY8VPsk\nPTPqQURLBTgDAAAEtElEQVRgnM2ZhDFKjLNpkzLOV8Q8ydy98juY2Y8lvcTd3cxM0o/d/YaI1624\n+3zlNx4SxtmsSRjnJIxRYpxNa9s469bYvyfp363/+Vck/d+a1wMA1FS3xv6fJC2a2Q5Jz2uj1AIA\nGJFawe7ufyvptgovPVXnfYeIcTZrEsY5CWOUGGfTWjXOWjV2AMD4oaUAALTMyIJ9ktoRmNm7zewJ\nM/uWmY3t7loze5+ZuZntG/VY0pjZh9f/Pf6Dmf25mb1k1GPqZWZvNbN/NLNvm9kHRj2eNGZ2s5n9\nTzN7bP3/jydGPaYsZjZtZl81s8+MeixZzOwlZvbQ+v8vHzezN456TGnM7L3r/3t/08z+zMyuy3v+\nKO/YJ6IdgZm9WdIdkl7t7v9G0kdGPKRUZnazpF+TdG7UY8nxeUmvcvefk/RPkj444vH8f2Y2LelP\nJP17SbdK+o9mdutoR5XqiqT3ufutkt4g6bfHdJySdELS46MeRIFFSX/l7q+U9GqN4XjN7EZJvytp\n3t1fJWla0jvzXjPKYJ+UdgTHJf2+u78gSe7+wxGPJ8sfSXq/wr/XseTun3P3K+vffknSTaMcT5/X\nS/q2u3/H3S9J+oTCB/pYcffvu/tX1v/8LwpBdONoR7WVmd0k6dclfWzUY8liZi+W9EsKGyvl7pfc\n/UejHVWmHZJ2ra9AnFVBXo4y2N8j6cNm9l2Fu+CxuXvr83JJv2hmj5jZ35jZ60Y9oH5mdoekp939\n66MeSwl3SfrLUQ+ix42Svtvz/VMaw8DsZWYHJb1G0iOjHUmqP1a40bg26oHkuEXSqqT710tGHzOz\n3aMeVD93f1ohI89J+r7CRtDP5b2m7jr2XE21Ixi0gnHukLRX4dfe10l60Mx+2oe8nKhgjB9SKMOM\nXN443f1/rD9nQaGksDzMsbWJme2R9ClJ73H350Y9nl5m9jZJP3T3R83sl0c9nhw7JL1W0rvd/REz\nW5T0AUn/ZbTD2szMfkrht8dbJP1I0ifN7LC7L2W9ZqDB7u6ZQW1mf6pQg5OkT2qEv7IVjPO4pIfX\ng/zvzeyaQl+J1WGNT8oeo5n9rML/4F8PXR10k6SvmNnr3f0HQxyipPx/l5JkZu+S9DZJtw/7w7HA\n05Ju7vn+pvXHxo6Z7VQI9WV3f3jU40nxJklvN7NDkq6TdIOZLbn74RGPq99Tkp5y9+Q3nocUgn3c\nvEXSP7v7qiSZ2cMKDRczg32UpZhJaUfwF5LeLElm9nJJMxqjZkHu/g13f6m7H3T3gwr/Z33tKEK9\niJm9VeHX87e7+4VRj6fPlyX9jJndYmYzCpNTnx7xmLZY78n0cUmPu/sfjno8adz9g+5+0/r/H98p\n6a/HMNS1/t/Id80saax1u6THRjikLOckvcHMZtf/979dBZO8A71jLzAp7Qjuk3TfejfKS5KOjtmd\n5iT575JeJOnz679dfMndf2u0Qwrc/YqZ/Y6kzyqsOrjP3b814mGleZOkI5K+YWZfW3/sQ+5+ZoRj\nmmTvlrS8/mH+HUl3jng8W6yXiR6S9BWFEuZXVbADlZ2nANAy7DwFgJYh2AGgZQh2AGgZgh0AWoZg\nB4CWIdgBoGUIdgBoGYIdAFrm/wEmRt49wQUdoAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114e32780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mu2 = np.array([2, 1])\n",
    "S2 = np.array([[1,2],[0.5,3]])\n",
    "\n",
    "X2 = np.random.multivariate_normal(mu2, S2, 300)\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "plt.plot(X2[:, 0], X2[:, 1], 'ob')\n",
    "plt.xlim([-8, 8])\n",
    "plt.ylim([-8, 8])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "U, L, V, m1 = pca(X1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 18.85384366,  16.36829919])"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ -3.15084580e-02,   2.88581710e-02,  -4.76768453e-02, ...,\n",
       "          1.06676514e-01,   2.40346349e-02,  -2.16188771e-02],\n",
       "       [ -1.51375814e-02,  -6.78252771e-02,   1.47314710e-02, ...,\n",
       "          4.10079440e-02,   9.67535445e-02,  -2.19823845e-01],\n",
       "       [ -1.26859764e-02,   4.82050501e-02,   9.97633944e-01, ...,\n",
       "          4.42274623e-03,  -3.43643526e-05,   1.59509748e-03],\n",
       "       ..., \n",
       "       [  9.12369051e-02,  -6.86938222e-02,   4.20684011e-03, ...,\n",
       "          9.87306739e-01,  -6.74103802e-03,   1.19528564e-02],\n",
       "       [  9.49172591e-02,   3.08591199e-02,  -3.38450307e-04, ...,\n",
       "         -6.16949004e-03,   9.90414835e-01,   2.10571637e-02],\n",
       "       [ -1.98213521e-01,  -9.80683362e-02,   2.25622896e-03, ...,\n",
       "          1.05714825e-02,   2.08617637e-02,   9.53193380e-01]])"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "U"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.42319191,  0.90604007],\n",
       "       [ 0.90604007,  0.42319191]])"
      ]
     },
     "execution_count": 185,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(300, 300)"
      ]
     },
     "execution_count": 186,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "U.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "COLORS = ['r', 'c', 'y']\n",
    "\n",
    "def plot_pca(X, V, mu, L):\n",
    "    plt.plot(X[:, 0], X[:, 1], 'ob')\n",
    "\n",
    "    for d in range(V.shape[1]):\n",
    "        l = np.sqrt(L[d])\n",
    "        print(l)\n",
    "        p1 = mu - l * V[:, d] \n",
    "        p2 = mu + l * V[:, d]\n",
    "        plt.plot([p1[0], p2[0]], [p1[1], p2[1]], color=COLORS[d], linewidth=5)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.74535650254\n",
      "3.68804717257\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([ 33.00912134,  13.60169195])"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAFpCAYAAACI3gMrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmQXeV55/Hv05ukbkndqCWZReoWYHbEKrAxmwg4IZiY\ncmUpewQhJhMFyDjY5YwrtirJLNGMl2xMZSBRMISgtj2x4zixBxzbE4PZBIjFkkCAMWhDgPZu7Vs/\n88fpq77LOXc7526nf5+qW1Kfvvfct0XzO+99zruYuyMiIunR1ugGiIhIshTsIiIpo2AXEUkZBbuI\nSMoo2EVEUkbBLiKSMokEu5l9xsxeNrM1ZvZ1M5ucxHlFRKRysYPdzE4Cfh9Y4O7nAu3Ax+OeV0RE\nqpNUKaYDmGJmHUA3sDmh84qISIViB7u7vw38GbABeAcYdvcfxD2viIhUpyPuCczsOOAm4GRgF/BN\nM7vZ3ZfnPW8xsBigp6fn4jPPPDPuW4uITCjPP//8NnefVep5sYMduA54y923ApjZt4EPATnB7u7L\ngGUACxYs8JUrVybw1iIiE4eZrS/neUnU2DcAHzSzbjMz4FpgbQLnFRGRKiRRY38G+BbwArB67JzL\n4p5XRESqk0QpBnf/E+BPkjiXiIjEo5mnIiIpo2AXEUkZBbuISMoo2EVEUkbBLiKSMgp2EZGUUbCL\niKSMgl1EJGUU7CIiKaNgFxFJGQW7iEjKKNhFRFJGwS4ikjIKdhGRlFGwi4ikjIJdRCRlFOwiIimj\nYBcRSRkFu4hIyijYRURSRsEuIpIyCnYRkZRRsIuIpIyCXUQkZRTsIiIpo2AXEUkZBbuISMoo2EVE\nUkbBLiKSMgp2EZGUUbCLiKSMgl1EJGUSCXYz6zOzb5nZq2a21swuS+K8IiJSuY6EznM38H13/zUz\n6wK6EzqviIhUKHawm1kvcBXwWwDufgg4FPe8IiJSnSRKMScDW4EHzOxFM7vPzHryn2Rmi81spZmt\n3Lp1awJvKyIiYZII9g7gIuBed78Q2Av8Yf6T3H2Zuy9w9wWzZs1K4G1FRCRMEsG+Cdjk7s+Mff0t\ngqAXEZEGiB3s7v4usNHMzhg7dC3wStzziohIdZIaFfMpYGhsRMybwCcTOq+IiFQokWB395eABUmc\nS0RE4kmqxy4ilbjmGpgzBxYuDB6nnAJmjW6VpISCXaTe3n4bHn00+Pvy5cGfAwPw2msweXLDmiXp\nobViROrtsccKj/X3K9QlMQp2kXrL9NazLVxY71ZIiinYReotrMeuYJcEKdhF6mnzZnj99dxjZnDl\nlY1pj6SSgl2knsJ66xdcAMcdV/+2SGop2EXqSfV1qQMFu0g9hQX71VfXvRmSbgp2kXpRfV3qRMEu\nUi9h9fXzz4cZM+rfFkk1BbtIvai+LnWiYBepF41flzpRsIvUwzvvBGvBZFN9XWpEwS5SD6qvSx0p\n2EXqQfV1qSMFu0g9aPy61JGCXaTWourrV13VmPZI6inYRWotrL5+3nmqr0vNKNhFak3DHKXOFOwi\ntaYbp1JnCnaRWnr3XXj11dxjqq9LjSnYRWpJ9XVpAAW7SC1pmKM0gIJdpJZUX5cGULCL1EpYfR1U\nX5eaU7CL1EpUfb2/v/5tkQlFwS5SKxq/Lg2iYBepFdXXpUEU7CK18N57sHZt4XHV16UOFOwitaD6\nujSQgl2kFjR+XRoosWA3s3Yze9HMvpfUOUValurr0kBJ9tjvAkKKiiITjOrr0mCJBLuZzQE+AtyX\nxPlEWlpYfX3+fJg5s/5tkQkpqR77XwGfA0ajnmBmi81spZmt3Lp1a0JvK9KENH5dGix2sJvZjcAW\nd3++2PPcfZm7L3D3BbNmzYr7tiLNS/V1abAkeuyXAx81s3XAN4BfMLPlCZxXpPVs2QKvvFJ4XPV1\nqaPYwe7un3f3Oe4+D/g48O/ufnPslom0ItXXpQloHLtIkjR+XZpAR5Inc/dHgUeTPKdIS1F9XZqA\neuwiSVF9XZqEgl0kKT/5SeGxc88FjQKTOlOwiyRFZRhpEgp2kaQo2KVJKNhFkrBlC7z8cuFx1del\nARTsIkkIq6+fc47q69IQCnaRJKgMI01EwS6SBAW7NBEFu0hcW7eqvi5NRcEuEldUfX327Pq3RQQF\nu0h8KsNIk1Gwi8SlYJcmo2AXiWPrVlizpvC46uvSQAp2mVCGhmDePGhrC/4cGop5wrD6+tlnq74u\nDZXosr0izWxoCBYvhn37gq/Xrw++Bli0qMqTqgwjTUg9dpkwliwZD/WMffuC41VTsEsTUrDLhLFh\nQ2XHS9q2Lby+rh2TpMEU7DJhDAxUdrwk1delSSnYZcJYuhS6u3OPdXcHx6uiMow0KQW7TBiLFsGy\nZTA4CGbBn8uW6cappI+5e93fdMGCBb5y5cq6v69IYrZtC1+S99134X3vq397ZEIws+fdfUGp56nH\nLlKNsPr6WWcp1KUpKNhFqqEyjDQxBbtINRTs0sQU7CKV2r4dVq8uPK7x69IkFOwilVJ9XZqcgl2k\nUirDSJNTsItUSsEuTU7BLlKJ7dth1arC41p/XZqIgl2kEmH19TPPhOOPr39bRCIo2EUqoTKMtAAF\nu0glHnus8JiCXZpM7GA3s7lm9mMze8XMXjazu5JomEi1Et/+LmPHjvD6ehnj12vWJpEQSWyNdwT4\nrLu/YGbTgOfN7Ifu/koC5xapSE22v8v4yU8gf9G8MurrNW2TSIjYPXZ3f8fdXxj7+25gLXBS3POK\nVCPO9ndDQzBzZrCkr1nw95yedZX19ZpsySdSRKI1djObB1wIPBPyvcVmttLMVm7dujXJtxU5ptj2\nd8XKIUND8MlPBqMZM7Zvh9tuy3peWLCXUYZJfEs+kRISW4/dzKYCjwFL3f3bxZ6r9dilVubNC0od\n+fr7Yf/+3J5zd/f4RhtRr4NgQ451L+wIuvD5/79s3gwnnFBVmwYHYd26oi8VyVHX9djNrBP4J2Co\nVKiLlBLnRmPU9ndQvBxSrPe8YQPh9fUzzigZ6sXaVPWWfCIlJDEqxoCvAmvd/S/iN0kmsjvvhFtu\nCXq47uM3GssN96jt73bsCH9+JtCLbWg9MACv/s2jhd8oc5hj4lvyiZSQRI/9cuAW4BfM7KWxxw0J\nnFcmmKEh+Ju/KewYV3qjcdGioMQxOhr8uWhRdHBnji9dCp2dhd/v6oIbboCDPygcv/5Ex8JYbRKp\nlSRGxTzh7ubu57n7BWOPh5NonEwsS5YUhnpG3BuNpcohixbBAw8EtfiM/n64/3548rs7mO8/LTjn\n737t6qI3YzVuXRpFM0+lbkqFXbHwLlYqKUe55ZCpU8e/f/fdwfdP3vQ4beRecV7lDF7ZeUJoySgz\nbr3acpJIbO5e98fFF1/sMrEsX+7e3e0eRF3w6O4OjmcMDuZ+P/Mwy31eUu0ZHAzOPTjofscd0e27\nb9qnCxp1L78b2tbBweifY3Aw2Z9BJh5gpZeRseqxS11ETdK59dbxHvwNNxSWS8zg9tuTrUmH9ajv\nvTd61MzHZjxacI7HCB+/vmFD+ePWVa6RWlGwS11Ehd3Ro+Ph+uCDQdBnl0seegjuuSf6vJWEY+a5\nN99cGOJRRtbvoG99YX199XHhwT4wADNmhJ8ru5ykco3UVDnd+qQfKsVMPFHliTjlijvuCEopYeWT\nckot5Tw+yncKD55+emRp6Y473Lu6ws/V3z9eUlK5RqpBmaUYBbtUJT84S9XAw4Iwqp5ezvmXLy8M\n9ewArSbEwx5/zmcKDy5eHNnGUhewzIUnqu2Zn18kjIJdaqacG6Fhr+nvzw2wqB5rnButST+e58LC\ng1/7WuTPGfVz5f+M6rFLNcoNdtXYpWKVrlaYqSdnL7DV2Vk4ISgzrryc89djAa0+dnIBLxV+o8jC\nX+UMy9ywIbhRHCbquEglFOxSsahQXb8+/OZfWFAfOgTTp4ePKy92/syN0rYiv7k9PWX9GCVdSeH4\n9Z93nA4nnhj5mrCJUPkGBuDhiCl8y5bpBqrEp2CXihXrlYaN7IgK6h07wqfZFzt/ZhTJ0aPRzzl8\nOHx5gEot5NGCY6NXRvfWIXciVJjMp5Jio4Q0OkbiUrBLxYr1SsNKMlFB7R4+RDFq3ZZyZT4NZC8P\nUI2wYN9y1sKSr1u0KPrfaN++YLile+H38p+jse1StXIK8Uk/dPO09S1fHn1zMH9kx/Ll0UMAo268\nZt9oreaRPbqmvb3y1/exw49SeCf01MmbypoFm9TN3VI3pTM/YyUjlKR1oVExUmuVjOwoFdT5ryk3\n+KJCO/t85YxUyX/8Cv9ScPA1Tit75Eo171nuv022akYoSesqN9hVipGqVbKBRNR66Bn5W9eVo7s7\nqEeXakM1C4hdTeEyvY+y8Fhbs2XabQYdHcGf5f4MECwNXEyxEUDaT1VClZP+ST/UY6+9J3ft8l96\n6SX/03Xr/NGdO33fkSM1eZ9yywClShOVTCrKf69yJjNFlYKietYruajg4CcYcnDv6ck9d2dnvB55\nf3/xTzTFeuya6DSxUGaPPbE9TyuhPU9r70/XreOPsjbU7DTjkmnTuKK3lyt7e7m8t5fjkhg6Uqah\noWBnpLBfN7NgfZXsce5RKt0ndGgI7rqrvHNn9LKLHcwoGOp4EpvYzEm0tY2Pypk5s/i529uDUT+T\nJsGBA+W3ISN7X9Yw2k91YqnrnqfSfB4fHs75+rA7T42M8OWNG/mVNWuY8eSTzH/uOe58/XW+9t57\nbKwmdSqwaFGwSqNZ7vHM6o2lSjVQ+T6hQ0Nw222VhTqEj19/ndPYzElAENSZRcdKnXt0NHi8732V\ntSHj1luLr2yp/VQljII9hY6MjvLUyEjJ563Zu5d7N29m0dq1DKxYweDTT3PzK6/wt5s38/LevYxW\n+WkuasXFe+4JVmsMW70xqg7e3l7ePqFh77lkSTD0sVJhwxzzl+l1D+8p58v8XNXOlI2ayJSh/VQl\nTEejGyDJW7V3L3uKzeCJsOHgQYa2bGFoyxYAZnR0cEVv77HyzUXTptFV4q5gZvmAzA29zHK0EIRN\nJnCWLAnC7q67gh77nj2F5+rqgmnTSvfmo96z3KV584UFe+bGaSW6usZ7zgMD5V0I8pVzQcj+dxUB\nVGNPo6PurN6zh8eHh3lieJjHh4d5p5qua54pbW18YPp0rhwL+8umT2daR27foFTNNz+Eo0ydCgcP\nBrNIM6LqzaXq3JWIqq/PYSNvM6eic3V2BvuoLlpU/s8dZnAwuEAovKXcGruCfQJwd946cIDHh4d5\nfNcunhge5rX9+2Oftw24cOrUoEff18cVvb2cMLkr8gbp6Gh08Odrbw9fNiD/puDQUDBLMyk38l2+\ny0dzjv2M93M6P6vqfNntzZSHqum5l7qJKhODgl2K2nLoEE+O9eYfHx7mxd27qbx4U6jj3SkceaEX\nVvfCql7YPAWwYwHX1hY+MqZcmQtERpK9dYCv8Af8AX+ec+zv+I8s5u+qOl9+ezPyR+v098Nv/EZQ\nU48Kfo10kXKDXTX2CWp2VxcfmzWLj82aBcCeI0dYMTJyrHSzYmSEfWGJVMKR4/fDDfvhhneDA9u7\naF/by1Xn9vLC7l7mDk5lwzorfpIi8m+yxg31/E8GSdXXMzLtzfTWN2wIji1dCtu2hb8m6uJXj6WK\nJR3UY5dQh0dHeXGsTp8p32w/ciT2eSePtnPoxemMvtQLq/tg7TQ41F7Wa8PKEfnDJyuVeb17UF/f\nTj/t5F7Qqqmvw3h7obC+Xqy0Us59ivyLhEo0E4NKMZIod+fVffuO9egfHx5mXRJj3w8b9rNp+E/H\nyjdremF37sQps+gAixvsMN5r/wjf43v8Ss73qqmvZyZcHTgAe/dGPy+qtBJ2o7Xai4Ski4Jdam7T\ngQM8kTXyZtXuvcnMjHirG1b1wepe+jb1svPVyZFPTbLGHlZfv4/f5ne4r6LzLF8On/xk7oieMJn6\ne1gPHMJ75ZppOrEp2KXuBs4+zMbpIzB/OHicOQJd8X+/+o9M4voTe4+Nvjmru5u2sa56kqNinmMB\nC3g+59jNPMQQ5b9Bf38wVLOckS+ZYYyV9MCj6u9RN2klXRTsUncFJYTOo0y6YDc3LRlm98nDPDk8\nzEgVE6fyzejo4PKxSVNX9PZy4xnT2PFeeR8VurrCZ6NOZ5gdzCior89lA5uYW9a5Ozqgt7e8TxCZ\n8I4a/hjVA1ePfWLTqBipu/xZpQMntrP0rj4W3dQHBBOn5l61l3dm7hrv1c+qfOLUjiNH+O727Xx3\nLEG7vtZG25rMDdleeHk67A//1Z42LfgzP3yv4ImCUH+DU8sO9bY2OHKk/IXMMuWWqJ591AiYqB6+\n1oaRbAp2SVSx6e3tZnzl9qksXjyVfd+ZAzgcfwDOG6bjomGOnDUMA5VPzTzUNgrn7QoeAEeBn08d\nH0u/uhd2TgKigzfuMMdyyyDLl+fORI0StXZOwcVTo2IkhEoxUndRS+l2dcHhnkP4OcPjPfrTd0N5\noyGL2zQlN+jfDiZOZTzLJVxC7u9kpfX1Uvr7x8euF5uBq1EuEqWuNXYzux64m+B/wfvc/YvFnq9g\nl6hg6++H/fuzSg2Tj8LZIzB/rHxz9ghMSeAu4Y7OIOBX99Gzqp3tPx9g0mjuOP1K6uvlyPTWofgw\nzezniWSrW7CbWTvwOvBhYBPwHPAJd38l6jUKdik2uuOhh4psjtE+Cu/fA+cN03nxMIfPGIa+EuMK\nyzBt714ue+UVrly1iitWr2bW2gOce+jV0Of29ARtr3RBr+yft6MjfC2c9vagVg+aiCSF6nnz9FLg\nDXd/c+yNvwHcBEQGu0jUMraZ2nLkGmVH2+C16fDadA5/cy7gMHdf0Js/b6x8c2LlE6d29/Twg0su\n4QeXXAJA++Gj8PoL4+WbrIlTxSYdRenvz/sxIgYHZY6XWv5YpJgkppOcBGzM+nrT2DGRSMV2/gnb\noDmawcYeePhE+OJZsOiD8OuXwX87G/75RHijB6qo3BztbIdzRuDjG+F/rIF/fRLufxY+/Tpc9x7M\nLv/i0dkJd9+de2xwMPy5meNJblIdtfGJpFfdRsWY2WJgMcBANdvGS6oUG91xyy0xT75tEvx4dvAA\n6DkchPR5MSdOnbwveNy0Ofj6vUljdfqxXv36HvDC4vlVVxX2sksNW4wa7ljpQmDq+U9MSdTYLwP+\ni7v/0tjXnwdw9/8Z9RrV2CXfnXcGI0ESmL9UWudROGM3zB9m1nlvcOjc/QxPnRr/vCMdQckmE/Sv\nT4MjwYfiO+4ItgDMFrZ07913J7t0gCY0pUs9b552ENw8vRZ4m+Dm6X9w95ejXqNgl2x33gn33tuY\n9/4Sn+OzbX/Oy/Pm8fh55/HEuefyb+d9kJ2zeuKf/GBbsHrl6l7aXu5j5xPTmZ6141TYYl+dnTB9\nehD2Zrk3XKsZBqklCNKl3sMdbwD+imC44/3uXnQenIJdskWNEIHxlRIh2Q01Mp7hUi7luZxjN/MP\nDB3/a7k3ZAer3EA1Sxtw/tSpXDm2HMJnPtzLplWTir4mE+5h2+OVM2pGPfZ00Vox0jKKjenO/vUs\nd1u9ck1jhJ0cV7CUwADr2UjefaDeQ3Du8HjYn7YHOhL4fyczcSpTvsmbOAXhIVxsad/88Ncyv+mh\nYJeWUe6Y7six7VX6ZR7mYT6Sc+xNTuZU3iz94slH4ayslSzPGU544tTYRiRv9GDeVlA2qaQnrvHw\n6aFFwKRlLF4cXmOfPHl8aF5+rzMJlawP09UFv/3b8Ld/O1abPtAOLx4XPGB84tT8rOUQjqti4tSM\nw3D1tuABsK+dSW9O57+vC1ay/MD06XS3t1c0aqbY+j2STuqxS0Pk9yLf/3748Y8Lb+h1d8OUKfWr\nr/8mD/IQv5lzLLu+PXVquROUHObuH18KYf4wnBR/x6lOMy6eNo1X/k8vI0+MTZwaGd9xSrXzdFMp\nRppWVN23VgEeZhoj7GAGHeTWgAZZxwaCWUJmcPvtucMUY23F138wt0d/6p5kpgiu64ZVvXS93suX\nb+7jro9H7zglrU3BLk0r6Zug1bieR3iEG3KOhdXXM6NSMksClHvh6ekZ79nnD1s8dqz7CLOvGeaD\ni4fZOWeYZ0dGOJjA/49zJ00Kdpsae5zd03NsxylpbaqxS9OqdPZkMZUGbka59fVMzlZ6/pkzg3p8\n1L0Bd2BvB1u+18+P/r2fZcvgh58YZeXu3Ty+axdPDA/z5MgIu44cKXxxCRsPHuTrW7bw9S1bADhu\nbMepTNgvmDaNrrYkPipIs1KPXWouv56+Z08yJZfubrj1VnjwweI3VsN6zCv4AB/g2ZxjYfX1aplF\nL3QWJqw2PurOy3v38vjYZuHfeHlXVTtO5Zvc1sal06Yd21rwQ729OROnpHmpFCNNIWp2pVn43qOl\ntLcHN1gHBuCGG0ovQ5AJzMzFZf166GsbYeto8fp6XP39sGNH+KzPMOXMBB2c52w4cCB34tS85CZO\nXZG1j+wJk4pPnJLGULBLSfUY31xsQ42pU6urtWdmo46MwOEiIwoza7vn/0w//twjXPOV3Pr6W8zj\nFN6qvDERurqCC1i5S/yWM5ol7CI55fhD3P7XI7SdH5Rvnt+zhyMJ/D996uTJXNnXdyzsT5syBVOd\nvuFUY5ei6rXqX1Q9ffv2YJu4qOBvb4/uibuXV8rJ5Nu8eUE7ZsyAAwfgj/Y+yjV5z61kf9NyHDo0\nPrmqFLPg00cp4StidrHoV2cCMwHYe/Qoz46MHCvfPD08zN4qFoX5+YED/Pzdd/n7d98FYHZn5/gN\n2b4+zu/poUN1+qalHvsEVa81RKLeJ9ObhsJeaKW93WI6Owt79WH19Vv5e/6BW+O/YZWKrQkTx5HR\nUV7as4cnxoL+8eFhthb7mFOmqe3tXDZ9Olf09nJjfz8XTZuWQGulFJVipKh6rfo3NBSsrx72Xvn1\n70yvulSJJY6p7GYnxxXU1+fxFuuZl+h7FfvUUUwt13Jxd362fz+PDw8HYb9rFz8/EG/i1BcGBlh6\nyikJtVCKUbBLUfVc9S+qNBt2EammNFOJX+L7fJ9fzjmWdH0dxkfs3HdfdRepes4g3XzwIE9m9eh/\numcPlaTCI/Pnc33+3n9SE+UGu4pkE1SxremSFrUNXNhGWlE1+aNHCy8QXV2VzwStZH2Yag0OBj3u\ne+6BBx4o3O+0HEmO9S/lxEmT+PXZs/lfp53GiwsWsPOKK3hk/nyWDAxwVW8vk4r8IxtwWW9v/Ror\nZVGwT1CLFgXhMzgYhGMmjGrx8b+Si0ixXRPdx4N8cBDuv7/yttQ62DM97cy/46JFwU3i5csruwi5\nB6temtV/n9Lejg6u7+/nT085hccuvJDhK6/kyQsv5IunnMKN/f30ZY15P3/qVHo1Br75uHvdHxdf\nfLHLxLJ8ufvgoLtZ8Ofy5dHP6+52D6It/DE4OP78wcHiz5061b29Pfj7dBvxw7QXPOmCvreKnqOS\nR9TP5e5+xx3Bz1/Nebu7i5+7no6Ojvqq3bv9nk2b/MF33ml0cyYUYKWXkbGqsUvTyZ5MFCUzegSi\np+13dQWxmKlxh9XXGRxkaOm6yBu8lejvD3rnxcRZV14rN4pq7NKyFi0KAiyqNg+54+4zJSUIbrJC\n8PW0abk3LsPKMCxcCBSWiirV3R1sRF1KpjRTjXrW3aW1KdilaYXV5rPt2xf07DMXAvdgUpB78PWO\nHbnPv5rHCs7x9KSFLF5cesx8sfp4Nfcnil20ohS7/yCSTcEuTW3KlOLfL9aLzQ7CHvZwSd6mGgD/\n+f8uLLkz0+Bg8TLN0qXBBaatrfwbnWEXrc7OoHwUplYjliSdFOzSlDJLHpSqRUf1YoeGglUkMy7n\nyYJJSXtmDvLk2/OKnt8sKPsUmz1/223Bc9zHS0Slwj1sVNIDDwQjfcLKStp8WiqhcUrSlJYsKb3H\naVQvNmyxrLD6+nd2Lix6/uzlfovNxs1fpTK7RFRM1F6kCnCJSz12aUrFSiylxt2HXRTCgv3/Hb06\n9Pzd3cEIlzijZDZsCC4w8+ZVVqIRSYJ67NKUojapKGfIX/5FIaq+HjUxadmyYH2bOLq767N6pkgY\n9dilKcVZ8iC/7h5WX9/UPsC6kEW/BgeD4I07AmX//sJPDZkSjUitKdilKcVZ8iD/ohBWhjl02UK6\nu3PHMGZfOEoNtYTiN1SjavIaiy71oGCXppUZnz46mrv+Sjmvu/XW8VElC0PGr59y28KiF47MhSVq\nAa+2tuI3VDPvnS/OJwHV7KVcWlJAUid7VEwPe9jJcXSSt53Rm2/CySeXPFfUMsLFZEbT5G+iHWed\n9bCRPrVct12ak5YUkAkre1TMh3iqMNQHBoLELkOlpZPsMM9fjTJOCIeN9FHNXqJoVIykTnYYR64P\nU+YaulGjc8KEbQaS2e4u7uJdURcY1ewljHrskjrZdezQYL86fPx6mHKn8Xd3R+/wlET4RtXmtX6M\nhFGwS1OKc6MwM6Ilavx6ZkXHcixaVHoHJLPgZm0lO0VVqp47XknrixXsZvYVM3vVzFaZ2T+bWV9S\nDZOJK3OjsNL1VzIyI1o+Njukvj53bs5N03IuIHffXXzoozs8/HBtw7eeO15JCpSzG0fUA/hFoGPs\n718CvlTO67SDkhQTtStSe3vpHZhyfP7zhSe55ZZj3w7brSlqp6LMDlBROxyZ5T6vonaKlIl676Bk\nZh8Dfs3dS/YhNNxRimlrK71OS1lD/S6/HJ56KvfYV78aLMdI9FDGYjc7q3mNSFIaMdzxNuCRIg1a\nbGYrzWzl1q1bE3xbSZtyatIlh/rt3QvPPlt4PKu+Xs1Ik6VLg3XTs3V2jpdbNIlImkHJYDezH5nZ\nmpDHTVnPWQIcASJ/jd19mbsvcPcFs2bNSqb1kkrlTOeHEqNNnnoq2E4pW159vdqRJvkjJTNfx703\nIJKUkuPY3f26Yt83s98CbgSu9aTqOjKhZcorS5YE4d3WFj6UsGgAP/po4bG88etLl4bP5ix2s3PJ\nksL11w8dGv/0EDWJSDc5pZ7ijoq5Hvgc8FF3L7Etgkj5steJefDBKkabhAV73vj1akaaRH1KWL8+\neiKTJhFZ5YL3AAALyUlEQVRJvcW6eWpmbwCTgMwGZivc/fZSr9PNU6nU0NB4D35gIAj1yADeuxf6\n+gpLMW+8AaeeGqsd1awdoxurkpRyb57GWlLA3d8f5/Ui5YraRi5UWH19zhw45ZTY7Qgr3xSjSUTS\nCJp5KunzWOEyvZWsD1NMdvmmGE0ikkZSsEv6RN04TUim/h8V7oOD4WvIayik1IuCXdKljPHrSalk\nCQENhZR6UrBLujz9NBw+nHssofp6vkpG1Wg9daknBbukS9QwxwTq69kyZZVbbgm+fuih4tv3aT11\nqScFu6RLjevrUF1ZReupSz0p2CU96lRfr6asovXUpZ4U7JIeYfX1k06KPSkpfzRLNTNMtZ661JP2\nPJX0qMH49UzZJdNDX78+d8PqbKXKKhVNshKJQT12SY8a1NfDyi7uhdcKlVWkmSjYJR327YNnnik8\nHjPYo8or7iqrSPNSKUbSoUb19YEB7ZgkrUc9dkmHBMevZ98s3bMHurpyv6+yizQ7BbukQ0L19fwx\n6tu3B3/296vsIq1DpRhpfQnW18Nulh4+DFOnwrZt1TVPpN7UY5fWt2JFYX39xBPh/ZVvF6Cp/5IG\nCnZpfWXsb1ouTf2XNFCwS+tLcPy6pv5LGijYpbUlPH5dU/8lDXTzVFrbihVw6FDusRNOqKq+nqGp\n/9Lq1GOX1pZgfV0kLRTs0trqsP66SKtRsEvr2r+/6vq6NpaWNFONXVpXVH39tNOKvixsKd7Fi4O/\nq7YuaaAeu7SuKuvr2lha0k7BLq2ryvq6ZpdK2inYpTXt3x+UYvKVEeyaXSppp2CX1hRWXz/++JL1\nddDsUkk/Bbu0phjj1zW7VNJOo2KkNcUcv67ZpZJm6rFL6zlwoCb7m4qkhYJdWs+KFXDwYO6x44+H\n009vTHtEmkwiwW5mnzUzN7OZSZxPpCitDyNSVOxgN7O5wC8CGgUs9aH1YUSKSqLH/pfA5wBP4Fwi\nxR04ED5+/eqrEzm91pCRNIg1KsbMbgLedvefWomPwWa2GFgMMKCZIFKtsPr6+94HZ5wR+9RaQ0bS\nomSP3cx+ZGZrQh43AV8A/ricN3L3Ze6+wN0XzJo1K267ZaKqYX1da8hIWpTssbv7dWHHzWw+cDKQ\n6a3PAV4ws0vd/d1EWymSUcP6utaQkbSouhTj7quB2ZmvzWwdsMDdtyXQLpFCUfX1hIJ9YCAov4Qd\nF2klGscureOZZ2pWXwetISPpkViwu/s89dalpmo8fl1ryEhaaK0YaR11GL+uNWQkDVSKkdZw4AA8\n/XTh8YTGr4ukiYJdWkNYfX32bDjzzMa0R6SJKdilNTRgfRjNQpVWpRq7tIbHHis8VsP1YTQLVVqZ\neuzS/KLq6zUMds1ClVamYJfm9+yzQbhnq3F9XbNQpZUp2KX5NaC+HjXbVLNQpRUo2KX5hQV7jYc5\nahaqtDIFuzS3BtTXQbNQpbVpVIw0t7D6+qxZcNZZNX9rzUKVVqUeuzQ37W8qUjEFuzS3Oo9fF0kD\nBbs0r4MH4amnCo8r2EWKUrBL82pgfV2klSnYpXmpvi5SFQW7NK8GjF8XSQMFuzQn1ddFqqZgl+YU\nVl+fORPOPrsx7RFpIQp2aU6qr4tUTcEuzUnj10WqpmCX5qP6ukgsCnZpPs89B/v35x5TfV2kbAp2\naT6qr4vEomCX5qPx6yKxKNiluai+LhKbgl2ai+rrIrEp2KW5hA1zvPpqaNOvqki59H+LNJe+Pjjn\nnNxjKsOIVERb40lz+b3fCx5bt8JPfhLcSP3whxvdKpGWomCX5jRrFvzqrwYPEamISjEiIikTO9jN\n7FNm9qqZvWxmX06iUSIiUr1YpRgzuwa4CTjf3Q+a2exkmiUiItWK22O/A/iiux8EcPct8ZskIiJx\nxA3204ErzewZM3vMzC6JeqKZLTazlWa2cuvWrTHfVkREopQsxZjZj4DjQ761ZOz1M4APApcA/2hm\np7i75z/Z3ZcBywAWLFhQ8H0REUlGyWB39+uivmdmdwDfHgvyZ81sFJgJqEsuItIgcUsx3wGuATCz\n04EuYFvcRomISPXiTlC6H7jfzNYAh4Bbw8owIiJSP7GC3d0PATcn1BYREUmAZp6KiKSMgl1EJGUU\n7CIiKaNgFxFJGQW7iEjKKNhFRFJGwS4ikjIKdhGRlFGwi4ikjIJdRCRlFOwiIimjYBcRSRkFu4hI\nyijYRURSRsEuIpIyCnYRkZRRsIuIpIyCXUQkZRTsIiIpo2AXEUkZBbuISMoo2EVEUkbBLiKSMgp2\nEZGUUbCLiKSMgl1EJGUU7CIiKaNgFxFJGQW7iEjKKNhFRFJGwS4ikjKxgt3MLjCzFWb2kpmtNLNL\nk2qYiIhUJ26P/cvAf3X3C4A/HvtaREQaKG6wOzB97O+9wOaY5xMRkZg6Yr7+08C/mdmfEVwkPhS/\nSSIiEkfJYDezHwHHh3xrCXAt8Bl3/ycz+w3gq8B1EedZDCwe+/Kgma2prsl1NRPY1uhGlEHtTE4r\ntBHUzqS1SjvPKOdJ5u5Vv4OZDQN97u5mZsCwu08v43Ur3X1B1W9cJ2pnslqhna3QRlA7k5a2dsat\nsW8Grh77+y8AP4t5PhERiSlujf13gLvNrAM4wHipRUREGiRWsLv7E8DFVbx0WZz3rSO1M1mt0M5W\naCOonUlLVTtj1dhFRKT5aEkBEZGUaViwt9JyBGb2KTN71cxeNrOmnV1rZp81MzezmY1uSxgz+8rY\nv+MqM/tnM+trdJuymdn1Zvaamb1hZn/Y6PaEMbO5ZvZjM3tl7Pfxrka3KYqZtZvZi2b2vUa3JYqZ\n9ZnZt8Z+L9ea2WWNblMYM/vM2H/vNWb2dTObXOz5jeyxt8RyBGZ2DXATcL67nwP8WYObFMrM5gK/\nCGxodFuK+CFwrrufB7wOfL7B7TnGzNqB/w38MnA28AkzO7uxrQp1BPisu58NfBD4vSZtJ8BdwNpG\nN6KEu4Hvu/uZwPk0YXvN7CTg94EF7n4u0A58vNhrGhnsrbIcwR3AF939IIC7b2lwe6L8JfA5gn/X\npuTuP3D3I2NfrgDmNLI9eS4F3nD3N939EPANggt6U3H3d9z9hbG/7yYIopMa26pCZjYH+AhwX6Pb\nEsXMeoGrCCZW4u6H3H1XY1sVqQOYMjYCsZsSednIYP808BUz20jQC26a3lue04ErzewZM3vMzC5p\ndIPymdlNwNvu/tNGt6UCtwGPNLoRWU4CNmZ9vYkmDMxsZjYPuBB4prEtCfVXBB2N0UY3pIiTga3A\nA2Mlo/vMrKfRjcrn7m8TZOQG4B2CiaA/KPaauOPYi0pqOYJaK9HODmAGwcfeS4B/NLNTvM7DiUq0\n8QsEZZiGK9ZOd/+XsecsISgpDNWzbWliZlOBfwI+7e4jjW5PNjO7Edji7s+b2cJGt6eIDuAi4FPu\n/oyZ3Q38IfBHjW1WLjM7juDT48nALuCbZnazuy+Pek1Ng93dI4PazP6BoAYH8E0a+JGtRDvvAL49\nFuTPmtkowboSW+vVPohuo5nNJ/gP/tNgVQfmAC+Y2aXu/m4dmwgU/7cEMLPfAm4Erq33xbGEt4G5\nWV/PGTvWdMyskyDUh9z9241uT4jLgY+a2Q3AZGC6mS1395sb3K58m4BN7p75xPMtgmBvNtcBb7n7\nVgAz+zbBgouRwd7IUkyrLEfwHeAaADM7HeiiiRYLcvfV7j7b3ee5+zyCX9aLGhHqpZjZ9QQfzz/q\n7vsa3Z48zwGnmdnJZtZFcHPqXxvcpgJjazJ9FVjr7n/R6PaEcffPu/ucsd/HjwP/3oShztj/IxvN\nLLOw1rXAKw1sUpQNwAfNrHvsv/+1lLjJW9MeewmtshzB/cD9Y6tRHgJubbKeZiv5a2AS8MOxTxcr\n3P32xjYp4O5HzOw/Af9GMOrgfnd/ucHNCnM5cAuw2sxeGjv2BXd/uIFtamWfAobGLuZvAp9scHsK\njJWJvgW8QFDCfJESM1A181REJGU081REJGUU7CIiKaNgFxFJGQW7iEjKKNhFRFJGwS4ikjIKdhGR\nlFGwi4ikzP8H6LYlnsQTRqgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114e42b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(6,6))\n",
    "plot_pca(X1, V, m1, L)\n",
    "plt.xlim([-8, 8])\n",
    "plt.ylim([-8, 8])\n",
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "U, L, V, m2 = pca(X2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.74535650254\n",
      "3.68804717257\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([ 33.00912134,  13.60169195])"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAFpCAYAAACI3gMrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4XPV95/H3Vzdbsi0ZywYCtmRzC3cCFgQCxiLQlHWz\nyXafNksrUwJttJhtStI87ZPETa9Rm7TdbdltgboNqYNnkzSENNk8TpNA4mu4xFyCzZ2AZa6xbGMJ\nS77I0m//OBppLufMnJlzNDd9Xs8zj6wzZ878BPZ3fvqe7+/7M+ccIiJSO+rKPQAREYmXAruISI1R\nYBcRqTEK7CIiNUaBXUSkxiiwi4jUmFgCu5l90syeNrNdZvZVM5sdx3VFRKRwkQO7mZ0K/B7Q5Zw7\nH6gHboh6XRERKU5cqZgGoNnMGoAW4I2YrisiIgWKHNidc68DfwvsAd4EBp1zP4h6XRERKU5D1AuY\n2QnAh4FlwEHgG2a22jm3IeO8XqAXYM6cOcvPPvvsqG8tIjKjPPbYY/ucc4vynRc5sAPXAa845wYA\nzOx+4H1AWmB3zq0D1gF0dXW5HTt2xPDWIiIzh5n1hzkvjhz7HuByM2sxMwOuBZ6N4boiIlKEOHLs\njwD3AY8DOyeuuS7qdUVEpDhxpGJwzv0J8CdxXEtERKLRylMRkRqjwC4iUmMU2EVEaowCu4hIjVFg\nFxGpMQrsIiI1RoFdRKTGKLCLiNQYBXYRkRqjwC4iUmMU2EVEaowCu4hIjVFgFxGpMQrsIiI1RoFd\nRKTGKLCLiNQYBXYRkRqjwC4iUmMU2EVEaowCu4jkNzpa7hFIARTYRSS3vXuhrQ2uvRb+4i9g61Y4\nerTco5IcGso9ABGpcFu2wOHD8KMfeQ+A5cthx47yjksCacYuIrlt2pR97IorSj4MCU+BXURy8wvs\n3d2lHoUUQIFdRILt3QtPP519/OqrSz8WCU2BXUSCbdmSfey882DRotKPRUJTYBeRYErDVCUFdhEJ\npsBelRTYRcTfwIDy61VKgV1E/AXl1088sfRjkYIosIuIP6VhqpYCu4j4U2CvWrEEdjObb2b3mdlz\nZvasmWlZmkg1GxiAXbuyjyu/XhXi6hVzB/AfzrlfM7MmoCWm64pIOfjl1889V/n1KhE5sJtZG3A1\n8FEA59wx4FjU64pIGSkNU9XiSMUsAwaAL5vZE2b2L2Y2J/MkM+s1sx1mtmNgYCCGtxWRaaPAXtXi\nCOwNwCXAXc65i4Fh4NOZJznn1jnnupxzXYu0HFmkcu3b559fX7my9GORosQR2F8DXnPOPTLx/X14\ngV5EqpHy61UvcmB3zr0FvGpm7544dC3wTNTrikiZKA1T9eKqivk4kJioiHkZuDmm64pIqSmwV71Y\nArtz7kmgK45riUgZ7dsHO3dmH1f9elXRylMRmeKXXz/nHDjppNKPRYqmwC4iU5SGqQkK7CIyZfPm\n7GMK7FVHgV1khkokYOlSqKvzvn7j7v3w1FPZJ6p+veoosIvMQIkE9PZCfz8453395u3Kr9cKBXaR\nGWjtWhgZST92xbFN2ScqDVOVFNhFZqA9e7KPdbMp+6DSMFVJgV1kBuroSP9+Afu5COXXa4UCu8gM\n1NcHLSm7JlyNT3797LPh5JMLvnbmTdlEouhhSpEU2EVmoJ4eWLcOOjvBDD40b1P2SUXk1/1uyvb2\nKriXmgK7yAzV0wO7d8P4ONx8Wjz16343ZUdGvONSOgrsIjPdgQOx1a/73ZTNdVymhwK7SI3Lm/Pe\nssXLm6QqMr+eeVM2qa5O6ZhSUmAXqWGhct4x9odZtcr/+NiYcu2lpMAuUoXCVp6Eynn7BPZt9cWV\nOW7cGPyccu2lo8AuUmUKqTzJm/M+cADnk1+/6Z6VRc2u8+XSlWsvDQV2kSpTSOVJUM578viWLVhG\nfv053s3Lh9/FTTcVnjoJer+wz0s8FNhFqkwhlSeZC5HA+76vb+IbnzTMJrqB4vLifu/n+74yrRTY\nRapM3ll4isyFSJ2d3vc9PRMn+PRfTwZ2KDwvnvp+APX13tes95VpZS6zzKkEurq63I4dO0r+viK1\nIJljT03HtLQUETgPHICFC7NKHd/FG7zFuya/N/MWMUn5mdljzrm8+0trxi5SZfLOwnNIrab52Nlb\ns4L6c7w7LaiD8uLVSIFdpAqltgPYvTt8UE+tpjl3YFPWOdsaurOO9fdDQwPcdlvUUUupKLCLlEi5\nux5mVtP49V8/u3flZH481dgY3HWXgnu1UI5dpARiy4tHUFc3lXk5gQPsYyF1ZPz7f/11OOUUGhq8\nYJ6pvh6OH5/+sYo/5dhFKkiY2vPpntGn5spXsDU7qJ91FpxyCuAf1HMdl8qiwC5SAvlqz0vRxzy1\nxnwludv0JssUMwUdl8qiwC5SAvlqz0vRxzy1msZ3f9OUwN7b63+NoONSWRTYRUogaAXoqlVe2qW/\n3/91QTP9fGmbXM+3jr3Ne3gy65r3759q/HXnnbBmzdQMvb7e+/7OO4N/xihjkpg550r+WL58uROZ\naTZscK6z0zkz7+uaNc61tDjnJV/8H52d/tfJfF1Li3c81/PJ9/vPfDvrjZ7jrLRrFPOzFTOmYt9v\npgJ2uBAxVoFdpEw6O3MH9aDAF/S65IdA0PP19d7X/8kns578Jz4W+EES5WfJN6Zi32+mChvYlYoR\nKZNcLWxzrSbNdyM2KK2TrGjxy68n+8MU21Y36HX9/cWlmiQaBXaRMgm6odrZmXs1ab4bsWbB7zkf\n//z6ZlbmvHY+uV7X3x88JrUrmB6xBXYzqzezJ8zsu3FdU6SW5W2pW+TrXI41h1f71K+/wJm8wamR\n2urmatebHFNmcFcb3+kT54z9duDZGK8nUtOKbeYVpQnY1T7165vpjtxWN7Ndrx/nihuzFC6WlgJm\nthhYD/QBv++c+2Cu89VSQGT6LFwI+/f7P7eD5Szn8fSDiQT85m/G9v5BOfVkikmKV+qWAn8P/CEQ\n2LXZzHrNbIeZ7RgYGIjpbUUk0x13QFNT9vE2DnIxT2Q/sbK4jauDFJtikvhEDuxm9kFgr3PusVzn\nOefWOee6nHNdixYtivq2IhKgpwfuuSc97dHe7t8f5gXOpG7JqbEuGIqSKpJ4NMRwjSuBD5nZKmA2\n0GpmG5xzq2O4togUoacnPZAmErD/o5sgozPjZlbiUnrTJF8b9/tLaUWesTvnPuOcW+ycWwrcAPxI\nQV0kOr8l+MUuy+/pgZ7Fm7KOR9nfVCpXHDN2EYlZZv/2/n645RavsmR0dOpYby9s3w4bN3qLfTo6\nvFx21mz54EHa+7Pz68n69SQtGKoNsS5Qcs5tylcRIyL5+XV7PHZsKqgnjYzA3XeHaPe7bVtWgfuL\nnMHrLE47Vo4FQ2oOFj+tPBWpQIXMnDMrln1TKps2Zb1uW3132vflqFwpRR/6mUiBXaQCRZ05Z30w\n+AT2s3q7y165Uoo+9DORArtImeRKQfjVgjc1QWNj+rFQPVgOHoQnsvPrV352Jbt3w/j41MKhUqdE\n8jU0k+IosIuUQb4UhF8t+D33wJe/nH7s1ltDLAbats2L3qlOPx0WT+XXg8Zz222l24c1zHEJKUxv\n37gf6scuM12c/ckzN/DI6uH+qU9lv9Fv/3ao8Zhl94hfsybP+xU4dm3AER4h+7HH0iumUOoVIzNd\nXZ1/F0az7Ml1ZF1d8FjGwvB774XVU8tNgsbjxyz93JaWaPn5RMLLqecs1xQgfK8YBXaRMihZo6zB\nQViwIPvTYs8eWLIk73jCUoOv0ih1EzARKUDJGmUF5ddTgnrQeHJt2JFJNzsriwK7SBmUolFWIgH/\n9Jubsp/o7vYdz003QX299319Pbz//eGDvW52VhYFdpESySxvBNLKDeMO6r29cMnQpuwnfQJ7IgHr\n10/tizo2Bg895AX7gqtwpOyUYxcpgczeLxD9pmMuS5fC2/2DHGAB9ZnbJGTk15Pnh835p97sXLDA\nO3bggG58loJy7CIVpNQrLPfsgavYlhXUf85pLF2xZPK3huuug4aG4Bunfrnznh4v2N97Lxw+7O3W\npHYAlUWBXSQG+RpZlXqFZUcHdLMp6/gmutMWIT344FT6Jeg6QdQOoHIpsItEFKaRVbErLBMJbw9T\nM++xcGG4GXFfX3BgDytf7lztACqXArtIRGFmrrnKG4Nm+4mE14M9dWPq/fvh5pvzB/fGkUEuydy0\nmuz+60HCVOmoHUDlUmAXiSjMzDWovBGCZ/tr13o92DONjuZPd2z8o+2++fVXyR916+vDVelo0+rK\npcAuElHYmWvypmNqeWOu2X6ulaD50h3n7d2UdSxsGqa7O1zjL21aXbkU2EUiijJzLTZPnS/d8UtN\nm7KOBQX2uokoUF8P117r1a+H3fjC78NKyk+BXSSiKDPXYvPUuT40vv7PQ1xw7LGs40H59SVLvCB+\n/Di89JIqXWqBArtIDIqdueaa7SeX92cyC75+IgFf/3h2/frLLAvMr6f+dpDrNwjtTVo9FNhFyijX\nbL+31/81t97qfzyR8FoAXH50U9ZzufLrqb8dBP2msGCB9iatJgrsImUWNNu/805Ysya9MdeaNd7x\nTMla+rGxwuvXDx2aCtBBv0GAUjTVRL1iRGrAwoVejfs8hjjAAhpIX07ayW6G2zs5cgSGh7Nfn9q3\nxm/jixtvLOHGIBJIvWJEakyuhUzJRUxXsj0rqL/MMvbQyf79XlCv8/lXnzr79vsNQouRqosCu0gJ\nFXsDMlfbgtR0SJg0TNAMO1eJpRYjVRcFdpESCdNTJsjttwfnuFMDcpT+MLlm31qMVF0U2EWmgV/z\nrlzBOTmTN/Pa6JpNzehTUy2ZUnuiz2OI5YSvX09lln/2nZmiAZU/VirdPBWJWSLhNeoaHQ3/GjP/\nm5NNTTBvXnBgb2+HoSHvva7ne3yPVWnPv8JSTuOVUGMoJBT4bRzS2Aitrdp0Yzrp5qlImaxdW1hQ\nh+CgeuxYcFBPSr5XUBrG72Zpps7O/Oek8utxMzqqTTcqhQK7SMxK1Y+8vd2bHScFBfZrroENG6aC\nt9+G1Km17GGE+RlV514+CuwiMZuOEkC/ipQ77ph6r7m8E5hff/BB78+7d3uz6Xvv9T4UUu3f7/V+\nX7gwXM48mdfPR5tulIcCu0jM+vq8fHNc2tuDK1KSZYh+9euvsJR+lgLpM+eeHpg7N/t9kmmfOFMp\nqnMvj8iB3cyWmNmPzewZM3vazG6PY2Ai1aqnB7785fRZcXt79iw5jMZGb2aerEi59VZ47TVYvdqr\nntm+3esPc02eMsf+/vQgnavXe1KuVEpqCiiI6tzLpyGGaxwHPuWce9zM5gGPmdkPnXPPxHBtmSGc\nc5hf8rdK9fRkV4T4VZIkq2E6O6eC4O23T90wbW2dOve22+Cuu6a+Hxvzvp8zBx4IUb8e1FQsl6AP\ngI4O/+fq671ySFXFlJlzLtYH8G3gl3Kds3z5cieS6gv9/e6CRx91a55/3v3ft95yew4fLveQpsWG\nDc51djoHztXXe187O73jyedbWrzjqY/29uxjycdchtwo9VlPdPJK1rmdnVPvH+ZRXx/8c2SOs6Vl\n6ueQ6QHscCHicBwz9klmthS4GHjE57leoBegQ4k3ybD54EF2Dg+zc3iYu954A4DOWbNYMX8+V7W1\nsaKtjXNaWqp+Vp+cwabO3JP5bPAvI4TcJY9++fXddE7m11OFScGkGhvzP578OTKbhWmGXhliW6Bk\nZnOBzUCfc+7+XOdqgZKkGnOOBdu2MRQURSa0NzRw5USQv6qtjUvmzaMpTJF2hVm61D/AdnZ6QbLQ\nf5J/xaf5NF9MO/av3MTN/GvRY0xqb4d9+8Kf79cZUsE+PiVdoGRmjcA3gUS+oC6SadfwcN6gDrD/\n+HG+s38/f/Dyy1zxxBPM37aN9z/5JH/8yiv88MAB3jl+vASjzVZoY6+gEsBkD5lCrWRz1rGw/WHC\nCvMzRumFI/GKPGM373fj9cAB59wnwrxGM3ZJ9ZW33uKm556LfJ164D1z506mb65qa+OkpqboA8zB\n74Zoam/zzHPXri08HZLLHA5xkPlZqZilvOKbiimUmVf3HuZnzPWbSLK3jEQTdsYeR2C/CtgK7ITJ\njRY/65zbGPQaBXbJtPfYMbYPDrJ1cJBtg4M8/s475J/D53dmczMrUtI3pzc3Y2axpQzCBjO/D4BC\nNTV5teapPsD3+T7Xpx3bTSfL2E0ckqtVw/yMdXXajGO6hQ3skW+eOue2AdV9R0vK7sSmJn510SJ+\nddEiAA4dP87DQ0Nsmwj2Dw8NMVJEdHjx8GFePHyYe956C4CTm5roONjGE/+vjdHGNrC59Pfb5M3L\nQoN7rs2fIfosvb196sZpY6MX3A8dmno+SpvefJJ16Dfe6P985s8eVAKpWonSi7UqRiQucxsauG7B\nAq6bWLs+Oj7OE4cOTc7otw0Osq/QTlvAW8eO8VbLANw64B0YroenWxl5aj6f+kob//WGeTQnNxkN\nISiY1dVNteyN8ktx6kIgvy3t4g7sfnXoQR9MmQG7r88/ZaNFSqWntr1SlZxzPDcyMjmj3zY4yCtH\njkS+bqMZXfPmTaZv3tfWxoIc/QHiSLEUKyi/voyX2c2yoq7plzYp5D7Cbbd5x8fGvA+J3l7/zbel\nOCVLxYiUg5lxzpw5nDNnDh875RQAXjtyZHI2v3VwkJ3DwxQ6bRl1joeGhnhoaIi/fvVVAM6fM2cy\nR7+irY0ls2envaa5eSro1dWVLp/sV7/eTwe7I9w09UubhK1ZTyRg/fqp2vexMe/7K69UyWOpacYu\nNevg6Cg/GRqanNE/OjTEsRj+vnfMmsWKtjaanp/PVz/bxpHnW8CV/jbTX/IZPsMX0o6t57f4KOuL\nul5jo9fjptggrKqY6acZu8x48xsbWdXezqqJ7ltHxsb46TvvsG1wkK8/O8hT44O4lsJrb/YcPUpi\n7144YS/cBQw2wK422NkGT7XBi/PguP8SkZaW+NI2cdevR13Um+9GspSOZuwyY405x67hYbYePDiZ\nvnkjs56wGEfq4NlWL9DvbINnWmGkYbLR1+rVxV12zpypG6hzOMTbnEAj6YuyouTXIdrsWjP26acZ\nu0ge9WZcNHcuF82dy+8uXoxzjt1HjrA15Ybsc8VMr2ePw8UHvQdgDi6eN5cVbW08+lQb1j4ft7/w\nhVMLF04F9vfxk6ygHjW/DtFm135VMTC1O5Py7KWjwC4ywcxY1tzMsuZmfuvkkwEYOHYs7YZsMQun\nnMHjhw7x+KFD0P463Ae82uylb56amNW/3kyu5SB1delBN7jMMVo+JUrNeTJwp7YdBu/Pxa4TkOIo\nFSNSgOGxMR4eGppM32w7MMTRuhjKYPY3eQE+Gex/PhfG04N0Z+dUqmM77+N9PJT2/C18iS9zS9FD\nCCphLJRSMtOnZC0FiqHALrVidHycv7rvEH//4CBvLxmk7sJBxlsLXziVZbjey80nZ/TPttI+r579\n+4Pz66fxc17htKLeLpn/j2NGrdYC00c5dpES+Lev1vHF3lZGRlqBJYzjmH3WCPUXDzK8bBAuGIRT\nilg4NWcMLn3bewCMGvufnwe72jjzqZd5Z1czC955Z/L0PSxhN8vSbrC2t8PRo+ktCDKFmaUX2ldH\nrQXKT4FdJILsjTGMIy/MgRfmAN7CKRYe9QL8BQe9r6cNF94wu9HB+UNw/hBP3tBIO9/hvFdeYcVT\nT3HVrl3sfeoM3F5Laztw+LC3H+r69bm348sX1IM2BQl6nVoLlJ9SMSIRBKUdgrS3w3jLKG+/awgu\nHKThkkGOnz4ETTH8O/zFrKla+p1t0D+Hzg6jr6/4TpbF5ssTifSbqO3tU5tyS/GUihEpgaC0g5/G\nRvjIR2D9+kZ4tR0ebfey5E1jcNY7cOFE6ub8QZhbRNPik47CSXvhur3e94MN9D/dxnfntnH09Dbc\nG/Mo9FeFKIuODh+e+rMqY0pLM3aREILyzH4NsoI6Ora3w9y5/h8Eaa+pc7B0eCLQH8QuGsS1x7Bw\n6mgdPDuPhufa+P3r5rP2v7TS2pB7bhc0Y8+3ZZ4qY6aHqmJEYpKvu2Fm0A+awRffwtfByUfgwkG6\nLniAQxeM8VxyB4wI6oCL5s5N24jk5Fmz0s5JJODmmyGzQ3JTE9xzT/DsW5Ux00OBXSQmhc4+c81y\nUxfuFGMbV3IlP2GgrY3t55/P1gsv5CsXfIh9ZzZDQ/R/y2c0N092sVzR1sYZzc0sWmS+4841+9aM\nfXoosIvEpNDZZ9AMv64ud+lhPi0Mc5D5WfXrp/MSb56wlE/80xCzLh1k68GDPPjmEDRHnxqf1NjI\nLx6YuBm7cz68NAfGvTx96s+f+VvLqlXZ1ThxLYCayXTzVCQmhdZlB/UvL7b5V5Jff5hXWcxYx2n8\n818aPb9+AnACiQT8+JZxxpcdmiiznHicUPjCqV+MjsLKfd4DYMTbcYqdbZy4t42RsVa+9bX6rJLI\n9eu9UsuNG6PvKyuF04xdJI9CdhDKJWpb3L/gj/gj0ovBv9O2mt+bf2/OmbLHwZLDNHUd5Pg5g4yf\nOwinxrPjlL04j2M72qZaIgx5O04p7RI/zdhFYhJ2B6F8wuTYGxuhtdXb63TBgvTz/Rp/fXuwm/5B\n78/9/XD33f5po/p6Y/1ftbB2bQv930pZOHW+N5tv7DrI8SXDBe8XMuocnDHkPW7wdpxidws81Ub/\nrjb6j8ynM2PHKZl+mrGLlIhfhUl9PcyfPxXIwftzRweccQY8+KB3rIVh3uYEmkhPp5zOS7zM6Xnf\nO5kPz3W/YPG7j/Nq66BXZnn+IJwTz8KpJbNmTd6QvaqtjfPmzKEu6q8vM5Rm7CIlUEgfldSZf3+/\nF9THxrzadm/hUnqeOjWvfwUPZQX1V1nMyyGbfjnn9XPP/C0gqaMD9jzfAM5bOAVA4xi8+9BkLX3z\npYOM1BW+cOrVo0f56t69fHWvt3DqhIYGrkzZQ3b5vHnMqiu0x4LkosAuUqRi+qjAVGVMctPnXCmU\nJL80zGZWUkj/9f37vRl7UxOkbhSV7OOS/MCZNFrv5cx3teG+CrMXOh56ZXiyN/3Wgwd5vYgdp94+\nfpzv7t/Pdyc+YWbX1XHZvHmTM/r3tbXlXTgluSkVIzKh0C6GhdZq+92EDWsrV3EV29OO/Q7/zJf4\nncIvxtRvC6mNwMKMLzVcOOfoz9hx6tkYNnRNLpxKTd+8K2Ph1EylVIxIAfLNvv2CfiF9VBIJr/xv\nrIgWMC0McxmPZh2PsnH12Jh3ozb1wyv5NWxZppmxtLmZ7fc384O1J7NnDyw+9xi//mdD1F100Ntx\n6tAhjhc4eRwHnjh0iCcOHeL/vP46AKfPns2K+fMng/2Zzc2Y8vSBNGMXIffs268Nba72AJkz9rAz\n9aBrXssDPMAvpR17jVNZwqtE3QrPr+fLwoX+eXi/c/OVgg6PjfHI0NDkjP6hwUGGY+gpcGJjY9qM\n/j1z59IwA/L0WnkqUoBc1SKFdHD0q28P+tDIfN1NN8Fdd2U/9+d8js/x+bRjG+jhRjaEG1QemT93\nIgG33JKehw/qDVNoOmp0fJyfHTqUlr4ZyGxEU4Q5dXVckdIK4b2trbTU10e+bqVRKkakALlWl4Zp\nUQvBG1fke31qr/KNG7PHEbxx9fQopG6/0La+jXV1dLW20tXayieXLME5xwuHD3s3ZA966ZuXjxS+\ncGp4fJwH3n6bB972dpxqMGN5Mk8/kcJpb2ws+LrVSjN2EXKnFLKqRXzk6loYNKutr/dKHFMD5m23\npVfINDPCQeZnlTqeyYu8xBn5f7A88rXfzWc6mn29cfSot1H4xKz+Z4cOEUeUOqelhRVtbfz3U07h\nknnzYrhi6YWdsdd+UkokhJ4eL4h3dnpBurNzKqXS1+cF+Vxy7ee5alV2O4GWluygnkh4x1LnWn71\n669xKi+lLEpqb/cehWpq8n5TiMLvv03UbfBOmTWLj5x4Iv/7zDN5oquLt6+6iu9dcAGf7ehgRVsb\ns4q8afrsyAjr3nyTN4so0aw2sQR2M7vezJ43s5fM7NNxXFOk1Hp6vFnm+Lj3NbVaJBn0ITtINzZ6\ntel1dd4MNpGYes4vWJt5+fTM1Eb2/qnh6tcPH/YWOOX78EnV2Zm7n3pYuT4Q49LW0MD17e30nXYa\nWy6+mMEVK9h28cV84bTT+JUFC5hfQM27AVe2tsY3uErlnIv0AOqBnwOnAU3Az4Bzc71m+fLlTqRa\nbdjgXGenc2bOtbc719TknBe6vUdLi3eOc955qc8lH52d2df1O28LV2Ud/B3W+V5vwwbn6uv9r+M3\ntun+b5Mc03QbGx93T73zjvvH115zNzz9tDt1+3bHj3/s+7jw0Uenf0DTCNjhwsTlMCflvABcAXw/\n5fvPAJ/J9RoFdqkV+QK3mf/zZunX2bAh+9xmht1RGrNefAYvBL5nrqCeGminIwBv2OB9cJTyg8TP\n+Pi4e2VkxN375puu97nn3DmPPDIZ2G97/vnSDiZmYQN7HFUxpwKvpnz/GvDeGK4rUvHyVYWE7eW+\ndm122aFffv11TvG9aWqW+wZv6s3MYlsh5OOXShoZ8Y6Xsg97cuHU0uZmVp98MgD7jh1j+9AQHTNk\nBWvJbp6aWa+Z7TCzHQMDA6V6W5Fpk0h4eXU/ycDtd3OxqSk7J+/3AbGSzVnHvDLH7JuHmR8KqTJv\nZgYF4NWrs+8RhJFI5K7VD1suOp0WNjXx4YULubhKq2EKFUdgfx1YkvL94oljaZxz65xzXc65rkWL\nFsXwtiLxSwYpvxuhmef19vq3CEgNpJk3F9vbvSC8f7/3NTlbTrbsTRVH/brfzcxcgTY5nrDBPfnf\nIddvC7kqhmSahMnX5HrgLXJ6GVjG1M3T83K9Rjl2qURhcsTJ3HRQHru+PndOOei17e3p793MsDtC\nU9aJQfn1sDdoc40hzGsLvVY5cuy1jFLdPPXei1XAC3jVMWvzna/ALpUo341Qv8Cf76Zoplw3U1Or\nWq7hwayTXuMUB+OhgnqugBrHz5Hv58m8WSvxCBvYY8mxO+c2OufOcs6d7pyLsDRBpHzy3Qj1y01n\nypd2CHp+ubsNAAATjklEQVS+o8NLlyRXr+arX8/VBiVfLXlmXX4h4wx7XvJmrTavLg+tPBWZkCvo\nQv6bgKm59dRc/cKF3qOuzrtpmtmyJPm6RGJq8VOu/HpTk5fX9lvxuWFDekANGsfatd57btgQbeXo\ndKw8lRiEmdbH/VAqRkotTN12vhx7rnxyZo14vlRHXV366zZscK5xomR9NiO++fUzed7NmePcmjVT\nY0mmbvx+pnzjSP5sUWvay7EoaaailDn2Qh8K7FJKhSycyRWkwl4nzM1J8AJ5e7v3XqkrRrv5UdbJ\nr/Mut+bW8YJ+ljhvkkplUGAXmVDIsv58wsxOc91QDPP4U/4462CC33BNTd4HQdifJcw4wt4klcoQ\nNrArxy41r9Ce4bkENQpLFbVuOyi/fuyY/85G4NWRZ9aehxmHasxrkwK71Lx8N0XjFqbNb5DZHOZy\nHs46HmZhUnJhUb6VoEm6yVm7FNil5pW6ciNMOWGQy3mYWaT3C3+Tk3mRM/O+dmQEbr89/0rQpLjb\n60rlUGCXmjcdPcPztR5Ipmxy7Qnh10Y8uMwx3OYS+/fnr7UHr7WBgnrt0p6nMiP09MQXyArpjhjU\n3bGz06tpz8yZBwX25A5JQTn2QjQ0RN85SSqbZuwiBcrVnjZTrjTQgQPpx4Py61vrurnjDi8Y58rd\nt7SE2yKvyJ3lpIoosIsUqJAqm1xpoMybt+/lEd/8+rPjZ03+RpDZKbK9Pf26+YI/wOio/4eQ1A6l\nYkQKFHbzjKSgNFBfX3pKJ1d+PfkbQdj+K2vX5r6BWgk90mX6aMYuUqC4qmwyZ/PXz9qUdU5qmWPY\nYJy8cetccGWO6tdrmwK7SEjJSpgbb4Tm5uw0SK6ZdFAVzeSCp5EjeevXMzfjCLMpiJp0zVBhlqfG\n/VBLAak2UTZqzvXaZIuCbn6ctd7/TU5yqf3Xm5pyNxorpv+NVBdCthQw79zS6urqcjt27Cj5+4oU\nK2glZ+om0X4SCbjpJv8t9Nrb4fBhL8f+J/wpf8qfpT3/Nf4bv8HXsl6zb1/x45HqZmaPOee68p2n\nm6ciIRTTbybXvqiQXpMedn/T/fuDN7/ONx6ZOZRjF8ngl7supt9MmB2XAGaRP7+eed2gksZie9RI\nbVFgF0mRnGX393uZ6+Sq0lWrCr8JGXb2/F4eYTZH044dbjuJ53l34HUPH/a/VtBxmVkU2EVSBK0q\n3bix8H4zYUsK/dIwzdd3097uv0R0wYKpvVEzjY/nr5SR2qfALpIiV+46TC/2VGHb9/oFdrq7A1eR\n5usXk/nbhoL7zKPALpIizt7tyQVI9fXB58ziCFfwUPYTK1dmtf8tpsdLUA8bqW0K7CIp4l7Q09MT\nnDYB//w6J54IZ589+frdu73gXmxlsiplZh4FdpEU09G7PWi2X18P1wSkYTKn5/mCc2en2gfIFAV2\nkQyF5NKLXdZv5tW3/7JPfxi6u7MO5QrOyd8o/N6nqcnr+z4dN1PD/OxSJmGWp8b9UEsBqQXFLOsH\nb2k/ODeLw26E2VmtBM7hGdfZ6dyaNVOtANrbvZYCGae69vb090ttH9De7lxjY3FtEOL82SU+qKWA\nyPQqZll/6mtWsIUtrEx7/hecyMm8hd9WeI2N0NrqbdDR0TGV91+71kvVJI8lf8OYzrYDamlQHmFb\nCigVI1KkYpb1pz5X6P6mo6Pe12SKCPwXUyVTItPZdkAtDSqbArtIkYLy3gsWBOeeU18Ttj9MqmSv\nGMi/RV+cpZthr6EbtZVBgV2kSEE3K4eGgmfRfX3ejdOg+vXNGakZP8nAnW/WPJ292NXnvbIpsIsU\nya80ct68qZRJUuosuqfHC/iX8SjNHEk7by+LeJZz8r5vMnDnmzVPR+lm0nReW6JTYBeJILM08sAB\n//NSZ9ednfnz652dMHeu/7WSgTvMrLnQNgiFmM5rSzSRAruZ/Y2ZPWdmT5nZt8xsflwDE6lGYXLP\nfX1wbd2mrHM20U1TE2zY4AXKu+/OHbijzJpVg17jwtREBj2ADwANE3/+IvDFMK9THbvUqlD13UeO\nuNHG7Pr1a9/1dFYd+HRsa6ca9OpFqevYzexXgV9zzuWdL6iOXWpZIhFcWw7A1q1w9dVpr9lXt4gT\nx39BR6dlnx8z1aBXr3JsjXcL8PUcA+oFegE6VBMlNaynJ09g3rQp69CPxrtx2GQVTfI600E16LUv\nb47dzB4ws10+jw+nnLMWOA4EZuqcc+ucc13Oua5FixbFM3qRMomUo/YJ7Kn169Pdalc16LUv74zd\nOXddrufN7KPAB4FrXVx5HZEKltw+L7k4qKBZ9tGj8JOfZB3OrF+fztlzX1/6+EE16LUmalXM9cAf\nAh9yzoXYtlek+uVb8ZnTo4/CkfT69QEW8gznph3r6Ji+yhXVoNe+qHXs/wDMA35oZk+a2d0xjEmk\nokXKUfukYbbWd5PaH6alxds8O7MPzOrVXiCOI8irBr22RQrszrkznHNLnHPvmXjcGtfARCpVpBz1\n5s1Zh5as7s6aPW/cmP1bQZL2MpV8tPJUpEBF90kJyK9f+gfdWbPnfLN/7WUquSiwixSo6Bz1T38K\nhw+nH1u4kMQT52bl0sPM/vfs0QpS8aeNNkRK5fOfh899Lu1Q/2W/xrm7vpFVoXLTTV5LgVz/PNvb\nvc+JzNfqRmjt0kYbIjGKZWbsc+P0Sy+u9K2w2bgRbr01a0/rSclUUNHVOVLTFNhF8kjWrQf1WA8l\nIL/+rbe7fU/fswfuvBPuvddL9QDU13tfk6mfMJ0kZWZSYBfJI1LdelJAfv1Qx7m+p6f2VN+92/tA\nOX7c+5q8waoVpBJEgV0kj1h6q/ikYVi5ks//ZV3ROxGtWpWdqtEKUgEFdpG8YpkZ+9Sv091ddIVN\nIgHr16ffXDXzbrrqxqkosIvkEXl/z2PHYPv2rMMXfLybpUu9Pxe6CtQvPeScd9NVRIFdJI/IvVV8\n8usDLORpzi16Fala70ouCuwiIUTqreKTX9/MStzEP79iShR141RyUWAXmW4BgT1VoTPtyOkhqWkK\n7CLTKSC/nrqxBhQ+01brXcklzq3xRCSTT359H+08zXmT3zc2FjfTzrsFn8xYmrGLpIi9qZZf/3Wb\nyq9DcNsAkWIpsItM8GsdcOONcNttES7qU7/+I9ed9v2xY+rvIvFSYBeZEFQbfvfdRc7cQ+bXwfsQ\nUetdiYsCu8iEoMoU54qcUe/YkfVJcaAuPb+e+T7aHUnioMAuMiFXZUpRC3988uuHlq+kuSX3Pzu1\n3pWoFNhFJvT1Bd/ILGrhj09g71i9Mq1MMYhWkEoUCuwiE3p6/De3KGrhT0B+Pdn4K7mKNdlrPZNW\nkEoUCuwiKVI3t4i08Mcnv86CBXD++WmHtIJUpoMWKIlkiGXhj1+b3pUrvdKXjPcCL6e+Z483U+/r\n08IjiUaBXWQ6+G2s0d3te6pWkErclIoRidvoKGzbln08ILCLxE2BXSRuIfPrItNFgV0kbn5pmKuv\nzsqvi0wX/U0TiVsB+XWR6aDALhIn5delAiiwi8TJL79+wglwwQXlGY/MSArsInEKWb8uMp30t00k\nTsqvSwWIJbCb2afMzJnZwjiuJ1KVlF+XChE5sJvZEuADgPrRycz22GMwPJx+TPl1KYM4Zux/B/wh\n4GK4lkj1Uv26VIhIf+PM7MPA6865n4U4t9fMdpjZjoGBgShvK1KZlF+XCpG3CZiZPQCc7PPUWuCz\neGmYvJxz64B1AF1dXZrdS21Rfl0qSN7A7py7zu+4mV0ALAN+Zt7OBIuBx83sMufcW7GOUqTSPf64\nf379wgvLMx6Z0Ypu2+uc2wmcmPzezHYDXc65fTGMS6S6KL8uFUR/60Ti4BPY/3xLN4lEyUciEt9G\nG865pXFdS6SqBOTX73+7mxd7vT9rIw0pJc3YRaJ6/HE4dCjt0NvMZycXMDLibXsnUkoK7CJR+aRh\ntnA149QD3l6mIqWkwC4SlU9g30T35J87Oko3FBFQYBeJ5vhx3/x6MrC3tEBfX4nHJDOeArtIFD75\n9cG6+ezkQjo7Yd063TiV0outKkZkRvJJw7R98GqOf7u+9GMRmaAZu0gU6g8jFUgzdpEofvmXYWzM\ny7Mnt8RTYJcyU2AXieL2273H6KjXj33LFvWHkbJTYBeJQ2MjXH659xApM+XYRURqjAK7iEiNUWAX\nEakxCuwiIjVGgV1EpMYosIuI1BgFdhGRGqPALlKhEglYutTbNnXpUrTNnoSmBUoiFSiRgN7eqS4F\n/f3e96BukZKfZuwiFWjt2qmgnqRt9iQsBXaRChS0nZ622ZMwFNhFKlDQdnraZk/CUGAXqUB9fd62\neqm0zZ6EpcAuUoF6erxt9To7wQxtsycFUVWMSIXq6VEgl+Joxi4iUmMU2EVEaowCu4hIjVFgFxGp\nMQrsIiI1RoFdRKTGKLCLiNSYyIHdzD5uZs+Z2dNm9tdxDEpERIoXaYGSmV0DfBi4yDl31MxOjGdY\nIiJSrKgz9jXAF5xzRwGcc3ujD0lERKKIGtjPAlaY2SNmttnMLg060cx6zWyHme0YGBiI+LYiIhIk\nbyrGzB4ATvZ5au3E6xcAlwOXAv9mZqc551zmyc65dcA6gK6urqznRUQkHnkDu3PuuqDnzGwNcP9E\nIH/UzMaBhYCm5CIiZRI1FfPvwDUAZnYW0ATsizooEREpXtS2vfcA95jZLuAYcJNfGkZEREonUmB3\nzh0DVsc0FhERiYFWnoqI1BgFdhGRGqPALiJSYxTYRURqjAK7iEiNUWAXEakxCuwiIjVGgV1EpMYo\nsIuI1BgFdhGRGqPALiJSYxTYRURqjAK7iEiNUWAXEakxCuwiIjVGgV1EpMYosIuI1BgFdhGRGqPA\nLiJSYxTYRURqjAK7iEiNUWAXEakxCuwiIjVGgV1EpMYosIuI1BgFdhGRGqPALiJSYxTYRURqjAK7\niEiNUWAXEakxCuwiIjUmUmA3s/eY2cNm9qSZ7TCzy+IamIiIFCfqjP2vgT9zzr0H+OOJ70VEpIyi\nBnYHtE78uQ14I+L1REQkooaIr/8E8H0z+1u8D4n3RR+SiIhEkTewm9kDwMk+T60FrgU+6Zz7ppl9\nBPgScF3AdXqB3olvj5rZruKGXFILgX3lHkQIGmd8qmGMoHHGrVrG+e4wJ5lzruh3MLNBYL5zzpmZ\nAYPOudYQr9vhnOsq+o1LROOMVzWMsxrGCBpn3GptnFFz7G8AKyf+/H7gxYjXExGRiKLm2D8G3GFm\nDcARplItIiJSJpECu3NuG7C8iJeui/K+JaRxxqsaxlkNYwSNM241Nc5IOXYREak8aikgIlJjyhbY\nq6kdgZl93MyeM7OnzaxiV9ea2afMzJnZwnKPxY+Z/c3Ef8enzOxbZja/3GNKZWbXm9nzZvaSmX26\n3OPxY2ZLzOzHZvbMxN/H28s9piBmVm9mT5jZd8s9liBmNt/M7pv4e/msmV1R7jH5MbNPTvz/3mVm\nXzWz2bnOL+eMvSraEZjZNcCHgYucc+cBf1vmIfkysyXAB4A95R5LDj8EznfOXQi8AHymzOOZZGb1\nwD8C/wk4F/gNMzu3vKPydRz4lHPuXOBy4H9U6DgBbgeeLfcg8rgD+A/n3NnARVTgeM3sVOD3gC7n\n3PlAPXBDrteUM7BXSzuCNcAXnHNHAZxze8s8niB/B/wh3n/XiuSc+4Fz7vjEtw8Di8s5ngyXAS85\n5152zh0Dvob3gV5RnHNvOucen/jzO3iB6NTyjiqbmS0GfgX4l3KPJYiZtQFX4y2sxDl3zDl3sLyj\nCtQANE9UILaQJ16WM7B/AvgbM3sVbxZcMbO3DGcBK8zsETPbbGaXlntAmczsw8DrzrmflXssBbgF\n+F65B5HiVODVlO9fowIDZiozWwpcDDxS3pH4+nu8icZ4uQeSwzJgAPjyRMroX8xsTrkHlck59zpe\njNwDvIm3EPQHuV4TtY49p7jaEUy3PONsABbg/dp7KfBvZnaaK3E5UZ4xfhYvDVN2ucbpnPv2xDlr\n8VIKiVKOrZaY2Vzgm8AnnHND5R5PKjP7ILDXOfeYmXWXezw5NACXAB93zj1iZncAnwY+V95hpTOz\nE/B+e1wGHAS+YWarnXMbgl4zrYHdORcYqM3sK3g5OIBvUMZf2fKMcw1w/0Qgf9TMxvH6SgyUanwQ\nPEYzuwDvf/jPvK4OLAYeN7PLnHNvlXCIQO7/lgBm9lHgg8C1pf5wzON1YEnK94snjlUcM2vEC+oJ\n59z95R6PjyuBD5nZKmA20GpmG5xzq8s8rkyvAa8555K/8dyHF9grzXXAK865AQAzux+v4WJgYC9n\nKqZa2hH8O3ANgJmdBTRRQc2CnHM7nXMnOueWOueW4v1lvaQcQT0fM7se79fzDznnRso9ngw/Bc40\ns2Vm1oR3c+o7ZR5TlomeTF8CnnXO/a9yj8ePc+4zzrnFE38fbwB+VIFBnYl/I6+aWbKx1rXAM2Uc\nUpA9wOVm1jLx//9a8tzkndYZex7V0o7gHuCeiW6Ux4CbKmymWU3+AZgF/HDit4uHnXO3lndIHufc\ncTP7XeD7eFUH9zjnni7zsPxcCdwI7DSzJyeOfdY5t7GMY6pmHwcSEx/mLwM3l3k8WSbSRPcBj+Ol\nMJ8gzwpUrTwVEakxWnkqIlJjFNhFRGqMAruISI1RYBcRqTEK7CIiNUaBXUSkxiiwi4jUGAV2EZEa\n8/8B5KhPOvR3IcEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111b5bf98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(6,6))\n",
    "plot_pca(X2, V, m2, L)\n",
    "plt.xlim([-8, 8])\n",
    "plt.ylim([-8, 8])\n",
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.3613842,  0.932417 ],\n",
       "       [ 0.932417 , -0.3613842]])"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-10, 10)"
      ]
     },
     "execution_count": 213,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAD8CAYAAABgmUMCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGrxJREFUeJzt3X+MHOddx/HPx2ebYiciycXN77ukUpQ2QRCSk2lLQIWk\nJTFVQ1EBR6GkgHTYbRGVQChwUlUJVaJAQZSWGrcE0tg0BUHa0DotSVWUIjVtz5HjOL+IE+zErps4\nrsiPJjSx/eWPmcWb9czuPLsz+8P3fkkj787MzX5v1vd8dmaeedYRIQAAqlo26gIAAJOF4AAAJCE4\nAABJCA4AQBKCAwCQhOAAACSpJThs32T7adu72uadZvtO24/m/55a8rNX237E9m7bN9ZRDwCgOXUd\ncfyDpKs75t0o6asRcaGkr+bPX8X2lKRPSLpG0sWSrrN9cU01AQAaUEtwRMTdkr7XMftaSTfnj2+W\n9IsFP7pW0u6IeDwiXpZ0a/5zAIAxtbzBbZ8REQfyx9+VdEbBOudIerLt+T5JP1m0MdvzkuYlafXq\n1Ze//vWvr7FUADjxbd++/ZmIWDPodpoMjv8XEWF7oLFNImKzpM2SNDc3F4uLi7XUBgBLhe29dWyn\nyV5VT9k+S5Lyf58uWGe/pPPanp+bzwMAjKkmg+N2STfkj2+Q9IWCdb4t6ULbF9heKWl9/nMAgDFV\nV3fcz0r6hqSLbO+z/VuS/kTSW20/Kumq/Llsn217myRFxGFJ75f0FUkPSfqniHigjpoAAM2o5RpH\nRFxXsujKgnW/I2ld2/NtkrbVUQcAoHncOQ4ASEJwAACSEBwAgCQEBwAgCcEBAEhCcAAAkhAcAIAk\nBAcAIAnBAQBIQnAAAJIQHACAJAQHACAJwQEASEJwAACSEBwAgCQEBwAgCcEBAEhCcAAAkjQaHLYv\nsr2jbXrO9gc61nmL7Wfb1vlgkzUBAAZTy3eOl4mIRyRdKkm2pyTtl3Rbwapfj4i3N1kLAKAewzxV\ndaWkxyJi7xBfEwBQs2EGx3pJny1Z9mbbO23fYfuSIdYEAEg0lOCwvVLSOyT9c8HieyXNRMSPSfpr\nSZ8v2ca87UXbiwcPHmyuWABAV8M64rhG0r0R8VTngoh4LiJeyB9vk7TC9ukF622OiLmImFuzZk3z\nFQMACg0rOK5TyWkq22fadv54bV7ToSHVBQBI1GivKkmyvVrSWyX9dtu8DZIUEZskvUvSRtuHJb0k\naX1ERNN1AQD603hwRMT3JU13zNvU9vjjkj7edB0AgHpw5zgAIAnBAQBIQnAAAJIQHACAJAQHACAJ\nwQEASEJwAACSEBwAgCQEBwAgCcEBAEhCcAAAkhAcAIAkBAcAIAnBAQBIQnAAAJIQHACAJAQHACAJ\nwQEASEJwAACSNB4ctvfYvt/2DtuLBctt+2O2d9veafuypmsCAPRv+ZBe52cj4pmSZddIujCfflLS\nJ/N/AQBjaBxOVV0r6TORuUfSKbbPGnVRAIBiwwiOkHSX7e225wuWnyPpybbn+/J5r2J73vai7cWD\nBw82VCoAoJdhBMcVEXGpslNS77P9M/1sJCI2R8RcRMytWbOm3goBAJU1HhwRsT//92lJt0la27HK\nfknntT0/N58HABhDjQaH7dW2T249lvQ2Sbs6Vrtd0q/nvaveKOnZiDjQZF0AgP413avqDEm32W69\n1j9GxJdtb5CkiNgkaZukdZJ2S3pR0m80XBMAYACNBkdEPC7pxwvmb2p7HJLe12QdAID6jEN3XADA\nBCE4AABJCA4AQBKCAwCQhOAAACQhOAAASQgOAEASggMAkITgAAAkITgAAEkIDgBAEoIDAJCE4AAA\nJCE4AABJCA4AQBKCAwCQhOAAACQhOAAASRoNDtvn2f6a7QdtP2D7dwvWeYvtZ23vyKcPNlkTAGAw\njX7nuKTDkn4vIu61fbKk7bbvjIgHO9b7ekS8veFaAAA1aPSIIyIORMS9+ePnJT0k6ZwmXxMA0Kyh\nXeOwfb6kn5D0zYLFb7a90/Ydti8p+fl524u2Fw8ePNhgpQCAboYSHLZPkvQvkj4QEc91LL5X0kxE\n/Jikv5b0+aJtRMTmiJiLiLk1a9Y0WzAAoFTjwWF7hbLQ2BoR/9q5PCKei4gX8sfbJK2wfXrTdQEA\n+tN0rypL+jtJD0XEX5Ssc2a+nmyvzWs61GRdAID+Nd2r6qckvVvS/bZ35PP+SNKMJEXEJknvkrTR\n9mFJL0laHxHRcF0AgD41GhwR8Z+S3GOdj0v6eJN1AADqw53jAIAkBAcAIAnBAQBIQnAAAJIQHACA\nJAQHACAJwQEASEJwAACSEBwAgCQEBwAgCcEBAEhCcAAAkhAcAIAkBAcAIAnBAQBIQnAAI7B1q3T+\n+dKyZdm/W7eOuiKguqa/ARBAh61bpfl56cUXs+d792bPJen660dXF1AVRxzAkC0sHAuNlhdfzOYD\nk6Dx4LB9te1HbO+2fWPBctv+WL58p+3Lmq4JGKUnniiev3cvp64wGRoNDttTkj4h6RpJF0u6zvbF\nHatdI+nCfJqX9Mkma8JkOJGvAczMlC+LOHbq6kT6nXFiafqIY62k3RHxeES8LOlWSdd2rHOtpM9E\n5h5Jp9g+q+G6MMZa1wD27j0xG9IPf1hatar7Ov2cujqRwxbjpengOEfSk23P9+XzUteR7Xnbi7YX\nDx48WHuhGI2ixm5U1wCG1fBef720ebM0OyvZ5euVndIqcqKHLcbLxFwcj4jNETEXEXNr1qwZdTmo\nQVljt3dv8fopDWldtTQZHnv2SEePZgFSpHVKq0qgccEdw9R0cOyXdF7b83PzeanrYAKkfmIva+ym\nporX73ZtYFDDaHjL9k/RqatVq7L5VQOtLFSbDFssYRHR2KTsPpHHJV0gaaWk+yRd0rHOL0i6Q5Il\nvVHSt3pt9/LLLw+Mly1bIlatisiat2PT9HS2rIh9/PqtqXNbq1aVb6eoltnZbPuzs9V+rqwWu+oe\n6F1Tt9+prObZ2eK6Zmdfvf3p6eL1pqfrqR8nBkmLUUfbXsdGur6AtE7Sf0l6TNJCPm+DpA35Yyvr\nefWYpPslzfXaJsExXFUa4rIGrluj361R7Kfxb9XaT+iU1TI93buOQfZPZwB0qhpoBAeqmJjgaGIi\nOIanakPc7eihKAympyNe85pqIdPZMG/cWN5QpzTQnfWsXPnqn1m5MmLFiuLGuP1IYdD9075u5+9a\nFgidv0/TR0w4MRAcGEjVT/RVG+JuRxxlp5/KGrqNG19dZ5WfazXmKQ1053ZXrMi206vRbq170knl\ny1P2Tytotmw5PqiKfp+iYOr3iAZLC8GBvqWczunWwPfa5qBTr8a7qKZuy1vB0Os1W3ptr9eUsn+q\n/q6dwdrPe1qHfk8lYrQIDvSt26ff9kag2yf4qaljn5LbT/fUGRyjmNoDsdsRRa9paur4/b5xYz01\nll23GFZjPuyQQn0IDvSt1yfpViPQ6/TTihXHXxeY9Kl1ymuQ0GhN7eo+Iis66ujUVJBwWmxy1RUc\nzrY1Webm5mJxcXHUZUys888vv8muZXY2uwegrv8eZ58tHThQ3/aaMjWVTS+/PPi2ZmezezEk6YYb\npCNHBt9miy3dckv5MOydQ7dL2b0hmzdnjxcWsvd3ZiarMWU492XLit9HO7uhEePL9vaImBt4OwTH\n0lPUqHSys0alV8BgdGZns7vPi5x+unTo0PHzp6ell14qDpSq4VH2waNbPRgPdQXHxAw5gvq0j5VU\npvVJtNdgfBidzrvCW3em28WhIWXzB71Dft26tPk48RAcS1RrrKQtW8qHu2gFzPT0SEpED+1DsLQP\nTdKPlKFJtm1Lm48TD8GxxHWO1Do7e/xpi5deaua1u40Mi1fr3FetcG8pGmurSNmHgGXLqg/oyLhY\nIDiWsNapjXe/W3rhBem007I//oWFY41I1QYp1apV0oYNWYOF7mZnswvh3cK9SqM9PS391V8Vn348\nciQ7Ynnve3sPVFk22GSTg1BizNTRNWvYE91xB9ere2irS+6gN8EVTVNTx7qTjrr77bhPVe+P6NV1\nunNAxamp4vU63++yIWA673BfsYL7OCaBauqOy+e9JarXkUTrgmkTnyKPHJFuvrnaqZFJO521enW9\nR1FVezsVdWRo7bupqez9vOGGbN7CQnnX4OjoZFl24bzzfZm09wmDoTvuElXWF7/T9LT0/PP13NfQ\nqe57RU40qd1bW9+e+MQT2WnH//1f6fvfH7yOzvsz6I47ueiOi4FUPZI4dChr2Ju4FkFolLPTu7e2\nesrdckvWoaGO0JCO/7/CxXEQHEtUyj0ar7winXqqtHJlvTXMzHS/l2Qpizh2Oq/smwPL5tfZoaEo\nwLg4joEvkoxi4uJ4PToHKOw2SKGdrV/XRd/WKK91jAl1Ik/T08UDCm7cWD7Q4CAdGlav7n2BnEEO\nJ5cYq4prHE3odf56+fLBx1yypTe8QXroIU5V9Wtqqvh9aB3B9boRcOXKbN+/8sqxeatWST/8w8V3\nnXdev2i/ntLPeFcYDa5xoBHr1nW/2WzQ0Jiayu7fIDQGU/Y+PPFE+WnI1nWq2Vnpppukv//74+8N\n+d73yrfbrnU95ejR7F9CY2lpLDhs/5nth23vtH2b7VNK1ttj+37bO2xzGDEkRefHt27Nzqu3N+h2\n1o2z1TAMek3i6NFsaIomQ2PFiua2PS6mpornz8wUjwawZUsWNhFZsCwsZDd+StnF9Fbjz/ULVFLH\n+a6iSdLbJC3PH39E0kdK1tsj6fSUbXONYzBl56irfL/1oF9G1Lqm0tQ1gZUrJ+8LpVpfV9vrOlP7\ne3XlldVu1qv63qd+hzomk8b9BsCI+PeIOJw/vUfSuU29FtIU9bp58cXyEVXbT1MMMpDdqlXZqbAm\nhxk5erT89ygyPT36I5STTsrqfuaZbOp2M93sbHYE+I1vHH9k+KY3Ze9t0XAhrSPMX/u17qPjVhm7\nDBjKxXHb/ybpcxGxpWDZf0t6VtIRSX8bEZtLtjEvaV6SZmZmLt/LF0X0rerNfy3tF0YHvUPYHp9r\nG9PTWUP93vdKmzbVW9fUlPSa11S7lyL1Bruy5Z37tv2Lm6p8/wpfwnTiq+vi+PIBi7hL0pkFixYi\n4gv5OguSDksqG2DiiojYb/u1ku60/XBE3N25Uh4om6WsV9UgdS91ZV/QVPYlP+2jsJb15qmqzsZ5\n2bLBGrtDh7IGc2qq/jA7cqT6DXid1w8+/OHihv6FF7Ijh7Ib7Tp/h/YjiV73dXANAykGOmkQEVdF\nxI8WTK3QeI+kt0u6PkoObSJif/7v05Juk7R2kJrQW1Gvm1WrspFTe52mqPPrT9sVnTKypSuvLD/K\nqesTclO/UxWdwSyVfw/KoUNZoJx2WvXtP/FE7zu6i2oAummyV9XVkv5A0jsiovDzju3Vtk9uPVZ2\nQX1XUzUtVZ09qKTygOjVzbJbr6p+T2PNzmanjDq7h27YIO3ePT6ntuqwYkUWCL2uH1x/fXbto1Pr\nyKFsQMNOMzPdjya4hoG+1HGFvWiStFvSk5J25NOmfP7Zkrblj18n6b58ekDZKa6e26ZXVXV195Lp\nNRx76tS6C7rV22p2NnuNKq9TdFd153Dfo56WLTvWa6r1u1VV1vusdRd/+z7rdic5PaXQopp6VTUW\nHE1OBEd1Zd/T0N7FNlW373OoMrUaxNY2irqV9uqW2t4oFoVOHQGyenX5sunprOtv1XCs0k228/dI\nfe+KtlFlGZYOggOVdPvU2sR2yxrZXp+OU6YqDV+3hrfqa0RktbYCrv0LqNpfo/33KgvUbkFddkTQ\n7SgC6AfBgUqaOOLott0qN6XV0aBX1c/NhoM0zv0Edbf3aNAjBY400I7gQCVNnd/u9im5V0NVpTEv\nGxU2te7UkJqeHmzf9BPUTR0Vcm0DnQgOVNbUp85+t1v1+7HrqLuo8ewWXL2OaHrV1E9jPeyjwkG3\ni8lFcGBidWvMmzidUtTY9/Mpv2oopAZeU0cGTR3JYHIRHJhooz733s+n8SY/wTexPzjiQKe6goMv\ncsKStHXr8cN6tMZ2KrsZrmyMr3Ed56mf3xEnNr7ICRhAP6PATtp3VTDSLZrCEQdQEZ/gMek44sDE\nKvr2wUnAJ3ggM9Cw6kCqzk/te/dmz6XJaIBbA0ECSxlHHBiqsm8fbH1vBIDxR3BgqMq+G6LXd0YA\nGB8EB4Zq0nomATgewYGhKvv2Qb6BDpgcBAeGip5JwOSjVxWGjp5JwGTjiAMAkKSx4LD9Idv7be/I\np3Ul611t+xHbu23f2FQ9AIB6NH2q6i8j4s/LFtqekvQJSW+VtE/St23fHhEPNlwXAKBPoz5VtVbS\n7oh4PCJelnSrpGtHXBMAoIumg+N3bO+0fZPtUwuWnyPpybbn+/J5x7E9b3vR9uLBgwebqBUAUMFA\nwWH7Ltu7CqZrJX1S0uskXSrpgKSPDvJaEbE5IuYiYm7NmjWDbAoAMICBrnFExFVV1rP9KUlfLFi0\nX9J5bc/PzecBAMZUk72qzmp7+k5JuwpW+7akC21fYHulpPWSbm+qJgDA4JrsVfWnti+VFJL2SPpt\nSbJ9tqRPR8S6iDhs+/2SviJpStJNEfFAgzUBAAbUWHBExLtL5n9H0rq259skbWuqDgBAvUbdHRcA\nMGEIDgBAEoIDAJCE4AAAJCE4AABJCA4AQBKCAwCQhOAAACQhOAAASQgOAEASggMAkITgAAAkITgA\nAEkIDgBAEoIDAJCE4AAAJCE4AABJCA4AQJLGvjrW9uckXZQ/PUXS/0TEpQXr7ZH0vKQjkg5HxFxT\nNQEABtfkd47/auux7Y9KerbL6j8bEc80VQsAoD6NBUeLbUv6FUk/1/RrAQCaN4xrHD8t6amIeLRk\neUi6y/Z22/NDqAcAMICBjjhs3yXpzIJFCxHxhfzxdZI+22UzV0TEftuvlXSn7Ycj4u6C15qXNC9J\nMzMzg5QNABiAI6K5jdvLJe2XdHlE7Kuw/ockvRARf95tvbm5uVhcXKynSABYImxvr6MDUtOnqq6S\n9HBZaNhebfvk1mNJb5O0q+GaAAADaDo41qvjNJXts21vy5+eIek/bd8n6VuSvhQRX264JgDAABrt\nVRUR7ymY9x1J6/LHj0v68SZrAADUizvHAQBJCA4AQBKCAwCQhOAAACQhOAAASQgOAEASggMAkITg\nAAAkITgAAEkIDgBAEoIDAJCE4AAAJCE4AABJCA4AQBKCAwCQhOAAACQhOAAASQgOAEASggMAkGSg\n4LD9y7YfsH3U9lzHsj+0vdv2I7Z/vuTnT7N9p+1H839PHaQeAEDzBj3i2CXplyTd3T7T9sWS1ku6\nRNLVkv7G9lTBz98o6asRcaGkr+bPAQBjbKDgiIiHIuKRgkXXSro1In4QEf8tabektSXr3Zw/vlnS\nLw5SDwCgecsb2u45ku5pe74vn9fpjIg4kD/+rqQzyjZoe17SfP70B7Z31VFow06X9Myoi6iAOusz\nCTVK1Fm3Sanzojo20jM4bN8l6cyCRQsR8YU6ipCkiAjb0WX5Zkmb85oWI2KubN1xQZ31moQ6J6FG\niTrrNkl11rGdnsEREVf1sd39ks5re35uPq/TU7bPiogDts+S9HQfrwUAGKKmuuPeLmm97R+yfYGk\nCyV9q2S9G/LHN0iq7QgGANCMQbvjvtP2PklvkvQl21+RpIh4QNI/SXpQ0pclvS8ijuQ/8+m2rrt/\nIumtth+VdFX+vIrNg9Q9RNRZr0mocxJqlKizbkuqTkeUXlYAAOA43DkOAEhCcAAAkoxtcEzicCa2\nP2d7Rz7tsb2jZL09tu/P16ule1xinR+yvb+t1nUl612d7+Pdtod6V7/tP7P9sO2dtm+zfUrJeiPZ\nl732jTMfy5fvtH3ZsGprq+E821+z/WD+t/S7Beu8xfazbf8XPjjsOvM6ur6PY7I/L2rbTztsP2f7\nAx3rjGR/2r7J9tPt97dVbQP7+juPiLGcJL1B2c0q/yFprm3+xZLuk/RDki6Q9JikqYKf/1NJN+aP\nb5T0kSHX/1FJHyxZtkfS6SPctx+S9Ps91pnK9+3rJK3M9/nFQ6zxbZKW548/Uvb+jWJfVtk3ktZJ\nukOSJb1R0jdH8D6fJemy/PHJkv6roM63SPrisGtLfR/HYX8W/B/4rqTZcdifkn5G0mWSdrXN69kG\n9vt3PrZHHDHBw5nYtqRfkfTZYb1mA9ZK2h0Rj0fEy5JuVbZPhyIi/j0iDudP71F2L9C4qLJvrpX0\nmcjcI+mU/F6loYmIAxFxb/74eUkPqXgEh0kw8v3Z4UpJj0XE3hHW8P8i4m5J3+uYXaUN7OvvfGyD\no4tzJD3Z9nzg4Uwa8NOSnoqIR0uWh6S7bG/Ph1IZhd/JD/lvKjmErbqfh+E3lX3aLDKKfVll34zT\n/pPt8yX9hKRvFix+c/5/4Q7blwy1sGN6vY9jtT+VDeJa9sFwHPanVK0N7Gu/NjVWVSUek+FMUlSs\n+Tp1P9q4IiL2236tpDttP5x/YqhNtzolfVLSHyv7Y/1jZafVfrPO16+iyr60vSDpsKStJZtpfF9O\nOtsnSfoXSR+IiOc6Ft8raSYiXsivdX1e2Q27wzYx76PtlZLeIekPCxaPy/58lTrbQGnEwRETOJxJ\nr5ptL1c21PzlXbaxP//3adu3KTtcrPWPpOq+tf0pSV8sWFR1P/etwr58j6S3S7oy8hOyBdtofF8W\nqLJvGt9/VdheoSw0tkbEv3Yubw+SiNhm+29snx4RQx2wr8L7OBb7M3eNpHsj4qnOBeOyP3NV2sC+\n9usknqoa9+FMrpL0cETsK1poe7Xtk1uPlV0EHupIvx3nht9Z8vrflnSh7QvyT1jrle3TobB9taQ/\nkPSOiHixZJ1R7csq++Z2Sb+e9wZ6o6Rn204bDEV+re3vJD0UEX9Rss6Z+XqyvVZZm3BoeFVWfh9H\nvj/blJ5RGIf92aZKG9jf3/mwr/4n9BJ4p7LzbT+Q9JSkr7QtW1DWE+ARSde0zf+08h5YkqaVfTnU\no5LuknTakOr+B0kbOuadLWlb/vh1ynou3CfpAWWnZYa9b2+RdL+knfl/krM668yfr1PWE+exYdep\nrNPDk5J25NOmcdqXRftG0obWe6+s988n8uX3q61n4BBrvELZ6cidbftxXUed78/33X3KOiG8eQR1\nFr6P47Y/8zpWKwuCH2mbN/L9qSzIDkh6JW83f6usDazj75whRwAASSbxVBUAYIQIDgBAEoIDAJCE\n4AAAJCE4AABJCA4AQBKCAwCQ5P8AKkeukpaKzeYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110cb9390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "newX2 = (X2) @ V\n",
    "\n",
    "plt.plot(newX2[:, 0], newX2[:, 1], 'ob')\n",
    "plt.xlim([-10, 10])\n",
    "plt.ylim([-10, 10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Review Standardization (Input Transformation)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-10, 10)"
      ]
     },
     "execution_count": 200,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAD8CAYAAABgmUMCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHEpJREFUeJzt3X2MXfV95/H3x2M7zRi6xPaU5xmIisiSVcPCiGYDjWhD\nsmBlQ1N1s0ZDQkolx0AqR9pol8pShFRZatqmEmkCyKG0BE9DWTUkKHVCIdsVLVtaxgiIeSqGtYMd\nAsaoGGMaMP7uH+fc+vj6nHvPmXvPnXPnfl7S1ZyH3z335zOe872/Z0UEZmZmZS1Z6AyYmdlwceAw\nM7NKHDjMzKwSBw4zM6vEgcPMzCpx4DAzs0r6Ejgk3SbpZUnbM8dWSrpP0rPpz/cUvPdSSc9I2iHp\n+n7kx8zM6tOvEsefA5e2Hbse+GFEnAX8MN0/iqQx4OvAZcA5wBWSzulTnszMrAZ9CRwR8QDwatvh\ny4Hb0+3bgV/PeesFwI6IeD4i3gLuTN9nZmYNtbTGa58YES+m2z8FTsxJcyrwQmZ/N/DLeReTtA5Y\nB7BixYrz3/e+9/Uxq2Zmi9+2bdteiYiJXq9TZ+D4NxERknqa2yQiNgObAaanp2Nubq4veTMzGxWS\ndvXjOnX2qnpJ0skA6c+Xc9LsAU7P7J+WHjMzs4aqM3DcA1yVbl8FfDcnzcPAWZLOlLQcWJu+z8zM\nGqpf3XG/BfwDcLak3ZJ+G/h94KOSngUuSfeRdIqkrQARcQj4PHAv8BRwV0Q80Y88mZlZPfrSxhER\nVxSc+khO2p8AazL7W4Gt/ciHmZnVzyPHzcysEgcOMzOrxIHDzMwqceAwM7NKHDjMzKwSBw4zM6vE\ngcPMzCpx4DAzs0ocOMzMrBIHDjMzq8SBw8zMKnHgMDOzShw4zMysEgcOMzOrxIHDzMwqceAwM7NK\nHDjMzKwSBw4zM6uk1sAh6WxJj2Ze+yV9oS3NxZJey6T5Up15MjOz3vRlzfEiEfEMcC6ApDFgD3B3\nTtK/i4iP15kXMzPrj0FWVX0EeC4idg3wM83MrM8GGTjWAt8qOPchSY9L+r6k9w8wT2ZmVtFAAoek\n5cAngP+Vc/oRYDIifgn4E+A7BddYJ2lO0tzevXvry6yZmXU0qBLHZcAjEfFS+4mI2B8RB9LtrcAy\nSatz0m2OiOmImJ6YmKg/x2ZmlmtQgeMKCqqpJJ0kSen2BWme9g0oX2ZmVlGtvaoAJK0APgp8LnNs\nPUBE3AL8JnCNpEPAm8DaiIi682VmZvNTe+CIiDeAVW3Hbslsfw34Wt35MDOz/vDIcTMzq8SBw8zM\nKnHgMDOzShw4zMysEgcOMzOrxIHDzMwqceAwM7NKHDjMzKwSBw4zM6vEgcPMzCpx4DAzs0ocOMzM\nrBIHDjMzq8SBw8zMKnHgMDOzShw4zMysEgcOMzOrxIHDzMwqceAwM7NKag8cknZK+pGkRyXN5ZyX\npK9K2iHpcUnn1Z0nMzObv6UD+pxfjYhXCs5dBpyVvn4ZuDn9aWZmDdSEqqrLgW9G4iHgBEknL3Sm\nzMws3yACRwD3S9omaV3O+VOBFzL7u9NjR5G0TtKcpLm9e/fWlFUzM+tmEIHjoog4l6RK6jpJH57P\nRSJic0RMR8T0xMREf3NoZmal1R44ImJP+vNl4G7ggrYke4DTM/unpcfMzKyBag0cklZIOr61DXwM\n2N6W7B7gM2nvqg8Cr0XEi3Xmy8zM5q/uXlUnAndLan3WX0TEDyStB4iIW4CtwBpgB3AQ+K2a82Rm\nZj2oNXBExPPAB3KO35LZDuC6OvNhZmb904TuuGZmNkQcOMzMrBIHDjMzq8SBw8zMKnHgMDOzShw4\nzHo0OwtnnAFLliQ/Z2cXOkdm9RrU7Lhmi9LsLKxbBwcPJvu7diX7ADMzC5cvszq5xGHWg40bjwSN\nloMH4aqrXPKwxcuBw6wHP/5x/vF33klKHkXBo+7qLVefWZ0cOMx6MDlZfO7gwaRE0q5VvbVrF0Qc\nqd4q83AvExB6ub5ZGUpm/Bgu09PTMTd3zCq0ZgPX3sbRToLDh48+dsYZycO83dQU7NxZ7bPGx2Hz\n5qPbU+Z7fVv8JG2LiOler+MSh1lJRd/23/3u4vfklUjyHuqdjrcUtae0l2qKqs+KjptV5V5VZiXk\n9Z66+uqkKujtt/PfMz4OmzYde3xsLGkDyTveSdmAMDmZH4Q6VauZVeESh1kJed/233qrOGhMTR1b\nhdSSFzRax/PaLVolnaJa5faAsGlTErSyioKY2Xw4cJiVULWaZ+fOo4NGtpqrU8mivSE729CdJy8g\nzMwkQWtqKmlj6RTEzObDjeNmJRQ1OOeRYP162Lo1CTgrV8LrrycllLJaDdmdPndqKgkaDghWVr8a\nxx04zErI69G0fHm1YFBFqzfWkiX5VVR5vbXMunGvKrMByqv+ue22+j6v1W5R1KDthm5bSLUGDkmn\nS/pbSU9KekLShpw0F0t6TdKj6etLdebJrIy8rrczM0n10eHDR9owVq3q/2dLsGZNsu2Gbmuiurvj\nHgL+e0Q8Iul4YJuk+yLiybZ0fxcRH685L2al5HW9/fSn4cEH4aab6v/8CLj9drjwwiQ4PfhgUtpp\n9cbKjt1w+4YthFpLHBHxYkQ8km6/DjwFnFrnZ5r1Kq/rbQTcfHNSGpBg9Wq49lrYt6+ePLQmSrz2\nWrj11mO78HoaEVtIA2scl3QG8ADwHyJif+b4xcC3gd3AHuCLEfFEzvvXAesAJicnz99VtouLWUVF\nDdJN5GlErIqhahyXdBzwV8AXskEj9QgwGRG/BPwJ8J28a0TE5oiYjojpiYmJejNsI22YGp5//GPP\nhGuDV3vgkLSMJGjMRsS3289HxP6IOJBubwWWSVpdd77MimzalFRHDYOVKz0Trg1e3b2qBPwp8FRE\n/HFBmpPSdEi6IM1TTTXHZt3NzCQD+Jqu1duqzMSHZv1Ud4njQuDTwK9lutuukbReUutP8zeB7ZIe\nA74KrI1hHJVoQ6dTFc9NN9XT1bZfpGRW3qLGec+Ea3WqtTtuRPw90LHQHxFfA75WZz7M2pVZK/zV\nV4vfX3bU+JIl9Yzwjujco2uY2mls+HjkuI2kMmtbrFxZ/P4yQWN8HD73ufnlrxceIGh1c+CwkdKq\nnirqzZ2t4vnXf61+/VajemtG2vkOGFy1KrlGVZ4J1wbBgcNGRrcpyuHoKp433qj+GStXJsHjwAHY\nsKFc76z2NBJ86lPJtCNVene1xnQ4aFjdvAKgjYy86qmsflTxtNodyo4oX7YsWZ8jW7qJSEoNY2Pl\nByK6esoGySUOGxmdehrlVfHU3atKSlYQzKsSe+ed8lO2u3rKBs2Bw0ZGUU+jvCqeQQyg67XT+fg4\nbNmSlDQ2bvTIcRscBw4bGWWnKG+1hdQ1gWEvxsaSksqqVck4jiuvTGbune/IcU9XYvPhwGEjo+xa\n3N3aQhbK+Hgy3fodd8Cbbx4JbO0ll7Ijx7OdBTxdiVXhpWPNMmZnk2/xTTE2lgwgnJw8sr54mfXP\nyywtW3Qdz7i7eA3V7Lhmw6D1DbxJDh8+suIglAsaUG7keFFnAU9XYt04cJilmlhF1QoAZcagtJTt\nmuv1zG2+HDjMUk37pi0lgeKMM5LBhJ2CWvuI9TJdc72euc2XA4eNtGyvoiUN+2toNT/u2tW5h9fU\nVNJgHlFt5HjZzgJm7Txy3EZW+wy57et6D4NeG7JnZhworLqGfccyG5wmtmlU4WolWygOHDaymtam\n0U1rxlxXK9lCc1WVjazJyXK9lJpgfBxuvNGBwprBJQ4bWXm9iprIpQtrmtoDh6RLJT0jaYek63PO\nS9JX0/OPSzqv7jyZwbG9iqqsfdEPY2Pd03iNDWuiWgOHpDHg68BlwDnAFZLOaUt2GXBW+loH3Fxn\nnsya4ud+rnsXYDd+WxPVXeK4ANgREc9HxFvAncDlbWkuB74ZiYeAEySdXHO+bER0mv21fZK/QU/b\n9sYb3eeTMmuiugPHqcALmf3d6bGqaZC0TtKcpLm9e/f2PaO2+HSb/XUYuuNu2OBpz615hqZxPCI2\nR8R0RExPTEwsdHZsCOQFhuyU48PQHXffPk97bs1Td+DYA5ye2T8tPVY1jVll3WZ/HcbJ/MqutWFW\np7oDx8PAWZLOlLQcWAvc05bmHuAzae+qDwKvRcSLNefLRkBRYFi58sj05IPuSdUPw1BSssWt1sAR\nEYeAzwP3Ak8Bd0XEE5LWS1qfJtsKPA/sAL4BXFtnnmx05I3TWL4c9u8/MvAv4kjwOO64weYPYNmy\n6p8b4fYOW1i1jxyPiK0kwSF77JbMdgDX1Z0PGz2tsQ8bNybf0icn4cCBY2eajUi6xh44UH+eVqyA\n1auP5GfNGrjrruqf3WrvAI/xsMHz0rE2UpYsGXy323YRSWlhw4bO06WX4WVerQovHWs2D01oEL/2\n2qS00GvQALd32MJw4LCR0oSR2DffXG38yNRUMjNuniYEQhs9Dhw2MmZnh68ra6sq6sYbvcyrNYcD\nh42E7CjyYZENDF7m1ZrEjeM2ElrjNoaBlFRBbdrkwGD91a/GcS/kZCNhmBqRPfGhNZ2rqmwkDEsj\ncqc1OjrN9Gs2SA4cNhKGZbW/1qC+dt1m+jUbJAcOa7z5fNNufw8kjclF3VrLWr68t/d3cs01cNNN\n+ee6zfRrNkgOHNZo8/mmXfQegFdegS1bjg4g3Vbhy3rrrfn9O7qRioMGdJ/p12yQHDis0ebzTbvo\nPVdemZQ+HnwQ3nzzyLkmNEZ3a4MpOj8sbTe2uDhwWKMVfaPetau42qrTt/Bdu+CWWwaz8t+SJcns\nt2WsWdO5Oi6vjcYDAG2hOHBYo3X6Rl1UbdXtW/ighi4dPlxuvY8VK+D22ztXx3kAoDWJBwBao7Xa\nKzqVENpniC3znkEYG4N33umcZnwc3v3u/AkPPfOt9Ztnx7WRkP2mXaS9aqr1nl57UPWqU9DIlhpe\nfTU/jRu+rakcOKzxZmaSb95FwaOoamr//tqy1JOpqaQaa+fO5N/mhm8bNg4cNjSKloI9cODYRuWN\nG+Httweexa7yGrTd8G3Dpra5qiT9IfBfgLeA54Dfioh/yUm3E3gdeAc41I/6N1uc2peCXbkyKVW0\n2gey4zWaWs2T16Cdt8StJzi0JqutcVzSx4D/HRGHJH0ZICL+Z066ncB0RLxS9tpuHDconvG2VaXV\naTbcMg3X8zE1lb+ueeucG7ttITW+cTwi/iYiDqW7DwGn1fVZNpo6jabuVs0z36DRqXutF12yUTGo\nNo6rge8XnAvgfknbJBVM8QaS1kmakzS3d+/eWjJpw6VTo3I/q3lawWJqqvMYkNagRCg/5sIz3tow\n6ilwSLpf0vac1+WZNBuBQ0DRn8RFEXEucBlwnaQP5yWKiM0RMR0R0xMTE71k2xaJbo3KnbrwdrJ0\nadKVt/XQv+OOJGB06tnVkm1n2bnz6N5T7TzjrQ2tiKjtBXwW+AdgvGT6G4Avdkt3/vnnh42WLVsi\npqYipOTnli2dj7fOjY9HJI/l7q+8a7S75ppy15qa6p73qanu7zXrJ2Au+vFs78dFci8MlwJPAhMd\n0qwAjs9s/1/g0m7XduAYLXkBYHw8eYgXBY3se1sPaKncg76Tood9XhDqlPctW4rz03qvWb/1K3DU\n2atqB/AuoNW/5KGIWC/pFODWiFgj6b3A3en5pcBfRETXJkT3qhotRb2npKPbHMbHO8/fNDsLGzYc\n2+Op2/uyliwpN9dVq6F8Pj2/3PvK6jIMvap+MSJOj4hz09f69PhPImJNuv18RHwgfb2/TNCw0VPU\ne6r9Ad5tuvWZmSPrcbQe3GNjR97XrW1hdrb82h0HDiTpu/X8cu8rG0YeOW6NV2XqjTID/2Zmjjy0\nW91yuzVMtxqy87rxLlsGxx139LF9+5L0K1fmX6/V88sz3towcuCwxsv7Zl40niIvyGS7vK5enbyu\nvLLaAlF5i0NBUmL5sz/Ln1Cxlb5TqaI1D1en3ldmTePAYY2X9818/fpy1TztXV737csf1d3SKrG0\nj68oGoV++HCSv6KSzr59LlXY4lPbXFVm/TQzc+zD9sILu8/vVFRSKDI5eex6Hrt2HdsQn03f+lnU\ngA9u7LbFxSUOG1plqnmqTHY4Pp4s4XrVVccGm4hjq8eyJZxNm/KrzyI6N9ibDSMHDlvUyjasT00l\nAePWW4vnsYoornKamSnuptvUmXrN5suBwxa1TZuSXk9FxseT7rk7d8Jdd3Vew6M1vqKohFN1oSmz\nYeXAYYvazAz8/M8Xn8+O4ejUaN5tfMXsbDJ2o+z7PLmhDTMHDlv0itb0bslOTFikvSdUexffq68+\nNvCsWpXfg8qTG9qwq23KkTp5yhGrolN32jJWrUpGnLe097oqUjR1SKdpSNz7yurU+ClHzJoibwBh\nFTfeePR+2S6+naYbqXLcrGkcOGzRax9AODZW/f1ZZR/wnRaaqnLcrGkcOGwkZMd8dGvPaNfeeF3m\nAd+pMd2TG9qwc+CwkbN1a/7xovmv2huv8x78y5YdvWpgp2lFPLmhDTs3jtvI6bSmxtRUcUN6tvF6\ndrb7dCdmTePGcbNU1TERRVVNrcBQVPLItm14VlsbZQ4cNtQ6jYkoCijd2hjceG3WmWfHtaGW1zX2\n4MFkidg33zx6httWo3irdFBU1bRp07HjNNx4bXZEbSUOSTdI2iPp0fS1piDdpZKekbRD0vV15ccW\np07rYFRZqCnLjddmndXWOC7pBuBARPxRhzRjwD8DHwV2Aw8DV0TEk52u7cZxa6k6KlyCO+7IL1E4\nONhit1gaxy8AdkTE8xHxFnAncPkC58mGSFF7Rd5SrpBUSxVVb3ndDLNy6g4cvyPpcUm3SXpPzvlT\ngRcy+7vTY8eQtE7SnKS5vXv31pFXG0JF1Uo33ljcAO4pP8x601PgkHS/pO05r8uBm4H3AucCLwJf\n6eWzImJzRExHxPTExEQvl7JFJq9rbKd2CveaMutNT72qIuKSMukkfQP4Xs6pPcDpmf3T0mNmPctb\npxzca8qsV3X2qjo5s/tJYHtOsoeBsySdKWk5sBa4p648mYF7TZn1qs5xHH8g6VwggJ3A5wAknQLc\nGhFrIuKQpM8D9wJjwG0R8USNeTIDiksjZtZdbYEjIj5dcPwnwJrM/lagYNo5MzNrmoXujmtmZkPG\ngcPMzCpx4DAzs0ocOMzMrBIHDjMzq8SBw8zMKnHgMDOzShw4zMysEgcOMzOrxIHDzMwqceAwM7NK\nHDjMzKwSBw4zM6vEgcPMzCpx4DAzs0ocOMzMrBIHDjMzq8SBw8zMKqlt6VhJfwmcne6eAPxLRJyb\nk24n8DrwDnAoIqbrypOZmfWuzjXH/1trW9JXgNc6JP/ViHilrryYmVn/1BY4WiQJ+BTwa3V/lpmZ\n1W8QbRy/ArwUEc8WnA/gfknbJK0bQH7MzKwHPZU4JN0PnJRzamNEfDfdvgL4VofLXBQReyT9AnCf\npKcj4oGcz1oHrAOYnJzsJdtmZtYDRUR9F5eWAnuA8yNid4n0NwAHIuKPOqWbnp6Oubm5/mTSzGxE\nSNrWjw5IdVdVXQI8XRQ0JK2QdHxrG/gYsL3mPJmZWQ/qDhxraaumknSKpK3p7onA30t6DPgn4K8j\n4gc158nMzHpQa6+qiPhszrGfAGvS7eeBD9SZBzMz6y+PHDczs0ocOMzMrBIHDjMzq8SBw8zMKnHg\nMDOzShw4zMysEgcOMzOrxIHDzMwqceAwM7NKHDjMzKwSBw4zM6vEgcPMzCpx4DAzs0ocOMzMrBIH\nDjMzq8SBw8zMKnHgMDOzShw4zMysEgcOMzOrpKfAIem/SnpC0mFJ023nflfSDknPSPrPBe9fKek+\nSc+mP9/TS37MzKx+vZY4tgO/ATyQPSjpHGAt8H7gUuAmSWM5778e+GFEnAX8MN03M7MG6ylwRMRT\nEfFMzqnLgTsj4mcR8f+AHcAFBeluT7dvB369l/yYmVn9ltZ03VOBhzL7u9Nj7U6MiBfT7Z8CJxZd\nUNI6YF26+zNJ2/uR0ZqtBl5Z6EyU4Hz2zzDkEZzPfhuWfJ7dj4t0DRyS7gdOyjm1MSK+249MAERE\nSIoO5zcDm9M8zUXEdFHapnA++2sY8jkMeQTns9+GKZ/9uE7XwBERl8zjunuA0zP7p6XH2r0k6eSI\neFHSycDL8/gsMzMboLq6494DrJX0LklnAmcB/1SQ7qp0+yqgbyUYMzOrR6/dcT8paTfwn4C/lnQv\nQEQ8AdwFPAn8ALguIt5J33Nrpuvu7wMflfQscEm6X8bmXvI9QM5nfw1DPochj+B89ttI5VMRhc0K\nZmZmx/DIcTMzq8SBw8zMKmls4BjG6Uwk/aWkR9PXTkmPFqTbKelHabq+dI+rmM8bJO3J5HVNQbpL\n03u8Q9JAR/VL+kNJT0t6XNLdkk4oSLcg97LbvVHiq+n5xyWdN6i8ZfJwuqS/lfRk+re0ISfNxZJe\ny/xf+NKg85nmo+PvsSH38+zMfXpU0n5JX2hLsyD3U9Jtkl7Ojm8r+wyc1995RDTyBfx7ksEq/weY\nzhw/B3gMeBdwJvAcMJbz/j8Ark+3rwe+POD8fwX4UsG5ncDqBby3NwBf7JJmLL237wWWp/f8nAHm\n8WPA0nT7y0W/v4W4l2XuDbAG+D4g4IPAPy7A7/lk4Lx0+3jgn3PyeTHwvUHnrervsQn3M+f/wE+B\nqSbcT+DDwHnA9syxrs/A+f6dN7bEEUM8nYkkAZ8CvjWoz6zBBcCOiHg+It4C7iS5pwMREX8TEYfS\n3YdIxgI1RZl7cznwzUg8BJyQjlUamIh4MSIeSbdfB54ifwaHYbDg97PNR4DnImLXAubh30TEA8Cr\nbYfLPAPn9Xfe2MDRwanAC5n9nqczqcGvAC9FxLMF5wO4X9K2dCqVhfA7aZH/toIibNn7PAhXk3zb\nzLMQ97LMvWnS/UPSGcB/BP4x5/SH0v8L35f0/oFm7Ihuv8dG3U+SSVyLvhg24X5CuWfgvO5rXXNV\nlaKGTGdSRck8X0Hn0sZFEbFH0i8A90l6Ov3G0Ded8gncDPweyR/r75FUq13dz88vo8y9lLQROATM\nFlym9ns57CQdB/wV8IWI2N92+hFgMiIOpG1d3yEZsDtoQ/N7lLQc+ATwuzmnm3I/j9LPZyAscOCI\nIZzOpFueJS0lmWr+/A7X2JP+fFnS3STFxb7+kZS9t5K+AXwv51TZ+zxvJe7lZ4GPAx+JtEI25xq1\n38scZe5N7fevDEnLSILGbER8u/18NpBExFZJN0laHREDnbCvxO+xEfczdRnwSES81H6iKfczVeYZ\nOK/7OoxVVU2fzuQS4OmI2J13UtIKSce3tkkagQc6029b3fAnCz7/YeAsSWem37DWktzTgZB0KfA/\ngE9ExMGCNAt1L8vcm3uAz6S9gT4IvJapNhiItK3tT4GnIuKPC9KclKZD0gUkz4R9g8tl6d/jgt/P\njMIahSbcz4wyz8D5/Z0PuvW/Qi+BT5LUt/0MeAm4N3NuI0lPgGeAyzLHbyXtgQWsIlkc6lngfmDl\ngPL958D6tmOnAFvT7feS9Fx4DHiCpFpm0Pf2DuBHwOPpf5KT2/OZ7q8h6Ynz3KDzSdLp4QXg0fR1\nS5PuZd69Ada3fvckvX++np7/EZmegQPM40Uk1ZGPZ+7jmrZ8fj69d4+RdEL40ALkM/f32LT7meZj\nBUkg+HeZYwt+P0kC2YvA2+lz87eLnoH9+Dv3lCNmZlbJMFZVmZnZAnLgMDOzShw4zMysEgcOMzOr\nxIHDzMwqceAwM7NKHDjMzKyS/w97SHxv2TOG0AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114d0b278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "newX2 = (X2 - m2)\n",
    "\n",
    "plt.plot(newX2[:, 0], newX2[:, 1], 'ob')\n",
    "plt.xlim([-10, 10])\n",
    "plt.ylim([-10, 10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-10, 10)"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAD8CAYAAABgmUMCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGv1JREFUeJzt3X2sHfV95/HPx9d2E1/QApdbnn0hWkQWVgmFKzcb0SxZ\nSAreNG5Wu12QS5Om0q2dpCLaXVVkLUXRVpH6lK6SkgS5KSkJbmhXLQlKTFKIuiLRLkmukTHmKRjW\nBjsEbozKk1GM4bt/zJxyfDxzzvzOmTkP1++XNLrnzMyZ872/c+/ve2Z+D+OIEAAAVa0YdQAAgMlC\n4gAAJCFxAACSkDgAAElIHACAJCQOAECSWhKH7ZttP2t7d9u6U2zfZfux/OfJJa+9yvajtvfYvqGO\neAAAzanrjOOvJF3Vse4GSd+NiPMlfTd/fhTbU5I+L+lqSRdKutb2hTXFBABoQC2JIyLukfRcx+oN\nkm7JH98i6dcLXrpO0p6IeCIiDku6LX8dAGBMrWzw2KdFxNP5459KOq1gn7MkPdX2fL+kXy46mO0F\nSQuSND09felb3/rWGkMFgOVvx44dP4uI2UGP02Ti+GcREbYHmtskIrZK2ipJ8/Pzsbi4WEtsAHC8\nsL2vjuM02avqGdtnSFL+89mCfQ5IOqft+dn5OgDAmGoycdwh6YP54w9K+kbBPj+SdL7t82yvlnRN\n/joAwJiqqzvu1yT9X0kX2N5v+3ck/aGk99h+TNKV+XPZPtP2dkmKiCOSPibpO5IelvS3EfFgHTEB\nAJpRSxtHRFxbsumKgn1/Iml92/PtkrbXEQcAoHmMHAcAJCFxAACSkDgAAElIHACAJCQOAEASEgcA\nIAmJAwCQhMQBAEhC4gAAJCFxAACSkDgAAElIHACAJCQOAEASEgcAIAmJAwCQhMQBAEhC4gAAJCFx\nAACSNJo4bF9ge2fb8oLtj3fsc7nt59v2+WSTMQEABlPLPcfLRMSjki6WJNtTkg5Iur1g1+9FxPua\njAUAUI9hXqq6QtLjEbFviO8JAKjZMBPHNZK+VrLtnbZ32b7T9kVDjAkAkGgoicP2aknvl/S/Cjbf\nJ2ltRLxN0p9L+nrJMRZsL9peXFpaai5YAEBXwzrjuFrSfRHxTOeGiHghIl7KH2+XtMr2qQX7bY2I\n+YiYn52dbT5iAEChYSWOa1Vymcr26badP16Xx3RwSHEBABI12qtKkmxPS3qPpN9tW7dJkiLiJkn/\nUdJm20ckvSLpmoiIpuMCAPSn8cQRES9LmulYd1Pb4xsl3dh0HACAejByHACQhMQBAEhC4gAAJCFx\nAACSkDgAAElIHACAJCQOAEASEgcAIAmJAwCQhMQBAEhC4gAAJCFxAACSkDgAAElIHACAJCQOAEAS\nEgcAIAmJAwCQhMQBAEhC4gAAJGk8cdjea/sB2zttLxZst+3P2d5je5ftS5qOCQDQv5VDep93R8TP\nSrZdLen8fPllSV/MfwIAxtA4XKraIOkrkblX0km2zxh1UACAYsNIHCHpbts7bC8UbD9L0lNtz/fn\n645ie8H2ou3FpaWlhkIFAPQyjMRxWURcrOyS1Edtv6ufg0TE1oiYj4j52dnZeiMEAFTWeOKIiAP5\nz2cl3S5pXccuBySd0/b87HwdAGAMNZo4bE/bPrH1WNJ7Je3u2O0OSb+V9656h6TnI+LpJuMCAPSv\n6V5Vp0m63Xbrvf46Ir5te5MkRcRNkrZLWi9pj6RDkn674ZgAAANoNHFExBOS3l6w/qa2xyHpo03G\nAQCozzh0xwUATBASBwAgCYkDAJCExAEASELiAAAkIXEAAJKQOAAASUgcAIAkJA4AQBISBwAgCYkD\nAJCExAEASELiAAAkIXEAAJKQOAAASUgcAIAkJA4AQBISBwAgSaOJw/Y5tv/R9kO2H7R9fcE+l9t+\n3vbOfPlkkzEBAAbT6D3HJR2R9F8j4j7bJ0raYfuuiHioY7/vRcT7Go4FAFCDRs84IuLpiLgvf/yi\npIclndXkewIAmjW0Ng7b50r6JUk/KNj8Ttu7bN9p+6KS1y/YXrS9uLS01GCkAIBuhpI4bJ8g6e8k\nfTwiXujYfJ+ktRHxNkl/LunrRceIiK0RMR8R87Ozs80GDAAo1XjisL1KWdLYFhF/37k9Il6IiJfy\nx9slrbJ9atNxAcvdtm3SuedKK1ZkP7dtG3VEWC4abRy3bUl/KenhiPizkn1Ol/RMRITtdcqS2cEm\n4wKWu23bpIUF6dCh7Pm+fdlzSdq4cXRxYXlwRDR3cPsySd+T9ICk1/PV/13SWkmKiJtsf0zSZmU9\nsF6R9F8i4v90O+78/HwsLi42Fjcw6c49N0sWnebmpL17hx0NxoXtHRExP+hxGj3jiIjvS3KPfW6U\ndGOTcQDHmyefLF6/b1926WrtWunTn+bsA/1h5DiOW8u5DWDt2vJtEW9cukr5nZdzeSENiQPLXlGF\n12oD2Lev/4p0nH3609KaNd33OXRI2rKl2vGWe3khTaNtHE2hjQNVdTYSS1mF+uY3SwcLumA03Qaw\nbVtWWT/5ZPOXi9rfq+zf3JZef714WzvaTJaHuto4OOPAsrZly9FJQ8qeFyUNqbxtoA7D/ta+cWNW\nqb/+elbBF2ld0up1GaqsXJosL4wvEgcmSquCs6WVK7Of3a63p1Zs3doGBlWWxKpeLqqiLAEUXbpa\nsyZbXyWhlZXLKafUFzsmSERM3HLppZcGjj+33hqxZk1EVr0dvaxZk23vNDdXvP/MzLHHKjtGWSxz\ncxF29rPK6+ziWOyEQugRU7ffqSzmsjKamzv62KtWHbvP6tXVywyjJ2kxaqiDR54E+llIHMtL1Uq4\nrIJrr+g6j7V5c3ll2k/l34q3atJpf4+pqeK4p6Z6x1Al1ioJoOhYZeXZmdBmZqodH+OLxIFlIaUS\nLvvG3vnabttXrMiSSVEcnQmnrKJOqaB7xVNUWbcnwZQy6lY+7ccqOnOo8vs0fcaE5pE4MPbq/Jbc\nbd9+lvbkUbWCn5nJ9q1SQXeLt3WGUXYG0lpWrYo44YTqFXu38mklmrKzhrL9+/2sMJ5IHBhrg35L\nLvoW2883+CrfqqtWpq24qmzvdszW71blDKrX0lk+3c4meiWq9vjKzsoGaRdK1e+lRJQjcWCsdfu2\n3V4BdKtgiy4Zbd48eGU76qX1DX3QM6ipqaPL/NZbs8bqOmKcmSn+XIdVmQ87SR0vSBwYa92+TbdX\nACnf9qXq1+fHedm8Ofv93/SmwY/Vrs5Lea04e2kqkXBZrBl1JQ5GjqMRZSONW1ojjlesyKqE48mq\nVdn4k8OH6zvmzEz5oMZ+2dJXv1o+sr1sVP7WrdnjQUbIl/1dVB3pjmJ1jRwncaARRZVKu1YF0CvB\noFnT09LLL5dv7zalyKmnFiermRnplVeKE0rV5MEUJ81gyhGMtY0bs4piaqp4e2skcpXJ+NCcV1/N\nkkeZzpH37SP3y85wDh4cfIT8+vVp6zFcJA40ZuNG6ZZbyqe6aO2zdWv2LRXDd/iw9KY3ZYmgSPtU\nI+1Tk/QjZfqX7dvT1mO4SBxoVCsxzM1lldPcXPEli1deqfd9V6/O2hLQ23PPSZs2HZs82hO8VDzX\nVpGyLwErVlSf0JFJFccbiQONaV3WuO667PmmTdnP6647egK+qhVSVVNT0s03S1/+cvmssHjD2rXS\nF76QNYR3S/BVKu2ZGemzny2+/Pjaa9kZy0c+0vuGUGWTKjY5CSUS1NE1q9si6SpJj0raI+mGgu2W\n9Ll8+y5Jl/Q6Jt1xx1+VwXqtbrl1DIRrLUVTdlQd+DYpy/R0vb9T1S60vbr7tpf95s3lMXZ+3kXj\nM4oGM65axTiOQamm7riNnnHYnpL0eUlXS7pQ0rW2L+zY7WpJ5+fLgqQvNhkThqPKWUSrwbTOb5GR\ndxJsnxr8tdfqO/44ePnl+rowz8xU7+nUqyNDe9nfckt5uXfGXtZw3nnprKwdBsPX9KWqdZL2RMQT\nEXFY0m2SNnTss0HSV/KEeK+kk2yf0XBcaFjVa9H79kkvvZS1SdStVSGV9eyaZHWMZVizJrusVFV7\ne5XUvSI/dCi7FFVV59/Lli3HjnM5fLjee5egf00njrMkPdX2fH++LnUf2V6wvWh7cWlpqfZAUa+U\ns4iDB7NvoSkVTVVPPrn8zjjq8uY3p7+mdVfBubneZz0pya3z74XG8fE2MY3jEbE1IuYjYn52dnbU\n4aCH1PEZr74qnXxy/Wcea9fSQF7m4ME3LueV3TmwbH2dFbh97PgMGsfHW9OJ44Ckc9qen52vS90H\nE6aoG+7mzd0r8eeey3pD1aVVIdU9FcdycuiQdP31xbeO/chHym8p26sCt8u75U5PH32ZKyJrE2nv\nXdXtVrcYA3W0sJctklZKekLSeZJWS7pf0kUd+/x7SXcq6131Dkk/7HVcelVNtl4T2NXRY8iOOPPM\n+noeHY9L2efQ6rFW1muuNS172Qy3Ve8kyLTq9dOkzI4rab2kH0t6XNKWfN0mSZvyx1bW8+pxSQ9I\nmu91TBLHZNu8uXuXzDoqvSuuGH3Fu1yX1v1EWhV7e5LprOCLKn/uJDg6E5M4mlhIHJOhqNIo+hba\neeOgQacHn5tbfmM3RrF0O+Po57Pv9fkyZXrz6kocE9M4jsnSPq9RxBvXx6+//tjxHRFHz0E0yER2\na9Zkr2+yJ9X0dPeJAcdVlXamljVrpMsv7z0NSZGyz77VhkH7xTJQR/YZ9sIZx/hLPWtov0zR7xnH\nzEx25lL37WU740y9y15rJHXqTavqXDq/zXcbrd+Kt6gcp6fL2xzaL131ioH2i9EQl6owzlKnEWmv\nVPqtHMfx8lSrQmzifukpS2f7Qa/LRVWSd3u7VJXfjzaM0asrcXCpCo0o6645M9P7MkW/I717XZ5q\nYnR6L7/5m9kNj4ou0Q1T5+dRNs7mpZeyS0pVxmm0TxVSZYoZxmAsHyQONKLsOvZnP9t7mvUm2ifm\n5rIxIp1Tra9YIZ1wQv3v1+7gweGNJVm16tgEWdR+UHYflNagwFNOqfZ+rQTTK9HQhrG8kDhQi84R\nxlJ5gmhNW/H669nPzkn2ujXe9jPRXavS2rjxjanWWwPUVq7MvmWnGMfJ9lpl/OUvZwmy1/1PpGxd\nUdJsnTlUGfnfOovodjbRLQZMqDqudw17oY1jvJQN9Oq3wbPO9oD2bsCdjbFVruMXjTfpNmX4KJap\nqf7LutuYivYym5k5dprzXm0cg/wNoBmicRzjool++bfeOljl3KrgW7EVJYCUY3T2/qkruU1Pl2+b\nmaneg6tKJZ2SPIs+u149oegpNf5IHBgbTY0ETumZNTNzdKVVpVtut8RUpeKretbS7T0ijj6DmZo6\nejBkZ2Xc7WynW6IuOyMoKqfUMwUSxuQgcWBsNDUSuOy4Ve4gV7VCr+PySj/JY5DLOP0k6m6f0SAV\nP5eoJguJA2Ojqcqj27fkXhVdlbOVQSvNbnFWed9+9ZOomzorZPqQyULiwFhp6nJFv8ftdRZQ97fi\nzobksvetUlFXaUtITdRNVfBMWDhZSBxAF2WTKdbxjb+KfivqqkkhNaE2dVbIGcdkIXEAPYyy0bbf\nirrJiriJ8qCNY7LUlTicHWuyzM/Px+Li4qjDALrati2biuPJJ7MBcq1BiN2sWJFVv53stHt4D1M/\nvydGw/aOiJgf+DgkDmB8nHtuNg15p7m5bJQ9MIi6EgdTjgBjhHtVYBKQOLAsdc6d1bqJ0LhrTT5Y\nZa4pYFRWNnVg238i6dckHVZ2P/Hfjoh/Kthvr6QXJb0m6Ugdp1E4vrXuQNearK91BzppMirg1kSQ\nwLhq8ozjLkn/OiLeJunHkj7RZd93R8TFJA3UoejeEO33jgAwmMYSR0T8Q0QcyZ/eK+nspt4LaFd2\nb4gqNycC0Nuw2jg+LOnOkm0h6W7bO2wvlB3A9oLtRduLS0tLjQSJ5aHs3hDcgQ6ox0CJw/bdtncX\nLBva9tki6YiksubJyyLiYklXS/qo7XcV7RQRWyNiPiLmZ2dnBwkbyxw9k4BmDdQ4HhFXdttu+0OS\n3ifpiigZMBIRB/Kfz9q+XdI6SfcMEheOb62GZQalAc1oslfVVZJ+X9K/jYjC29jbnpa0IiJezB+/\nV9L/aComHD/omQQ0p8k2jhslnSjpLts7bd8kSbbPtL093+c0Sd+3fb+kH0r6VkR8u8GYAAADauyM\nIyL+Zcn6n0hanz9+QtLbm4oBAFA/Ro4DAJKQOAAASUgcAIAkJA4AQBISBwAgCYkDAJCExAEASELi\nAAAkIXEAAJKQOAAASUgcAIAkJA4AQBISBwAgCYkDAJCExAEASELiAAAkIXEAAJKQOAAASRpLHLY/\nZftAfr/xnbbXl+x3le1Hbe+xfUNT8QAA6tHYPcdz/zMi/rRso+0pSZ+X9B5J+yX9yPYdEfFQw3EB\nAPo06ktV6yTtiYgnIuKwpNskbRhxTACALppOHL9ne5ftm22fXLD9LElPtT3fn687hu0F24u2F5eW\nlpqIFQBQwUCJw/bdtncXLBskfVHSWyRdLOlpSZ8Z5L0iYmtEzEfE/Ozs7CCHAgAMYKA2joi4ssp+\ntv9C0jcLNh2QdE7b87PzdQCAMdVkr6oz2p5+QNLugt1+JOl82+fZXi3pGkl3NBUTAGBwTfaq+mPb\nF0sKSXsl/a4k2T5T0pciYn1EHLH9MUnfkTQl6eaIeLDBmAAAA2oscUTEdSXrfyJpfdvz7ZK2NxUH\nAKBeo+6OCwCYMCQOAEASEgcAIAmJAwCQhMQBAEhC4gAAJCFxAACSkDgAAElIHACAJCQOAEASEgcA\nIAmJAwCQhMQBAEhC4gAAJCFxAACSkDgAAElIHACAJCQOAECSxm4da/tvJF2QPz1J0j9FxMUF++2V\n9KKk1yQdiYj5pmICAAyuyXuO/+fWY9ufkfR8l93fHRE/ayoWAEB9GkscLbYt6Tck/bum3wsA0Lxh\ntHH8iqRnIuKxku0h6W7bO2wvDCEeAMAABjrjsH23pNMLNm2JiG/kj6+V9LUuh7ksIg7Y/kVJd9l+\nJCLuKXivBUkLkrR27dpBwgYADMAR0dzB7ZWSDki6NCL2V9j/U5Jeiog/7bbf/Px8LC4u1hMkABwn\nbO+oowNS05eqrpT0SFnSsD1t+8TWY0nvlbS74ZgAAANoOnFco47LVLbPtL09f3qapO/bvl/SDyV9\nKyK+3XBMAIABNNqrKiI+VLDuJ5LW54+fkPT2JmMAANSLkeMAgCQkDgBAEhIHACAJiQMAkITEAQBI\nQuIAACQhcQAAkpA4AABJSBwAgCQkDgBAEhIHACAJiQMAkITEAQBIQuIAACQhcQAAkpA4AABJSBwA\ngCQkDgBAEhIHACDJQInD9n+y/aDt123Pd2z7hO09th+1/aslrz/F9l22H8t/njxIPACA5g16xrFb\n0n+QdE/7StsXSrpG0kWSrpL0BdtTBa+/QdJ3I+J8Sd/NnwMAxthAiSMiHo6IRws2bZB0W0T8PCL+\nn6Q9ktaV7HdL/vgWSb8+SDwAgOatbOi4Z0m6t+35/nxdp9Mi4un88U8lnVZ2QNsLkhbypz+3vbuO\nQBt2qqSfjTqICoizPpMQo0ScdZuUOC+o4yA9E4ftuyWdXrBpS0R8o44gJCkiwnZ02b5V0tY8psWI\nmC/bd1wQZ70mIc5JiFEizrpNUpx1HKdn4oiIK/s47gFJ57Q9Pztf1+kZ22dExNO2z5D0bB/vBQAY\noqa6494h6Rrbv2D7PEnnS/phyX4fzB9/UFJtZzAAgGYM2h33A7b3S/o3kr5l+zuSFBEPSvpbSQ9J\n+rakj0bEa/lrvtTWdfcPJb3H9mOSrsyfV7F1kLiHiDjrNQlxTkKMEnHW7biK0xGlzQoAAByDkeMA\ngCQkDgBAkrFNHJM4nYntv7G9M1/22t5Zst9e2w/k+9XSPS4xzk/ZPtAW6/qS/a7Ky3iP7aGO6rf9\nJ7Yfsb3L9u22TyrZbyRl2atsnPlcvn2X7UuGFVtbDOfY/kfbD+X/S9cX7HO57efb/hY+Oew48zi6\nfo5jUp4XtJXTTtsv2P54xz4jKU/bN9t+tn18W9U6sK//84gYy0XSv1I2WOV/S5pvW3+hpPsl/YKk\n8yQ9Lmmq4PV/LOmG/PENkv5oyPF/RtInS7btlXTqCMv2U5L+W499pvKyfYuk1XmZXzjEGN8raWX+\n+I/KPr9RlGWVspG0XtKdkizpHZJ+MILP+QxJl+SPT5T044I4L5f0zWHHlvo5jkN5FvwN/FTS3DiU\np6R3SbpE0u62dT3rwH7/z8f2jCMmeDoT25b0G5K+Nqz3bMA6SXsi4omIOCzpNmVlOhQR8Q8RcSR/\neq+ysUDjokrZbJD0lcjcK+mkfKzS0ETE0xFxX/74RUkPq3gGh0kw8vLscIWkxyNi3whj+GcRcY+k\n5zpWV6kD+/o/H9vE0cVZkp5qez7wdCYN+BVJz0TEYyXbQ9LdtnfkU6mMwu/lp/w3l5zCVi3nYfiw\nsm+bRUZRllXKZpzKT7bPlfRLkn5QsPmd+d/CnbYvGmpgb+j1OY5VeSqbxLXsi+E4lKdUrQ7sq1yb\nmquqEo/JdCYpKsZ8rbqfbVwWEQds/6Kku2w/kn9jqE23OCV9UdIfKPtn/QNll9U+XOf7V1GlLG1v\nkXRE0raSwzRelpPO9gmS/k7SxyPihY7N90laGxEv5W1dX1c2YHfYJuZztL1a0vslfaJg87iU51Hq\nrAOlESeOmMDpTHrFbHulsqnmL+1yjAP5z2dt367sdLHWf5KqZWv7LyR9s2BT1XLuW4Wy/JCk90m6\nIvILsgXHaLwsC1Qpm8bLrwrbq5QljW0R8fed29sTSURst/0F26dGxFAn7KvwOY5FeeaulnRfRDzT\nuWFcyjNXpQ7sq1wn8VLVuE9ncqWkRyJif9FG29O2T2w9VtYIPNSZfjuuDX+g5P1/JOl82+fl37Cu\nUVamQ2H7Kkm/L+n9EXGoZJ9RlWWVsrlD0m/lvYHeIen5tssGQ5G3tf2lpIcj4s9K9jk930+21ymr\nEw4OL8rKn+PIy7NN6RWFcSjPNlXqwP7+z4fd+p/QS+ADyq63/VzSM5K+07Zti7KeAI9Kurpt/ZeU\n98CSNKPs5lCPSbpb0ilDivuvJG3qWHempO3547co67lwv6QHlV2WGXbZflXSA5J25X8kZ3TGmT9f\nr6wnzuPDjlNZp4enJO3Ml5vGqSyLykbSptZnr6z3z+fz7Q+orWfgEGO8TNnlyF1t5bi+I86P5WV3\nv7JOCO8cQZyFn+O4lWcex7SyRPAv2taNvDyVJbKnJb2a15u/U1YH1vF/zpQjAIAkk3ipCgAwQiQO\nAEASEgcAIAmJAwCQhMQBAEhC4gAAJCFxAACS/H+HAFH0pGPmeQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111a022e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "newX2 = (X2 - m2) @ V\n",
    "\n",
    "plt.plot(newX2[:, 0], newX2[:, 1], 'ob')\n",
    "plt.xlim([-10, 10])\n",
    "plt.ylim([-10, 10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-10, 10)"
      ]
     },
     "execution_count": 212,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAD8CAYAAABgmUMCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGZZJREFUeJzt3X+MHeV97/HPxwYnsaElGIffXhNdi8SpGkpWLjdJq6SQ\nFFZRKFWaa7QQEiJtnSZRIvWqIrIURbey1F9pRUoAOSkqATeUq5YGpSYUR72lrUrKGhljMBSH2mDH\ngbWtCxjSYMO3f8yceHx25ux59pw5v/b9klY7Z+Y5c75+1jvfnefXOCIEAEC7FvU7AADAcCFxAACS\nkDgAAElIHACAJCQOAEASEgcAIElXEoft22y/YHtnYd/pth+w/XT+/a0V773c9lO2d9u+oRvxAADq\n0607jr+UdHnTvhskfT8iVkv6fv76BLYXS/q6pCskrZF0te01XYoJAFCDriSOiHhQ0uGm3VdKuj3f\nvl3Sb5S8da2k3RHxTES8Jumu/H0AgAF1Uo3nPjMiDuTbP5Z0ZkmZcyU9V3i9T9Ivl53M9pSkKUla\ntmzZe97xjnd0MVQAGH3btm07GBErOj1PnYnjZyIibHe0tklEbJK0SZLGx8djenq6K7EBwEJhe283\nzlPnqKrnbZ8tSfn3F0rK7Jd0fuH1efk+AMCAqjNx3Cvpunz7OknfKSnzsKTVti+wvUTSuvx9AIAB\n1a3huN+W9G+SLrS9z/anJf2BpA/ZflrSZflr2T7H9hZJiohjkj4n6X5JuyTdHRGPdyMmAEA9utLH\nERFXVxy6tKTsjyRNFF5vkbSlG3EAAOrHzHEAQBISBwAgCYkDAJCExAEASELiAAAkIXEAAJKQOAAA\nSUgcAIAkJA4AQBISBwAgCYkDAJCExAEASELiAAAkIXEAAJKQOAAASUgcAIAkJA4AQBISBwAgSa2J\nw/aFtrcXvl6y/cWmMh+w/WKhzJfrjAkA0JmuPHO8SkQ8JekiSbK9WNJ+SfeUFP3niPhInbEAALqj\nl01Vl0r6YUTs7eFnAgC6rJeJY52kb1cce6/tHbbvs/2uHsYEAEjUk8Rhe4mkj0r6vyWHH5G0MiJ+\nUdKfS/q7inNM2Z62PT0zM1NfsACAlnp1x3GFpEci4vnmAxHxUkQcybe3SDrZ9hkl5TZFxHhEjK9Y\nsaL+iAEApXqVOK5WRTOV7bNsO99em8d0qEdxAQAS1TqqSpJsL5P0IUm/Xdi3XpIi4lZJH5P0GdvH\nJP1E0rqIiLrjAgDMT+2JIyJekbS8ad+the2bJN1UdxwAgO5g5jgAIAmJAwCQhMQBAEhC4gAAJCFx\nAACSkDgAAElIHACAJCQOAEASEgcAIAmJAwCQhMQBAEhC4gAAJCFxAACSkDgAAElIHACAJCQOAEAS\nEgcAIAmJAwCQhMQBAEhSe+Kwvcf2Y7a3254uOW7bX7O92/YO2xfXHRMAYP5O6tHnfDAiDlYcu0LS\n6vzrlyXdkn8HAAygQWiqulLStyLzkKTTbJ/d76AAAOV6kThC0lbb22xPlRw/V9Jzhdf78n0nsD1l\ne9r29MzMTE2hAgDm0ovE8f6IuEhZk9Rnbf/qfE4SEZsiYjwixlesWNHdCAEAbas9cUTE/vz7C5Lu\nkbS2qch+SecXXp+X7wMADKBaE4ftZbZPbWxL+rCknU3F7pX0iXx01SWSXoyIA3XGBQCYv7pHVZ0p\n6R7bjc/6q4j4nu31khQRt0raImlC0m5Jr0r6VM0xAQA6UGviiIhnJL27ZP+the2Q9Nk64wAAdM8g\nDMcFAAwREgcAIAmJAwCQhMQBAEhC4gAAJCFxAACSkDgAAElIHACAJCQOAEASEgcAIAmJAwCQhMQB\nAEhC4gAAJCFxAACSkDgAAElIHMAA2rxZWrVKWrQo+755c78jAo6r+wmAABJt3ixNTUmvvpq93rs3\ney1Jk5P9iwto4I4DGDAbNhxPGg2vvprtBwZBrYnD9vm2/9H2E7Yft/2FkjIfsP2i7e3515frjAkY\ndM8+m7Yf6LW67ziOSfrdiFgj6RJJn7W9pqTcP0fERfnX/6k5JmCgrVxZvn/RIvo6MBhqTRwRcSAi\nHsm3X5a0S9K5dX4m0E396KTeuFFaunT2/tdfz/o6SB7ot571cdheJemXJP2g5PB7be+wfZ/td1W8\nf8r2tO3pmZmZGiMFMo1O6r17pYjjndR1X7gnJ6VNm6TFi2cfK+vrYAQWes0RUf+H2KdI+idJGyPi\nb5uO/ZykNyLiiO0JSTdGxOpW5xsfH4/p6en6AsaCtnlzdnHeu7f8+NiYtGdP/XEsWpQlrGa29MYb\n2XbzCCwpu1vZtIkRWJjN9raIGO/0PLXfcdg+WdLfSNrcnDQkKSJeiogj+fYWSSfbPqPuuIAyxbuM\nKimd1J3cDVT1dRT3MwIL/VD3qCpL+gtJuyLiTyvKnJWXk+21eUyH6owLqFJ2IW5WdUFv1mlTV1lf\nx5Il0pEjxxNRVYJjBBbqVPcdx/skXSvp1wrDbSdsr7e9Pi/zMUk7bT8q6WuS1kUv2s+woFXdCcx1\nwV26NLugt6PTu4FGX8fYWNY8tXx5loAOHTqeiLI/uWZrN7kB81HrzPGI+BdJFf+1f1bmJkk31RkH\nUFQ2M/vaa6V//dfsgtuqb2Pjxvb7DuYzH6PRv/Lss1ksGzce709ZtSpLGkVVf2JNTLQXIzAfzBzH\nglN2JxAh3XprdsFtbh5aulS6887sAt6cNFr1YbTTR9F8rlZNWynNT1u2tF8WSEXiwEgru7BXXYAj\nsgtusXlobKx6hNJcF/qyPopWTV1zNW2lND/Rx4E69WQ4brcxHBftqBqq+pa3zG7yKWrVJFVsSlq0\nKJuU12z5cumUU7Iyp5+e7Tt0KJuX8frrx88vndgs1Wok19jY8T6N4q9s8+ti+V4MGcZw6dZwXBIH\nRlbVqKPly6XDh6v7B5rL3nhjtv2FL7ROOFVOPjm7wL/22on7Fy06Ph9Dqk4CrWL7+Mel229nHgfa\nMzTzOIBeaW6WqvoL/vBh6Z3vbO+chw5lHefXXz+/pCFJR4/OThrSiUlDSksakvSTn0jve1/7TWtA\nt3DHgZFQ1izVqhln377yZqZhQ5MUUnDHARRUjZQqMzExGklD6qwTnDWuMF88ARAjIeUCesst9cXR\na43O91Q8ZRCd4I4DI2GhzpR+6aXjdwopdxCscYVOcMeBkbBxo3TNNf2OoveOHj1+sU+5g+Apg+gE\ndxwYCZOT2fDUhWjv3uo7iGuuKb/7SJ3VDhSRODAybryxfDXZqoUAR8Xixa0nD5atyrtxY1Y3RUuW\ntL+AIxY2EgdGRvNqsmNj0m23SevXz/3eYdbOCLGy/ovmUWdDODIffcI8Doy8VpMBF5LikwOr6oR5\nIaONeRxAm0Y5aaQ0wxX7L+gcRydIHBhpmzePbh/H2Fj7zUv2ic/ooHMcnSBxYGSUzWPYsGE02+4b\nzwcZG2uvfON5I3ZWN1XPHaFzHO0gcWAklD0b45pr0pup3vzmeuLrtsbcjLJnftjSsmWz39NIoHv3\nZivqXncdiyNifmpPHLYvt/2U7d22byg5bttfy4/vsH1x3TFh9JTNY5iP//qvzs9Rt+JdRtlIsjvu\nmLsuXn1VuvvueuPE6Ko1cdheLOnrkq6QtEbS1bbXNBW7QtLq/GtK0gitJIReWSidumXNSZOTWbPV\nG29kTVDXXdde89yhQ9VPLwRaqfuOY62k3RHxTES8JukuSVc2lblS0rci85Ck02yfXXNcGDELpVO3\n1WNsTzklW8Bxviv/slYV2lV34jhX0nOF1/vyfallZHvK9rTt6ZmZma4HiuFW1tY/asbGWj/7/JVX\nOv+MhXLnhs4MTed4RGyKiPGIGF+xYkW/w8GAabT1j/J6VQcPljclpfbvjI1V19NCuXNDZ+pOHPsl\nnV94fV6+L7UMMKfJyezieued/Y6kHq+8In3qU7OXUU8ZOdaYGV62rhfDcdGuuhPHw5JW277A9hJJ\n6yTd21TmXkmfyEdXXSLpxYg4UHNcGGGTk+3Pbxg2R49mnd929iz0lKRhZ+VXrcpe86xyzFetiSMi\njkn6nKT7Je2SdHdEPG57ve3G0nNbJD0jabekb0j6nTpjwsIwyn0ejc7vuUZOFedzFJ+/XnxWR2M0\n1p49JA20j0UOMbIaM8effXY0Z49XGRvLEmcjEbCgIRq6tcghTwDESComjZUrpSNHsnkLo64sGbCg\nIbptaEZVAe0qW37k5Zf7HVX9qjq3WdAQ3UbiwMhojDK65prZw1Nfe60vIXXMzib2tTouze7cLi74\neOTI7Kf9MYIKnaCpCiOhcZfRjfWqBklEduFvdby5T6O5Lg4dkk4+OZu7cfhwdqdRLA+kInFgJHRr\nkcNuWrw4u4DfUvPqa8VRUpOT5XVx9Gh253LwYL2xYGFgVBVGwqJFgzdyas0aadeu3sY1NlY9t6P4\n6FgsTDw6Fiho1dHbr2VInniie0lj+fLjk/VaaTUhkM5wdAuJAyOhbMLf0qXZ8iOtOpeHwdKl2RIh\njcl685kVb9MZju4hcWAklD3QqDHKqJP5CmNj/Vm+pGy0VHFtqtTnqEfQGY7uoXMcI2NysvziuHJl\n+iNkpeNDVq+9tvPYUt1xx4n/luaRUhEnLiMyl1Fduwv9wR0HhlpxvsKqVeXLjs933apNm7Lvi3r8\nW1L23I2ykVIRWd/HXP825myg20gcGFplM8TLHn86n2d1NP5Cn5oqf6Jeo6lo+fLZk+s6UXWRr2pu\nO3x4dhPdZz7DqreoF4kDQ6vsr/Cqx59OTrbfSb5kSTbprmwGupTNz7jjjixZHTwoffrTaX0OixdL\nl156PDkV72je8pby97RaNqT4zPE9e6Sbb2bVW9SLxIGhlbp4Xzud5Ha2PEmrBRHfeOPEi/GWLe31\nNYyNZeWOHZO2bs0u6nfeKb35zcfLHDp04l1Tqw5xmqDQL3SOY2hVdXq3+uu8VSf5kiXtrWnVfP52\nElLVRX6uu6aqDvHmZUaAXuKOA0Orau5G1V/hZeWLw15PPXXuzyw7/1wT65Ytq+5naHV3VNUh3lg6\nvfl87QwUALqBxIGh1WruRrvlG30Ve/ZkHc2tVJ1/rlFbrZqxWt0dpTS5tTtQAOgG1qoCclVPyrNn\nz6to1nhwVFVTWNXT9spW9V26NEtQVecrOxdP+UM7Bn6tKtt/bPtJ2zts32P7tIpye2w/Znu7bbIB\n+qaqKWv9+rn7Ehojm6pGVzXuEpqbk6Tqu6aUpjie8odeqrOp6gFJvxARvyjpPyR9qUXZD0bERd3I\nhMB8VTVl3Xxz++do1fRU1ZwklQ+fTWmK4yl/6KWeNFXZvkrSxyJi1n9523skjUdE208KoKkKg6pb\nTU/d/FxGXqFh4Juqmlwv6b6KYyFpq+1ttqeqTmB7yva07emZmZlaggQ6NZ/FFrvRnJQ6UADoREd3\nHLa3Sjqr5NCGiPhOXmaDpHFJvxklH2b73IjYb/ttypq3Ph8RD7b6XO44MIzowEa/deuOo6MJgBFx\nWavjtj8p6SOSLi1LGvk59uffX7B9j6S1klomDmAYbdxY3pzE7G8MmzpHVV0u6fckfTQiSp8GbXuZ\n7VMb25I+LGlnXTEB/URzEkZFnUuO3CTpTZIecDZG8aGIWG/7HEnfjIgJSWdKuic/fpKkv4qI79UY\nE9BXVc8MAYZJbYkjIv5Hxf4fSZrIt5+R9O66YgAAdB9LjgA51noC2sPquIBmz4MoTs6jaQk4EXcc\ngNIeCgUsdCQOQKz1BKQgcQBirScgBYkDUPpDoYCFjMQBiMl5QApGVQE5JucB7eGOAwCQhMQBAEhC\n4gAAJCFxAACSkDgAAElIHACAJCQOAEASEgcAIAmJAwCQhMQBAEhSW+Kw/RXb+21vz78mKspdbvsp\n27tt31BXPACA7qh7rao/i4g/qTpoe7Gkr0v6kKR9kh62fW9EPFFzXACAeep3U9VaSbsj4pmIeE3S\nXZKu7HNMAIAW6k4cn7e9w/Zttt9acvxcSc8VXu/L981ie8r2tO3pmZmZOmIFALSho8Rhe6vtnSVf\nV0q6RdLbJV0k6YCkr3byWRGxKSLGI2J8xYoVnZwKANCBjvo4IuKydsrZ/oak75Yc2i/p/MLr8/J9\nAIABVeeoqrMLL6+StLOk2MOSVtu+wPYSSesk3VtXTACAztU5quqPbF8kKSTtkfTbkmT7HEnfjIiJ\niDhm+3OS7pe0WNJtEfF4jTEBADpUW+KIiGsr9v9I0kTh9RZJW+qKAwDQXf0ejgsAGDIkDgBAEhIH\nACAJiQMAkITEAQBIQuIAACQhcQAAkpA4AABJSBwAgCQkDgBAEhIHACAJiQMAkITEAQBIQuIAACQh\ncQAAkpA4AABJSBwAgCQkDgBAktoeHWv7ryVdmL88TdL/j4iLSsrtkfSypNclHYuI8bpiAgB0rs5n\njv+vxrbtr0p6sUXxD0bEwbpiAQB0T22Jo8G2JX1c0q/V/VkAgPr1oo/jVyQ9HxFPVxwPSVttb7M9\n1YN4AAAd6OiOw/ZWSWeVHNoQEd/Jt6+W9O0Wp3l/ROy3/TZJD9h+MiIeLPmsKUlTkrRy5cpOwgYA\ndMARUd/J7ZMk7Zf0nojY10b5r0g6EhF/0qrc+Ph4TE9PdydIAFggbG/rxgCkupuqLpP0ZFXSsL3M\n9qmNbUkflrSz5pgAAB2oO3GsU1Mzle1zbG/JX54p6V9sPyrp3yX9fUR8r+aYAAAdqHVUVUR8smTf\njyRN5NvPSHp3nTEAALqLmeMAgCQkDgBAEhIHACAJiQMAkITEAQBIQuIAACQhcQAAkpA4AABJSBwA\ngCQkDgBAEhIHACAJiQMAkITEAQBIQuIAACQhcQAAkpA4AABJSBwAgCQkDgBAEhIHACBJR4nD9m/Z\nftz2G7bHm459yfZu20/Z/vWK959u+wHbT+ff39pJPACA+nV6x7FT0m9KerC40/YaSeskvUvS5ZJu\ntr245P03SPp+RKyW9P38NQBggHWUOCJiV0Q8VXLoSkl3RcRPI+I/Je2WtLai3O359u2SfqOTeAAA\n9TuppvOeK+mhwut9+b5mZ0bEgXz7x5LOrDqh7SlJU/nLn9re2Y1Aa3aGpIP9DqINxNk9wxCjRJzd\nNixxXtiNk8yZOGxvlXRWyaENEfGdbgQhSRERtqPF8U2SNuUxTUfEeFXZQUGc3TUMcQ5DjBJxdtsw\nxdmN88yZOCLisnmcd7+k8wuvz8v3NXve9tkRccD22ZJemMdnAQB6qK7huPdKWmf7TbYvkLRa0r9X\nlLsu375OUtfuYAAA9eh0OO5VtvdJ+p+S/t72/ZIUEY9LulvSE5K+J+mzEfF6/p5vFobu/oGkD9l+\nWtJl+et2bOok7h4izu4ahjiHIUaJOLttQcXpiMpuBQAAZmHmOAAgCYkDAJBkYBPHMC5nYvuvbW/P\nv/bY3l5Rbo/tx/JyXRkelxjnV2zvL8Q6UVHu8ryOd9vu6ax+239s+0nbO2zfY/u0inJ9qcu56saZ\nr+XHd9i+uFexFWI43/Y/2n4i/136QkmZD9h+sfB/4cu9jjOPo+XPcUDq88JCPW23/ZLtLzaV6Ut9\n2r7N9gvF+W3tXgPn9XseEQP5Jemdyiar/D9J44X9ayQ9KulNki6Q9ENJi0ve/0eSbsi3b5D0hz2O\n/6uSvlxxbI+kM/pYt1+R9L/nKLM4r9u3S1qS1/maHsb4YUkn5dt/WPXz60ddtlM3kiYk3SfJki6R\n9IM+/JzPlnRxvn2qpP8oifMDkr7b69hSf46DUJ8l/wd+LGlsEOpT0q9KuljSzsK+Oa+B8/09H9g7\njhji5UxsW9LHJX27V59Zg7WSdkfEMxHxmqS7lNVpT0TEP0TEsfzlQ8rmAg2KdurmSknfisxDkk7L\n5yr1TEQciIhH8u2XJe1S+QoOw6Dv9dnkUkk/jIi9fYzhZyLiQUmHm3a3cw2c1+/5wCaOFs6V9Fzh\ndcfLmdTgVyQ9HxFPVxwPSVttb8uXUumHz+e3/LdV3MK2W8+9cL2yvzbL9KMu26mbQao/2V4l6Zck\n/aDk8Hvz/wv32X5XTwM7bq6f40DVp7JFXKv+MByE+pTauwbOq17rWquqLR6Q5UxStBnz1Wp9t/H+\niNhv+22SHrD9ZP4XQ9e0ilPSLZJ+X9kv6+8ra1a7vpuf34526tL2BknHJG2uOE3tdTnsbJ8i6W8k\nfTEiXmo6/IiklRFxJO/r+jtlE3Z7bWh+jraXSPqopC+VHB6U+jxBN6+BUp8TRwzhciZzxWz7JGVL\nzb+nxTn2599fsH2PstvFrv6StFu3tr8h6bslh9qt53lroy4/Kekjki6NvEG25By112WJduqm9vpr\nh+2TlSWNzRHxt83Hi4kkIrbYvtn2GRHR0wX72vg5DkR95q6Q9EhEPN98YFDqM9fONXBe9TqMTVWD\nvpzJZZKejIh9ZQdtL7N9amNbWSdwT1f6bWobvqri8x+WtNr2BflfWOuU1WlP2L5c0u9J+mhEvFpR\npl912U7d3CvpE/looEskvVhoNuiJvK/tLyTtiog/rShzVl5OttcquyYc6l2Ubf8c+16fBZUtCoNQ\nnwXtXAPn93ve697/hFECVylrb/uppOcl3V84tkHZSICnJF1R2P9N5SOwJC1X9nCopyVtlXR6j+L+\nS0nrm/adI2lLvv12ZSMXHpX0uLJmmV7X7R2SHpO0I/9PcnZznPnrCWUjcX7Y6ziVDXp4TtL2/OvW\nQarLsrqRtL7xs1c2+ufr+fHHVBgZ2MMY36+sOXJHoR4nmuL8XF53jyobhPDePsRZ+nMctPrM41im\nLBH8fGFf3+tTWSI7IOloft38dNU1sBu/5yw5AgBIMoxNVQCAPiJxAACSkDgAAElIHACAJCQOAEAS\nEgcAIAmJAwCQ5L8B0u37T86tpOAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110946a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "newX2 = (X2 - m2) @ V \n",
    "newX2 /= np.std(newX2, 0)\n",
    "\n",
    "plt.plot(newX2[:, 0], newX2[:, 1], 'ob')\n",
    "plt.xlim([-10, 10])\n",
    "plt.ylim([-10, 10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Input Transformation \n",
    "\n",
    "<img src=\"http://webpages.uncc.edu/mlee173/teach/itcs4156online/images/class/input_transform.png\" width=600/>\n",
    "<center>(Lecun, 1998)</center>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}