ML/PCA_SVD.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from numpy import linalg as la"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PCA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### example 1 \n",
    "<reference> https://www.fun-coding.org/recommend_basic5.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "numpy.random.randint(low, high=None, size=None) 이용 \n",
    "<br>변수(행)별로 평균을 0으로 centering한 행렬 X′를 만듬 (좌표계의 원점이 평균 벡터와 일치하도록 만듬)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* 원본 data\n",
      "[[5. 7. 4.]\n",
      " [6. 9. 7.]\n",
      " [0. 7. 8.]\n",
      " [3. 3. 0.]\n",
      " [2. 7. 2.]]\n"
     ]
    }
   ],
   "source": [
    "n, p = 5, 3\n",
    "X = np.random.randint(10, size = (n, p)).astype('float64')\n",
    "print(\"* 원본 data\")\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"* 각 dimension 별 평균\")\n",
    "print(np.mean(X, axis=0))\n",
    "X -= np.mean(X, axis=0)\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "공분산행렬은 다음 식으로 만들 수 있음\n",
    "$$Σ=cov(X)= X^T X/(n−1) \\propto X^T X $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5.7,  1.6, -0.3],\n",
       "       [ 1.6,  4.8,  5.6],\n",
       "       [-0.3,  5.6, 11.2]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "C = np.cov(X, rowvar=False)\n",
    "C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "공분산행렬을 기반으로 고유값과 고유벡터 구하기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 1.04228723,  6.17355322, 14.48415955]),\n",
       " array([[-0.31421288,  0.9472554 ,  0.06306729],\n",
       "        [ 0.82742512,  0.24068297,  0.50738485],\n",
       "        [-0.46544381, -0.21161032,  0.8594086 ]]))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l, principal_axes = la.eig(C)\n",
    "l, principal_axes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "고유값을 높은 순으로 정렬하고, 이에 대응하는 고유벡터와 순서를 맞춤"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 1, 0], dtype=int64)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idx = l.argsort()[::-1]\n",
    "idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([14.48415955,  6.17355322,  1.04228723]),\n",
       " array([[ 0.06306729,  0.9472554 , -0.31421288],\n",
       "        [ 0.50738485,  0.24068297,  0.82742512],\n",
       "        [ 0.8594086 , -0.21161032, -0.46544381]]))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l, principal_axes = l[idx], principal_axes[:, idx]\n",
    "l, principal_axes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "np.matmul(principal_axes.T, principal_axes) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "차원축소예 (고유값을 기준으로 가장 큰 d개의 고유 벡터 선택)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* principal axis(principal_compoents) 를 구함"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.06306729,  0.9472554 ],\n",
       "       [ 0.50738485,  0.24068297],\n",
       "       [ 0.8594086 , -0.21161032]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# d = 2 \n",
    "principal_axes[:, :2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* principal axis 를 기반으로 원본데이터 X에 대한 principal components 를 구함 \n",
    "  <br>그리고 top 2 개의 components 들에 대해서 dimensionality reduction을 수행"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 7.30466478e+00,  5.57461648e+00,  2.35913621e+00],\n",
       "       [ 1.09607276e+01,  6.36840686e+00,  2.30344214e+00],\n",
       "       [ 1.04269627e+01, -8.10176494e-03,  2.06842536e+00],\n",
       "       [ 1.71135642e+00,  3.56381508e+00,  1.53963672e+00],\n",
       "       [ 5.39664570e+00,  3.15607092e+00,  4.23266246e+00]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mapped_data = X.dot(principal_axes)\n",
    "mapped_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 7.30466478e+00,  5.57461648e+00],\n",
       "       [ 1.09607276e+01,  6.36840686e+00],\n",
       "       [ 1.04269627e+01, -8.10176494e-03],\n",
       "       [ 1.71135642e+00,  3.56381508e+00],\n",
       "       [ 5.39664570e+00,  3.15607092e+00]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mapped_data_reduced = X.dot(principal_axes[:, :2])\n",
    "mapped_data_reduced"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 7.30466478 10.96072755 10.42696269  1.71135642  5.3966457 ]\n",
      "[ 5.57461648  6.36840686 -0.00810176  3.56381508  3.15607092]\n"
     ]
    }
   ],
   "source": [
    "# 새로운 변수 z1 과 z2 (5개의 데이터에대한 각각의 dimension component를 갖고있는 vector)\n",
    "z1 = mapped_data_reduced[:,0]\n",
    "z2 = mapped_data_reduced[:,1]\n",
    "print(z1)\n",
    "print(z2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### example 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 4 \n",
    "d = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0., -2.],\n",
       "       [ 3., -3.],\n",
       "       [ 2.,  0.],\n",
       "       [-1.,  1.]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.array([[0,-2],[3,-3],[2,0],[-1,1]]).astype('float64')\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-1., -1.],\n",
       "       [ 2., -2.],\n",
       "       [ 1.,  1.],\n",
       "       [-2.,  2.]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 각 row마다 빼진다. 알아서 broadcasting 되어 계산됨\n",
    "X -= np.mean(X, axis=0)\n",
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ C =Σ=cov(X)= X^T X/(n−1) $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 3.33333333 -2.        ]\n",
      " [-2.          3.33333333]]\n",
      "[[10. -6.]\n",
      " [-6. 10.]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[10., -6.],\n",
       "       [-6., 10.]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "C = np.cov(X, rowvar=False)\n",
    "print(C)\n",
    "print(C*(n-1))\n",
    "np.matmul(X.T,X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.33333333 1.33333333]\n",
      "[[ 0.70710678  0.70710678]\n",
      " [-0.70710678  0.70710678]]\n"
     ]
    }
   ],
   "source": [
    "l, principal_axes = la.eig(C)\n",
    "print(l)\n",
    "print(principal_axes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.33333333 1.33333333]\n",
      "[[ 0.70710678  0.70710678]\n",
      " [-0.70710678  0.70710678]]\n"
     ]
    }
   ],
   "source": [
    "idx = l.argsort()[::-1]\n",
    "l, principal_axes = l[idx], principal_axes[:, idx]\n",
    "print(l)\n",
    "print(principal_axes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-2.22044605e-16, -1.41421356e+00],\n",
       "       [ 2.82842712e+00, -4.44089210e-16],\n",
       "       [ 2.22044605e-16,  1.41421356e+00],\n",
       "       [-2.82842712e+00,  4.44089210e-16]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mapped_data = X.dot(principal_axes)\n",
    "mapped_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVD\n",
    "<reference> \n",
    "<br> https://www.fun-coding.org/recommend_basic6.html\n",
    "<br> https://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix\n",
    "<br> https://twlab.tistory.com/54\n",
    "$$ R=UΣV^T $$\n",
    "* note that $RR^T$ 또는 $R^TR$은 symmetric matrix이므로, eigen decomposition이 가능 \n",
    "* U는 $RR^T$ 즉, $R$의 공분산 행렬의 고유벡터\n",
    "* V는 $R^TR$ 즉, $R^T$의 공분산 행렬의 고유벡터\n",
    "* $ΣΣ^T$ 또는 $Σ^TΣ=λ$ \n",
    "  따라서, singular value (특이값) $σ = \\sqrt{λ}$\n",
    "    \n",
    "> approach: \n",
    "  $RR^T$ 또는 $R^TR$ 은 symmetric square matrix 이므로 eigen decomposition이 가능하고,\n",
    "  <br>이것을 이용하여, U, V, λ 를 구한다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* 원본 data\n",
      "[[7. 0. 9.]\n",
      " [1. 9. 9.]\n",
      " [6. 7. 2.]\n",
      " [2. 8. 7.]\n",
      " [6. 5. 2.]]\n"
     ]
    }
   ],
   "source": [
    "user, item = 5, 3\n",
    "R = np.random.randint(10, size = (user, item)).astype('float')\n",
    "# user, item = 4, 2\n",
    "# R = np.array([[2,3],[1,4],[0,0],[0,0]]).astype('float')\n",
    "# R = np.array([[1,0,0,0,0],[0,0,2,0,3],[0,0,0,0,0],[0,2,0,0,0]])\n",
    "print(\"* 원본 data\")\n",
    "print(R)\n",
    "# print(\"* 각 dimension 별 평균\")\n",
    "# print(np.mean(R, axis=0))\n",
    "# R -= np.mean(R, axis=0)\n",
    "# print(R)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[21.1382289   8.18373463  7.08532049]\n",
      "[[-0.41433905  0.85547047 -0.2918952 ]\n",
      " [-0.56736422 -0.40176078 -0.40797702]\n",
      " [-0.39009474 -0.04848456  0.64441775]\n",
      " [-0.4953958  -0.29891228 -0.16444909]\n",
      " [-0.32989405  0.12271615  0.55320358]]\n",
      "[[-0.41528843  0.66401194  0.62179069]\n",
      " [-0.63626796 -0.70053058  0.32314081]\n",
      " [-0.65015275  0.26142886 -0.71341177]]\n",
      "[[21.1382289   0.          0.        ]\n",
      " [ 0.          8.18373463  0.        ]\n",
      " [ 0.          0.          7.08532049]]\n"
     ]
    }
   ],
   "source": [
    "# we now perform singular value decomposition of X\n",
    "# \"economy size\" (or \"thin\") SVD\n",
    "# U, s, Vt = la.svd(R, full_matrices=True)\n",
    "U, s, Vt = la.svd(R, full_matrices=False)\n",
    "V = Vt.T\n",
    "print(s)\n",
    "S = np.diag(s)\n",
    "print(U)\n",
    "print(V)\n",
    "print(S)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[7., 0., 9.],\n",
       "       [1., 9., 9.],\n",
       "       [6., 7., 2.],\n",
       "       [2., 8., 7.],\n",
       "       [6., 5., 2.]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 7.00000000e+00, -6.04388184e-15,  9.00000000e+00],\n",
       "       [ 1.00000000e+00,  9.00000000e+00,  9.00000000e+00],\n",
       "       [ 6.00000000e+00,  7.00000000e+00,  2.00000000e+00],\n",
       "       [ 2.00000000e+00,  8.00000000e+00,  7.00000000e+00],\n",
       "       [ 6.00000000e+00,  5.00000000e+00,  2.00000000e+00]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.matmul(np.matmul(U, S), V.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trancated SVD(즉 dimensionality reductino을 함): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[8.28596951, 0.66831046, 7.52454243],\n",
       "       [2.79737801, 9.93408631, 6.93777773],\n",
       "       [3.16096196, 5.52456936, 5.25737127],\n",
       "       [2.72449469, 8.37651544, 6.16875077],\n",
       "       [3.56281386, 3.73340933, 4.79630636]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# d = 2 \n",
    "np.matmul(np.matmul(U[:,:2], S[:2,:2]), V.T[:2,:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### numerical 하게 계산해봤다"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[126.  97. 110.]\n",
      " [ 97. 219. 161.]\n",
      " [110. 161. 219.]]\n",
      "[[130.  88.  60.  77.  60.]\n",
      " [ 88. 163.  87. 137.  69.]\n",
      " [ 60.  87.  89.  82.  75.]\n",
      " [ 77. 137.  82. 117.  66.]\n",
      " [ 60.  69.  75.  66.  65.]]\n"
     ]
    }
   ],
   "source": [
    "print(np.matmul(R.T, R))\n",
    "print(np.matmul(R, R.T))\n",
    "C1 = np.matmul(R.T, R)\n",
    "C2 = np.matmul(R, R.T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "eigenvalues = \n",
      " [446.82472096  66.97351257  50.20176648]\n",
      "eigenvectors, which is same with V = \n",
      " [[-0.41528843 -0.66401194 -0.62179069]\n",
      " [-0.63626796  0.70053058 -0.32314081]\n",
      " [-0.65015275 -0.26142886  0.71341177]]\n"
     ]
    }
   ],
   "source": [
    "# # C1 is a symmetric matrix and so it can be diagonalized:\n",
    "l, principal_axes = la.eig(C1)\n",
    "# sort results wrt. eigenvalues\n",
    "idx = l.argsort()[::-1]\n",
    "l, principal_axes = l[idx], principal_axes[:, idx]\n",
    "# the eigenvalues in decreasing order\n",
    "print (\"eigenvalues = \\n\", l)\n",
    "# a matrix of eigenvectors (each column is an eigenvector)\n",
    "V = principal_axes\n",
    "print (\"eigenvectors, which is same with V = \\n\", V)\n",
    "# principal_components = R.dot(principal_axes)\n",
    "# print (\"principal_components = \\n\", principal_components)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$R = U\\sum V^T$ 양변에 오른쪽에 $V$ 를 곱하면, \n",
    "<br> $RV = U\\sum = [s_1 U_1, s_2 U_2,  ... ,s_r U_r] $ 인데 각 column마다 대응되는 singular value 를 나누어주면 \n",
    "$U$ 를 구할 수 있다\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[21.1382289   8.18373463  7.08532049]\n",
      "[[-0.41433905 -0.85547047  0.2918952 ]\n",
      " [-0.56736422  0.40176078  0.40797702]\n",
      " [-0.39009474  0.04848456 -0.64441775]\n",
      " [-0.4953958   0.29891228  0.16444909]\n",
      " [-0.32989405 -0.12271615 -0.55320358]]\n"
     ]
    }
   ],
   "source": [
    "singulars = np.sqrt(l)\n",
    "print(singulars)\n",
    "U = np.matmul(R, V)\n",
    "# print(U)\n",
    "for i, sing in enumerate(singulars):\n",
    "    U[:,i] = U[:,i]/sing\n",
    "print(U)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[21.1382289 ,  0.        ,  0.        ],\n",
       "       [ 0.        ,  8.18373463,  0.        ],\n",
       "       [ 0.        ,  0.        ,  7.08532049]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "S = np.diag(singulars)\n",
    "S"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[7., 0., 9.],\n",
       "       [1., 9., 9.],\n",
       "       [6., 7., 2.],\n",
       "       [2., 8., 7.],\n",
       "       [6., 5., 2.]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "R"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[7.00000000e+00, 1.24779861e-15, 9.00000000e+00],\n",
       "       [1.00000000e+00, 9.00000000e+00, 9.00000000e+00],\n",
       "       [6.00000000e+00, 7.00000000e+00, 2.00000000e+00],\n",
       "       [2.00000000e+00, 8.00000000e+00, 7.00000000e+00],\n",
       "       [6.00000000e+00, 5.00000000e+00, 2.00000000e+00]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.matmul(np.matmul(U, S), V.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "일반적으로, 어떤 rating matrix R을 가지고, 추정한 SVD값과의 RMSE는 최소가 됨이 증명 되어있다. \n",
    "<br>하지만, R에 missing value 가 있을 때에는 SVD를 구할수 없어 prediction 된 U, S, V를 찾아야한다. \n",
    "따라서, 이 predication된 SVD 는 latent factor 모델을 이용한 추천시스템에 사용된다. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}