Mathematics/LinearAlgebra/intro_to_linear_algebra.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#What-&amp;-Why-of-Linear-Algebra\" data-toc-modified-id=\"What-&amp;-Why-of-Linear-Algebra-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>What &amp; Why of Linear Algebra</a></span></li><li><span><a href=\"#Different-Tensors\" data-toc-modified-id=\"Different-Tensors-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>Different Tensors</a></span><ul class=\"toc-item\"><li><span><a href=\"#Scalars,-Vectors,-Matrices:-It's-all-about-the-dimension\" data-toc-modified-id=\"Scalars,-Vectors,-Matrices:-It's-all-about-the-dimension-2.1\"><span class=\"toc-item-num\">2.1&nbsp;&nbsp;</span>Scalars, Vectors, Matrices: It's all about the dimension</a></span></li><li><span><a href=\"#Creating-with-NumPy\" data-toc-modified-id=\"Creating-with-NumPy-2.2\"><span class=\"toc-item-num\">2.2&nbsp;&nbsp;</span>Creating with NumPy</a></span><ul class=\"toc-item\"><li><span><a href=\"#Indexing-with-NumPy\" data-toc-modified-id=\"Indexing-with-NumPy-2.2.1\"><span class=\"toc-item-num\">2.2.1&nbsp;&nbsp;</span>Indexing with NumPy</a></span><ul class=\"toc-item\"><li><span><a href=\"#Different-parts-of-a-vector\" data-toc-modified-id=\"Different-parts-of-a-vector-2.2.1.1\"><span class=\"toc-item-num\">2.2.1.1&nbsp;&nbsp;</span>Different parts of a vector</a></span></li><li><span><a href=\"#Different-parts-of-a-matrix\" data-toc-modified-id=\"Different-parts-of-a-matrix-2.2.1.2\"><span class=\"toc-item-num\">2.2.1.2&nbsp;&nbsp;</span>Different parts of a matrix</a></span></li><li><span><a href=\"#Different-parts-of-a-tensor\" data-toc-modified-id=\"Different-parts-of-a-tensor-2.2.1.3\"><span class=\"toc-item-num\">2.2.1.3&nbsp;&nbsp;</span>Different parts of a tensor</a></span></li></ul></li></ul></li></ul></li><li><span><a href=\"#Basic-Properties\" data-toc-modified-id=\"Basic-Properties-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span>Basic Properties</a></span><ul class=\"toc-item\"><li><span><a href=\"#Shape\" data-toc-modified-id=\"Shape-3.1\"><span class=\"toc-item-num\">3.1&nbsp;&nbsp;</span>Shape</a></span></li><li><span><a href=\"#Transpose\" data-toc-modified-id=\"Transpose-3.2\"><span class=\"toc-item-num\">3.2&nbsp;&nbsp;</span>Transpose</a></span></li></ul></li></ul></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.367277Z",
     "start_time": "2019-10-16T15:21:21.365024Z"
    }
   },
   "outputs": [],
   "source": [
    "# Imports needed for this notebook\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# What & Why of Linear Algebra"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "- Study of \"vector spaces\"; relationship of **linear** relationships\n",
    "- Uses vectors, matrices, and tensors\n",
    "- Mapping & dimensionality (PCA)\n",
    "- Used in lots of ML applications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "We'll try to put abstract ideas into the formalism of linear algebra\n",
    " - images/pixels\n",
    " - language (NLP)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Different Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Scalars, Vectors, Matrices: It's all about the dimension"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "![different_tensors.png](images/different_tensors.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Creating with NumPy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.380622Z",
     "start_time": "2019-10-16T15:21:21.371905Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# Scalar\n",
    "s = np.arange(1)\n",
    "display(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.387284Z",
     "start_time": "2019-10-16T15:21:21.382214Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# Vector\n",
    "v = np.arange(4)\n",
    "display(v)\n",
    "\n",
    "# Other ways to define vector\n",
    "x = np.linspace(-np.pi, np.pi, 10)\n",
    "display(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.392587Z",
     "start_time": "2019-10-16T15:21:21.388923Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# Matrix\n",
    "M = np.arange(4 * 2).reshape((4, 2))\n",
    "display(M)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.397895Z",
     "start_time": "2019-10-16T15:21:21.394636Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# 3D Tensor\n",
    "T_3d = np.arange(4 * 2 * 3).reshape((4, 2, 3))\n",
    "display(T_3d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "### Indexing with NumPy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "#### Different parts of a vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.407147Z",
     "start_time": "2019-10-16T15:21:21.399748Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# For Vectors\n",
    "display(v[1:4])  # second to fourth element. Element 5 is not included\n",
    "display(v[::2])  # every other element\n",
    "display(v[:])    # print the whole vector\n",
    "display(v[::-1]) # reverse the vector!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "#### Different parts of a matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:53.929360Z",
     "start_time": "2019-10-16T15:21:53.925928Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(M[0, 0])   # element at first row and first column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:54.489263Z",
     "start_time": "2019-10-16T15:21:54.484756Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(M[-1, -1]) # elemenet last row and last column "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:56.104738Z",
     "start_time": "2019-10-16T15:21:56.100969Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(M[0, :])   # first row and all columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:22:05.427391Z",
     "start_time": "2019-10-16T15:22:05.424350Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(M[:, 0])   # all rows and first column "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:22:10.021962Z",
     "start_time": "2019-10-16T15:22:10.017942Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(M[:])      # all rows and all columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "#### Different parts of a tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.430604Z",
     "start_time": "2019-10-16T15:21:21.427554Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print(T_3d[0])      # 2D: First matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.430604Z",
     "start_time": "2019-10-16T15:21:21.427554Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print(T_3d[0, 0])   # 1D: First matrix's first vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.430604Z",
     "start_time": "2019-10-16T15:21:21.427554Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print(T_3d[0,0, 0]) # 0D: First matrix's first vector's first element"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.436755Z",
     "start_time": "2019-10-16T15:21:21.432883Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print(T_3d[0, 0, :])  # 1D: first matrix, first vector, all elements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.440823Z",
     "start_time": "2019-10-16T15:21:21.437945Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print(T_3d[0, :, 0])  # 1D: first matrix, all the vectors, just the fist element"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.444909Z",
     "start_time": "2019-10-16T15:21:21.442659Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print(T_3d[0, :, 1:]) # 1D: first matrix, all the vectors, all elements after the first"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "# Basic Properties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Shape\n",
    "\n",
    "Can help us know the dimensions and size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:22:53.144222Z",
     "start_time": "2019-10-16T15:22:53.138488Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print('Scalar:')\n",
    "s = np.array(100)\n",
    "print(s)\n",
    "display(s.shape)\n",
    "display(s.size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:23:04.680486Z",
     "start_time": "2019-10-16T15:23:04.674597Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print('Vector:')\n",
    "print(v)\n",
    "display(v.shape)\n",
    "display(v.size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.464503Z",
     "start_time": "2019-10-16T15:21:21.460332Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print('Matrix:')\n",
    "print(M)\n",
    "display(M.shape)\n",
    "display(M.size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.471780Z",
     "start_time": "2019-10-16T15:21:21.466891Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "print('3D Tensor:')\n",
    "print(T_3d)\n",
    "display(T_3d.shape)\n",
    "display(T_3d.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "hidden": true
   },
   "source": [
    "## Transpose"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "hidden": true
   },
   "source": [
    "![transpose_tensors.png](images/transpose_tensors.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.480646Z",
     "start_time": "2019-10-16T15:21:21.473815Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(M.shape)\n",
    "print(M)\n",
    "\n",
    "display(M.T.shape)\n",
    "# Alternative to: M.T\n",
    "print(np.transpose(M))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-16T15:21:21.489042Z",
     "start_time": "2019-10-16T15:21:21.481906Z"
    },
    "hidden": true
   },
   "outputs": [],
   "source": [
    "display(T_3d.shape)\n",
    "print(T_3d)\n",
    "\n",
    "display(T_3d.T.shape)\n",
    "print(T_3d.T)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}