00. NumPy Basics/1. NumPy Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy enables fast computation in Python, the underlying implementation is in C hence it's blazing fast. The Key feature of NumPy is the `ndarray` object. The data type should be homogenous, that is the array should contain elements of single data type.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vector: [1 2 3 4]\n",
      "Shape: (4,)\n",
      "Dim: 1\n",
      "Data type: int64\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "vector = np.array([1, 2, 3, 4])\n",
    "print(\"Vector: {}\".format(vector))\n",
    "# Every array will have a shape. That is, its dimensions\n",
    "print(\"Shape: {}\".format(vector.shape))\n",
    "# Print number of dimensions\n",
    "print(\"Dim: {}\".format(vector.ndim))\n",
    "print(\"Data type: {}\".format(vector.dtype))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The number of dimensions numpy uses is as follows:\n",
    "\n",
    "`(depth, rows, columns) `\n",
    "\n",
    "So a 3D array of 3 rows 2 columns and 2 depth will have following shape:\n",
    "\n",
    "`(2, 3, 2)`\n",
    "\n",
    "Which is somewhat counter intuitive since we expect \"2\" to be on the other side. This is the convention followed in mostly all libraries. E.g. OpenCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[ 1  2]\n",
      "  [ 3  4]\n",
      "  [ 5  6]]\n",
      "\n",
      " [[ 7  8]\n",
      "  [ 9 10]\n",
      "  [11 12]]]\n"
     ]
    }
   ],
   "source": [
    "v = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])\n",
    "v.shape = (2, 3, 2)\n",
    "print(v)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[ 0.  0.]\n",
      "  [ 0.  0.]\n",
      "  [ 0.  0.]]\n",
      "\n",
      " [[ 0.  0.]\n",
      "  [ 0.  0.]\n",
      "  [ 0.  0.]]]\n"
     ]
    }
   ],
   "source": [
    "v = np.zeros((2, 3, 2))\n",
    "print(v)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## arange\n",
    "The `arange` function is similar to Python's `range` function. The data type, if not specified, in many cases will be `np.foat64`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14]\n"
     ]
    }
   ],
   "source": [
    "a = np.arange(15)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## zeros, zeros_like\n",
    "`zeros(dim)` will return a np.array of `dim` dimensions initialised with 0. Note that `dim` should be a `tuple`. \n",
    "\n",
    "`zeros_like(array)` will return a np.array of same dimensions as of `array` initialised with zeros. \n",
    "\n",
    "#### same functionality is with `ones` , and `ones_like` except of course the initialization is done with ones. \n",
    "\n",
    "#### same functionality with `empty`, and `empty_like` which will create numpy arrays but won't initialise it with anything (hence, faster) By default, all the values in the array will have garbage values. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Zeros\n",
      "A: [[ 0.  0.  0.]\n",
      " [ 0.  0.  0.]\n",
      " [ 0.  0.  0.]]\n",
      "B: [[ 0.  0.  0.]\n",
      " [ 0.  0.  0.]\n",
      " [ 0.  0.  0.]]\n",
      "\n",
      "Ones\n",
      "A: [[ 1.  1.  1.]\n",
      " [ 1.  1.  1.]\n",
      " [ 1.  1.  1.]]\n",
      "B: [[ 1.  1.  1.]\n",
      " [ 1.  1.  1.]\n",
      " [ 1.  1.  1.]]\n",
      "\n",
      "Empty\n",
      "A: [[ 1.  1.  1.]\n",
      " [ 1.  1.  1.]\n",
      " [ 1.  1.  1.]]\n",
      "B: [[ 1.  1.  1.]\n",
      " [ 1.  1.  1.]\n",
      " [ 1.  1.  1.]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Zeros\")\n",
    "a = np.zeros((3, 3))\n",
    "print(\"A: {}\".format(a))\n",
    "b = np.zeros_like(a)\n",
    "print(\"B: {}\".format(b))\n",
    "print(\"\\nOnes\")\n",
    "a = np.ones((3, 3))\n",
    "print(\"A: {}\".format(a))\n",
    "b = np.ones_like(a)\n",
    "print(\"B: {}\".format(b))\n",
    "print(\"\\nEmpty\")\n",
    "a = np.empty((3, 3))\n",
    "print(\"A: {}\".format(a))\n",
    "b = np.empty_like(a)\n",
    "print(\"B: {}\".format(b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## astype\n",
    "`astype` will convert one data type to another. Also note that `astype` will __create a new copy of the input array (even if the data type is same)__. \n",
    "\n",
    "Also, converting from higher precision (like float) to lower precision (like int) will cause loss of information (decimal part is lost in case of float to int). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A: [ 1.   2.   3.   4.5  6.7], dtype: float64\n",
      "B: [1 2 3 4 6], dtype: int64\n"
     ]
    }
   ],
   "source": [
    "a = np.array([1, 2, 3, 4.5, 6.7])\n",
    "print(\"A: {}, dtype: {}\".format(a, a.dtype))\n",
    "b = a.astype(np.int)\n",
    "print(\"B: {}, dtype: {}\".format(b, b.dtype))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vectorization and vector-scalar operations\n",
    "Using for loops in code is not only prone to error but also is inefficient. We can use NumPy operations to circumvent such for loops. This process is called vectorization. \n",
    "\n",
    "#### Using operations on same sized arrays produce element wise operations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5 7 9]\n",
      " [5 7 9]]\n",
      "[[ 4 10 18]\n",
      " [ 4 10 18]]\n",
      "[[-3 -3 -3]\n",
      " [ 3  3  3]]\n"
     ]
    }
   ],
   "source": [
    "a = np.array([[1, 2, 3], [4, 5, 6]])\n",
    "b = np.array([[4, 5, 6], [1, 2, 3]])\n",
    "\n",
    "c = a + b\n",
    "print(c)\n",
    "\n",
    "c = a * b\n",
    "print(c)\n",
    "\n",
    "c = a - b\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using scalars with vectors will produce element wise operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[4 5 6]\n",
      " [7 8 9]]\n",
      "[[ 3  6  9]\n",
      " [12 15 18]]\n",
      "[[ 1.          0.5         0.33333333]\n",
      " [ 0.25        0.2         0.16666667]]\n"
     ]
    }
   ],
   "source": [
    "a = 3\n",
    "b = np.array([[1, 2, 3], [4, 5, 6]])\n",
    "\n",
    "c = a + b\n",
    "print(c)\n",
    "\n",
    "c = a * b\n",
    "print(c)\n",
    "\n",
    "c = 1.0 / b\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slicing \n",
    "\n",
    "You can slice by following syntax:\n",
    "```\n",
    "array[start_index:end_index] \n",
    "```\n",
    "For n-dimensional array:\n",
    "```\n",
    "array[start_index:end_index, start_index:end_index] \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Slicing NumPy arrays is similar to that of Python lists. One main distinction in Python list and NumPy array is that the slice __is not the copy, but the original array. Hence, if any operations on the slice will be reflected in the original array.__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19]\n",
      "[ 0  1  2  3  4  5  6  7  8  9  5  5  5  5  5 15 16 17 18 19]\n"
     ]
    }
   ],
   "source": [
    "a = np.arange(20)\n",
    "print(a)\n",
    "a[10:15] = 5\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to avoid above scenario, you can use `copy()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19]\n",
      "[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19]\n"
     ]
    }
   ],
   "source": [
    "a = np.arange(20)\n",
    "print(a)\n",
    "b = a[10:15].copy()\n",
    "b = 5\n",
    "# value in the original array doesn't change\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Slicing by `:` will take the entire axis. So:\n",
    "```\n",
    "1. arr2d[:, 0]         Will return array of shape (3, )\n",
    "2. arr2d[:, :1]        Will return array of shape (3, 1)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 4, 7])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arr2d = np.array([[1, 2, 3], \n",
    "                  [4, 5, 6], \n",
    "                  [7, 8, 9]])\n",
    "arr2d[:, 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boolean indexing\n",
    "Using boolen indexing, you can use it to filter or check if any entries have any specific values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True, False, False, False, False], dtype=bool)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([\"Mayur\", \"is\", \"an\", \"awesome\", \"coder\"])\n",
    "a == \"Mayur\" # Returns boolean array\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['is', 'an', 'awesome', 'coder'],\n",
       "      dtype='|S7')"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# lists entry where value != \"Mayur\" \n",
    "a[~(a == \"Mayur\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__You can use `|` for `or` and `&` for `and` but not Python's `and`, `or` will not work with NumPy's indexing. __"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True, False, False, False,  True], dtype=bool)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(a == \"Mayur\") | (a == \"coder\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fancy indexing\n",
    "You can index a list to print the array in the given order. For instance, you want to print the 1st row, 3rd row, and 2nd row in that order. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 1 1]\n",
      " [2 2 2]\n",
      " [3 3 3]\n",
      " [4 4 4]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[2, 2, 2],\n",
       "       [4, 4, 4],\n",
       "       [3, 3, 3]])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([[1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4]])\n",
    "print(a)\n",
    "a[[1, 3, 2]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transposing\n",
    "\n",
    "You can obtains the transpose of your matring using `matrix.T` where `matrix` is your matrix name. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2 3 4]\n",
      " [1 2 3 4]\n",
      " [1 2 3 4]]\n"
     ]
    }
   ],
   "source": [
    "a = np.array([[1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4]])\n",
    "print(a.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Universal Functions\n",
    "NumPy has variety of functions that can be applied to scalars as well as vectors. Some examples are sqrt, exp, log, log10, sin, cos, arcsin etc. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "485165195.41\n",
      "[[ 1.20093606  1.55455582]\n",
      " [ 2.45014857  1.87287753]]\n"
     ]
    }
   ],
   "source": [
    "a = 20\n",
    "b = np.random.rand(2, 2)\n",
    "print(np.exp(a))\n",
    "print(np.exp(b))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## meshgrid\n",
    "\n",
    "One of the most useful function is meshgrid. It's used to visualize data boundaries of your classifier. What you do is train your classifier, then create a meshgrid of every pixel in the plot, and then classify the pixel. When you give the pixel a specific color according to the labelled class you can clearly visualize the boundaries. \n",
    "\n",
    "Using meshgrid requires three steps. \n",
    "1. Create xs (1D array)\n",
    "2. Create ys (1D array)\n",
    "3. Create meshgrid (2D array) which corresponds to every pixel in the graph.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  1.        ,   1.09090909,   1.18181818, ...,   9.81818182,\n",
       "          9.90909091,  10.        ],\n",
       "       [  1.        ,   1.09090909,   1.18181818, ...,   9.81818182,\n",
       "          9.90909091,  10.        ],\n",
       "       [  1.        ,   1.09090909,   1.18181818, ...,   9.81818182,\n",
       "          9.90909091,  10.        ],\n",
       "       ..., \n",
       "       [  1.        ,   1.09090909,   1.18181818, ...,   9.81818182,\n",
       "          9.90909091,  10.        ],\n",
       "       [  1.        ,   1.09090909,   1.18181818, ...,   9.81818182,\n",
       "          9.90909091,  10.        ],\n",
       "       [  1.        ,   1.09090909,   1.18181818, ...,   9.81818182,\n",
       "          9.90909091,  10.        ]])"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xs = np.linspace(1, 10, 100)\n",
    "ys = np.linspace(1, 10, 100)\n",
    "xx, yy = np.meshgrid(xs, ys)\n",
    "# plot with xx and yy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# where\n",
    "if you have 3 arrays x, y, and condition then, `np.where` is replacement for using:\n",
    "```\n",
    "if condition: \n",
    "    use x\n",
    "else:\n",
    "    use y\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0,  3,  2,  3,  2, -5])"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = [0, -1, 2, 3, -4, -5]\n",
    "b = [9, 3, 4, 11, 2, 3]\n",
    "c = [True, False, True, True, False, True]\n",
    "np.where(c, a, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mean, sum, std\n",
    "NumPy provides variety of functions for statistical use. You can furthermore specify the axis you want to reduce. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.47420622  0.43247366  0.93400638]\n",
      " [ 0.75673826  0.02606293  0.96672388]\n",
      " [ 0.00133784  0.89357209  0.08779815]]\n",
      "0.508102157536\n",
      "0.508102157536\n",
      "0.376361414067\n"
     ]
    }
   ],
   "source": [
    "a = np.random.rand(3, 3)\n",
    "print(a)\n",
    "print(np.mean(a)) # both are fine\n",
    "print(a.mean())\n",
    "\n",
    "print(np.std(a))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}