notebooks/02 Scientific Computing Tools in Python.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%load_ext watermark\n",
    "%watermark  -d -u -a 'Andreas Mueller, Kyle Kastner, Sebastian Raschka' -v -p numpy,scipy,matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The use of watermark (above) is optional, and we use it to keep track of the changes while developing the tutorial material. (You can install this IPython extension via \"pip install watermark\". For more information, please see: https://github.com/rasbt/watermark)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SciPy 2016 Scikit-learn Tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "01.2 Jupyter Notebooks\n",
    "==================\n",
    "\n",
    "* You can run a cell by pressing ``[shift] + [Enter]`` or by pressing the \"play\" button in the menu.\n",
    "\n",
    "![](figures/ipython_run_cell.png)\n",
    "\n",
    "* You can get help on a function or object by pressing ``[shift] + [tab]`` after the opening parenthesis ``function(``\n",
    "\n",
    "![](figures/ipython_help-1.png)\n",
    "\n",
    "* You can also get help by executing ``function?``\n",
    "\n",
    "![](figures/ipython_help-2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numpy Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Manipulating `numpy` arrays is an important part of doing machine learning\n",
    "(or, really, any type of scientific computation) in python.  This will likely\n",
    "be a short review for most. In any case, let's quickly go through some of the most important features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Setting a random seed for reproducibility\n",
    "rnd = np.random.RandomState(seed=123)\n",
    "\n",
    "# Generating a random array\n",
    "X = rnd.uniform(low=0.0, high=1.0, size=(3, 5))  # a 3 x 5 array\n",
    "\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Note that NumPy arrays use 0-indexing just like other data structures in Python.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Accessing elements\n",
    "\n",
    "# get a single element \n",
    "# (here: an element in the first row and column)\n",
    "print(X[0, 0])\n",
    "\n",
    "# get a row \n",
    "# (here: 2nd row)\n",
    "print(X[1])\n",
    "\n",
    "# get a column\n",
    "# (here: 2nd column)\n",
    "print(X[:, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Transposing an array\n",
    "print(X.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\begin{bmatrix}\n",
    "    1 & 2 & 3 & 4 \\\\\n",
    "    5 & 6 & 7 & 8\n",
    "\\end{bmatrix}^T\n",
    "= \n",
    "\\begin{bmatrix}\n",
    "    1 & 5 \\\\\n",
    "    2 & 6 \\\\\n",
    "    3 & 7 \\\\\n",
    "    4 & 8\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Creating a row vector\n",
    "# of evenly spaced numbers over a specified interval.\n",
    "y = np.linspace(0, 12, 5)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Turning the row vector into a column vector\n",
    "print(y[:, np.newaxis])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Getting the shape or reshaping an array\n",
    "\n",
    "# Generating a random array\n",
    "rnd = np.random.RandomState(seed=123)\n",
    "X = rnd.uniform(low=0.0, high=1.0, size=(3, 5))  # a 3 x 5 array\n",
    "\n",
    "print(X.shape)\n",
    "print(X.reshape(5, 3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Indexing by an array of integers (fancy indexing)\n",
    "indices = np.array([3, 1, 0])\n",
    "print(indices)\n",
    "X[:, indices]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is much, much more to know, but these few operations are fundamental to what we'll\n",
    "do during this tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scipy Sparse Matrices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We won't make very much use of these in this tutorial, but sparse matrices are very nice\n",
    "in some situations.  In some machine learning tasks, especially those associated\n",
    "with textual analysis, the data may be mostly zeros.  Storing all these zeros is very\n",
    "inefficient, and representing in a way that only contains the \"non-zero\" values can be much more efficient.  We can create and manipulate sparse matrices as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from scipy import sparse\n",
    "\n",
    "# Create a random array with a lot of zeros\n",
    "rnd = np.random.RandomState(seed=123)\n",
    "\n",
    "X = rnd.uniform(low=0.0, high=1.0, size=(10, 5))\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# set the majority of elements to zero\n",
    "X[X < 0.7] = 0\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# turn X into a CSR (Compressed-Sparse-Row) matrix\n",
    "X_csr = sparse.csr_matrix(X)\n",
    "print(X_csr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Converting the sparse matrix to a dense array\n",
    "print(X_csr.toarray())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(You may have stumbled upon an alternative method for converting sparse to dense representations: `numpy.todense`; `toarray` returns a NumPy array, whereas `todense` returns a NumPy matrix. In this tutorial, we will be working with NumPy arrays, not matrices; the latter are not supported by scikit-learn.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CSR representation can be very efficient for computations, but it is not\n",
    "as good for adding elements.  For that, the LIL (List-In-List) representation\n",
    "is better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Create an empty LIL matrix and add some items\n",
    "X_lil = sparse.lil_matrix((5, 5))\n",
    "\n",
    "for i, j in np.random.randint(0, 5, (15, 2)):\n",
    "    X_lil[i, j] = i + j\n",
    "\n",
    "print(X_lil)\n",
    "print(type(X_lil))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_dense = X_lil.toarray()\n",
    "print(X_dense)\n",
    "print(type(X_dense))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Often, once an LIL matrix is created, it is useful to convert it to a CSR format\n",
    "(many scikit-learn algorithms require CSR or CSC format)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_csr = X_lil.tocsr()\n",
    "print(X_csr)\n",
    "print(type(X_csr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The available sparse formats that can be useful for various problems are:\n",
    "\n",
    "- `CSR` (compressed sparse row)\n",
    "- `CSC` (compressed sparse column)\n",
    "- `BSR` (block sparse row)\n",
    "- `COO` (coordinate)\n",
    "- `DIA` (diagonal)\n",
    "- `DOK` (dictionary of keys)\n",
    "- `LIL` (list in list)\n",
    "\n",
    "The [``scipy.sparse``](http://docs.scipy.org/doc/scipy/reference/sparse.html) submodule also has a lot of functions for sparse matrices\n",
    "including linear algebra, sparse solvers, graph algorithms, and much more."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another important part of machine learning is the visualization of data.  The most common\n",
    "tool for this in Python is [`matplotlib`](http://matplotlib.org).  It is an extremely flexible package, and\n",
    "we will go over some basics here.\n",
    "\n",
    "Since we are using Jupyter notebooks, let us use one of IPython's convenient built-in \"[magic functions](https://ipython.org/ipython-doc/3/interactive/magics.html)\", the \"matoplotlib inline\" mode, which will draw the plots directly inside the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Plotting a line\n",
    "x = np.linspace(0, 10, 100)\n",
    "plt.plot(x, np.sin(x));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Scatter-plot points\n",
    "x = np.random.normal(size=500)\n",
    "y = np.random.normal(size=500)\n",
    "plt.scatter(x, y);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Showing images using imshow\n",
    "# - note that origin is at the top-left by default!\n",
    "\n",
    "x = np.linspace(1, 12, 100)\n",
    "y = x[:, np.newaxis]\n",
    "\n",
    "im = y * np.sin(x) * np.cos(y)\n",
    "print(im.shape)\n",
    "\n",
    "plt.imshow(im);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Contour plots \n",
    "# - note that origin here is at the bottom-left by default!\n",
    "plt.contour(im);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 3D plotting\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "ax = plt.axes(projection='3d')\n",
    "xgrid, ygrid = np.meshgrid(x, y.ravel())\n",
    "ax.plot_surface(xgrid, ygrid, im, cmap=plt.cm.jet, cstride=2, rstride=2, linewidth=0);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many, many more plot types available.  One useful way to explore these is by\n",
    "looking at the [matplotlib gallery](http://matplotlib.org/gallery.html).\n",
    "\n",
    "You can test these examples out easily in the notebook: simply copy the ``Source Code``\n",
    "link on each page, and put it in a notebook using the ``%load`` magic.\n",
    "For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# %load http://matplotlib.org/mpl_examples/pylab_examples/ellipse_collection.py\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from matplotlib.collections import EllipseCollection\n",
    "\n",
    "x = np.arange(10)\n",
    "y = np.arange(15)\n",
    "X, Y = np.meshgrid(x, y)\n",
    "\n",
    "XY = np.hstack((X.ravel()[:,np.newaxis], Y.ravel()[:,np.newaxis]))\n",
    "\n",
    "ww = X/10.0\n",
    "hh = Y/15.0\n",
    "aa = X*9\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "\n",
    "ec = EllipseCollection(ww, hh, aa, units='x', offsets=XY,\n",
    "                       transOffset=ax.transData)\n",
    "ec.set_array((X+Y).ravel())\n",
    "ax.add_collection(ec)\n",
    "ax.autoscale_view()\n",
    "ax.set_xlabel('X')\n",
    "ax.set_ylabel('y')\n",
    "cbar = plt.colorbar(ec)\n",
    "cbar.set_label('X+Y')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}