003_data_visualization_02_low_level_mpl.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Low-level interface with `matplotlib`\n",
    "A potentially confusing feature of Matplotlib is its dual interfaces: a convenient MATLAB-style state-based interface of `pyplot`, and a more powerful object-oriented interface. \n",
    "\n",
    "Matplotlib was originally written as a Python alternative for MATLAB users, and much of its syntax reflects that fact. The MATLAB-style tools are contained in the `pyplot` (`plt`) interface. For example, the following code will probably look quite familiar to MATLAB users:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "x = np.linspace(0,20,1000)\n",
    "\n",
    "plt.figure()  # create a plot figure\n",
    "\n",
    "# create the first of two panels and set current axis\n",
    "plt.subplot(2, 1, 1) # (rows, columns, panel number)\n",
    "plt.plot(x, np.sin(x))\n",
    "\n",
    "# create the second panel and set current axis\n",
    "plt.subplot(2, 1, 2)\n",
    "plt.plot(x, np.cos(x));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This interface is fast and convenient for simple plots, it keeps track of the \"current\" figure and axes, which are where all `plt` commands are applied. But it is easy to run into problems. For example, once the second panel is created, how can we go back and add something to the first? This is possible within the MATLAB-style interface, but a bit clunky. Fortunately, there is a better way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# create figure\n",
    "fig = plt.figure()\n",
    "\n",
    "# add axes to the figure\n",
    "ax1 = fig.add_subplot(2,1,1)\n",
    "ax2 = fig.add_subplot(2,1,2)\n",
    "\n",
    "# call plot() method on the axes\n",
    "ax1.plot(x, np.sin(x))\n",
    "ax2.plot(x, np.cos(x));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we need to look a bit closer to what is going on and what structure does the `matplotlib` figure have. A figure is is a visual container, and it can be created with `plt.figure()` command. It has several parameters, the most used of which is `figsize`, where we can specify the size of the figure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# create a square figure\n",
    "fig = plt.figure(figsize=(5,5))\n",
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, by itself, `figure` doesn't show anything, it is an empty container. We need to populate it with `axes`. Main way to do it is to use method `add_subplot`. Think about this method as if it divides the figure in a grid and adds `axes` in one of the cells of that grid. The arguments this method requires are (1) number of rows of the grid, (2) number of columns of the grid, and (3) in which cell (counting left to right, top to bottom) create an axis. In the example above, we call `add_subplot` twice, each time we ask it to divide the figure in 2 rows and 1 column, and take the each of the axes in turn to plot sine and cosine. If you want just one set of axes on your figure, you can call `add_subplot(1,1,1)`. But here we will create multiple sets of axes. Start with one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ax1 = fig.add_subplot(2,1,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's examine our figure again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that now we have the axes. Axes are what holds the plots. Anything that draws on the axes is called an *Artist*. Very poetic.\n",
    "\n",
    ">**Note** that although the figure we created is a square (5 by 5 units) what we see now is only one set of axes, and they occupy only the top half of our figure, because that's how we defined it.\n",
    "\n",
    "Now let's add another set of axes. And just for the sake of it, instead of doing the second half as one plot, I will split it in two. For that I will need to change the grid in `add_subplot` to a 2 by 2, instead of 2 by 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# first and second cell on that grid are on the top half\n",
    "# so I take the third cell\n",
    "ax2 = fig.add_subplot(2,2,3)\n",
    "fig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# and the forth cell\n",
    "ax3 = fig.add_subplot(2,2,4)\n",
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's remind ourselves that this is a hierarchical structure. Figure contains axes, which in turn contain Artists. We can use method `get_children()` on both figure and axes to see what they contain. E.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig.get_children()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Figure contains 4 objects. A rectangle (which is the background of the figure) and 3 sets of axes, which we just created.\n",
    "\n",
    "Let's examine children of one of the axes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ax1.get_children()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Axes already contain a lot of things things from the start, which is can be evident from just looking at them. They contain spines, and ticks, and background rectangle and so on. Although we usually don't do it, we can change things directly on on these objects. For example, I can take the first element from that list above and change something, for example color. Almost all objects in `matplotlib` have method `set_color()` specifically for that purpose:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ax1.get_children()[0].set_color('red')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# now let's re-examine our figure\n",
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that one of the sides of the box became red. This shows you that potentially you have control over any detail of your figures, plots and axes, at a very low level. Everything is very transparent here.\n",
    "\n",
    "However, it would be very cumbersome to change all of these things manually every time we need. It would require a lot of code and settings to get everything just the way you want it. To simplify interaction with the plots, `matplotlib` has higher-level methods.\n",
    "\n",
    "Let's start with methods that plot something on the axes and then go to the settings. These are already familiar to us from the `pyplot` API, e.g. `plot`, `scatter`, `violinplot`, `boxplot`, `hist`, etc. But now instead of calling them as functions, we call them as methods on the axes where we want to create them! Very straightforward:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ax2.plot(np.sin(x))\n",
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can easily go back and plot something on another set of axes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ax1.scatter([1,4,2,3,5],[1,2,3,4,5])\n",
    "fig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ax3.violinplot([1,2,3,4,5,23,4,3,1,5,7,3])\n",
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With regards to the settings of the axes, the full list of methods can be found on `matplotlib` [Axes documentation](https://matplotlib.org/api/axes_api.html) (not that each method has many settings, which are not listed on that page, but rather in the details of the method). \n",
    "\n",
    "But since the library is extensively used, specific things can be easily found by searching \"matplotlib `<`what you want to do`>`\", usually with comprehensive explanations on Stackoverflow. Let's see some examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# remove X and Y axes\n",
    "ax1.set_axis_off()\n",
    "fig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# turn grid on\n",
    "ax2.grid('on')\n",
    "fig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# get ticks on the X axis and then set the ticks 1/1000 \n",
    "# of the original values and rotate them to 45 degrees\n",
    "t = ax2.get_xticks()\n",
    "ax2.set_xticklabels(t/1000, rotation=45)\n",
    "fig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# set the title of the subplots\n",
    "ax2.set_title('This is sine wave')\n",
    "ax3.set_title('This is violin plot')\n",
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the possibilities are limitless with `matplotlib`, and hopefully the structure seems transparent to you too.\n",
    "\n",
    "Let's see another example with real data. We will plot the scatter of the Iris dataset, and the histogram for each of the species separately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "iris = pd.read_csv('data/iris.csv')\n",
    "iris.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(8,8))\n",
    "scat = fig.add_subplot(2,1,1)\n",
    "# sharex will bind the X axis of the histograms to the x axis \n",
    "# of the scatterplot, so that their limits will be always the same\n",
    "hist = [fig.add_subplot(2,3,4, sharex=scat),\n",
    "        fig.add_subplot(2,3,5, sharex=scat),\n",
    "        fig.add_subplot(2,3,6, sharex=scat)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "species_colors = {'setosa':     plt.cm.Set1.colors[0],\n",
    "                  'versicolor': plt.cm.Set1.colors[1],\n",
    "                  'virginica':  plt.cm.Set1.colors[2]}\n",
    "\n",
    "for i, (sp, data) in enumerate(iris.groupby('species')):\n",
    "    scat.plot(data['sepal length'],data['sepal width'],'o',\n",
    "              color=species_colors[sp])\n",
    "    hist[i].hist(data['sepal length'], \n",
    "                 color=species_colors[sp])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "fig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color='DarkSeaGreen'>Exercise</font>\n",
    "Using brain sizes dataset (`data/brain_size.csv`), create a compound figure with 3 axes in 1 row, on each of them create a scatterplot of one of the IQ scores (Y axis) based on MRI_Count (X axis). If you feel like being challenged, color the points according to the gender.\n",
    "\n",
    "Further adjustements:\n",
    "- set the title of each subplot to be the IQ score type on that plot\n",
    "- set the X axis label on each plot to 'MRI_count'\n",
    "- show the legend which would say the gender is displayed in which color; show the legend outside the figure on the right (you'd have to search how to do that)\n",
    "- plot the mean score for each group as a horisontal dashed line"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "brains = pd.read_csv('data/brain_size.csv')\n",
    "brains.replace({'.':np.nan}, inplace=True)\n",
    "\n",
    "brains.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> [Brain sizes dataset [1]](http://wps.aw.com/wps/media/objects/14/15269/projects/ch6_iq/brain.html): \n",
    "\n",
    "> *Gender*: Male or Female\n",
    "\n",
    "> *FSIQ*: Full Scale IQ scores based on the four Wechsler (1981) subtests\n",
    "\n",
    "> *VIQ*: Verbal IQ scores based on the four Wechsler (1981) subtests\n",
    "\n",
    "> *PIQ*: Performance IQ scores based on the four Wechsler (1981) subtests\n",
    "\n",
    "> *Weight*: body weight in pounds\n",
    "\n",
    "> *Height*: height in inches\n",
    "\n",
    "> *MRICount*: total pixel Count from the 18 MRI scans\n",
    "\n",
    "> [1] Willerman, L., Schultz, R., Rutledge, J. N., and Bigler, E. (1991), \"In Vivo Brain Size and Intelligence,\" Intelligence, 15, 223-228\n",
    "\n",
    "[Solution](https://anotepad.com/notes/qgk6j6)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6",
   "language": "python",
   "name": "py36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}