notebooks/02.02-Loading-External-Datasets-in-Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--BOOK_INFORMATION-->\n",
    "<a href=\"https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv\" target=\"_blank\"><img align=\"left\" src=\"data/cover.jpg\" style=\"width: 76px; height: 100px; background: white; padding: 1px; border: 1px solid black; margin-right:10px;\"></a>\n",
    "*This notebook contains an excerpt from the book [Machine Learning for OpenCV](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv) by Michael Beyeler.\n",
    "The code is released under the [MIT license](https://opensource.org/licenses/MIT),\n",
    "and is available on [GitHub](https://github.com/mbeyeler/opencv-machine-learning).*\n",
    "\n",
    "*Note that this excerpt contains only the raw code - the book is rich with additional explanations and illustrations.\n",
    "If you find this content useful, please consider supporting the work by\n",
    "[buying the book](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv)!*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "< [Dealing with Data Using Python's NumPy Package](02.01-Dealing-with-Data-Using-Python-NumPy.ipynb) | [Contents](../README.md) | [Visualizing Data Using Matplotlib](02.03-Visualizing-Data-Using-Matplotlib.ipynb) >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading External Datasets in Python\n",
    "\n",
    "Thanks to the SciPy community, there are tons of resources out there for getting your hands on some data.\n",
    "\n",
    "A particularly useful resource comes in the form of the sklearn.datasets package of scikit-learn. This package comes pre-installed with some small datasets that do not require to download any files from external websites. These datasets include:\n",
    "- <tt>load_boston</tt>: The Boston dataset contains housing prices in different suburbs of Boston, along with a number of interesting features such as per capita crime rate by town, proportion of residential land and non-retail business, etc.\n",
    "- <tt>load_iris</tt>: The Iris dataset contains three different types of iris flowers (setosa, versicolor, and virginica), along with four features describing the width and length of the sepals and petals.\n",
    "- <tt>load_diabetes</tt>: The diabetes dataset lets you classify patients as having diabetes or not, based on features such as patient age, sex, body mass index, average blood pressure, and six blood serum measurements.\n",
    "- <tt>load_digits</tt>: The digits dataset contains 8x8 pixel images of digits 0-9.\n",
    "- <tt>load_linnerud</tt>: The Linnerud dataset contains three physiological and three exercies variables measured on twenty middle-aged men in a fitness club.\n",
    "\n",
    "Even better, scikit-learn allows you to download datasets directly from external repositories, such as:\n",
    "- <tt>fetch_olivetti_faces</tt>: The Olivetta face dataset contains of ten different images each of 40 distinct subjects.\n",
    "- <tt>fetch_20newsgroups</tt>: The 20 newsgroup dataset contains around 18,000 newsgroups post on 20 topics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even better, it is possible to download datasets directly from the machine learning database at http://mldata.org. \n",
    "\n",
    "For example, to download the MNIST dataset of handwritten digits, simply type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnist = datasets.fetch_mldata('MNIST original')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this might take a while, depending on your internet connection.\n",
    "\n",
    "The MNIST database contains a total of 70,000 examples of handwritten digits (28x28 pixel images, labeled from 0 to 9). Data and labels are delivered in two separate containers, which we can inspect as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(70000, 784)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist.data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(70000,)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist.target.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we can see that mnist.data contains 70,000 images of 28 x 28 = 784 pixels each.\n",
    "Labels are stored in `mnist.target`, where there is only one label per image.\n",
    "\n",
    "We can further inspect the values of all targets, but we don't just want to print them all.\n",
    "Instead, we are interested to see all distinct target values, which is easy to do with NumPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9.])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(mnist.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--NAVIGATION-->\n",
    "< [Dealing with Data Using Python's NumPy Package](02.01-Dealing-with-Data-Using-Python-NumPy.ipynb) | [Contents](../README.md) | [Visualizing Data Using Matplotlib](02.03-Visualizing-Data-Using-Matplotlib.ipynb) >"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}