PythonDataBasics_DataFrame.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "\n",
    "## Tabular Data Structures / DataFrames in Python for Engineers and Geoscientists \n",
    "### Michael Pyrcz, Associate Professor, University of Texas at Austin \n",
    "\n",
    "#### Contacts: [Twitter/@GeostatsGuy](https://twitter.com/geostatsguy) | [GitHub/GeostatsGuy](https://github.com/GeostatsGuy) | [www.michaelpyrcz.com](http://michaelpyrcz.com) | [GoogleScholar](https://scholar.google.com/citations?user=QVZ20eQAAAAJ&hl=en&oi=ao) | [Book](https://www.amazon.com/Geostatistical-Reservoir-Modeling-Michael-Pyrcz/dp/0199731446)\n",
    "\n",
    "This is a tutorial for / demonstration of **Tabular Data Structures in Python**.  In Python, the common tool for dealing with Tabular Data Structures is the DataFrame from the pandas Python package.    \n",
    "\n",
    "This tutorial includes the methods and operations that would commonly be required for Engineers and Geoscientists working with Tabular Data Structures for the purpose of:\n",
    "\n",
    "1. Data Checking and Cleaning\n",
    "2. Data Mining / Inferential Data Analysis\n",
    "3. Data Analytics / Building Predictive Models with Geostatistics and Machine Learning\n",
    "\n",
    "##### Tabular Data Structures\n",
    "\n",
    "In Python we will commonly store our data in two formats, tables and arrays.  For sampled data with typically multiple features $1,\\ldots,m$ over $1,\\ldots,n$ samples we will work with tables.  For exhaustive maps and models usually representing a single feature on a regular grid over $1,\\ldots,n_{i}$ for $i = 1,\\ldots,n_{dim}$ we will work with arrays.\n",
    "\n",
    "pandas package provides a convenient DataFrame object for working with data in a table and numpy package provides a convenient ndarray object for working with gridded data. In the following tutorial we will focus on DataFrames although we will utilize ndarrays a couple of times.  There is another section on Gridded Data Structures that focuses on ndarrays.\n",
    "\n",
    "#### Project Goal\n",
    "\n",
    "Learn the basics for working with Tabular Data Structures in Python.\n",
    "\n",
    "#### Load the required libraries\n",
    "\n",
    "The following code loads the required libraries.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os                                                   # to set current working directory \n",
    "import numpy as np                                          # arrays and matrix math\n",
    "import pandas as pd                                         # DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you get a package import error, you may have to first install some of these packages. This can usually be accomplished by opening up a command window on Windows and then typing 'python -m pip install [package-name]'. More assistance is available with the respective package docs.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set the working directory\n",
    "\n",
    "I always like to do this so I don't lose files and to simplify subsequent read and writes (avoid including the full address each time).  Also, in this case make sure to place the required (see below) data file in this directory.  When we are done with this tutorial we will write our new dataset back to this directory.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "os.chdir(\"c:/PGE337\")                                       # set the working directory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loading Data \n",
    "\n",
    "Let's load the provided multivariate, spatial dataset.  '2D_MV_200wells.csv' is available at https://github.com/GeostatsGuy/GeoDataSets.  It is a comma delimited file with X and Y coordinates,facies 1 and 2 (1 is sandstone and 2 interbedded sand and mudstone), porosity (fraction), permeability (mDarcy) and acoustic impedance (kg/m2s*10^6). We load it with the pandas 'read_csv' function into a data frame we called 'df' and then preview it by printing a slice and by utilizing the 'head' DataFrame member function (with a nice and clean format, see below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     X      Y  facies  porosity  permeability  acoustic_impedance\n",
      "0   565  1485       1    0.1184         6.170               2.009\n",
      "1  2585  1185       1    0.1566         6.275               2.864\n",
      "2  2065  2865       2    0.1920        92.297               3.524\n",
      "3  3575  2655       1    0.1621         9.048               2.157\n",
      "4  1835    35       1    0.1766         7.123               3.979\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2585</td>\n",
       "      <td>1185</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity  permeability  acoustic_impedance\n",
       "0   565  1485       1    0.1184         6.170               2.009\n",
       "1  2585  1185       1    0.1566         6.275               2.864\n",
       "2  2065  2865       2    0.1920        92.297               3.524\n",
       "3  3575  2655       1    0.1621         9.048               2.157\n",
       "4  1835    35       1    0.1766         7.123               3.979"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"2D_MV_200wells.csv\")                      # read a .csv file in as a DataFrame\n",
    "print(df.iloc[0:5,:])                                       # display first 4 samples in the table as a preview\n",
    "df.head()                                                   # we could also use this command for a table preview "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is useful to review the summary statistics of our loaded DataFrame.  That can be accomplished with the 'describe' DataFrame member function.  We transpose to switch the axes for ease of visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>X</th>\n",
       "      <td>200.0</td>\n",
       "      <td>2053.400000</td>\n",
       "      <td>1113.524641</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>1112.500000</td>\n",
       "      <td>2160.000000</td>\n",
       "      <td>2915.00000</td>\n",
       "      <td>3955.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Y</th>\n",
       "      <td>200.0</td>\n",
       "      <td>1876.150000</td>\n",
       "      <td>1137.580160</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>920.000000</td>\n",
       "      <td>1855.000000</td>\n",
       "      <td>2782.50000</td>\n",
       "      <td>3995.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>facies</th>\n",
       "      <td>200.0</td>\n",
       "      <td>1.330000</td>\n",
       "      <td>0.471393</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.00000</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>porosity</th>\n",
       "      <td>200.0</td>\n",
       "      <td>0.149300</td>\n",
       "      <td>0.032948</td>\n",
       "      <td>0.050000</td>\n",
       "      <td>0.132175</td>\n",
       "      <td>0.150150</td>\n",
       "      <td>0.17420</td>\n",
       "      <td>0.223200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>permeability</th>\n",
       "      <td>200.0</td>\n",
       "      <td>25.287462</td>\n",
       "      <td>64.470135</td>\n",
       "      <td>0.015820</td>\n",
       "      <td>1.366750</td>\n",
       "      <td>4.825500</td>\n",
       "      <td>14.59700</td>\n",
       "      <td>463.641000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>acoustic_impedance</th>\n",
       "      <td>200.0</td>\n",
       "      <td>3.000435</td>\n",
       "      <td>0.592201</td>\n",
       "      <td>2.009000</td>\n",
       "      <td>2.483250</td>\n",
       "      <td>2.964500</td>\n",
       "      <td>3.52700</td>\n",
       "      <td>3.984000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>porosity100</th>\n",
       "      <td>200.0</td>\n",
       "      <td>14.930005</td>\n",
       "      <td>3.294792</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>13.217500</td>\n",
       "      <td>15.015000</td>\n",
       "      <td>17.42000</td>\n",
       "      <td>22.320000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>permpor</th>\n",
       "      <td>200.0</td>\n",
       "      <td>134.387088</td>\n",
       "      <td>313.959651</td>\n",
       "      <td>0.259685</td>\n",
       "      <td>10.435609</td>\n",
       "      <td>27.722167</td>\n",
       "      <td>89.78552</td>\n",
       "      <td>2152.465181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>perm_cutoff</th>\n",
       "      <td>200.0</td>\n",
       "      <td>25.186850</td>\n",
       "      <td>64.507743</td>\n",
       "      <td>0.000100</td>\n",
       "      <td>1.026000</td>\n",
       "      <td>4.725500</td>\n",
       "      <td>14.59700</td>\n",
       "      <td>463.641000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    count         mean          std        min          25%  \\\n",
       "X                   200.0  2053.400000  1113.524641  25.000000  1112.500000   \n",
       "Y                   200.0  1876.150000  1137.580160  35.000000   920.000000   \n",
       "facies              200.0     1.330000     0.471393   1.000000     1.000000   \n",
       "porosity            200.0     0.149300     0.032948   0.050000     0.132175   \n",
       "permeability        200.0    25.287462    64.470135   0.015820     1.366750   \n",
       "acoustic_impedance  200.0     3.000435     0.592201   2.009000     2.483250   \n",
       "porosity100         200.0    14.930005     3.294792   5.000000    13.217500   \n",
       "permpor             200.0   134.387088   313.959651   0.259685    10.435609   \n",
       "perm_cutoff         200.0    25.186850    64.507743   0.000100     1.026000   \n",
       "\n",
       "                            50%         75%          max  \n",
       "X                   2160.000000  2915.00000  3955.000000  \n",
       "Y                   1855.000000  2782.50000  3995.000000  \n",
       "facies                 1.000000     2.00000     2.000000  \n",
       "porosity               0.150150     0.17420     0.223200  \n",
       "permeability           4.825500    14.59700   463.641000  \n",
       "acoustic_impedance     2.964500     3.52700     3.984000  \n",
       "porosity100           15.015000    17.42000    22.320000  \n",
       "permpor               27.722167    89.78552  2152.465181  \n",
       "perm_cutoff            4.725500    14.59700   463.641000  "
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe().transpose()   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is straightforward to extract subsets from a DataFrame to make a new DataFrame.  This is useful for cleaning up data by removing features that are no longer of interest.  If the samples are in random order then the first $n_{s}$ samples are a random sample of size $n_{s}$.  Below we make a new DataFrame, 'df_subset', with the rows 0 to 4 and columns 2 to 6 and another new DataFrame, 'df_no_location', with the X and Y coordinates removed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   facies  porosity  permeability  acoustic_impedance\n",
       "0       1    0.1184         6.170               2.009\n",
       "1       1    0.1566         6.275               2.864\n",
       "2       2    0.1920        92.297               3.524\n",
       "3       1    0.1621         9.048               2.157\n",
       "4       1    0.1766         7.123               3.979"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_subset = df.iloc[0:5,2:7]                                # make a new dataframe with just the first 4 samples and no X,Y\n",
    "df_subset.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   facies  porosity  permeability  acoustic_impedance\n",
       "0       1    0.1184         6.170               2.009\n",
       "1       1    0.1566         6.275               2.864\n",
       "2       2    0.1920        92.297               3.524\n",
       "3       1    0.1621         9.048               2.157\n",
       "4       1    0.1766         7.123               3.979"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_no_location = df[['facies','porosity','permeability','acoustic_impedance']] \n",
    "df_no_location.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also easy to add a column to our data frame.  Note, we assume that the array is in the same order as the DataFrame.  This could be an issue if any rows were removed form either before adding etc.  To demonstrate we make a 1D numpy array of zeros using the 'zeros' function and add it to our DataFrame with the feature name indicated as 'zero'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>zero</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2585</td>\n",
       "      <td>1185</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "      <td>6.2750</td>\n",
       "      <td>high</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity  permeability  acoustic_impedance  \\\n",
       "0   565  1485       1    0.1184         6.170               2.009   \n",
       "1  2585  1185       1    0.1566         6.275               2.864   \n",
       "2  2065  2865       2    0.1920        92.297               3.524   \n",
       "3  3575  2655       1    0.1621         9.048               2.157   \n",
       "4  1835    35       1    0.1766         7.123               3.979   \n",
       "\n",
       "   perm_cutoff tporosity  zero  \n",
       "0       0.0001       low   0.0  \n",
       "1       6.2750      high   0.0  \n",
       "2      92.2970      high   0.0  \n",
       "3       9.0480      high   0.0  \n",
       "4       7.1230      high   0.0  "
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zeros = np.zeros(200)                                       # make a array of zeros\n",
    "df['zero'] = pd.Series(zeros)                               # add the array to our DataFrame\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also remove unwanted columns without having to subset the DataFrame.  We do this with the 'drop' member function of the DataFrame object. We just have the give the column name and by indicating axis=1 we specify to drop a column instead of a row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2585</td>\n",
       "      <td>1185</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "      <td>6.2750</td>\n",
       "      <td>high</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity  permeability  acoustic_impedance  \\\n",
       "0   565  1485       1    0.1184         6.170               2.009   \n",
       "1  2585  1185       1    0.1566         6.275               2.864   \n",
       "2  2065  2865       2    0.1920        92.297               3.524   \n",
       "3  3575  2655       1    0.1621         9.048               2.157   \n",
       "4  1835    35       1    0.1766         7.123               3.979   \n",
       "\n",
       "   perm_cutoff tporosity  \n",
       "0       0.0001       low  \n",
       "1       6.2750      high  \n",
       "2      92.2970      high  \n",
       "3       9.0480      high  \n",
       "4       7.1230      high  "
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.drop('zero',axis=1)                                      # remove the zero column\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We may want to make new features by using mathematical operators applied to existing features.  For example we can make a porosity feature that is in percentage instead of fraction (called 'porosity100') or a ratio of permeability divided by porosity (called 'permpor') may be useful for subsequent calculations such as the Lorenz Coefficient.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>porosity100</th>\n",
       "      <th>permpor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "      <td>11.84</td>\n",
       "      <td>52.111486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2585</td>\n",
       "      <td>1185</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "      <td>6.2750</td>\n",
       "      <td>high</td>\n",
       "      <td>15.66</td>\n",
       "      <td>40.070243</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "      <td>19.20</td>\n",
       "      <td>480.713542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "      <td>16.21</td>\n",
       "      <td>55.817397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "      <td>17.66</td>\n",
       "      <td>40.334088</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity  permeability  acoustic_impedance  \\\n",
       "0   565  1485       1    0.1184         6.170               2.009   \n",
       "1  2585  1185       1    0.1566         6.275               2.864   \n",
       "2  2065  2865       2    0.1920        92.297               3.524   \n",
       "3  3575  2655       1    0.1621         9.048               2.157   \n",
       "4  1835    35       1    0.1766         7.123               3.979   \n",
       "\n",
       "   perm_cutoff tporosity  porosity100     permpor  \n",
       "0       0.0001       low        11.84   52.111486  \n",
       "1       6.2750      high        15.66   40.070243  \n",
       "2      92.2970      high        19.20  480.713542  \n",
       "3       9.0480      high        16.21   55.817397  \n",
       "4       7.1230      high        17.66   40.334088  "
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['porosity100'] = df['porosity']*100                      # add a new column with porosity in percentage\n",
    "df['permpor'] = df['permeability']/df['porosity']           # add a new feature with ratio of perm / por \n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could also use conditional statements when assigning values to a new feature.  For example, we could have a categorical porosity measure for high and low porosity, called 'tporosity'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>porosity100</th>\n",
       "      <th>permpor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "      <td>11.84</td>\n",
       "      <td>52.111486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2585</td>\n",
       "      <td>1185</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "      <td>6.2750</td>\n",
       "      <td>high</td>\n",
       "      <td>15.66</td>\n",
       "      <td>40.070243</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "      <td>19.20</td>\n",
       "      <td>480.713542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "      <td>16.21</td>\n",
       "      <td>55.817397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "      <td>17.66</td>\n",
       "      <td>40.334088</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity  permeability  acoustic_impedance  \\\n",
       "0   565  1485       1    0.1184         6.170               2.009   \n",
       "1  2585  1185       1    0.1566         6.275               2.864   \n",
       "2  2065  2865       2    0.1920        92.297               3.524   \n",
       "3  3575  2655       1    0.1621         9.048               2.157   \n",
       "4  1835    35       1    0.1766         7.123               3.979   \n",
       "\n",
       "   perm_cutoff tporosity  porosity100     permpor  \n",
       "0       0.0001       low        11.84   52.111486  \n",
       "1       6.2750      high        15.66   40.070243  \n",
       "2      92.2970      high        19.20  480.713542  \n",
       "3       9.0480      high        16.21   55.817397  \n",
       "4       7.1230      high        17.66   40.334088  "
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['tporosity'] = np.where(df['porosity']>=0.12, 'high', 'low') # conditional statement assign a new feature\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's an example where we use a conditional statement to assign a very low permeability value (0.0001 mD) for all porosity values below a threshold. Of course, this is for demonstration, in practice a much lower porosity threshold would likely be applied.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>porosity100</th>\n",
       "      <th>permpor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "      <td>11.84</td>\n",
       "      <td>52.111486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2585</td>\n",
       "      <td>1185</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1566</td>\n",
       "      <td>6.275</td>\n",
       "      <td>2.864</td>\n",
       "      <td>6.2750</td>\n",
       "      <td>high</td>\n",
       "      <td>15.66</td>\n",
       "      <td>40.070243</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "      <td>19.20</td>\n",
       "      <td>480.713542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "      <td>16.21</td>\n",
       "      <td>55.817397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "      <td>17.66</td>\n",
       "      <td>40.334088</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity  permeability  acoustic_impedance  \\\n",
       "0   565  1485       1    0.1184         6.170               2.009   \n",
       "1  2585  1185       1    0.1566         6.275               2.864   \n",
       "2  2065  2865       2    0.1920        92.297               3.524   \n",
       "3  3575  2655       1    0.1621         9.048               2.157   \n",
       "4  1835    35       1    0.1766         7.123               3.979   \n",
       "\n",
       "   perm_cutoff tporosity  porosity100     permpor  \n",
       "0       0.0001       low        11.84   52.111486  \n",
       "1       6.2750      high        15.66   40.070243  \n",
       "2      92.2970      high        19.20  480.713542  \n",
       "3       9.0480      high        16.21   55.817397  \n",
       "4       7.1230      high        17.66   40.334088  "
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['perm_cutoff'] = np.where(df['porosity']>=0.12, df['permeability'],0.0001) # conditional statement assign a new feature\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What about missing or invalid values?  Let's assign a single porosity value to NaN, 'not a number', indicating a missing or eroneous value.  We will then check for the number of NaN values in our DataFrame.  Then we can search for and display the sample with the NaN porosity value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     X      Y  facies  porosity  permeability  acoustic_impedance  \\\n",
      "1  2585  1185       1       NaN         6.275               2.864   \n",
      "\n",
      "   perm_cutoff tporosity  porosity100    permpor  \n",
      "1        6.275      high        15.66  40.070243  \n"
     ]
    }
   ],
   "source": [
    "df.set_value(1,'porosity',np.NaN)                           # let's give ourselves a NaN / missing value in our table\n",
    "df.isnull().sum().sum()                                     # let's check for missing values in our table\n",
    "nan_rows = df[df['porosity'].isnull()]                      # find the row with missing values\n",
    "print(nan_rows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that sample 1 has a NaN porosity value.  Now we may choose to remove the sample with the NaN.  The 'dropna' DataFrame member function will remove all samples with NaN entries from the entire DataFrame.  By visualizing the index at the left of the DataFrame preview you can confirm that sample 1 is removed (it jumps from 0 to 2)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>permeability</th>\n",
       "      <th>acoustic_impedance</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>porosity100</th>\n",
       "      <th>permpor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "      <td>11.84</td>\n",
       "      <td>52.111486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "      <td>19.20</td>\n",
       "      <td>480.713542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "      <td>16.21</td>\n",
       "      <td>55.817397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "      <td>17.66</td>\n",
       "      <td>40.334088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3375</td>\n",
       "      <td>2525</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1239</td>\n",
       "      <td>1.468</td>\n",
       "      <td>2.337</td>\n",
       "      <td>1.4680</td>\n",
       "      <td>high</td>\n",
       "      <td>12.39</td>\n",
       "      <td>11.848265</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity  permeability  acoustic_impedance  \\\n",
       "0   565  1485       1    0.1184         6.170               2.009   \n",
       "2  2065  2865       2    0.1920        92.297               3.524   \n",
       "3  3575  2655       1    0.1621         9.048               2.157   \n",
       "4  1835    35       1    0.1766         7.123               3.979   \n",
       "5  3375  2525       1    0.1239         1.468               2.337   \n",
       "\n",
       "   perm_cutoff tporosity  porosity100     permpor  \n",
       "0       0.0001       low        11.84   52.111486  \n",
       "2      92.2970      high        19.20  480.713542  \n",
       "3       9.0480      high        16.21   55.817397  \n",
       "4       7.1230      high        17.66   40.334088  \n",
       "5       1.4680      high        12.39   11.848265  "
      ]
     },
     "execution_count": 180,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.dropna()                                            # drop any rows (samples) with atleast one missing value        \n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may want to rename the feature names, column titles.  This is accomplished with the 'rename' DataFrame member function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>perm</th>\n",
       "      <th>ai</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>porosity100</th>\n",
       "      <th>permpor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565</td>\n",
       "      <td>1485</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "      <td>11.84</td>\n",
       "      <td>52.111486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "      <td>19.20</td>\n",
       "      <td>480.713542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575</td>\n",
       "      <td>2655</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "      <td>16.21</td>\n",
       "      <td>55.817397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "      <td>17.66</td>\n",
       "      <td>40.334088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3375</td>\n",
       "      <td>2525</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1239</td>\n",
       "      <td>1.468</td>\n",
       "      <td>2.337</td>\n",
       "      <td>1.4680</td>\n",
       "      <td>high</td>\n",
       "      <td>12.39</td>\n",
       "      <td>11.848265</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     X      Y  facies  porosity    perm     ai  perm_cutoff tporosity  \\\n",
       "0   565  1485       1    0.1184   6.170  2.009       0.0001       low   \n",
       "2  2065  2865       2    0.1920  92.297  3.524      92.2970      high   \n",
       "3  3575  2655       1    0.1621   9.048  2.157       9.0480      high   \n",
       "4  1835    35       1    0.1766   7.123  3.979       7.1230      high   \n",
       "5  3375  2525       1    0.1239   1.468  2.337       1.4680      high   \n",
       "\n",
       "   porosity100     permpor  \n",
       "0        11.84   52.111486  \n",
       "2        19.20  480.713542  \n",
       "3        16.21   55.817397  \n",
       "4        17.66   40.334088  \n",
       "5        12.39   11.848265  "
      ]
     },
     "execution_count": 181,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.rename(columns={'acoustic_impedance': 'ai','permeability':'perm'}) # rename columns of the \n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One could extract samples into a new DataFrame with multiple criteria.  This is shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>perm</th>\n",
       "      <th>ai</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>porosity100</th>\n",
       "      <th>permpor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065</td>\n",
       "      <td>2865</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1920</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.297</td>\n",
       "      <td>high</td>\n",
       "      <td>19.20</td>\n",
       "      <td>480.713542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2295</td>\n",
       "      <td>1325</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1790</td>\n",
       "      <td>31.933</td>\n",
       "      <td>3.491</td>\n",
       "      <td>31.933</td>\n",
       "      <td>high</td>\n",
       "      <td>17.90</td>\n",
       "      <td>178.396648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>3715</td>\n",
       "      <td>3045</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1914</td>\n",
       "      <td>116.781</td>\n",
       "      <td>2.187</td>\n",
       "      <td>116.781</td>\n",
       "      <td>high</td>\n",
       "      <td>19.14</td>\n",
       "      <td>610.141066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>545</td>\n",
       "      <td>3765</td>\n",
       "      <td>1</td>\n",
       "      <td>0.1817</td>\n",
       "      <td>14.311</td>\n",
       "      <td>3.045</td>\n",
       "      <td>14.311</td>\n",
       "      <td>high</td>\n",
       "      <td>18.17</td>\n",
       "      <td>78.761695</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1385</td>\n",
       "      <td>2415</td>\n",
       "      <td>2</td>\n",
       "      <td>0.1774</td>\n",
       "      <td>22.578</td>\n",
       "      <td>2.711</td>\n",
       "      <td>22.578</td>\n",
       "      <td>high</td>\n",
       "      <td>17.74</td>\n",
       "      <td>127.271702</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      X      Y  facies  porosity     perm     ai  perm_cutoff tporosity  \\\n",
       "2   2065  2865       2    0.1920   92.297  3.524       92.297      high   \n",
       "6   2295  1325       1    0.1790   31.933  3.491       31.933      high   \n",
       "7   3715  3045       2    0.1914  116.781  2.187      116.781      high   \n",
       "13   545  3765       1    0.1817   14.311  3.045       14.311      high   \n",
       "15  1385  2415       2    0.1774   22.578  2.711       22.578      high   \n",
       "\n",
       "    porosity100     permpor  \n",
       "2         19.20  480.713542  \n",
       "6         17.90  178.396648  \n",
       "7         19.14  610.141066  \n",
       "13        18.17   78.761695  \n",
       "15        17.74  127.271702  "
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_extract = df.loc[(df['porosity'] > 0.12) & (df['perm'] > 10.0)] # extract with multiple conditions to a new table\n",
    "df_extract.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also simple to build a brandnew DataFrame from a set of 1D arrays.  Note, they must have the same size and be sorted consistently.  We will extract porosity and perm as arrays (if you remove '.values' they are extracted as Series, an array retaining the feature name).  We then use the pandas DataFrame command to make a new DataFrame with each 1D array and the column names specified as 'porosity' and 'permeabilty'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>permeability</th>\n",
       "      <th>porosity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6.170</td>\n",
       "      <td>0.1184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>92.297</td>\n",
       "      <td>0.1920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9.048</td>\n",
       "      <td>0.1621</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.123</td>\n",
       "      <td>0.1766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.468</td>\n",
       "      <td>0.1239</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   permeability  porosity\n",
       "0         6.170    0.1184\n",
       "1        92.297    0.1920\n",
       "2         9.048    0.1621\n",
       "3         7.123    0.1766\n",
       "4         1.468    0.1239"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "por = df['porosity'].values                                 # extract porosity column as vector\n",
    "perm = df['perm'].values                                    # extract permeability column as vector\n",
    "df_new = pd.DataFrame({'porosity': por, 'permeability': perm}) # make a new DataFrame from the vectors\n",
    "df_new.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can reach in and retrieve the actual raw information in the DataFrame including the column names and actual values as an numpy array.  We can't edit them like this, but we can access and use this information.  This included 'index' with information about the index (i.e. index from start to stop with step), 'columns' with the names of the features and 'values' with the data table entries as a 2D array.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Int64Index([  0,   2,   3,   4,   5,   6,   7,   8,   9,  10,\n",
      "            ...\n",
      "            190, 191, 192, 193, 194, 195, 196, 197, 198, 199],\n",
      "           dtype='int64', length=199)\n",
      "Index(['X ', 'Y', 'facies', 'porosity', 'perm', 'ai', 'perm_cutoff',\n",
      "       'tporosity', 'porosity100', 'permpor'],\n",
      "      dtype='object')\n",
      "[[565 1485 1 ..., 'low' 11.84 52.111486486486484]\n",
      " [2065 2865 2 ..., 'high' 19.2 480.71354166666674]\n",
      " [3575 2655 1 ..., 'high' 16.21 55.81739666872301]\n",
      " ..., \n",
      " [375 1705 1 ..., 'high' 13.209999999999999 18.198334595003786]\n",
      " [3795 535 1 ..., 'low' 6.0920000000000005 0.25968483256730135]\n",
      " [3455 1645 1 ..., 'high' 15.049999999999999 6.578073089700997]]\n"
     ]
    }
   ],
   "source": [
    "print(df.index)                                             # get information about the index\n",
    "print(df.columns)                                           # get the list of feature names\n",
    "print(df.values)                                            # get the 2D array with all the table data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another method for getting a list of the DataFrame feature names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['X ',\n",
       " 'Y',\n",
       " 'facies',\n",
       " 'porosity',\n",
       " 'perm',\n",
       " 'ai',\n",
       " 'perm_cutoff',\n",
       " 'tporosity',\n",
       " 'porosity100',\n",
       " 'permpor']"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(df)                                                    # get a list with the feature names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's inteact with the DataFrame more surgically, one feature and sample at a time.  Here we retrieve the 4th column feature name and the porosity value for sample #1.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "porosity\n",
      "Porosity value for sample number 1 is 0.1566.\n"
     ]
    }
   ],
   "source": [
    "col2_name = df.columns[3]                                   # get the name of the 4th feature (porosity)\n",
    "print(col2_name)                                          \n",
    "por1 = df.values[1,3]                                       # get the value for sample 1 of the 4th feature (porosity)\n",
    "print('Porosity value for sample number 1 is ' + str(por1) + '.') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also manually change values.  This is done with the 'iloc' pandas DataFrame member function for integer-location based selection.  We look up the porosity value for sample 1 and then we use the 'set_value' DataFrame member function to change the value to 0.1000.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The value of porosity for sample 2 is 0.1621.\n",
      "The value of porosity for sample 2 is now 0.1000.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>facies</th>\n",
       "      <th>porosity</th>\n",
       "      <th>perm</th>\n",
       "      <th>ai</th>\n",
       "      <th>perm_cutoff</th>\n",
       "      <th>tporosity</th>\n",
       "      <th>porosity100</th>\n",
       "      <th>permpor</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>565.0</td>\n",
       "      <td>1485.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.1184</td>\n",
       "      <td>6.170</td>\n",
       "      <td>2.009</td>\n",
       "      <td>0.0001</td>\n",
       "      <td>low</td>\n",
       "      <td>11.84</td>\n",
       "      <td>52.111486</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2065.0</td>\n",
       "      <td>2865.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.1000</td>\n",
       "      <td>92.297</td>\n",
       "      <td>3.524</td>\n",
       "      <td>92.2970</td>\n",
       "      <td>high</td>\n",
       "      <td>19.20</td>\n",
       "      <td>480.713542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3575.0</td>\n",
       "      <td>2655.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.1621</td>\n",
       "      <td>9.048</td>\n",
       "      <td>2.157</td>\n",
       "      <td>9.0480</td>\n",
       "      <td>high</td>\n",
       "      <td>16.21</td>\n",
       "      <td>55.817397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1835.0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.1766</td>\n",
       "      <td>7.123</td>\n",
       "      <td>3.979</td>\n",
       "      <td>7.1230</td>\n",
       "      <td>high</td>\n",
       "      <td>17.66</td>\n",
       "      <td>40.334088</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3375.0</td>\n",
       "      <td>2525.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.1239</td>\n",
       "      <td>1.468</td>\n",
       "      <td>2.337</td>\n",
       "      <td>1.4680</td>\n",
       "      <td>high</td>\n",
       "      <td>12.39</td>\n",
       "      <td>11.848265</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       X        Y  facies  porosity    perm     ai  perm_cutoff tporosity  \\\n",
       "0   565.0  1485.0     1.0    0.1184   6.170  2.009       0.0001       low   \n",
       "2  2065.0  2865.0     2.0    0.1000  92.297  3.524      92.2970      high   \n",
       "3  3575.0  2655.0     1.0    0.1621   9.048  2.157       9.0480      high   \n",
       "4  1835.0    35.0     1.0    0.1766   7.123  3.979       7.1230      high   \n",
       "5  3375.0  2525.0     1.0    0.1239   1.468  2.337       1.4680      high   \n",
       "\n",
       "   porosity100     permpor  \n",
       "0        11.84   52.111486  \n",
       "2        19.20  480.713542  \n",
       "3        16.21   55.817397  \n",
       "4        17.66   40.334088  \n",
       "5        12.39   11.848265  "
      ]
     },
     "execution_count": 187,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "por = df.iloc[2]['porosity']                               # get the value for sample 1 of the porosity feature\n",
    "print('The value of porosity for sample 2 is ' + str(por) + '.')\n",
    "df.set_value(2,'porosity',0.1000)                          # set the value for sample 1 of the porosity feature\n",
    "print('The value of porosity for sample 2 is now 0.1000.')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It may be useful to write the DataFrame out for storage or curation and / or to be utilize with another platform (even R or Excel!).  It is easy to write the DataFrame back to a comma delimited file.  We have the 'to_csv' DataFrame member function to accomplish this.  The file will write to the working directory (another reason we set that at the beginning).  Go to that folder and open this new file with TextPad, Excel or any other program that opens .txt files to check it out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"2D_MV_200wells_out.csv\")                      # write out the df DataFrame to a comma delimited file "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### More Exercises\n",
    "\n",
    "There are so many more exercises and tests that one could attempt to gain experience with the pandas package, DataFrames objects in Python. I'll end here for brevity, but I invite you to continue. Check out the docs at https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html.  I'm always happy to discuss,\n",
    "\n",
    "*Michael*\n",
    "\n",
    "Michael Pyrcz, Ph.D., P.Eng. Associate Professor The Hildebrand Department of Petroleum and Geosystems Engineering, Bureau of Economic Geology, The Jackson School of Geosciences, The University of Texas at Austin\n",
    "On twitter I'm the @GeostatsGuy.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}