M01S03.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# STUDY GROUP - M01S03\n",
    "## Series vs DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Objectives\n",
    "You will be able to:\n",
    "* Understand and explain what Pandas Series and DataFrames are and how they differ from dictionaries and lists\n",
    "* Create Series & DataFrames from dictionaries and lists\n",
    "* Perform data manipulation on series and DataFrames using methods and attibutes\n",
    "* Perform data manipulation on series and DataFrames using lambda fucntions, def functions, .map()/.apply()/.applymap()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Manipulation in Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why pandas series and DataFrames instead of built-in python data types of lists and dictionaries?\n",
    "* Series and DataFrames have a range of built in methods which make standard practices and procedures streamlined. \n",
    "\n",
    "What are series and DataFrames?\n",
    "* “Series” — one-dimensional array\n",
    "* “Data Frames” - two-dimensional array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas Methods for Importing & Exporting Data\n",
    "\n",
    "- pd.read_csv()\n",
    "- pd.read_excel()\n",
    "- pd.read_json()\n",
    "- pd.DataFrame.from_dict()\n",
    "\n",
    "- df.to_csv()\n",
    "- df.to_excel()\n",
    "- df.to_json()\n",
    "- df.to_dict()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### .map(), .apply(), .applymap() with def function vs lambda function "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- both useful for applying an operation to all values in a series or DataFrame\n",
    "    * mapping a user-defined function allows for easy re-use\n",
    "    * lambda function is 1-time-use\n",
    "    \n",
    "https://sites.google.com/site/prgrammnote/python/difference-between-map-applymap-and-apply-methods-in-pandas\n",
    "\n",
    "https://chrisalbon.com/python/data_wrangling/pandas_apply_operations_to_dataframes/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pandas Methods for Formatting and Cleaning Data\n",
    "\n",
    "* df['col'].astype()\n",
    "* pd.to_datetime()\n",
    "* df.rename()\n",
    "* df.drop()\n",
    "* df.set_index()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**when manipulating and formatting data it is good practice to preview changes before overriting data**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Accessing Data in Pandas\n",
    "Methods:\n",
    "* .head() - first 5 rows of DataFrame\n",
    "* .tail() - last 5 rows of DataFrame\n",
    "\n",
    "And attributes:\n",
    "* .index\n",
    "* .columns\n",
    "* .dtypes\n",
    "* .shape\n",
    "* .loc - select by label\n",
    "* .iloc - select by index\n",
    "\n",
    "Confused on methods vs attributes? Check out these links:\n",
    "\n",
    "https://www.quora.com/What-is-the-difference-between-methods-and-attributes-in-Python\n",
    "\n",
    "https://stackoverflow.com/questions/28798781/differences-between-data-attributes-and-method-attributes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Boolean Masking for Data Selection\n",
    "\n",
    "df.loc[df_conditional, desired col]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_wine\n",
    "import pandas as pd \n",
    "\n",
    "data = load_wine()\n",
    "df = pd.DataFrame(data.data, columns=data.feature_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>alcohol</th>\n",
       "      <th>malic_acid</th>\n",
       "      <th>ash</th>\n",
       "      <th>alcalinity_of_ash</th>\n",
       "      <th>magnesium</th>\n",
       "      <th>total_phenols</th>\n",
       "      <th>flavanoids</th>\n",
       "      <th>nonflavanoid_phenols</th>\n",
       "      <th>proanthocyanins</th>\n",
       "      <th>color_intensity</th>\n",
       "      <th>hue</th>\n",
       "      <th>od280/od315_of_diluted_wines</th>\n",
       "      <th>proline</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>11.96</td>\n",
       "      <td>1.09</td>\n",
       "      <td>2.30</td>\n",
       "      <td>21.0</td>\n",
       "      <td>101.0</td>\n",
       "      <td>3.38</td>\n",
       "      <td>2.14</td>\n",
       "      <td>0.13</td>\n",
       "      <td>1.65</td>\n",
       "      <td>3.21</td>\n",
       "      <td>0.99</td>\n",
       "      <td>3.13</td>\n",
       "      <td>886.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>11.66</td>\n",
       "      <td>1.88</td>\n",
       "      <td>1.92</td>\n",
       "      <td>16.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>1.61</td>\n",
       "      <td>1.57</td>\n",
       "      <td>0.34</td>\n",
       "      <td>1.15</td>\n",
       "      <td>3.80</td>\n",
       "      <td>1.23</td>\n",
       "      <td>2.14</td>\n",
       "      <td>428.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>11.84</td>\n",
       "      <td>2.89</td>\n",
       "      <td>2.23</td>\n",
       "      <td>18.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>1.72</td>\n",
       "      <td>1.32</td>\n",
       "      <td>0.43</td>\n",
       "      <td>0.95</td>\n",
       "      <td>2.65</td>\n",
       "      <td>0.96</td>\n",
       "      <td>2.52</td>\n",
       "      <td>500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>11.84</td>\n",
       "      <td>0.89</td>\n",
       "      <td>2.58</td>\n",
       "      <td>18.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>2.20</td>\n",
       "      <td>2.21</td>\n",
       "      <td>0.22</td>\n",
       "      <td>2.35</td>\n",
       "      <td>3.05</td>\n",
       "      <td>0.79</td>\n",
       "      <td>3.08</td>\n",
       "      <td>520.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>11.65</td>\n",
       "      <td>1.67</td>\n",
       "      <td>2.62</td>\n",
       "      <td>26.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>1.92</td>\n",
       "      <td>1.61</td>\n",
       "      <td>0.40</td>\n",
       "      <td>1.34</td>\n",
       "      <td>2.60</td>\n",
       "      <td>1.36</td>\n",
       "      <td>3.21</td>\n",
       "      <td>562.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>11.64</td>\n",
       "      <td>2.06</td>\n",
       "      <td>2.46</td>\n",
       "      <td>21.6</td>\n",
       "      <td>84.0</td>\n",
       "      <td>1.95</td>\n",
       "      <td>1.69</td>\n",
       "      <td>0.48</td>\n",
       "      <td>1.35</td>\n",
       "      <td>2.80</td>\n",
       "      <td>1.00</td>\n",
       "      <td>2.75</td>\n",
       "      <td>680.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>11.62</td>\n",
       "      <td>1.99</td>\n",
       "      <td>2.28</td>\n",
       "      <td>18.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>3.02</td>\n",
       "      <td>2.26</td>\n",
       "      <td>0.17</td>\n",
       "      <td>1.35</td>\n",
       "      <td>3.25</td>\n",
       "      <td>1.16</td>\n",
       "      <td>2.96</td>\n",
       "      <td>345.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>11.81</td>\n",
       "      <td>2.12</td>\n",
       "      <td>2.74</td>\n",
       "      <td>21.5</td>\n",
       "      <td>134.0</td>\n",
       "      <td>1.60</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.14</td>\n",
       "      <td>1.56</td>\n",
       "      <td>2.50</td>\n",
       "      <td>0.95</td>\n",
       "      <td>2.26</td>\n",
       "      <td>625.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103</th>\n",
       "      <td>11.82</td>\n",
       "      <td>1.72</td>\n",
       "      <td>1.88</td>\n",
       "      <td>19.5</td>\n",
       "      <td>86.0</td>\n",
       "      <td>2.50</td>\n",
       "      <td>1.64</td>\n",
       "      <td>0.37</td>\n",
       "      <td>1.42</td>\n",
       "      <td>2.06</td>\n",
       "      <td>0.94</td>\n",
       "      <td>2.44</td>\n",
       "      <td>415.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>11.61</td>\n",
       "      <td>1.35</td>\n",
       "      <td>2.70</td>\n",
       "      <td>20.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>2.74</td>\n",
       "      <td>2.92</td>\n",
       "      <td>0.29</td>\n",
       "      <td>2.49</td>\n",
       "      <td>2.65</td>\n",
       "      <td>0.96</td>\n",
       "      <td>3.26</td>\n",
       "      <td>680.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>11.46</td>\n",
       "      <td>3.74</td>\n",
       "      <td>1.82</td>\n",
       "      <td>19.5</td>\n",
       "      <td>107.0</td>\n",
       "      <td>3.18</td>\n",
       "      <td>2.58</td>\n",
       "      <td>0.24</td>\n",
       "      <td>3.58</td>\n",
       "      <td>2.90</td>\n",
       "      <td>0.75</td>\n",
       "      <td>2.81</td>\n",
       "      <td>562.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>112</th>\n",
       "      <td>11.76</td>\n",
       "      <td>2.68</td>\n",
       "      <td>2.92</td>\n",
       "      <td>20.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>1.75</td>\n",
       "      <td>2.03</td>\n",
       "      <td>0.60</td>\n",
       "      <td>1.05</td>\n",
       "      <td>3.80</td>\n",
       "      <td>1.23</td>\n",
       "      <td>2.50</td>\n",
       "      <td>607.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113</th>\n",
       "      <td>11.41</td>\n",
       "      <td>0.74</td>\n",
       "      <td>2.50</td>\n",
       "      <td>21.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>2.48</td>\n",
       "      <td>2.01</td>\n",
       "      <td>0.42</td>\n",
       "      <td>1.44</td>\n",
       "      <td>3.08</td>\n",
       "      <td>1.10</td>\n",
       "      <td>2.31</td>\n",
       "      <td>434.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>115</th>\n",
       "      <td>11.03</td>\n",
       "      <td>1.51</td>\n",
       "      <td>2.20</td>\n",
       "      <td>21.5</td>\n",
       "      <td>85.0</td>\n",
       "      <td>2.46</td>\n",
       "      <td>2.17</td>\n",
       "      <td>0.52</td>\n",
       "      <td>2.01</td>\n",
       "      <td>1.90</td>\n",
       "      <td>1.71</td>\n",
       "      <td>2.87</td>\n",
       "      <td>407.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>116</th>\n",
       "      <td>11.82</td>\n",
       "      <td>1.47</td>\n",
       "      <td>1.99</td>\n",
       "      <td>20.8</td>\n",
       "      <td>86.0</td>\n",
       "      <td>1.98</td>\n",
       "      <td>1.60</td>\n",
       "      <td>0.30</td>\n",
       "      <td>1.53</td>\n",
       "      <td>1.95</td>\n",
       "      <td>0.95</td>\n",
       "      <td>3.33</td>\n",
       "      <td>495.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120</th>\n",
       "      <td>11.45</td>\n",
       "      <td>2.40</td>\n",
       "      <td>2.42</td>\n",
       "      <td>20.0</td>\n",
       "      <td>96.0</td>\n",
       "      <td>2.90</td>\n",
       "      <td>2.79</td>\n",
       "      <td>0.32</td>\n",
       "      <td>1.83</td>\n",
       "      <td>3.25</td>\n",
       "      <td>0.80</td>\n",
       "      <td>3.39</td>\n",
       "      <td>625.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>121</th>\n",
       "      <td>11.56</td>\n",
       "      <td>2.05</td>\n",
       "      <td>3.23</td>\n",
       "      <td>28.5</td>\n",
       "      <td>119.0</td>\n",
       "      <td>3.18</td>\n",
       "      <td>5.08</td>\n",
       "      <td>0.47</td>\n",
       "      <td>1.87</td>\n",
       "      <td>6.00</td>\n",
       "      <td>0.93</td>\n",
       "      <td>3.69</td>\n",
       "      <td>465.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>11.87</td>\n",
       "      <td>4.31</td>\n",
       "      <td>2.39</td>\n",
       "      <td>21.0</td>\n",
       "      <td>82.0</td>\n",
       "      <td>2.86</td>\n",
       "      <td>3.03</td>\n",
       "      <td>0.21</td>\n",
       "      <td>2.91</td>\n",
       "      <td>2.80</td>\n",
       "      <td>0.75</td>\n",
       "      <td>3.64</td>\n",
       "      <td>380.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127</th>\n",
       "      <td>11.79</td>\n",
       "      <td>2.13</td>\n",
       "      <td>2.78</td>\n",
       "      <td>28.5</td>\n",
       "      <td>92.0</td>\n",
       "      <td>2.13</td>\n",
       "      <td>2.24</td>\n",
       "      <td>0.58</td>\n",
       "      <td>1.76</td>\n",
       "      <td>3.00</td>\n",
       "      <td>0.97</td>\n",
       "      <td>2.44</td>\n",
       "      <td>466.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     alcohol  malic_acid   ash  alcalinity_of_ash  magnesium  total_phenols  \\\n",
       "74     11.96        1.09  2.30               21.0      101.0           3.38   \n",
       "75     11.66        1.88  1.92               16.0       97.0           1.61   \n",
       "77     11.84        2.89  2.23               18.0      112.0           1.72   \n",
       "84     11.84        0.89  2.58               18.0       94.0           2.20   \n",
       "87     11.65        1.67  2.62               26.0       88.0           1.92   \n",
       "88     11.64        2.06  2.46               21.6       84.0           1.95   \n",
       "94     11.62        1.99  2.28               18.0       98.0           3.02   \n",
       "96     11.81        2.12  2.74               21.5      134.0           1.60   \n",
       "103    11.82        1.72  1.88               19.5       86.0           2.50   \n",
       "109    11.61        1.35  2.70               20.0       94.0           2.74   \n",
       "110    11.46        3.74  1.82               19.5      107.0           3.18   \n",
       "112    11.76        2.68  2.92               20.0      103.0           1.75   \n",
       "113    11.41        0.74  2.50               21.0       88.0           2.48   \n",
       "115    11.03        1.51  2.20               21.5       85.0           2.46   \n",
       "116    11.82        1.47  1.99               20.8       86.0           1.98   \n",
       "120    11.45        2.40  2.42               20.0       96.0           2.90   \n",
       "121    11.56        2.05  3.23               28.5      119.0           3.18   \n",
       "124    11.87        4.31  2.39               21.0       82.0           2.86   \n",
       "127    11.79        2.13  2.78               28.5       92.0           2.13   \n",
       "\n",
       "     flavanoids  nonflavanoid_phenols  proanthocyanins  color_intensity   hue  \\\n",
       "74         2.14                  0.13             1.65             3.21  0.99   \n",
       "75         1.57                  0.34             1.15             3.80  1.23   \n",
       "77         1.32                  0.43             0.95             2.65  0.96   \n",
       "84         2.21                  0.22             2.35             3.05  0.79   \n",
       "87         1.61                  0.40             1.34             2.60  1.36   \n",
       "88         1.69                  0.48             1.35             2.80  1.00   \n",
       "94         2.26                  0.17             1.35             3.25  1.16   \n",
       "96         0.99                  0.14             1.56             2.50  0.95   \n",
       "103        1.64                  0.37             1.42             2.06  0.94   \n",
       "109        2.92                  0.29             2.49             2.65  0.96   \n",
       "110        2.58                  0.24             3.58             2.90  0.75   \n",
       "112        2.03                  0.60             1.05             3.80  1.23   \n",
       "113        2.01                  0.42             1.44             3.08  1.10   \n",
       "115        2.17                  0.52             2.01             1.90  1.71   \n",
       "116        1.60                  0.30             1.53             1.95  0.95   \n",
       "120        2.79                  0.32             1.83             3.25  0.80   \n",
       "121        5.08                  0.47             1.87             6.00  0.93   \n",
       "124        3.03                  0.21             2.91             2.80  0.75   \n",
       "127        2.24                  0.58             1.76             3.00  0.97   \n",
       "\n",
       "     od280/od315_of_diluted_wines  proline  \n",
       "74                           3.13    886.0  \n",
       "75                           2.14    428.0  \n",
       "77                           2.52    500.0  \n",
       "84                           3.08    520.0  \n",
       "87                           3.21    562.0  \n",
       "88                           2.75    680.0  \n",
       "94                           2.96    345.0  \n",
       "96                           2.26    625.0  \n",
       "103                          2.44    415.0  \n",
       "109                          3.26    680.0  \n",
       "110                          2.81    562.0  \n",
       "112                          2.50    607.0  \n",
       "113                          2.31    434.0  \n",
       "115                          2.87    407.0  \n",
       "116                          3.33    495.0  \n",
       "120                          3.39    625.0  \n",
       "121                          3.69    465.0  \n",
       "124                          3.64    380.0  \n",
       "127                          2.44    466.0  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[df[\"alcohol\"]<12] # boolean indexing\n",
    "#df['alcoho'] < 12\n",
    "# new_df = df[df['alcohol'] < 12] #boolean masking\n",
    "# new_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}