2022-08Aug-02.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome and agenda\n",
    "\n",
    "1. Getting started\n",
    "    - What is data analysis?\n",
    "    - What is Pandas?\n",
    "    - Descriptive statistics\n",
    "    - Using Pandas series\n",
    "    - Broadcasting \n",
    "    - Mask arrays\n",
    "    - Useful methods\n",
    "2. Data types and data frames\n",
    "    - dtypes (different types of data we can store)\n",
    "    - `NaN` (\"not a number\")\n",
    "    - Data frames\n",
    "    - Querying with boolean/mask indexes\n",
    "    - Reading CSV data\n",
    "3. Real-world data\n",
    "    - CSV\n",
    "    - Online data\n",
    "    - Sorting\n",
    "    - Grouping\n",
    "    - Pivot tables\n",
    "    - Joining\n",
    "    - Cleaning data\n",
    "4. Text and dates\n",
    "    - Text\n",
    "    - Dates\n",
    "5. Visualization\n",
    "    - Line plots\n",
    "    - Bar plots\n",
    "    - Histograms\n",
    "    - Pie plots\n",
    "    - Scatter plots\n",
    "    - Boxplots\n",
    "    \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What is data?\n",
    "\n",
    "If we're talking about data analysis, then we should really know what data is!\n",
    "\n",
    "The nouns in the computer world are data.\n",
    "    - Files (lots of types of files)\n",
    "    - Logs (information about who did what, and when, on which computer)\n",
    "    - Preferences (e.g., Netflix)\n",
    "    - Store inventories and purchasing histories\n",
    "    \n",
    "Thanks to mobile devices, and computers, and companies all getting interconnected, there is a **LOT** of data out there.  \n",
    "\n",
    "The problem isn't finding data. The problem is understanding what our data really means, and doing something useful with it.\n",
    "\n",
    "The scientific method means: I ask a question, and I try to answer that question as best as possible, using techniques that others have demonstrated are reliable.  Data science is all about applying that method to the world of data.\n",
    "\n",
    "I want to be able to ask a question, and then answer it.  I'll use data in order to do that.\n",
    "\n",
    "Some examples:\n",
    "    - What products are selling best?\n",
    "    - What products are selling best in each country?  In each age demographic? In each country vs. each age demographic?\n",
    "    - Which employees are bringing in the most sales?\n",
    "    - Which universities' graduates earn the most money 10 years after graduating?\n",
    "    - Which stocks/bonds/investments have done best over the last 10 years? 50 years?  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data science\n",
    "\n",
    "The idea is: Use scientific principles to ask questions and answer them with data.\n",
    "\n",
    "I divide the world of data science into three pieces:\n",
    "\n",
    "- Data analysis — use the data we've collected to understand the past and present\n",
    "- Data engineering — there's so much data out there, in so many different formats and sources, and getting it in a timely, organized way to our team's computers is hard -- data engineers solve these problems\n",
    "- Machine learning — learn from the past, to make predictions about the future\n",
    "\n",
    "\n",
    "How are we going to gather the data, and ask questions?  We're going to read it into data structures on our computer, and then use methods on those data structures to create queries.  We'll be using Python and Pandas in this course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Think about data\n",
    "\n",
    "What data does Amazon have about its products? What data does Amazon have about you? How does Amazon use this data in its business?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python -- why is this a language for data analytics?\n",
    "\n",
    "It's also a very good choice:\n",
    "- Easy to read\n",
    "- Easy to learn\n",
    "- Lots of support\n",
    "- Open source (cheap or free)\n",
    "\n",
    "It's a very bad choice, in many ways:\n",
    "- It runs slowly (relative to many other languages)\n",
    "- It uses lots of memory\n",
    "\n",
    "Data analytics uses *lots* of data, often many hundreds of megabytes -- or more!   It's not at all unusual to have data sets that are several GB in size.  If the same data in Python is 10x bigger than the data in C, then you can handle more data in C, even if it's a harder language to work with.\n",
    "\n",
    "The reason is something called \"NumPy.\" This is a Python module, written mostly in C.  It allows us to work with data in C format (i.e., very fast, very small), using a very thin layer of Python on top of it.  NumPy is super fast and super efficient, but it lets us work with friendly Python code.\n",
    "\n",
    "The best of both worlds!  (Almost)\n",
    "\n",
    "NumPy can still be a bit low level and hard to work with.  We, in this class, will be using Pandas.  Pandas is (mostly) a wrapper around NumPy, making it far friendlier and easier to work with.\n",
    "\n",
    "Just the Pandas library for Python has between 5-10 million users.  People at companies and organizations around the world are using Pandas more and more to analyze their data:\n",
    "\n",
    "- E-commerce companies\n",
    "- Manufacturers\n",
    "- Banks and financial institutions\n",
    "- Marketing companies\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data structures in Pandas\n",
    "\n",
    "Pandas mostly ignores Python data structures, in favor of its own:\n",
    "\n",
    "- Series (1-dimensional data)\n",
    "- Data frame (2-dimensional data)\n",
    "\n",
    "Assuming that you have loaded Pandas and Jupyter onto your computer, you can say:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np          # we will use, occasionally, some of the low-level NumPy functionality via \"np\"\n",
    "import pandas as pd         # load the Pandas module into memory, and make it available via the \"pd\" namespace\n",
    "from pandas import Series   # I also want to use the Series name by itself, rather than pd.Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's say I want to create a series\n",
    "\n",
    "# I use a Python list of integers to create my series\n",
    "s = Series([10, 20, 30, 40, 50, 60, 70])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.series.Series"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(s)   # in Jupyter, if an expression is on the final line of a cell, we get the value back w/o print"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# A series works a lot like a list, in many ways\n",
    "\n",
    "s[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "60"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "t = Series([50, 40, 20, 30, 88, 22, 16])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[10, 20, 30, 40, 50, 60]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what happens when I add together two Python lists?\n",
    "mylist1 = [10, 20, 30]\n",
    "mylist2 = [40, 50, 60]\n",
    "\n",
    "mylist1 + mylist2  # we get a new list -- all elements of mylist1, followed by all elements of mylist2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's look at s and t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "5    60\n",
       "6    70\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    50\n",
       "1    40\n",
       "2    20\n",
       "3    30\n",
       "4    88\n",
       "5    22\n",
       "6    16\n",
       "dtype: int64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     60\n",
       "1     60\n",
       "2     50\n",
       "3     70\n",
       "4    138\n",
       "5     82\n",
       "6     86\n",
       "dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what happens when I add s and t together?\n",
    "\n",
    "# we get a new series -- the new series has 7 elements, with the index 0-6\n",
    "# the new series, at index 0, is s[0] + t[0]\n",
    "# the new series, at index 1, is s[1] + t[1]\n",
    "\n",
    "# the addition took places as vectors!\n",
    "\n",
    "s + t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   -40\n",
       "1   -20\n",
       "2    10\n",
       "3    10\n",
       "4   -38\n",
       "5    38\n",
       "6    54\n",
       "dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s - t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     500\n",
       "1     800\n",
       "2     600\n",
       "3    1200\n",
       "4    4400\n",
       "5    1320\n",
       "6    1120\n",
       "dtype: int64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s * t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.200000\n",
       "1    0.500000\n",
       "2    1.500000\n",
       "3    1.333333\n",
       "4    0.568182\n",
       "5    2.727273\n",
       "6    4.375000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s / t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    10\n",
       "3    10\n",
       "4    50\n",
       "5    16\n",
       "6     6\n",
       "dtype: int64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s % t  # remainder from dividing s/t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Series operations\n",
    "\n",
    "1. Create a series containing three elements — your birth year, month, and day.\n",
    "2. Create a second series containing three elements -- birth year, month, and day -- from someone you know.\n",
    "3. Show the difference between your two ages in year, month, day"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "me = Series([1970, 7, 14])\n",
    "sis = Series([1971, 8, 22])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    1\n",
       "2    8\n",
       "dtype: int64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sis - me"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers (2946636120.py, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"/var/folders/rr/0mnyyv811fs5vyp22gf4fxk00000gn/T/ipykernel_3159/2946636120.py\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    me = Series([1970, 07, 14])\u001b[0m\n\u001b[0m                       ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers\n"
     ]
    }
   ],
   "source": [
    "me = Series([1970, 07, 14])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Different number bases\n",
    "\n",
    "If you want to enter numbers in a different number base, you can do that in Python:\n",
    "\n",
    "- base 2 (binary): Use 0b, as in 0b101010\n",
    "- base 16 (hex): Use 0x, as in 0xab12cd34\n",
    "- base 8 (octal): Use 0o, as in 0o12345"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next up:\n",
    "\n",
    "- Descriptive statistics\n",
    "- Aggregate methods\n",
    "- Five-number summary\n",
    "- Mean and standard deviation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lists vs. arrays\n",
    "\n",
    "Python normally uses lists.  Many people say to me, \"Oh, those are just arrays, right?\"  No, because there are two basic differences between lists and arrays:\n",
    "\n",
    "1. Arrays have a set length, that cannot be changed once they're created.\n",
    "2. All elements in an array must be of the same type.\n",
    "\n",
    "Since Pandas series are based on NumPy arrays, you might think that they cannot have different types in them.  But it turns out that they can, thanks to a bit of Python magic!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = Series([10, 20, 30, 40, 50])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10.5\n",
       "1    20.3\n",
       "2    30.4\n",
       "3    40.0\n",
       "4    50.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = Series([10.5, 20.3, 30.4, 40, 50])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       a\n",
       "1      bc\n",
       "2    defg\n",
       "3      hi\n",
       "dtype: object"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# a dtype of \"object\" means: Something in the Python world, I don't know what\n",
    "s = Series(['a', 'bc', 'defg', 'hi'])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# once we have a dtype of object, anything is possible\n",
    "# we can have a mix, then, and Pandas will live with it\n",
    "# but we should to avoid it\n",
    "\n",
    "s = Series([10, 20, 'abc', 'def', 30.5, 60.8, [1,2,3,4]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0              10\n",
       "1              20\n",
       "2             abc\n",
       "3             def\n",
       "4            30.5\n",
       "5            60.8\n",
       "6    [1, 2, 3, 4]\n",
       "dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you can install NumPy using pip, just as you did for Pandas:\n",
    "# pip install numpy\n",
    "\n",
    "# this is not a Python command -- you need to run it on the command line"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ages of people in my family\n",
    "\n",
    "# how can we describe this?\n",
    "\n",
    "s = Series([52, 50, 21, 19, 16])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "31.6"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How can we describe a bunch of data points, in a way that will make sense?\n",
    "\n",
    "# one possibility: mean (average) -- sum, divided by the number of data points\n",
    "\n",
    "# Pandas series have a bunch of methods for this, and other *descriptive statistics*.\n",
    "\n",
    "s.mean()   # mean is great... but also flawed, because it is easily skewed with one or two outliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# very often, we don't use the mean.  Rather, we use the *median*.\n",
    "# the median is calculated in this way: Take all values, from the smallest to the largest,\n",
    "# and line them up, sorted.  Take the middle value or (if there is an even number of values)\n",
    "# the average (mean) of the two middle values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21.0"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.median()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# why are mean and median floats, when the numbers in s are integers?\n",
    "# In Python 3, division with / always returns a float, even if the numbers are all integers\n",
    "# if you want an integer result, use //, known as \"floordiv,\" which removes (not rounds!) any decimals "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on method median in module pandas.core.generic:\n",
      "\n",
      "median(axis: 'int | None | lib.NoDefault' = <no_default>, skipna=True, level=None, numeric_only=None, **kwargs) method of pandas.core.series.Series instance\n",
      "    Return the median of the values over the requested axis.\n",
      "    \n",
      "    Parameters\n",
      "    ----------\n",
      "    axis : {index (0)}\n",
      "        Axis for the function to be applied on.\n",
      "    skipna : bool, default True\n",
      "        Exclude NA/null values when computing the result.\n",
      "    level : int or level name, default None\n",
      "        If the axis is a MultiIndex (hierarchical), count along a\n",
      "        particular level, collapsing into a scalar.\n",
      "    numeric_only : bool, default None\n",
      "        Include only float, int, boolean columns. If None, will attempt to use\n",
      "        everything, then use only numeric data. Not implemented for Series.\n",
      "    **kwargs\n",
      "        Additional keyword arguments to be passed to the function.\n",
      "    \n",
      "    Returns\n",
      "    -------\n",
      "    scalar or Series (if level specified)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# how do I get help on things in Pandas?\n",
    "# My favorite way: help(function)\n",
    "\n",
    "help(s.median)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17.812916661793487"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# standard deviation tells us how much the data varies from the mean\n",
    "# a standard deviation of 0 means: all data points are the same\n",
    "# often we talk about mean+1sd mean-1sd\n",
    "\n",
    "s.std()   # calculate the standard deviation on our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "31.6"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49.41291666179349"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean() + s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13.787083338206514"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean() - s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 68% of the data points in s should be between 49 and 13"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    52\n",
       "1    50\n",
       "2    21\n",
       "3    19\n",
       "4    16\n",
       "dtype: int64"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "67.22583332358698"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean() + 2 * s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-4.025833323586973"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean() - 2 * s.std()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Descriptive statistics\n",
    "\n",
    "We can now understand our data in a variety of ways:\n",
    "\n",
    "- Mean, which tells us the arithmetic average — easy to understand, and often useful, and the reference point for the standard deviation. However, it's easy to skew with outliers.\n",
    "- Standard deviation, which tells us how much the data spreads out from the mean. A high std means that the data is all over the place, whereas a low one means that it's concentrated around the mean.\n",
    "- Median, which is the middle value.\n",
    "\n",
    "You can also think of the median as the 50% point in our data, if we line things up from lowest to highest.  We can similarly calculate the 25% mark and the 75% mark in our data.  These are often known as the 1Q and 3Q values.  Pandas lets us calculate these pretty easily:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "19.0"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.quantile(0.25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21.0"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.quantile(0.5)  # median"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50.0"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.quantile(0.75)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    52\n",
       "1    50\n",
       "2    21\n",
       "3    19\n",
       "4    16\n",
       "dtype: int64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can also see how much the data is spread out using the IQR (inter-quartile range)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we might also want to know the minimum and maximum values in our data:\n",
    "\n",
    "s.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "52"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Descriptive statistics summary\n",
    "\n",
    "For a series `s`, we can call:\n",
    "\n",
    "- `s.mean()`\n",
    "- `s.std()`\n",
    "- `s.quantile(0.25)`\n",
    "- `s.quantile(0.50)` or `s.median()`\n",
    "- `s.quantile(0.75)`\n",
    "- `s.min()`\n",
    "- `s.max()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Weather forecast\n",
    "\n",
    "1. Go to a Web site that shows the 10-day forecast for your area, Create a series containing the high temperatures for each day in the next 10 days.\n",
    "2. Calculate each of the descriptive statistics.  Are there are any obvious outliers (very hot or very cold) in the coming 10 days?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on method quantile in module pandas.core.series:\n",
      "\n",
      "quantile(q=0.5, interpolation='linear') method of pandas.core.series.Series instance\n",
      "    Return value at the given quantile.\n",
      "    \n",
      "    Parameters\n",
      "    ----------\n",
      "    q : float or array-like, default 0.5 (50% quantile)\n",
      "        The quantile(s) to compute, which can lie in range: 0 <= q <= 1.\n",
      "    interpolation : {'linear', 'lower', 'higher', 'midpoint', 'nearest'}\n",
      "        This optional parameter specifies the interpolation method to use,\n",
      "        when the desired quantile lies between two data points `i` and `j`:\n",
      "    \n",
      "            * linear: `i + (j - i) * fraction`, where `fraction` is the\n",
      "              fractional part of the index surrounded by `i` and `j`.\n",
      "            * lower: `i`.\n",
      "            * higher: `j`.\n",
      "            * nearest: `i` or `j` whichever is nearest.\n",
      "            * midpoint: (`i` + `j`) / 2.\n",
      "    \n",
      "    Returns\n",
      "    -------\n",
      "    float or Series\n",
      "        If ``q`` is an array, a Series will be returned where the\n",
      "        index is ``q`` and the values are the quantiles, otherwise\n",
      "        a float will be returned.\n",
      "    \n",
      "    See Also\n",
      "    --------\n",
      "    core.window.Rolling.quantile : Calculate the rolling quantile.\n",
      "    numpy.percentile : Returns the q-th percentile(s) of the array elements.\n",
      "    \n",
      "    Examples\n",
      "    --------\n",
      "    >>> s = pd.Series([1, 2, 3, 4])\n",
      "    >>> s.quantile(.5)\n",
      "    2.5\n",
      "    >>> s.quantile([.25, .5, .75])\n",
      "    0.25    1.75\n",
      "    0.50    2.50\n",
      "    0.75    3.25\n",
      "    dtype: float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(s.quantile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "33.2"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "33.5"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.median()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9189365834726815"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32.281063416527324"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean() - s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34.11893658347268"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.mean() + s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    10.000000\n",
       "mean     33.200000\n",
       "std       0.918937\n",
       "min      32.000000\n",
       "25%      32.250000\n",
       "50%      33.500000\n",
       "75%      34.000000\n",
       "max      34.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we can get all of these descriptive statistics with the \"describe\" method\n",
    "\n",
    "s.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I'm going to use NumPy to generate some random numbers, and use those to create a series\n",
    "\n",
    "np.random.seed(0)   # always start random numbers from the same place, so we all get the same values\n",
    "s = Series(np.random.randint(0, 100, 10))   # get 10 random ints from 0-100, and create a series from them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    44\n",
       "1    47\n",
       "2    64\n",
       "3    67\n",
       "4    67\n",
       "5     9\n",
       "6    83\n",
       "7    21\n",
       "8    36\n",
       "9    87\n",
       "dtype: int64"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# to retrieve a value and index n, I could use s[n]\n",
    "# but it's much better to use s.loc[n]\n",
    "\n",
    "s.loc[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3    67\n",
       "5     9\n",
       "7    21\n",
       "dtype: int64"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# note the double square brackets!\n",
    "# outer ones are for s.loc\n",
    "# inner ones mean: I'm asking for 3 values\n",
    "\n",
    "s.loc[[3, 5, 7]]  # this is a \"fancy\" index, where I ask for three different values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4    67\n",
       "5     9\n",
       "6    83\n",
       "7    21\n",
       "dtype: int64"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# I can also use a slice (just like in Python)\n",
    "\n",
    "s.loc[4:7]  # starting at 4, until 7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     44\n",
       "1     47\n",
       "2     64\n",
       "3     67\n",
       "4     67\n",
       "5    999\n",
       "6     83\n",
       "7     21\n",
       "8     36\n",
       "9     87\n",
       "dtype: int64"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# can I set values in this way? YES!\n",
    "\n",
    "s.loc[5] = 999\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if I assign to a fancy index, all of the elements are changed to the new value\n",
    "\n",
    "s.loc[[3,5,7]] = 888\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     44\n",
       "1     47\n",
       "2     64\n",
       "3    888\n",
       "4     67\n",
       "5    888\n",
       "6     83\n",
       "7    888\n",
       "8     36\n",
       "9     87\n",
       "dtype: int64"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this is the series I created with the high temps in Modi'in (my city)\n",
    "\n",
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    33\n",
       "1    34\n",
       "2    34\n",
       "3    34\n",
       "4    34\n",
       "5    34\n",
       "6    33\n",
       "7    32\n",
       "8    32\n",
       "9    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I don't want to have an index that's 0-9.  I want an index that is easier to understand.\n",
    "# For example, MMDD\n",
    "\n",
    "# Pandas allows for this!\n",
    "\n",
    "# pass index = and a list of values (strings, integers, anything!)\n",
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32],\n",
    "          index=['0802', '0803', '0804', '0805', '0806', \n",
    "                '0807', '0808', '0809', '0810', '0811' ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what will the weather be like on August 9th?\n",
    "\n",
    "s.loc['0809']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what will the high temperature be on August 8th through 10th?\n",
    "\n",
    "s.loc['0808':'0810']  # use a slice"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# or, if I prefer, just state them explicitly with a fancy index\n",
    "s.loc[['0808', '0809', '0810']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.loc[['0808', '0809', '0810']] = 98,99,100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802     33\n",
       "0803     34\n",
       "0804     34\n",
       "0805     34\n",
       "0806     34\n",
       "0807     34\n",
       "0808     98\n",
       "0809     99\n",
       "0810    100\n",
       "0811     32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32],\n",
    "          index=['0802', '0803', '0804', '0805', '0806', \n",
    "                '0807', '0808', '0809', '0810', '0811' ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrieving via indexes\n",
    "\n",
    "We've now seen that we can use `.loc` to retrieve one or more values from our series, using the index.  If the index is the default (numbers), we can use it via the positions. But if the index is custom, either strings or integers, we can still use it.\n",
    "\n",
    "But what if we want to retrieve by position, even though we have our own string index? We can do that with the `.iloc` accessor.  It works just like `.loc`, but uses the numeric positions (starting at 0), rather than our own custom index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc['0804']  # retrieving via .loc and the index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.iloc[2]       # retrieving via .iloc and the position"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Weather, with dates\n",
    "\n",
    "1. Recreate your weather series, using MMDD-style strings as your indexes.\n",
    "2. Retrieve, via the index, the high temperature on August 5th\n",
    "3. Retrieve, via the index, the high temperatures  on August 4th through 9th.\n",
    "4. What is the max temp going to be from August 9th through 11th?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if I want to use my index to retrieve values, I use .loc\n",
    "# if I want to use the positions to retrieve values, I use .iloc \n",
    "\n",
    "# .iloc and .loc are exactly the same if I don't have a special/new/custom index, if I just\n",
    "# use the default.\n",
    "\n",
    "# .iloc will always use integers, starting at 0.  \n",
    "# .loc can use integers, but it can also use strings and other types"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make sure that your index values are strings, not integers\n",
    "\n",
    "# '0805'  not 0805"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if you assign a list (or other iterable) with the right number of values \n",
    "# to s.index, you're set\n",
    "\n",
    "s.index = [2,4,6,8,10,12,14,16, 18, 20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2     33\n",
       "4     34\n",
       "6     34\n",
       "8     34\n",
       "10    34\n",
       "12    34\n",
       "14    33\n",
       "16    32\n",
       "18    32\n",
       "20    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Weather, with dates\n",
    "\n",
    "1. Recreate your weather series, using MMDD-style strings as your indexes.\n",
    "2. Retrieve, via the index, the high temperature on August 5th\n",
    "3. Retrieve, via the index, the high temperatures  on August 4th through 9th.\n",
    "4. What is the max temp going to be from August 9th through 11th?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "# method 1 for creating my series of temps with dates:\n",
    "# create the series\n",
    "# pass the keyword argument index with a list of strings\n",
    "\n",
    "s = Series([33, 34, 34c, 34, 34, 34, 33, 32, 32, 32],\n",
    "          index=['0802', '0803', '0804', '0805', '0806', \n",
    "                '0807', '0808', '0809', '0810', '0811' ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "# method 2 for creating the series with dates\n",
    "\n",
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32])\n",
    "\n",
    "s.index = ['0802', '0803', '0804', '0805', '0806', \n",
    "           '0807', '0808', '0809', '0810', '0811' ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc['0805']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc['0804':'0809']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc['0809':'0811'].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     3.0\n",
       "mean     32.0\n",
       "std       0.0\n",
       "min      32.0\n",
       "25%      32.0\n",
       "50%      32.0\n",
       "75%      32.0\n",
       "max      32.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc['0809':'0811'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next up\n",
    "\n",
    "1. Broadcasting \n",
    "2. Mask arrays \n",
    "3. More about indexes\n",
    "4. Useful methods\n",
    "\n",
    "10-minute break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    44\n",
       "b    47\n",
       "c    64\n",
       "d    67\n",
       "e    67\n",
       "f     9\n",
       "g    83\n",
       "h    21\n",
       "i    36\n",
       "j    87\n",
       "dtype: int64"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a series of 10 random integers (via NumPy)\n",
    "\n",
    "np.random.seed(0)\n",
    "s = Series(np.random.randint(0, 100, 10),\n",
    "          index=list('abcdefghij'))  \n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a     88\n",
       "b     94\n",
       "c    128\n",
       "d    134\n",
       "e    134\n",
       "f     18\n",
       "g    166\n",
       "h     42\n",
       "i     72\n",
       "j    174\n",
       "dtype: int64"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s + s    # we'll get a new series back, with + happening on each index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    54\n",
       "b    57\n",
       "c    74\n",
       "d    77\n",
       "e    77\n",
       "f    19\n",
       "g    93\n",
       "h    31\n",
       "i    46\n",
       "j    97\n",
       "dtype: int64"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what if, instead of adding two series objects, I add a series to a scalar (individual) value?\n",
    "\n",
    "s + 10   # broadcasting -- each element in s ran +10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    41\n",
       "b    44\n",
       "c    61\n",
       "d    64\n",
       "e    64\n",
       "f     6\n",
       "g    80\n",
       "h    18\n",
       "i    33\n",
       "j    84\n",
       "dtype: int64"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s - 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    132\n",
       "b    141\n",
       "c    192\n",
       "d    201\n",
       "e    201\n",
       "f     27\n",
       "g    249\n",
       "h     63\n",
       "i    108\n",
       "j    261\n",
       "dtype: int64"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s * 3\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    14.666667\n",
       "b    15.666667\n",
       "c    21.333333\n",
       "d    22.333333\n",
       "e    22.333333\n",
       "f     3.000000\n",
       "g    27.666667\n",
       "h     7.000000\n",
       "i    12.000000\n",
       "j    29.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s / 3    # truediv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    14\n",
       "b    15\n",
       "c    21\n",
       "d    22\n",
       "e    22\n",
       "f     3\n",
       "g    27\n",
       "h     7\n",
       "i    12\n",
       "j    29\n",
       "dtype: int64"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s // 3    # floordiv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a     85184\n",
       "b    103823\n",
       "c    262144\n",
       "d    300763\n",
       "e    300763\n",
       "f       729\n",
       "g    571787\n",
       "h      9261\n",
       "i     46656\n",
       "j    658503\n",
       "dtype: int64"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s ** 3   # to the 3rd power"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    2\n",
       "b    2\n",
       "c    1\n",
       "d    1\n",
       "e    1\n",
       "f    0\n",
       "g    2\n",
       "h    0\n",
       "i    0\n",
       "j    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s % 3   # remainder = modulus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Convert our temperatures\n",
    "\n",
    "Assign `s` to be our 10-day forecast:\n",
    "\n",
    "- If your forecast is in Celsius, use broadcasting to get a new series in Fahrenheit\n",
    "- If your forecast is in Fahrenheit, use broadcasting to get a new series in Celsius\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [],
   "source": [
    "# °F = (°C × 9/5) + 32 \n",
    "# C =  (°F − 32) x 5/9 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    44\n",
       "b    47\n",
       "c    64\n",
       "d    67\n",
       "e    67\n",
       "f     9\n",
       "g    83\n",
       "h    21\n",
       "i    36\n",
       "j    87\n",
       "dtype: int64"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32],\n",
    "          index=['0802', '0803', '0804', '0805', '0806', \n",
    "                '0807', '0808', '0809', '0810', '0811' ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    91.4\n",
       "0803    93.2\n",
       "0804    93.2\n",
       "0805    93.2\n",
       "0806    93.2\n",
       "0807    93.2\n",
       "0808    91.4\n",
       "0809    89.6\n",
       "0810    89.6\n",
       "0811    89.6\n",
       "dtype: float64"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s * (9/5) + 32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    10.000000\n",
       "mean     91.760000\n",
       "std       1.654086\n",
       "min      89.600000\n",
       "25%      90.050000\n",
       "50%      92.300000\n",
       "75%      93.200000\n",
       "max      93.200000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# since the result is a series, I can run lots of other methods:\n",
    "\n",
    "(s * (9/5) + 32).describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "f_temps = s * (9/5) + 32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33.0\n",
       "0803    34.0\n",
       "0804    34.0\n",
       "0805    34.0\n",
       "0806    34.0\n",
       "0807    34.0\n",
       "0808    33.0\n",
       "0809    32.0\n",
       "0810    32.0\n",
       "0811    32.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(f_temps - 32) * (5/9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0803    34\n",
       "0804    34\n",
       "0805    34\n",
       "0806    34\n",
       "0807    34\n",
       "0808    33\n",
       "0809    32\n",
       "0810    32\n",
       "0811    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s  # s is an instance of Series, and it has access to the Series methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    33\n",
       "0808    33\n",
       "0806    34\n",
       "dtype: int64"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we can use any index values we want in fancy indexing\n",
    "s.loc[['0802', '0808', '0806']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's try something else\n",
    "s = Series([10, 20, 30, 40, 50])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    30\n",
       "3    40\n",
       "0    10\n",
       "dtype: int64"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what if I try fancy indexing with this s?\n",
    "\n",
    "s.loc[[2, 3, 0]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "2    30\n",
       "3    40\n",
       "dtype: int64"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what if, instead of passing a list of integers (for the indexes), I pass a list of boolean (True/False) values?\n",
    "\n",
    "# Pandas will return all elements of our series s, that correspond \n",
    "# to a True value.  It will drop/ignore all elements that correspond\n",
    "# to a False value.\n",
    "\n",
    "s.loc[[True, False, True, True, False]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    40\n",
       "1    50\n",
       "2    60\n",
       "3    70\n",
       "4    80\n",
       "dtype: int64"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s + 30  # this broadcasts with our series, giving us a new series, with everything +30"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s < 30 # this broadcasts the < operator to our series, returning... a series of booleans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2    False\n",
       "3     True\n",
       "4     True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s > 30"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s < s.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "dtype: int64"
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# each time I run a comparison between a series and a scalar value, I get back\n",
    "# a series of booleans\n",
    "\n",
    "# what happens if I then apply that series of booleans as a mask index to my series?\n",
    "\n",
    "s.loc[s<30]  # what are the elements of s that are less than 30?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc[2:4]   # this means: s, from index 2 to index 4 -- we're passing a slice object to s.loc[]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc[[2,3,4]]  # this means, s, indexes 2,3, and 4 -- we're passing a list object to s.loc[]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mask indexes\n",
    "\n",
    "If I want selected values from my series, a standard way to do this is with a \"mask index\":\n",
    "\n",
    "- We create a boolean series, based on broadcasting a comparison operator (`<`, `==`, etc.)\n",
    "- We apply that boolean series as an index on the series.\n",
    "\n",
    "The result: We get only those values for which the comparison is `True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s < s.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "dtype: int64"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# show all values in s that are < mean\n",
    "s[s < s.mean()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    20\n",
       "2    30\n",
       "3    40\n",
       "4    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# show all values in s that are >= 15\n",
    "\n",
    "s[s >= 15]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    -6\n",
       "b    -3\n",
       "c    14\n",
       "d    17\n",
       "e    17\n",
       "f   -41\n",
       "g    33\n",
       "h   -29\n",
       "i   -14\n",
       "j    37\n",
       "dtype: int64"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(0)\n",
    "s = Series(np.random.randint(-50, 50, 10), \n",
    "          index=list('abcdefghij'))\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "c    14\n",
       "d    17\n",
       "e    17\n",
       "g    33\n",
       "j    37\n",
       "dtype: int64"
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find positive numbers in s\n",
    "s.loc[s > 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    -6\n",
       "b    -3\n",
       "f   -41\n",
       "h   -29\n",
       "i   -14\n",
       "dtype: int64"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find negative numbers in s\n",
    "s.loc[s < 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    -6\n",
       "c    14\n",
       "i   -14\n",
       "dtype: int64"
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find even numbers in s\n",
    "s.loc[s%2 == 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b    -3\n",
       "d    17\n",
       "e    17\n",
       "f   -41\n",
       "g    33\n",
       "h   -29\n",
       "j    37\n",
       "dtype: int64"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find odd numbers in s\n",
    "s.loc[s%2 == 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Family ages\n",
    "\n",
    "1. Create a series in which the values are ages of people in your family, and the index contains their names.\n",
    "2. Find all people who are below the mean age.\n",
    "3. Find all people who are above the mean age + 1 std.\n",
    "4. Find all people whose ages are odd."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RL      52\n",
       "SF      50\n",
       "AMLF    21\n",
       "SBLF    19\n",
       "ADLF    16\n",
       "dtype: int64"
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = Series([52, 50, 21, 19, 16],\n",
    "          index=['RL', 'SF', 'AMLF', 'SBLF', 'ADLF'])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AMLF    21\n",
       "SBLF    19\n",
       "ADLF    16\n",
       "dtype: int64"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find people below the mean age\n",
    "s.loc[s < s.mean()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RL    52\n",
       "SF    50\n",
       "dtype: int64"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find people above mean age + 1 std\n",
    "s.loc[s >  s.mean() + s.std()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AMLF    21\n",
       "SBLF    19\n",
       "dtype: int64"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc[s%2 == 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['AMLF', 'SBLF'], dtype=object)"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the index back, rather than the values?\n",
    "s.loc[s%2 == 1].index.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     17\n",
       "1     79\n",
       "2      4\n",
       "3     42\n",
       "4     58\n",
       "      ..\n",
       "95    19\n",
       "96    46\n",
       "97    42\n",
       "98    56\n",
       "99    60\n",
       "Length: 100, dtype: int64"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# combine queries\n",
    "\n",
    "np.random.rand(0)\n",
    "s = Series(np.random.randint(0, 100, 100))  # 100 numbers, each between 0-100\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4     58\n",
       "13    82\n",
       "21    84\n",
       "23    68\n",
       "25    68\n",
       "28    76\n",
       "29    52\n",
       "30    78\n",
       "34    58\n",
       "39    48\n",
       "44    64\n",
       "47    94\n",
       "49    50\n",
       "52    48\n",
       "55    98\n",
       "63    58\n",
       "67    98\n",
       "68    62\n",
       "70    94\n",
       "72    82\n",
       "77    50\n",
       "81    58\n",
       "85    86\n",
       "90    80\n",
       "92    54\n",
       "98    56\n",
       "99    60\n",
       "dtype: int64"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's find even numbers greater than the mean\n",
    "# first, find s>s.mean(), returning a boolean series (same index as s)\n",
    "# next, find s%2==0, returning a boolean series (same index as s)\n",
    "\n",
    "# use & to get a new boolean series with the same index as s, where the values\n",
    "# are True if both input series are True\n",
    "\n",
    "s.loc[(s>s.mean()) & \n",
    "      (s%2==0)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Combining conditions\n",
    "\n",
    "If you want to get a new boolean series based on two existing boolean series:\n",
    "\n",
    "- Use `&` for `and`, `|` for `or`, and `~` for `not`\n",
    "- Put `()` around every clause in your condition\n",
    "- Then apply the resulting boolean series on your series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Very big and very small numbers\n",
    "\n",
    "1. Generate a series with 100 random integers (as I've done, using `np.random.randint`) from 0-1000.\n",
    "2. Find the numbers that are either very big (i.e., > mean + std) or very small (i.e., < mean - std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     684\n",
       "1     559\n",
       "2     629\n",
       "3     192\n",
       "4     835\n",
       "     ... \n",
       "95    398\n",
       "96    611\n",
       "97    565\n",
       "98    908\n",
       "99    633\n",
       "Length: 100, dtype: int64"
      ]
     },
     "execution_count": 180,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(0)\n",
    "s = Series(np.random.randint(0, 1000, 100))\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     False\n",
       "1     False\n",
       "2     False\n",
       "3     False\n",
       "4      True\n",
       "      ...  \n",
       "95    False\n",
       "96    False\n",
       "97    False\n",
       "98     True\n",
       "99    False\n",
       "Length: 100, dtype: bool"
      ]
     },
     "execution_count": 181,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# boolean series indicating which numbers are very big\n",
    "s > s.mean() + s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     False\n",
       "1     False\n",
       "2     False\n",
       "3      True\n",
       "4     False\n",
       "      ...  \n",
       "95    False\n",
       "96    False\n",
       "97    False\n",
       "98    False\n",
       "99    False\n",
       "Length: 100, dtype: bool"
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# boolean series indicating which numbers are very small\n",
    "s < s.mean() - s.std()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3     192\n",
       "4     835\n",
       "8       9\n",
       "14     70\n",
       "22     87\n",
       "23    174\n",
       "25    849\n",
       "28    845\n",
       "29     72\n",
       "31    916\n",
       "32    115\n",
       "33    976\n",
       "36    847\n",
       "39    850\n",
       "40     99\n",
       "41    984\n",
       "42    177\n",
       "46    147\n",
       "47    910\n",
       "50    961\n",
       "58    151\n",
       "62    882\n",
       "63    183\n",
       "64     28\n",
       "66    128\n",
       "67    128\n",
       "68    932\n",
       "69     53\n",
       "70    901\n",
       "78     42\n",
       "81    888\n",
       "84    999\n",
       "85    937\n",
       "86     57\n",
       "88    870\n",
       "89    119\n",
       "92     82\n",
       "93     91\n",
       "94    896\n",
       "98    908\n",
       "dtype: int64"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find numbers that are either very big or very small\n",
    "s.loc[(s > s.mean() + s.std()) |\n",
    "      (s < s.mean() - s.std())]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Series([], dtype: int64)"
      ]
     },
     "execution_count": 185,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what about 2 * std?\n",
    "\n",
    "s.loc[(s > s.mean() + 2 * s.std()) |\n",
    "      (s < s.mean() - 2 * s.std())]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next up\n",
    "\n",
    "1. More on indexes\n",
    "2. Useful methods on our series\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    684\n",
       "b    559\n",
       "c    629\n",
       "d    192\n",
       "e    835\n",
       "f    763\n",
       "g    707\n",
       "h    359\n",
       "i      9\n",
       "j    723\n",
       "dtype: int64"
      ]
     },
     "execution_count": 187,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(0)\n",
    "s = Series(np.random.randint(0, 1000, 10),\n",
    "          index=list('abcdefghij'))\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "j    277\n",
       "i    754\n",
       "h    804\n",
       "g    599\n",
       "f     70\n",
       "e    472\n",
       "d    600\n",
       "c    396\n",
       "b    314\n",
       "a    705\n",
       "dtype: int64"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = Series(np.random.randint(0, 1000, 10),\n",
    "          index=list('jihgfedcba'))\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    1389\n",
       "b     873\n",
       "c    1025\n",
       "d     792\n",
       "e    1307\n",
       "f     833\n",
       "g    1306\n",
       "h    1163\n",
       "i     763\n",
       "j    1000\n",
       "dtype: int64"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the addition will take place via the indexes, not the positions\n",
    "s + t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1389"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "684+705"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    777\n",
       "b    916\n",
       "c    115\n",
       "d    976\n",
       "e    755\n",
       "a    709\n",
       "b    847\n",
       "c    431\n",
       "d    448\n",
       "e    850\n",
       "dtype: int64"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# index entries don't have to be unique!\n",
    "\n",
    "t = Series(np.random.randint(0, 1000, 10),\n",
    "          index=list('abcdeabcde'))\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    777\n",
       "a    709\n",
       "dtype: int64"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# retrieve an index that refers to multiple values, and you'll get a series back\n",
    "t.loc['a']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    777\n",
       "a    709\n",
       "c    115\n",
       "c    431\n",
       "dtype: int64"
      ]
     },
     "execution_count": 194,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.loc[['a', 'c']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    1461.0\n",
       "a    1393.0\n",
       "b    1475.0\n",
       "b    1406.0\n",
       "c     744.0\n",
       "c    1060.0\n",
       "d    1168.0\n",
       "d     640.0\n",
       "e    1590.0\n",
       "e    1685.0\n",
       "f       NaN\n",
       "g       NaN\n",
       "h       NaN\n",
       "i       NaN\n",
       "j       NaN\n",
       "dtype: float64"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what will happen when we add s (index a-j) and t (index a-e, a-e) now?\n",
    "s + t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    777\n",
       "b    916\n",
       "c    115\n",
       "d    976\n",
       "e    755\n",
       "a    709\n",
       "b    847\n",
       "c    431\n",
       "d    448\n",
       "e    850\n",
       "dtype: int64"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "709"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# use iloc when the index would make things ambiguious, or impossible\n",
    "t.iloc[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "\"Cannot get left slice bound for non-unique label: 'a'\"",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m/var/folders/rr/0mnyyv811fs5vyp22gf4fxk00000gn/T/ipykernel_3159/4171544057.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# what if I try to get t.loc['a':'c']?\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'a'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m'c'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/usr/local/lib/python3.10/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m    965\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    966\u001b[0m             \u001b[0mmaybe_callable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapply_if_callable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 967\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmaybe_callable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    968\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    969\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_is_scalar_access\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.10/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_getitem_axis\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m   1178\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mslice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1179\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_validate_key\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1180\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_slice_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1181\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0mcom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mis_bool_indexer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1182\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getbool_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.10/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_get_slice_axis\u001b[0;34m(self, slice_obj, axis)\u001b[0m\n\u001b[1;32m   1212\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1213\u001b[0m         \u001b[0mlabels\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mobj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1214\u001b[0;31m         \u001b[0mindexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlabels\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mslice_indexer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mslice_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mslice_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstop\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mslice_obj\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1215\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1216\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindexer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mslice\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.10/site-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mslice_indexer\u001b[0;34m(self, start, end, step, kind)\u001b[0m\n\u001b[1;32m   6272\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_deprecated_arg\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkind\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"kind\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"slice_indexer\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6273\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6274\u001b[0;31m         \u001b[0mstart_slice\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_slice\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mslice_locs\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstep\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   6275\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6276\u001b[0m         \u001b[0;31m# return a slice\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.10/site-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mslice_locs\u001b[0;34m(self, start, end, step, kind)\u001b[0m\n\u001b[1;32m   6482\u001b[0m         \u001b[0mstart_slice\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6483\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mstart\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6484\u001b[0;31m             \u001b[0mstart_slice\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_slice_bound\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"left\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   6485\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mstart_slice\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6486\u001b[0m             \u001b[0mstart_slice\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.10/site-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mget_slice_bound\u001b[0;34m(self, label, side, kind)\u001b[0m\n\u001b[1;32m   6409\u001b[0m             \u001b[0mslc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmaybe_booleans_to_slice\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mslc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mview\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"u1\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6410\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mslc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndarray\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6411\u001b[0;31m                 raise KeyError(\n\u001b[0m\u001b[1;32m   6412\u001b[0m                     \u001b[0;34mf\"Cannot get {side} slice bound for non-unique \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6413\u001b[0m                     \u001b[0;34mf\"label: {repr(original_label)}\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyError\u001b[0m: \"Cannot get left slice bound for non-unique label: 'a'\""
     ]
    }
   ],
   "source": [
    "# what if I try to get t.loc['a':'c']?\n",
    "\n",
    "t.loc['a':'c']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    777\n",
       "a    709\n",
       "c    115\n",
       "c    431\n",
       "d    976\n",
       "d    448\n",
       "dtype: int64"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.loc[['a', 'c', 'd']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count      6.000000\n",
       "mean     576.000000\n",
       "std      305.947708\n",
       "min      115.000000\n",
       "25%      435.250000\n",
       "50%      578.500000\n",
       "75%      760.000000\n",
       "max      976.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 203,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.loc[['a', 'c', 'd']].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Index types\n",
    "\n",
    "- Default is numeric, starting at 0, and going until the length - 1 (much like Python lists, strings, tuples).\n",
    "- You can also have a range index, specified with `range(start, end)` or `range(start, end, step)`.\n",
    "- You can have a string index, specified with a list of strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'], dtype='object')"
      ]
     },
     "execution_count": 204,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['a', 'b', 'c', 'd', 'e', 'a', 'b', 'c', 'd', 'e'], dtype='object')"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [],
   "source": [
    "t.index = range(10, 30, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10    777\n",
       "12    916\n",
       "14    115\n",
       "16    976\n",
       "18    755\n",
       "20    709\n",
       "22    847\n",
       "24    431\n",
       "26    448\n",
       "28    850\n",
       "dtype: int64"
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=10, stop=30, step=2)"
      ]
     },
     "execution_count": 208,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10    777\n",
       "12    916\n",
       "14    115\n",
       "16    976\n",
       "18    755\n",
       "20    709\n",
       "22    847\n",
       "24    431\n",
       "26    448\n",
       "28    850\n",
       "dtype: int64"
      ]
     },
     "execution_count": 209,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    777\n",
       "b    916\n",
       "c    115\n",
       "d    976\n",
       "e    755\n",
       "a    709\n",
       "b    847\n",
       "c    431\n",
       "d    448\n",
       "e    850\n",
       "dtype: int64"
      ]
     },
     "execution_count": 210,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.index = list('abcdeabcde')\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a     777\n",
       "b     916\n",
       "c     115\n",
       "d     976\n",
       "e    5555\n",
       "a     709\n",
       "b     847\n",
       "c     431\n",
       "d     448\n",
       "e    5555\n",
       "dtype: int64"
      ]
     },
     "execution_count": 211,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.loc['e'] = 5555  # this will assign to two elements\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    684\n",
       "b    559\n",
       "c    629\n",
       "d    192\n",
       "e    835\n",
       "f    763\n",
       "g    707\n",
       "h    359\n",
       "i      9\n",
       "j    723\n",
       "dtype: int64"
      ]
     },
     "execution_count": 212,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's set all values in s that are less than the mean to be 50\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.loc[s<s.mean()] = 50  # we can assign here, using our boolean index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    684\n",
       "b    559\n",
       "c    629\n",
       "d     50\n",
       "e    835\n",
       "f    763\n",
       "g    707\n",
       "h     50\n",
       "i     50\n",
       "j    723\n",
       "dtype: int64"
      ]
     },
     "execution_count": 214,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Weekend temps\n",
    "\n",
    "1. Recreate our series of high temperatures, but instead of dates as the index, use day names (`Mon`, `Tue`).\n",
    "2. What will be the mean temperature on weekends (Sat-Sun).\n",
    "3. What will be the mean temperature on weekdays?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32],\n",
    "          index=['Tue', 'Wed', 'Thu', 'Fri', 'Sat', \n",
    "                'Sun', 'Mon', 'Tue', 'Wed', 'Thu' ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34.0"
      ]
     },
     "execution_count": 217,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc[['Sat', 'Sun']].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Mon    33\n",
       "Tue    33\n",
       "Tue    32\n",
       "Wed    34\n",
       "Wed    32\n",
       "Thu    34\n",
       "Thu    32\n",
       "Fri    34\n",
       "dtype: int64"
      ]
     },
     "execution_count": 219,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc[['Mon', 'Tue', 'Wed', 'Thu', 'Fri']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sat    34\n",
       "Sun    34\n",
       "dtype: int64"
      ]
     },
     "execution_count": 221,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# indexes are series (mostly), too!\n",
    "# we can run the \"isin\" method on a series, which returns True/False for each value\n",
    "\n",
    "# here, we're saying: get a boolean series indicating whether each value of s's index\n",
    "#  is in ['Sat', 'Sun']\n",
    "# apply that as a mask index on s\n",
    "\n",
    "s.loc[s.index.isin(['Sat', 'Sun'])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Tue    33\n",
       "Wed    34\n",
       "Thu    34\n",
       "Fri    34\n",
       "Mon    33\n",
       "Tue    32\n",
       "Wed    32\n",
       "Thu    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 223,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# tilde (~) flips the logic (True->False, False->True) on a boolean series\n",
    "# here, we do the same as above, but flip the logic, so that we get\n",
    "# the opposite elements\n",
    "\n",
    "s.loc[~s.index.isin(['Sat', 'Sun'])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "metadata": {},
   "outputs": [],
   "source": [
    "temps = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32],\n",
    "          index=['Tue', 'Wed', 'Thu', 'Fri', 'Sat', \n",
    "                'Sun', 'Mon', 'Tue', 'Wed', 'Thu' ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     684\n",
       "1     559\n",
       "2     629\n",
       "3     192\n",
       "4     835\n",
       "     ... \n",
       "95    398\n",
       "96    611\n",
       "97    565\n",
       "98    908\n",
       "99    633\n",
       "Length: 100, dtype: int64"
      ]
     },
     "execution_count": 225,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.random.seed(0)\n",
    "s = Series(np.random.randint(0, 1000, 100))\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    684\n",
       "1    559\n",
       "2    629\n",
       "3    192\n",
       "4    835\n",
       "dtype: int64"
      ]
     },
     "execution_count": 227,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what if I just want to look at the first part of a series? Or the last part?\n",
    "# I can use the head and tail methods.  By default, they show 5 elements, but\n",
    "# can pass an integer to change that\n",
    "\n",
    "s.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "95    398\n",
       "96    611\n",
       "97    565\n",
       "98    908\n",
       "99    633\n",
       "dtype: int64"
      ]
     },
     "execution_count": 228,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 229,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    684\n",
       "1    559\n",
       "2    629\n",
       "3    192\n",
       "4    835\n",
       "5    763\n",
       "6    707\n",
       "dtype: int64"
      ]
     },
     "execution_count": 229,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.head(7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34    5\n",
       "32    3\n",
       "33    2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 231,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# I want to know: What are the most common values in my series?\n",
    "# the value_counts method does this for us\n",
    "\n",
    "# if I call temps.value_counts(), each of the distinct values in temps becomes\n",
    "# an index element, and the number of times it appears becomes the value\n",
    "\n",
    "# it's ordered, from most to least common\n",
    "\n",
    "# it's a series, so we can do series things on it -- like .head()\n",
    "\n",
    "temps.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34    0.5\n",
       "32    0.3\n",
       "33    0.2\n",
       "dtype: float64"
      ]
     },
     "execution_count": 232,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what percentage of the time was each value in temps?\n",
    "\n",
    "temps.value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([684, 559, 629, 192, 835, 763, 707, 359,   9, 723, 277, 754, 804,\n",
       "       599,  70, 472, 600, 396, 314, 705, 486, 551,  87, 174, 849, 677,\n",
       "       537, 845,  72, 777, 916, 115, 976, 755, 709, 847, 431, 448, 850,\n",
       "        99, 984, 177, 797, 659, 147, 910, 423, 288, 961, 265, 697, 639,\n",
       "       544, 543, 714, 244, 151, 675, 510, 459, 882, 183,  28, 802, 128,\n",
       "       932,  53, 901, 550, 488, 756, 273, 335, 388, 617,  42, 442, 888,\n",
       "       257, 321, 999, 937,  57, 291, 870, 119, 779, 430,  82,  91, 896,\n",
       "       398, 611, 565, 908, 633])"
      ]
     },
     "execution_count": 234,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what are the unique (different) values in a series?\n",
    "s.unique() # returns a NumPy array!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([33, 34, 32])"
      ]
     },
     "execution_count": 237,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "temps.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([34, 32, 33], dtype='int64')"
      ]
     },
     "execution_count": 238,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# another way -- get the index from value_counts()\n",
    "temps.value_counts().index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RL      52\n",
       "SF      50\n",
       "AMLF    21\n",
       "SBLF    19\n",
       "ADLF    16\n",
       "dtype: int64"
      ]
     },
     "execution_count": 239,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = Series([52, 50, 21, 19, 16],\n",
    "          index=['RL', 'SF', 'AMLF', 'SBLF', 'ADLF'])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RL       NaN\n",
       "SF      -2.0\n",
       "AMLF   -29.0\n",
       "SBLF    -2.0\n",
       "ADLF    -3.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 240,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what is the difference between each person's age and the next person's age?\n",
    "\n",
    "s.diff()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RL           NaN\n",
       "SF     -0.038462\n",
       "AMLF   -0.580000\n",
       "SBLF   -0.095238\n",
       "ADLF   -0.157895\n",
       "dtype: float64"
      ]
     },
     "execution_count": 241,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.pct_change()  # silly with ages!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise: Change in weather\n",
    "\n",
    "1. Using our high-temperature series, what are the 3 most common temperatures in your 10-day forecats?\n",
    "2. On what day will there be the greatest numeric (not percentage) change in temperature?  How about the smallest change?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = Series([33, 34, 34, 34, 34, 34, 33, 32, 32, 32],\n",
    "          index=['0802', '0803', '0804', '0805', '0806', \n",
    "                '0807', '0808', '0809', '0810', '0811' ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 243,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34    5\n",
       "32    3\n",
       "33    2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 243,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34    5\n",
       "32    3\n",
       "dtype: int64"
      ]
     },
     "execution_count": 244,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.value_counts().head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0802    NaN\n",
       "0803    1.0\n",
       "0804    0.0\n",
       "0805    0.0\n",
       "0806    0.0\n",
       "0807    0.0\n",
       "0808   -1.0\n",
       "0809   -1.0\n",
       "0810    0.0\n",
       "0811    0.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 245,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.diff()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0803    34\n",
       "dtype: int64"
      ]
     },
     "execution_count": 248,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find the day(s) with the greatest change in temp\n",
    "\n",
    "s.loc[s.diff() == s.diff().max()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0808    33\n",
       "0809    32\n",
       "dtype: int64"
      ]
     },
     "execution_count": 249,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# find the day(s) with the smallest change in temp\n",
    "\n",
    "s.loc[s.diff() == s.diff().min()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next week:\n",
    "\n",
    "1. dtypes -- what are they, and why do we care?\n",
    "2. Data frames (2D data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}