orig_base_r_to_tidyverse.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>Transitioning into the tidyverse</center>\n",
    "### <center>Rebecca Barter (@rlbarter)</center>\n",
    "\n",
    "\n",
    "<center><img src=\"images/horst_tidyverse.png\" alt=\"Tidyverse\" style=\"width: 400px;\" align=\"middle\"/>\n",
    "(Image by Allison Horst)\n",
    "</center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "## Slides \n",
    "[https://github.com/rlbarter/tidyverse_tutorial](https://github.com/rlbarter/tidyverse_tutorial)\n",
    "\n",
    "\n",
    "## Blog post \n",
    "[http://www.rebeccabarter.com/blog/2019-08-05_base_r_to_tidyverse/](http://www.rebeccabarter.com/blog/2019-08-05_base_r_to_tidyverse/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> What is the tidyverse? </center>\n",
    "<br>\n",
    "<center>A set of packages (<i>ggplot2, dplyr, purrr, tidyr, readr, tibble</i>)</center>\n",
    "<br>\n",
    "<center>\n",
    "... and a way of thinking about \"tidy\" analysis.\n",
    "</center>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center>Two ways to keep up with the tidyverse:</center>\n",
    "\n",
    "\n",
    "<center>\n",
    "    <table>\n",
    "        <tr>\n",
    "         <td><img src=\"images/twitter.png\" alt=\"Tidyverse\" style=\"width: 100px;\"/></td>\n",
    "         <td><img src=\"images/rstudio.png\" alt=\"Tidyverse\" style=\"width: 200px;\"/></td>\n",
    "        </tr>        \n",
    "    </table>\n",
    "</center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>Loading the tidyverse</center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "library(dplyr)\n",
    "library(ggplot2)\n",
    "library(purrr)\n",
    "library(tidyr)\n",
    "library(readr)\n",
    "library(tibble)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center>Or you can just load the tidyverse package:</center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "library(tidyverse) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Entering the tidyverse </center>\n",
    "<br>\n",
    "<center>The fundamental object type of the tidyverse is the <b>data frame</b></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center><b>Your data frame is the universe</b> whose objects are the columns</center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "# Alcohol consumption in Australia and New Zealand\n",
    "library(DAAG)\n",
    "grog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center>Tidy coding involves <b>minimizing defining new objects</b></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## <center> Part 1 </center>\n",
    "<center> piping, dplyr and ggplot2 </center>\n",
    "\n",
    "\n",
    "<center>\n",
    "    <table>\n",
    "        <tr>\n",
    "            <td><img src=\"images/pipe.png\" alt=\"Pipe\" style=\"width: 200px;\"/></td>\n",
    "            <td><img src=\"images/dplyr.png\" alt=\"dplyr\" style=\"width: 200px;\"/></td>\n",
    "            <td><img src=\"images/ggplot2.png\" alt=\"ggplot2\" style=\"width: 200px;\"/></td>\n",
    "        </tr>        \n",
    "     </table>\n",
    "</center>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "## <center> Part 2 </center>\n",
    "<center>tidyr, purrr, readr, tibbles, lubridate, forcats, stringr </center>\n",
    "\n",
    "<center>\n",
    "    <table>\n",
    "        <tr>\n",
    "            <td><img src=\"images/tidyr.jpg\" alt=\"tidyr\" style=\"width: 200px;\"/></td>\n",
    "            <td><img src=\"images/purrr.jpg\" alt=\"purrr\" style=\"width: 200px;\"/></td>\n",
    "            <td><img src=\"images/readr.png\" alt=\"readr\" style=\"width: 170px;\"/></td>\n",
    "            <td><img src=\"images/tibble.png\" alt=\"tibble\" style=\"width: 165px;\"/></td>\n",
    "            <td><img src=\"images/lubridate.png\" alt=\"lubridate\" style=\"width: 165px;\"/></td>\n",
    "            <td><img src=\"images/forcats.png\" alt=\"forcats\" style=\"width: 165px;\"/></td>\n",
    "            <td><img src=\"images/stringr.jpg\" alt=\"stringr\" style=\"width: 165px;\"/></td>\n",
    "        </tr>        \n",
    "     </table>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>Gapminder data!</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let's load the gapminder dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# to download the data directly:\n",
    "gapminder_orig <- read.csv(\n",
    "    \"https://raw.githubusercontent.com/swcarpentry/r-novice-gapminder/gh-pages/_episodes_rmd/data/gapminder-FiveYearData.csv\"\n",
    ")\n",
    "# define a copy of the original dataset that we will clean and play with \n",
    "gapminder <- gapminder_orig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# check the dimension of the data\n",
    "dim(gapminder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# show the first 6 rows of the data\n",
    "head(gapminder, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Piping </center>\n",
    "\n",
    "<center><img src=\"images/pipe.png\" alt=\"Tidyverse\" style=\"width: 200px;\" align=\"middle\"/></center>\n",
    "\n",
    "<center>&ensp;&ensp;Chain functions together &emsp; &emsp;  Easy to read  &emsp;&emsp; Reduces intermediate objects  </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>The pipe: %>%</center>\n",
    "<br>\n",
    "<center>Read the pipe as <i>\"and then\"</i> </center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>%  \n",
    "  filter(continent == \"Americas\", year == 2007) %>%\n",
    "  select(country, year, lifeExp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### What is the pipe doing?\n",
    "\n",
    "The object on the *left-hand-side* of the `%>%` is used as the *first argument* of the function on the *right-hand-side*.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "Which is easier to read?\n",
    "\n",
    "The unpiped version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "gapminder_filtered <- filter(gapminder, continent == \"Americas\", year == 2007)\n",
    "gapminder_filtered_selected <- select(gapminder_filtered, country, lifeExp)\n",
    "gapminder_filtered_selected"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "or the piped version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(continent == \"Americas\", year == 2007) %>%\n",
    "  select(country, lifeExp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>dplyr</center>\n",
    "\n",
    "<center><img src=\"images/dplyr.png\" alt=\"dplyr\" style=\"width: 200px;\" align=\"middle\"/></center>\n",
    "\n",
    "<center> the data manipulation package </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> dplyr: indexing </center>\n",
    "\n",
    "The tidyverse rarely uses `[,]` or `$` indexing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Instead, indexing is done using \n",
    "\n",
    "- `filter()` for rows \n",
    "\n",
    "- `select()` for columns.\n",
    "\n",
    "Variable names inside tidyverse functions are **unquoted**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>dplyr::select()</center>\n",
    "\n",
    "`select()` is for indexing columns. Specify the column names that you want to keep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  select(country, gdpPercap) %>%\n",
    "  head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "or specify those that you want to remove"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  select(-continent) %>%\n",
    "  head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>dplyr::filter()</center>\n",
    "\n",
    "`filter()` is for indexing rows that satisfy a condition. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  filter(pop > 1000000000) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "You can specify multiple conditions "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  filter(pop > 1000000000, year == 1992) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> dplyr: readability </center>\n",
    "\n",
    "Compare dplyr code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(continent == \"Americas\", year == 2007) %>%\n",
    "  select(country, lifeExp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "with one potential version of equivalent base R code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "continent_year_index <- which(gapminder[\"continent\"] == \"Americas\" & gapminder[\"year\"] == 2007)\n",
    "gapminder[continent_year_index, c(\"country\", \"lifeExp\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> dplyr::mutate() </center>\n",
    "\n",
    "`mutate()` lets you add new variables.\n",
    "\n",
    "The new variable is *usually* a function of existing variables.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  mutate(gdp = gdpPercap * pop) %>%\n",
    "  head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Note that if you want to actually modify your data, you would need to redefine the `gapminder` data frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder <- gapminder %>% \n",
    "  mutate(gdp = gdpPercap * pop) \n",
    "gapminder %>% head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> dplyr::arrange() </center>\n",
    "\n",
    "`arrange()` lets you reorder the rows of your data frame based on a specified column/variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    " gapminder %>% \n",
    "  arrange(lifeExp) %>%\n",
    "  head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To arrange in *descending* order, you need to wrap the variable in the `desc()` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  arrange(desc(lifeExp)) %>%\n",
    "  head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> dplyr::group_by() </center>\n",
    "\n",
    "`group_by()` lets you to apply functions separately within groups defined by a categorical variable. \n",
    "\n",
    "After you're done, remember to `ungroup()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  group_by(continent) %>%\n",
    "  filter(lifeExp > mean(lifeExp)) %>%\n",
    "  ungroup() \n",
    "#  count(continent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# compare with the ungrouped version\n",
    "gapminder %>%\n",
    "  filter(lifeExp > mean(lifeExp)) \n",
    "#  count(continent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> dplyr::summarise() </center>\n",
    "\n",
    "`summarise()` lets you aggregate across rows of the data frame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  summarise(mean_lifeExp = mean(lifeExp),\n",
    "            total_gdp = sum(gdp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "`summarise()` also works really nicely with `group_by()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  group_by(year) %>%\n",
    "  summarise(mean_lifeExp = mean(lifeExp),\n",
    "            total_gdp = sum(gdp)) %>%\n",
    "  ungroup()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> More dplyr functions </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`rename()` for renaming variables of the data frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  rename(gdp_per_capita = gdpPercap,\n",
    "         life_exp = lifeExp) %>%\n",
    "  head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "`distinct()` for extracting the distinct values of a variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  distinct(continent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "`sample_n()` and `sample_frac()` for taking random samples of rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  sample_n(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "`count()` for counting the number of rows with each value of a categorical variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  count(continent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "`transmute()` mutate and select at the same time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  transmute(gdp = gdpPercap * pop) %>%\n",
    "  head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Advanced dplyr practitioners will eventually want to learn about [*scoped verbs*](http://www.rebeccabarter.com/blog/2019-01-23_scoped-verbs/). \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> ggplot2 </center>\n",
    "\n",
    "\n",
    "<center><img src=\"images/ggplot2.png\" alt=\"ggplot2\" style=\"width: 200px;\" align=\"middle\"/></center>\n",
    "\n",
    "<center>the data visualisation package</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center>ggplot2</center>\n",
    "\n",
    "Based on the **layered grammar of graphics**.\n",
    "\n",
    "**Geom**etric objects are created based on **aes**thetic mappings from data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ggplot(gapminder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> ggplot2 with pipes </center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  filter(year == 2007) %>%\n",
    "  ggplot()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Geom layers </center>\n",
    "\n",
    "Layers are added using `+` (rather than `%>%`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() \n",
    "  # add a points layer on top (for gdpPercap and lifeExp)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Geom layers </center>\n",
    "\n",
    "You can layer multiple geom layers ontop of one another."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() +\n",
    "  geom_point(aes(x = gdpPercap, y = lifeExp)) +\n",
    "  # ((((( add a smoothed LOESS layer (geom_smooth) ))))) \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> Global aesthetic mappings </center>\n",
    "\n",
    "Instead of specifying a separate `aes()` function for each layer, you can specify a global aesthetic function in the `ggplot()` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  # (((( specify global aesthetic mappings ))))\n",
    "  ggplot() +\n",
    "  geom_point(aes(x = gdpPercap, y = lifeExp)) +\n",
    "  geom_smooth(aes(x = gdpPercap, y = lifeExp), method = \"loess\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> ggplot2::geom_line() </center>\n",
    "\n",
    "Line plots are great for visualizing time series. \n",
    "\n",
    "Plotting the average life expectency by year."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  # ((((( calculate the average life expectency for each year )))))\n",
    "\n",
    "  # ((((( plot points and line )))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> Grouping line plots </center>\n",
    "\n",
    "What if we want a separate line for each continent?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ((((((( create separate line for each continent )))))))\n",
    "gapminder %>%\n",
    "  # calculate average life expectancy\n",
    "  group_by(year) %>%\n",
    "  summarise(avg_lifeExp = mean(lifeExp)) %>%\n",
    "  ungroup() %>%\n",
    "  # plot points and lines\n",
    "  ggplot(aes(x = year, y = avg_lifeExp)) +\n",
    "  geom_point() +\n",
    "  geom_line()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> More aesthetic mappings </center>\n",
    "\n",
    "We have only seen x- and y-position mappings so far. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To change the color of all points - add a `col` argument *outside* the `aes()` function "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() +\n",
    "  # (((((( change color of all points to cornflowerblue ))))))\n",
    "  geom_point(aes(x = gdpPercap, \n",
    "                 y = lifeExp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## <center> Using a variable to colour the points </center>\n",
    "\n",
    "To specify color based on a variable - add a `col` argument *inside* the `aes()` function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() +\n",
    "  # ((((( change color based on continent )))))\n",
    "  geom_point(aes(x = gdpPercap, \n",
    "                 y = lifeExp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> Changing the colors with a scale layer </center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() +\n",
    "  geom_point(aes(x = gdpPercap, \n",
    "                 y = lifeExp, \n",
    "                 col  = continent)) + \n",
    "  # manually choose the colors for the groups\n",
    "  scale_colour_manual(values = c(\"orange\", \"red4\", \"purple\", \"darkgreen\", \"blue\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> Other aesthetic mappings </center>\n",
    "\n",
    "Specifying the **size** based on a variable "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() +\n",
    "  geom_point(aes(x = gdpPercap, \n",
    "                 y = lifeExp, \n",
    "                 col = continent, \n",
    "                 size = pop)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> Line plots: grouping and color </center>\n",
    "\n",
    "Color can also group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  group_by(continent, year) %>%\n",
    "  summarise(avg_lifeExp = mean(lifeExp)) %>%\n",
    "  # specify global aesthetic mappings\n",
    "  ggplot() +\n",
    "  # add a points layer on top\n",
    "  geom_line(aes(x = year, y = avg_lifeExp, group = continent))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Other types of geoms </center>\n",
    "\n",
    "## Histograms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  ggplot() + \n",
    "  geom_histogram(aes(x = lifeExp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Boxplots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  ggplot() +\n",
    "  geom_boxplot(aes(x = continent, y = lifeExp))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Faceting: making grids of plots </center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>%\n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() +\n",
    "  geom_point(aes(x = gdpPercap, y = lifeExp)) \n",
    "  # ((((( Create a facet of plots by continent )))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Customizing ggplot2 </center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  filter(year == 2007) %>%\n",
    "  ggplot() +\n",
    "  # add scatter points\n",
    "  geom_point(aes(x = gdpPercap, y = lifeExp, col = continent, size = pop),\n",
    "             alpha = 0.7) +\n",
    "  # add some text annotations for the very large countries\n",
    "  geom_text(aes(x = gdpPercap, y = lifeExp + 3, label = country),\n",
    "            col = \"grey50\",\n",
    "            data = filter(gapminder, year == 2007, pop > 1000000000 | country %in% c(\"Nigeria\", \"United States\"))) +\n",
    "  # clean the axes names and breaks\n",
    "  scale_x_log10(limits = c(200, 60000)) +\n",
    "  # change labels\n",
    "  labs(title = \"GDP versus life expectancy in 2007\",\n",
    "       x = \"GDP per capita (log scale)\",\n",
    "       y = \"Life expectancy\",\n",
    "       size = \"Population\",\n",
    "       col = \"Continent\") +\n",
    "  # change the size scale\n",
    "  scale_size(range = c(0.1, 10),\n",
    "             # remove size legend\n",
    "             guide = \"none\") +\n",
    "  # add a nicer theme\n",
    "  theme_classic() +\n",
    "  # place legend at top and grey axis lines\n",
    "  theme(legend.position = \"top\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "gapminder_life_exp_diff <- gapminder %>%\n",
    "  # filter to the starting and ending years only\n",
    "  filter(year == 1952 | year == 2007) %>%\n",
    "  # ensure that the data are arranged so that 1952 is first and 2007 is second \n",
    "  # within each year\n",
    "  arrange(country, year) %>%\n",
    "  # for country, add a variable corresponding to the difference between life \n",
    "  # expectency in 2007 and 1952\n",
    "  group_by(country) %>%\n",
    "  mutate(lifeExp_diff = lifeExp[2] - lifeExp[1],\n",
    "         # also calculate the largest population for the country (based on the two years)\n",
    "         max_pop = max(pop)) %>%\n",
    "  ungroup() %>%\n",
    "  # arrange in order of the biggest difference in life expectency\n",
    "  arrange(lifeExp_diff) %>%\n",
    "  # restrict to countries with a population of at least 30,000 so we can fit \n",
    "  # the plot in a reasonable space\n",
    "  filter(max_pop > 50000000) %>%\n",
    "  # redefine the country varaible so that it does not have the additional \n",
    "  # country levels corresponding to countries that were removed in the previous\n",
    "  # step\n",
    "  mutate(country = droplevels(country)) %>%\n",
    "  select(country, year, continent, lifeExp, lifeExp_diff)\n",
    "gapminder_life_exp_diff %>%\n",
    "  mutate(country = fct_inorder(country)) %>%\n",
    "  # for each country define a varaible for min and max life expectancy\n",
    "  group_by(country) %>%\n",
    "  mutate(max_lifeExp = max(lifeExp),\n",
    "         min_lifeExp = min(lifeExp)) %>% \n",
    "  ungroup() %>%\n",
    "  ggplot() +\n",
    "  # plot a horizontal line from min to max life expectency for each country\n",
    "  geom_segment(aes(x = min_lifeExp, xend = max_lifeExp, \n",
    "                   y = country, yend = country,\n",
    "                   col = continent), alpha = 0.5, size = 7) +\n",
    "  # add a point for each life expectancy data point\n",
    "  geom_point(aes(x = lifeExp, y = country, col = continent), size = 8) +\n",
    "  # add text of the country name as well as the max and min life expectency \n",
    "  geom_text(aes(x = min_lifeExp + 0.7, y = country, \n",
    "                label = paste(country, round(min_lifeExp))), \n",
    "            col = \"grey50\", hjust = \"right\") +\n",
    "  geom_text(aes(x = max_lifeExp - 0.7, y = country, \n",
    "                label = round(max_lifeExp)), \n",
    "            col = \"grey50\", hjust = \"left\") +\n",
    "  # ensure that the left-most text is not cut off \n",
    "  scale_x_continuous(limits = c(20, 85)) +\n",
    "  # choose a different colour palette\n",
    "  scale_colour_brewer(palette = \"Pastel2\") +\n",
    "  # set the title\n",
    "  labs(title = \"Change in life expectancy\",\n",
    "       subtitle = \"Between 1952 and 2007\",\n",
    "       x = \"Life expectancy (in 1952 and 2007)\",\n",
    "       y = NULL, \n",
    "       col = \"Continent\") +\n",
    "  # remove the grey background\n",
    "  theme_classic() +\n",
    "  # remove the axes and move the legend to the top\n",
    "  theme(legend.position = \"top\", \n",
    "        axis.line = element_blank(),\n",
    "        axis.ticks = element_blank(),\n",
    "        axis.text = element_blank())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <center> Part 2: everything else </center>\n",
    "\n",
    "## <center> tidyr, purrr, readr, tibbles, lubridate, forcats, stringr </center>\n",
    "\n",
    "\n",
    "<center>\n",
    "    <table>\n",
    "        <tr>\n",
    "            <td><img src=\"images/tidyr.jpg\" alt=\"tidyr\" style=\"width: 200px;\"/></td>\n",
    "            <td><img src=\"images/purrr.jpg\" alt=\"purrr\" style=\"width: 200px;\"/></td>\n",
    "            <td><img src=\"images/readr.png\" alt=\"readr\" style=\"width: 170px;\"/></td>\n",
    "            <td><img src=\"images/tibble.png\" alt=\"tibble\" style=\"width: 165px;\"/></td>\n",
    "            <td><img src=\"images/lubridate.png\" alt=\"lubridate\" style=\"width: 165px;\"/></td>\n",
    "            <td><img src=\"images/forcats.png\" alt=\"forcats\" style=\"width: 165px;\"/></td>\n",
    "            <td><img src=\"images/stringr.jpg\" alt=\"stringr\" style=\"width: 165px;\"/></td>\n",
    "        </tr>        \n",
    "     </table>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> tidyr </center>\n",
    "\n",
    "<center><td><img src=\"images/tidyr.jpg\" alt=\"tidyr\" style=\"width: 200px;\"/></td></center>\n",
    "\n",
    "### <center> Convert data between long and wide form </center>\n",
    "\n",
    "- `gather()` and `spread()`\n",
    "\n",
    "- `pivot_longer()` and `pivot_wider()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% \n",
    "  filter(country == \"Australia\", year > 2000) \n",
    "  #gather(key = \"variable\", value = \"value\", -country, -year) \n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> purrr </center>\n",
    "\n",
    "<center> <td><img src=\"images/purrr.jpg\" alt=\"purrr\" style=\"width: 200px;\"/></td> </center>\n",
    "\n",
    "### <center> Iterate. Tidyverse's apply functions </center>\n",
    "\n",
    "- `map()` functions iterate over elements of list, vector or columns of data frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gapminder %>% map_df(class) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> readr </center>\n",
    "\n",
    "<center> <td><img src=\"images/readr.png\" alt=\"readr\" style=\"width: 170px;\"/></td> </center>\n",
    "\n",
    "### <center>  Load in data </center>\n",
    "\n",
    "`read_csv()`, `read_delim()`, ....\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> tibble </center>\n",
    "\n",
    "<center> <td><img src=\"images/tibble.png\" alt=\"tibble\" style=\"width: 165px;\"/></td> </center>\n",
    "\n",
    "### <center> Fancy data frames </center>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> lubridate </center>\n",
    "\n",
    "<center> <td><img src=\"images/lubridate.png\" alt=\"lubridate\" style=\"width: 165px;\"/></td> </center>\n",
    "\n",
    "### <center> Handle dates and times </center>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "library(lubridate)\n",
    "mdy(\"August 2nd 2019\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mdy(\"8/2/19\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mdy_hms(\"August 2nd 2019, 1:21:30 pm\") - mdy_hms(\"August 1st 2019, 11:23:33 am\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> forcats </center>\n",
    "\n",
    "<center> <td><img src=\"images/forcats.png\" alt=\"forcats\" style=\"width: 165px;\"/></td> </center>\n",
    "\n",
    "### <center> Handle factors </center>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "library(repr)\n",
    "options(repr.plot.width=8, repr.plot.height=2.5)\n",
    "\n",
    "gapminder %>%\n",
    "  filter(pop > 100000000) %>%\n",
    "  group_by(country) %>% \n",
    "  summarise(avg_lifeExp = mean(lifeExp)) %>%\n",
    "  arrange(desc(avg_lifeExp)) %>% ############ Why doesn't this work?\n",
    "  ggplot() +\n",
    "  geom_bar(aes(x = country, y = avg_lifeExp), stat = \"identity\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# <center> stringr </center>\n",
    "\n",
    " <center> <td><img src=\"images/stringr.jpg\" alt=\"stringr\" style=\"width: 165px;\"/></td> </center>\n",
    "\n",
    "### <center> do things with strings </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<center> <td><img src=\"images/stringr_table.png\" alt=\"stringr\" style=\"width: 800px;\"/></td> </center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://stringr.tidyverse.org/articles/from-base.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Blog post\n",
    "### http://www.rebeccabarter.com/blog/2019-08-05_base_r_to_tidyverse/"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.5.2"
  },
  "rise": {
   "scroll": true,
   "start_slideshow_at": "selected",
   "theme": "sky"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}