raincloud_tutorial_python.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "#sns.set(style=\"darkgrid\")\n",
    "#sns.set(style=\"whitegrid\")\n",
    "#sns.set_style(\"white\")\n",
    "sns.set(style=\"whitegrid\",font_scale=2)\n",
    "import matplotlib.collections as clt\n",
    "\n",
    "\n",
    "import ptitprince as pt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Python tutorial will follow the footsteps of the R tutorial and guide you in the creation and customization of Raincloud plots.\n",
    "\n",
    "The Python implementation of Raincloud is a package named PtitPrince, written on the top of seaborn, a Python plotting library written to obtain good-looking plots from pandas dataframes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default this tutorial will not save versions of the figures you create, but you can change that in the following cell by changing `savefigs` to `True` and setting the figures directory to whatever you'd prefer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "savefigs = True\n",
    "figs_dir = '../figs/tutorial_python'\n",
    "\n",
    "if savefigs:\n",
    "    # Make the figures folder if it doesn't yet exist\n",
    "    if not os.path.isdir('../figs/tutorial_python'):\n",
    "        os.makedirs('../figs/tutorial_python')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def export_fig(axis,text, fname):\n",
    "    if savefigs:\n",
    "        axis.text()\n",
    "        axis.savefig(fname, bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As first step, we will load the same dataset used before and visualize the distribution of each measure as a simple barplot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv (\"simdat.csv\", sep= \",\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "f, ax = plt.subplots(figsize=(7, 7))\n",
    "\n",
    "sns.barplot(x = \"group\", y = \"score\", data = df, capsize= .1)\n",
    "plt.title(\"Figure P1\\n Bar Plot\")\n",
    "\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP01.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plot can give the reader a first idea of the dataset: which group has bigger mean value, whether this difference is likely to be significant or not.\n",
    "Only the mean of each group score and the standard deviation is visualized in this plot. \n",
    "\n",
    "\n",
    "To have an idea of the distribution of our dataset we can plot a \"cloud\", a smoothed version of the histogram:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plotting the clouds\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "dy=\"group\"; dx=\"score\"; ort=\"h\"; pal = sns.color_palette(n_colors=1)\n",
    "\n",
    "ax=pt.half_violinplot( x = dx, y = dy, data = df, palette = pal, bw = .2, cut = 0.,\n",
    "                      scale = \"area\", width = .6, inner = None, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P2\\n Basic Rainclouds\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP02.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To have a more precise idea of the distribution and illustrate potential outliers or other patterns within the data, we now add the \"rain\", a simple monodimensional representation of the data points:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# adding the rain\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "ax=pt.half_violinplot( x = dx, y = dy, data = df, palette = pal, bw = .2, cut = 0.,\n",
    "                      scale = \"area\", width = .6, inner = None, orient = ort)\n",
    "ax=sns.stripplot( x = dx, y = dy, data = df, palette = pal, edgecolor = \"white\",\n",
    "                 size = 3, jitter = 0, zorder = 0, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P3\\n Raincloud Without Jitter\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP03.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# adding jitter to the rain\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "ax=pt.half_violinplot( x = dx, y = dy, data = df, palette = pal, bw = .2, cut = 0.,\n",
    "                      scale = \"area\", width = .6, inner = None, orient = ort)\n",
    "ax=sns.stripplot( x = dx, y = dy, data = df, palette = pal, edgecolor = \"white\",\n",
    "                 size = 3, jitter = 1, zorder = 0, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P4\\n Raincloud with Jittered Data\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP04.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This gives a good idea of the distribution of the data points, but the median and the quartiles are not obvious, making it hard to determine statistical differences at a glance. Hence we add an \"empty\" boxplot to show median, quartiles and outliers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#adding the boxplot with quartiles\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "ax=pt.half_violinplot( x = dx, y = dy, data = df, palette = pal, bw = .2, cut = 0.,\n",
    "                      scale = \"area\", width = .6, inner = None, orient = ort)\n",
    "ax=sns.stripplot( x = dx, y = dy, data = df, palette = pal, edgecolor = \"white\",\n",
    "                 size = 3, jitter = 1, zorder = 0, orient = ort)\n",
    "ax=sns.boxplot( x = dx, y = dy, data = df, color = \"black\", width = .15, zorder = 10,\\\n",
    "            showcaps = True, boxprops = {'facecolor':'none', \"zorder\":10},\\\n",
    "            showfliers=True, whiskerprops = {'linewidth':2, \"zorder\":10},\\\n",
    "               saturation = 1, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P5\\n Raincloud with Boxplot\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP05.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can set a color palette to characterize the two groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#adding color\n",
    "pal = \"Set2\"\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "\n",
    "ax=pt.half_violinplot( x = dx, y = dy, data = df, palette = pal, bw = .2, cut = 0.,\n",
    "                      scale = \"area\", width = .6, inner = None, orient = ort)\n",
    "ax=sns.stripplot( x = dx, y = dy, data = df, palette = pal, edgecolor = \"white\",\n",
    "                 size = 3, jitter = 1, zorder = 0, orient = ort)\n",
    "ax=sns.boxplot( x = dx, y = dy, data = df, color = \"black\", width = .15, zorder = 10,\\\n",
    "            showcaps = True, boxprops = {'facecolor':'none', \"zorder\":10},\\\n",
    "            showfliers=True, whiskerprops = {'linewidth':2, \"zorder\":10},\\\n",
    "               saturation = 1, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P6\\n Tweaking the Colour of Your Raincloud\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP06.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a very nice plot, but written in far too many lines! We can use the function `pt.Raincloud` to add some automation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#same thing with a single command: now x **must** be the categorical value\n",
    "dx = \"group\"; dy = \"score\"; ort = \"h\"; pal = \"Set2\"; sigma = .2\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "\n",
    "pt.RainCloud(x = dx, y = dy, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .6, ax = ax, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P7\\n Using the pt.Raincloud function\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP07.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 'move' parameter can be used to shift the rain below the boxplot, giving better visibility of the raw data in some instances: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#moving the rain below the boxplot\n",
    "dx = \"group\"; dy = \"score\"; ort = \"h\"; pal = \"Set2\"; sigma = .2\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .6, ax = ax, orient = ort, move = .2)\n",
    "\n",
    "plt.title(\"Figure P8\\n Rainclouds with Shifted Rain\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP08.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Further, the raincloud function works equally well with a list or np.array, if you prefer to use those instead of a dataframe input: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Usage with a list/np.array input\n",
    "dx = list(df[\"group\"]); dy = list(df[\"score\"])\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, palette = pal, bw = sigma,\n",
    "                 width_viol = .6, ax = ax, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P9\\n Rainclouds with List/Array Inputs\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP09.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For some data, you may want to flip the orientation of the raincloud to a 'petit prince' plot. You can do this with the 'orient' flag in the pt.RainCloud Function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Changing orientation\n",
    "dx=\"group\"; dy=\"score\"; ort=\"v\"; pal = \"Set2\"; sigma = .2\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .5, ax = ax, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P10\\n Flipping your Rainclouds\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP10.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also change the smoothing kernel used to generate the probability distribution function of the data. To do this, you  adjust the sigma parameter: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#changing cloud smoothness\n",
    "dx=\"group\"; dy=\"score\"; ort=\"h\"; pal = \"Set2\"; sigma = .05\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .6, ax = ax, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P11\\n Customizing Raincloud Smoothness\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP11.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, using the pointplot flag you can add a line connecting group mean values. This can be useful for more complex datasets, for example repeated measures or factorial data. Below we illustrate a few different approaches to plotting such data using rainclouds, by changing the hue, opacity, or dodge element of the individual plots:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#adding a red line connecting the groups' mean value (useful for longitudinal data)\n",
    "dx=\"group\"; dy=\"score\"; ort=\"h\"; pal = \"Set2\"; sigma = .2\n",
    "f, ax = plt.subplots(figsize=(7, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .6, ax = ax, orient = ort,\n",
    "               pointplot = True)\n",
    "\n",
    "plt.title(\"Figure P12\\n Adding Lineplots to Emphasize Factorial Effects\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP12.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another flexible option is to use Facet Grids to separate different groups or factor levels, illustrated below: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Rainclouds with FacetGrid\n",
    "g = sns.FacetGrid(df, col = \"gr2\", height = 6)\n",
    "g = g.map_dataframe(pt.RainCloud, x = \"group\", y = \"score\", data = df,\n",
    "                    orient = \"h\")\n",
    "\n",
    "g.fig.subplots_adjust(top=0.75)\n",
    "g.fig.suptitle(\"Figure P13\\n Using FacetGrid for More Complex Designs\",  fontsize=26)\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP13.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an alternative, it is possible to use the hue input for plotting different sub-groups directly over one another, facilitating their comparison:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Hue Input for Subgroups\n",
    "dx=\"group\"; dy=\"score\"; dhue = \"gr2\"; ort=\"h\"; pal = \"Set2\"; sigma = .2\n",
    "f, ax = plt.subplots(figsize=(12, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, hue = dhue, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .7, ax = ax, orient = ort)\n",
    "\n",
    "plt.title(\"Figure P14\\n Rainclouds with Subgroups\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP14.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To improve the readability of this plot, we adjust the alpha-level using the associated flag (0-1 alpha intensity): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setting alpha level\n",
    "f, ax = plt.subplots(figsize=(12, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, hue = dhue, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .7, ax = ax, orient = ort , alpha = .65)\n",
    "\n",
    "plt.title(\"Figure P15\\n Adjusting Raincloud Alpha Level\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP15.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rather than letting the two boxplots obscure one another, we can set the dodge flag to true, adding interpretability: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#The Doge Flag\n",
    "f, ax = plt.subplots(figsize=(12, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, hue = dhue, data = df, palette = pal, bw = sigma,\n",
    "                 width_viol = .7, ax = ax, orient = ort , alpha = .65, dodge = True)\n",
    "\n",
    "plt.title(\"Figure P16\\n The Boxplot Dodge Flag\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP16.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we may want to add a traditional line-plot to our graph to aid in the detection of factorial main effects and interactions. As an example, we've plotted the mean within each boxplot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#same, with dodging and line\n",
    "f, ax = plt.subplots(figsize=(12, 5))\n",
    "ax=pt.RainCloud(x = dx, y = dy, hue = dhue, data = df, palette = pal, bw = sigma, width_viol = .7,\n",
    "                ax = ax, orient = ort , alpha = .65, dodge = True, pointplot = True)\n",
    "\n",
    "plt.title(\"Figure P17\\n Dodged Boxplots with Lineplots\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP17.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the same plot, but now with the individual observations moved below the boxplots again using the 'move' parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#moving the rain under the boxplot\n",
    "f, ax = plt.subplots(figsize=(12, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, hue = dhue, data = df, palette = pal, bw = sigma, width_viol = .7,\n",
    "                ax = ax, orient = ort , alpha = .65, dodge = True, pointplot = True, move = .2)\n",
    "\n",
    "plt.title(\"Figure P18\\n Shifting the Rain with the Move Parameter\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP18.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As our last example, we'll consider a complex repeated measures design with two groups and three timepoints. The goal is to illustrate our complex interactions and main-effects, while perserving the transparent nature of the raincloud plot: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load in the repeated data\n",
    "df_rep = pd.read_csv (\"repeated_measures_data.csv\", sep= \",\")\n",
    "df_rep.columns = [\"score\",  \"timepoint\", \"group\"]\n",
    "df_rep.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the repeated measures data\n",
    "dx = \"group\"; dy = \"score\"; dhue = \"timepoint\"; ort = \"h\"; pal = \"Set2\"; sigma = .2\n",
    "f, ax = plt.subplots(figsize=(12, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, hue = dhue, data = df_rep, palette = pal, bw = sigma, width_viol = .7,\n",
    "                ax = ax, orient = ort , alpha = .65, dodge = True, pointplot = True, move = .2)\n",
    "\n",
    "plt.title(\"Figure P19\\n Repeated Measures Data - Example 1\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP19.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function is flexible enough that you can flip the ordering of the factors around simply by changing which variable informs the hue parameter: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Now with the group as hue\n",
    "dx = \"timepoint\"; dy = \"score\"; dhue = \"group\"\n",
    "f, ax = plt.subplots(figsize=(12, 5))\n",
    "\n",
    "ax=pt.RainCloud(x = dx, y = dy, hue = dhue, data = df_rep, palette = pal, bw = sigma, width_viol = .7,\n",
    "                ax = ax, orient = ort , alpha = .65, dodge = True, pointplot = True, move = .2)\n",
    "\n",
    "plt.title(\"Figure P20\\n  Repeated Measures Data - Example 2\")\n",
    "if savefigs:\n",
    "    plt.savefig('../figs/tutorial_python/figureP20.png', bbox_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it! Hopefully this tutorial has given you an idea of some of the different ways you can produce raincloud plots in Python."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}