01.13 Statistical Analysis with Pingouin.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Statistical Analysis with Pingouin<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Exercise-1\" data-toc-modified-id=\"Exercise-1-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Exercise 1</a></span></li><li><span><a href=\"#Outlook\" data-toc-modified-id=\"Outlook-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>Outlook</a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[**_Pingouin_**](https://pingouin-stats.org) is a compact package that provides the most important test tools for a significance study. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![img/pingouin.png](img/pingouin.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Info:</b> It is worth visiting the <i>Pingouin</i> website (follow link above). The website provides a very good overview of available significance tests and also a decision tree that helps selecting the correct test for the respective data set.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Screenshots from [pingouin-stats.org/guidelines.html](https://pingouin-stats.org/guidelines.html):\n",
    "![img/pingouin.png](img/pingouin_1.png)\n",
    "![img/pingouin.png](img/pingouin_2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's recap our last example from the _Pandas_ chapter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example from the Pandas chapter (just with the Violin plot):\n",
    "import pandas as pd\n",
    "import os\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pingouin as pg\n",
    "\n",
    "# Define file paths:\n",
    "file_path = \"Data/Pandas_1/\"\n",
    "file_name_1 = \"Group_A_data.xls\"\n",
    "file_name_2 = \"Group_B_data.xls\"\n",
    "\n",
    "file_1 = os.path.join(file_path, file_name_1)\n",
    "file_2 = os.path.join(file_path, file_name_2)\n",
    "\n",
    "# Read the Excel files with Pandas into a Pandas Dataframe:\n",
    "Group_A_df = pd.read_excel(file_1, index_col=0)\n",
    "Group_B_df = pd.read_excel(file_2, index_col=0)\n",
    "\n",
    "# Broadcast the DataFrame data into the appropriate variables:\n",
    "Group_A = Group_A_df[\"Data\"].values\n",
    "Group_B = Group_B_df[\"Data\"].values\n",
    "\n",
    "# VIOLIN-PLOTS:\n",
    "fig=plt.figure(3, figsize=(5,6))\n",
    "fig.clf()\n",
    "\n",
    "plt.violinplot([Group_A, Group_B], showmedians=True)\n",
    "\n",
    "plt.xticks([1,2], labels=[\"A\", \"B\"])\n",
    "plt.xlabel(\"Groups\")\n",
    "plt.ylabel(\"measurements\")\n",
    "plt.title(\"Violin plot\")\n",
    "plt.tight_layout\n",
    "# plt.ylim(-40, 40)\n",
    "#plt.show()\n",
    "fig.savefig(\"violinplot with data.pdf\", dpi=120)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'and change file_path to:'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\"\"Google Colab Users:\n",
    "If you are running the notebook in Google Colab , you need to enable\n",
    "the access to your files in your Google Drive. First, upload the \n",
    "provided excel files to your Google Drive; put them, e.g., into a folder\n",
    "called \"Data/Pandas_1\" in the root-directory of your Google Drive. Then, \n",
    "execute the following commands.\n",
    "\"\"\"\n",
    "#from google.colab import drive\n",
    "#drive.mount('/content/drive')\n",
    "\n",
    "\"\"\"and change file_path to:\"\"\"\n",
    "#file_path = \"/content/drive/My Drive/Data/Pandas_1/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now want to know, whether the difference between the two groups is significant or not. Let's assume, that our data is normally distributed and the two samples are independent. The corresponding test would be an unpaired, two-sample student's t-test. The corresponding _Pingouin_ command is the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "test_result = pg.ttest(Group_A, Group_B, paired=False)\n",
    "#print(test_result)\n",
    "display(test_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the output of this _Pingouin_ command is not just a single value, e.g., the _p_-value, but a table. To be more correct, this table actually is a _Pandas_ DataFrame. Besides the _p_-value (`p-val`), the DataFrame contains other useful statistical properties:\n",
    "- `T` : _T_-value\n",
    "- `p-val` : _p_-value\n",
    "- `dof` : degrees of freedom\n",
    "- `cohen-d` : Cohen's _d_ effect size\n",
    "- `CI95%` : 95% confidence intervals of the difference in means\n",
    "- `power` : achieved power of the test ( = 1 - type II error)\n",
    "- `BF10` : Bayes Factor of the alternative hypothesis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, that for DataFrames with many keys, by default the `print` command doesn't print out all of them. To get a list of all available keys, use the command `DataFrame.keys()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# applied to our test_result-DataFrame:\n",
    "print(test_result.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To access the data of a specific key, just use the same syntax as for dictionaries and add \"`.values`\" in order to \"extract\" the value from the DataFrame structure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"the p-value of our test is:\", \n",
    "      f\"{test_result['p-val'].values}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Just recently explored: There is the display-command,\n",
    "# that always prints out Pandas tables/DataFrames in \n",
    "# formatted shape:\n",
    "display(test_result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" You can install pingouin also from within Jupyter. \n",
    "    Uncomment and run the following cell ones (and then \n",
    "    comment it again):\n",
    "\"\"\"\n",
    "#import sys\n",
    "#!{sys.executable} -m pip install pingouin"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1\n",
    "\n",
    "Copy the solution of the _Pandas_ example above into a new script. Extend your script by the following functionalities:\n",
    "    \n",
    "1. Apply the significance test to the `Group_A` and `Group_B` data.\n",
    "2. Write an _if_-statement that checks, whether the _p_-value of the significance test is lower or greater than 0.05. If the _p_-value is lower, then print out \"_there is a significant difference_\" together with the according _p_-value, otherwise print out \"_no significant difference_\" together with the according _p_-value.\n",
    "3. Plug-in the significance check-up into a function, e.g. named `unpaired_two_sample_ttest(...)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "code_folding": []
   },
   "outputs": [],
   "source": [
    "# Your solution 1.1 and 1.2 here:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary>Toggle solution</summary>\n",
    "\n",
    "```python\n",
    "# Solution 1.1 and 1.2:\n",
    "test_result = pg.ttest(Group_A, Group_B, paired=False)\n",
    "p_value = test_result[\"p-val\"].values\n",
    "\n",
    "if p_value>0.05:\n",
    "    print(f\"no significant difference (p-value: {p_value})\")\n",
    "else:\n",
    "    print(f\"here is a significant difference (p-value: {p_value})\")\n",
    "``` \n",
    "<script src=\"https://gist.github.com/username/a39a422ebdff6e732753b90573100b16.js\"></script>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your solution 1.3 here:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary>Toggle solution</summary>\n",
    "\n",
    "```python\n",
    "# Solution 1.3:\n",
    "import pandas as pd\n",
    "import os\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pingouin as pg\n",
    "\n",
    "# Define file paths:\n",
    "file_path = \"Data/Pandas_1/\"\n",
    "file_name_1 = \"Group_A_data.xls\"\n",
    "file_name_2 = \"Group_B_data.xls\"\n",
    "file_name_3 = \"Group_B2_data.xls\"\n",
    "\n",
    "file_1 = os.path.join(file_path, file_name_1)\n",
    "file_2 = os.path.join(file_path, file_name_2)\n",
    "file_3 = os.path.join(file_path, file_name_3)\n",
    "\n",
    "# Read the Excel files with Pandas into a Pandas Dataframe:\n",
    "Group_A_df = pd.read_excel(file_1, index_col=0)\n",
    "Group_B_df = pd.read_excel(file_2, index_col=0)\n",
    "Group_B2_df= pd.read_excel(file_3, index_col=0)\n",
    "\n",
    "# Broadcast the DataFrame data into the approproate variables:\n",
    "Group_A = Group_A_df[\"Data\"].values\n",
    "Group_B = Group_B_df[\"Data\"].values\n",
    "Group_B2= Group_B2_df[\"Data\"].values\n",
    "\n",
    "\n",
    "# DEFINE PLOT-FUNCTIONS:\n",
    "def plot_violin(A, B, fignum, figaspect, the_title):\n",
    "    fig = plt.figure(fignum, figsize=figaspect)\n",
    "    fig.clf()\n",
    "\n",
    "    x_ticks_A = np.ones(len(A))\n",
    "    x_ticks_B = np.ones(len(B))\n",
    "\n",
    "    plt.violinplot([A, B], showmedians=True)\n",
    "    plt.xticks([1,2], labels=[\"A\", \"B\"])\n",
    "    plt.xlabel(\"Groups\")\n",
    "    plt.ylabel(\"measurements\")\n",
    "    plt.title(the_title)\n",
    "    plt.tight_layout\n",
    "    # plt.ylim(-40, 40)\n",
    "    plt.show()\n",
    "    \n",
    "    \n",
    "# DEFINE SIGNFICANCE-TEST-FUNCTIONS:\n",
    "def unpaired_two_sample_ttest(A, B):\n",
    "    test_result = pg.ttest(A, B, paired=False)\n",
    "    p_value = test_result[\"p-val\"].values\n",
    "    if p_value>0.05:\n",
    "        print(f\"no significant difference (p-value: {p_value})\")\n",
    "    else:\n",
    "        print(f\"significant difference (p-value: {p_value})\")\n",
    "\n",
    "\n",
    "# Our actual analysis:\n",
    "plot_violin(Group_A, Group_B, 7, (5,6), \"Group A and B data\")\n",
    "print(\"Group_A vs. Group_B:\")\n",
    "unpaired_two_sample_ttest(Group_A, Group_B)\n",
    "\n",
    "plot_violin(Group_A, Group_B2, 7, (5,6), \"Group A and B2 data\")\n",
    "print(\"Group_A vs. Group_B2:\")\n",
    "unpaired_two_sample_ttest(Group_A, Group_B2)\n",
    "``` \n",
    "<script src=\"https://gist.github.com/username/a39a422ebdff6e732753b90573100b16.js\"></script>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "norm_A= pg.normality(Group_A)\n",
    "display(norm_A)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "norm_A[\"normal\"].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can let Python decide, whether to apply a student's t-test for \n",
    "# normally distributed data or a Mann-Whitney U test for not-normally\n",
    "# distributed data:\n",
    "\n",
    "if norm_A[\"normal\"].values:\n",
    "    print(\"Group A is normally distributed\")\n",
    "    # ttest\n",
    "    test_result = pg.ttest(Group_A, Group_B, paired=False)\n",
    "    test_type=\"ttest\"\n",
    "else:\n",
    "    print(\"Group A is not normally distributed\")\n",
    "    # mwu test\n",
    "    test_result = pg.mwu(Group_A, Group_B)\n",
    "    test_type=\"mwu\" # can be further used e.g. for an \n",
    "                    # additional print-out\n",
    "test_result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outlook\n",
    "Now imagine, that you have several Excel files to be read and processed. In this case it's highly recommended to plug-in, e.g., the data import and the analysis part into **_for_-loops**, while the imported data have to be plugged into iterable **_NumPy_ Arrays**. Also think about that you have several data project folders and you don't want to look into each of them. In order prevent your routine from crashing, you could implement an **_if_-statement** that checks during the data import, whether files could be imported or not. And, whenever possible, plug-in complex and repetitive calculations and procedures into a **function definition** in order to keep your code readble and to save lines of unnecessary code."
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Statistical Analysis with Pingouin",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  },
  "vscode": {
   "interpreter": {
    "hash": "4dbe93ddc4bed5499d29720f38f7446a34b15f2f3201b881b9bf1c1a4baa393f"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}