src/notebooks/web-heatmap-comparison.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## About\n",
    "\n",
    "[Heatmap](https://python-graph-gallery.com/heatmap/) is a graphical representation of data where the individual values contained in a matrix are represented as colors. This is very useful to display the **concentration of values** between two dimensions, without looking at the individual values.\n",
    "\n",
    "This chart has been created by [Joseph Barbier](https://barbierjoseph.com). Thanks to him for accepting sharing its work here!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries\n",
    "\n",
    "First, we need to install the following libraries:\n",
    "\n",
    "- [matplotlib](https://python-graph-gallery.com/matplotlib/): for plot customization\n",
    "- [seaborn](https://python-graph-gallery.com/seaborn/): for creating the plot\n",
    "- [pandas](https://python-graph-gallery.com/pandas/): for data manipulation\n",
    "- [highlight_text](https://python-graph-gallery.com/advanced-custom-annotations-matplotlib/): for annotations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.patches import FancyArrowPatch\n",
    "import matplotlib.colors as mcolors\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "from highlight_text import ax_text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "The type of data needed when creating a [heatmap](https://python-graph-gallery.com/heatmap/) is a **matrix**. Each individual cell in the matrix **represents a value** that will be visualized using a color scale.\n",
    "\n",
    "In this post we need 2 datasets:\n",
    "\n",
    "- **original** heatmap\n",
    "- **normalized** heatmap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = 'https://raw.githubusercontent.com/holtzy/The-Python-Graph-Gallery/master/static/data/heatmap_data.csv'\n",
    "heatmap_data = pd.read_csv(path, index_col=0)\n",
    "\n",
    "path = 'https://raw.githubusercontent.com/holtzy/The-Python-Graph-Gallery/master/static/data/heatmap_data_norm.csv'\n",
    "heatmap_data_norm = pd.read_csv(path, index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple double heatmap\n",
    "\n",
    "We start by creating a **figure with 2 subplots**.\n",
    "\n",
    "The first subplot will contain the **original heatmap**, and the second subplot will contain the **normalized heatmap**.\n",
    "\n",
    "We use the `heatmap()` function from [seaborn](https://python-graph-gallery.com/seaborn/) to create the heatmaps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axs = plt.subplots(ncols=2, figsize=(16, 10))\n",
    "\n",
    "# iterate over the datasets\n",
    "for i, data in enumerate([heatmap_data, heatmap_data_norm]):\n",
    "    \n",
    "    # plot the heatmap\n",
    "    sns.heatmap(\n",
    "        data,\n",
    "        ax=axs[i],\n",
    "        cbar=False\n",
    "    )\n",
    "\n",
    "# save and show\n",
    "plt.savefig('../../static/graph/web-heatmap-comparison-1.png', bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![preview](https://github.com/holtzy/The-Python-Graph-Gallery/blob/master/static/graph/web-heatmap-comparison-1.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom color map and remove axis\n",
    "\n",
    "The next step is to customize the **color map** and **remove** the axis.\n",
    "\n",
    "- the **colormap** is created using the `LinearSegmentedColormap.from_list()` function, and then we pass it as an argument to the `cmap` parameter of the `heatmap()` function\n",
    "\n",
    "- the **axis** is removed using the `set_axis_off()` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a custom colormap\n",
    "cmap = mcolors.LinearSegmentedColormap.from_list(\"\", [\"#2a9d8f\", \"#e9c46a\", \"#e76f51\"])\n",
    "\n",
    "fig, axs = plt.subplots(ncols=2, figsize=(16, 10))\n",
    "\n",
    "# iterate over the datasets\n",
    "for i, data in enumerate([heatmap_data, heatmap_data_norm]):\n",
    "    \n",
    "    # plot the heatmap\n",
    "    sns.heatmap(\n",
    "        data,\n",
    "        ax=axs[i],\n",
    "        cmap=cmap,\n",
    "        cbar=False\n",
    "    )\n",
    "\n",
    "    # remove the axis\n",
    "    axs[i].set_axis_off()\n",
    "\n",
    "# save and show\n",
    "plt.savefig('../../static/graph/web-heatmap-comparison-2.png', bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![preview](https://github.com/holtzy/The-Python-Graph-Gallery/blob/master/static/graph/web-heatmap-comparison-2.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add region labels\n",
    "\n",
    "The next step is to add **region labels** to the heatmaps.\n",
    "\n",
    "For this, we **iterate over the index** of the dataset (where the region names are stored) and add the text to the plot using the `text()` function. We have to specify:\n",
    "\n",
    "- the **x** and **y** coordinates\n",
    "- the **text** to display\n",
    "- the **horizontal alignment**\n",
    "- the **vertical alignment**\n",
    "- the **font size**\n",
    "- the **font weight**\n",
    "\n",
    "*Warning*: make sure **both** of the heatmaps have the **same region names in the same order**, otherwise the labels will not match the regions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a custom colormap\n",
    "cmap = mcolors.LinearSegmentedColormap.from_list(\"\", [\"#2a9d8f\", \"#e9c46a\", \"#e76f51\"])\n",
    "\n",
    "fig, axs = plt.subplots(ncols=2, figsize=(16, 10))\n",
    "\n",
    "# iterate over the datasets\n",
    "for i, data in enumerate([heatmap_data, heatmap_data_norm]):\n",
    "    \n",
    "    # plot the heatmap\n",
    "    sns.heatmap(\n",
    "        data,\n",
    "        ax=axs[i],\n",
    "        cmap=cmap,\n",
    "        cbar=False\n",
    "    )\n",
    "\n",
    "    # remove the axis\n",
    "    axs[i].set_axis_off()\n",
    "\n",
    "    # add the region names\n",
    "    for j,region in enumerate(data.index):\n",
    "        axs[i].text(\n",
    "            0.4, # x axis position\n",
    "            j+0.5, # y axis position\n",
    "            f\"{region}\", # text\n",
    "            ha='left',\n",
    "            va='center',\n",
    "            fontsize=17,\n",
    "            fontweight='light',\n",
    "        )\n",
    "\n",
    "# save and show\n",
    "plt.savefig('../../static/graph/web-heatmap-comparison-3.png', bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![preview](https://github.com/holtzy/The-Python-Graph-Gallery/blob/master/static/graph/web-heatmap-comparison-3.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add legend\n",
    "\n",
    "Now let's add a **legend** to the heatmaps so that it's more understandable.\n",
    "\n",
    "We add both `Consommation d'énergie` and `Consommation d'énergie par habitant` to the legend using the `text()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a custom colormap\n",
    "cmap = mcolors.LinearSegmentedColormap.from_list(\"\", [\"#2a9d8f\", \"#e9c46a\", \"#e76f51\"])\n",
    "\n",
    "fig, axs = plt.subplots(ncols=2, figsize=(16, 10))\n",
    "\n",
    "# iterate over the datasets\n",
    "for i, data in enumerate([heatmap_data, heatmap_data_norm]):\n",
    "    \n",
    "    # plot the heatmap\n",
    "    sns.heatmap(\n",
    "        data,\n",
    "        ax=axs[i],\n",
    "        cmap=cmap,\n",
    "        cbar=False\n",
    "    )\n",
    "\n",
    "    # remove the axis\n",
    "    axs[i].set_axis_off()\n",
    "\n",
    "    # add the region names\n",
    "    for j,region in enumerate(data.index):\n",
    "        axs[i].text(\n",
    "            0.4, # x axis position\n",
    "            j+0.5, # y axis position\n",
    "            f\"{region}\", # text\n",
    "            ha='left',\n",
    "            va='center',\n",
    "            fontsize=17,\n",
    "            fontweight='light',\n",
    "        )\n",
    "\n",
    "    # description of each heatmap\n",
    "    if i==0: # first heatmap\n",
    "        text = \"Consommation d'énergie\"\n",
    "    else: # second heatmap\n",
    "        text = \"Consommation d'énergie par habitant\"\n",
    "    ax_text(\n",
    "        0, 12.4,\n",
    "        f\"<{text}>\",\n",
    "        ha='left', va='center',\n",
    "        fontsize=12, fontweight='light',\n",
    "        color='black',\n",
    "        highlight_textprops=[\n",
    "            {\"style\": \"italic\"}\n",
    "        ],\n",
    "        ax=axs[i]\n",
    "    )\n",
    "\n",
    "    # date for reference\n",
    "    ax_text(\n",
    "        0, -0.2,\n",
    "        \"<2011>\",\n",
    "        ha='left', va='center',\n",
    "        fontsize=12, fontweight='light',\n",
    "        color='black',\n",
    "        highlight_textprops=[\n",
    "            {\"style\": \"italic\"}\n",
    "        ],\n",
    "        ax=axs[i]\n",
    "    )\n",
    "    ax_text(\n",
    "        10, -0.2,\n",
    "        \"<2021>\",\n",
    "        ha='left', va='center',\n",
    "        fontsize=12, fontweight='light',\n",
    "        color='black',\n",
    "        highlight_textprops=[\n",
    "            {\"style\": \"italic\"}\n",
    "        ],\n",
    "        ax=axs[i]\n",
    "    )\n",
    "\n",
    "# save and show\n",
    "plt.savefig('../../static/graph/web-heatmap-comparison-4.png', bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![preview](https://github.com/holtzy/The-Python-Graph-Gallery/blob/master/static/graph/web-heatmap-comparison-4.png?raw=true)\n",
    "\n",
    "We have the main components of the plot! It only misses a title and a bunch of annotations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final chart with annotations\n",
    "\n",
    "All annotations relies on the `ax_text()` function from the `highlight_text` library. In this [post](https://python-graph-gallery.com/advanced-custom-annotations-matplotlib/) we explain **how to use it** in detail.\n",
    "\n",
    "The arrows are created using the `FancyArrowPatch()` function from `matplotlib.patches`. You can **learn more** about them in this [post](https://python-graph-gallery.com/how-to-create-and-custom-arrows-matplotlib/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a custom colormap\n",
    "cmap = mcolors.LinearSegmentedColormap.from_list(\"\", [\"#2a9d8f\", \"#e9c46a\", \"#e76f51\"])\n",
    "\n",
    "fig, axs = plt.subplots(ncols=2, figsize=(16, 10))\n",
    "\n",
    "# iterate over the datasets\n",
    "for i, data in enumerate([heatmap_data, heatmap_data_norm]):\n",
    "    \n",
    "    # plot the heatmap\n",
    "    sns.heatmap(\n",
    "        data,\n",
    "        ax=axs[i],\n",
    "        cmap=cmap,\n",
    "        cbar=False\n",
    "    )\n",
    "\n",
    "    # remove the axis\n",
    "    axs[i].set_axis_off()\n",
    "\n",
    "    # add the region names\n",
    "    for j,region in enumerate(data.index):\n",
    "        axs[i].text(\n",
    "            0.4, # x axis position\n",
    "            j+0.5, # y axis position\n",
    "            f\"{region}\", # text\n",
    "            ha='left',\n",
    "            va='center',\n",
    "            fontsize=17,\n",
    "            fontweight='light',\n",
    "        )\n",
    "\n",
    "    # description of each heatmap\n",
    "    if i==0: # first heatmap\n",
    "        text = \"Consommation d'énergie\"\n",
    "    else: # second heatmap\n",
    "        text = \"Consommation d'énergie par habitant\"\n",
    "    ax_text(\n",
    "        0, 12.4,\n",
    "        f\"<{text}>\",\n",
    "        ha='left', va='center',\n",
    "        fontsize=12, fontweight='light',\n",
    "        color='black',\n",
    "        highlight_textprops=[\n",
    "            {\"style\": \"italic\"}\n",
    "        ],\n",
    "        ax=axs[i]\n",
    "    )\n",
    "\n",
    "    # date for reference\n",
    "    ax_text(\n",
    "        0, -0.2,\n",
    "        \"<2011>\",\n",
    "        ha='left', va='center',\n",
    "        fontsize=12, fontweight='light',\n",
    "        color='black',\n",
    "        highlight_textprops=[\n",
    "            {\"style\": \"italic\"}\n",
    "        ],\n",
    "        ax=axs[i]\n",
    "    )\n",
    "    ax_text(\n",
    "        10, -0.2,\n",
    "        \"<2021>\",\n",
    "        ha='left', va='center',\n",
    "        fontsize=12, fontweight='light',\n",
    "        color='black',\n",
    "        highlight_textprops=[\n",
    "            {\"style\": \"italic\"}\n",
    "        ],\n",
    "        ax=axs[i]\n",
    "    )\n",
    "\n",
    "# title\n",
    "text = \"L'île de France pollue t'elle vraiment <plus> ?\"\n",
    "ax_text(\n",
    "    0, -4,\n",
    "    text,\n",
    "    ha='left', va='center',\n",
    "    fontsize=24, fontweight='bold',\n",
    "    color='black',\n",
    "    highlight_textprops=[\n",
    "        {\"color\": \"#e76f51\"}\n",
    "    ],\n",
    "    ax=axs[0]\n",
    ")\n",
    "\n",
    "# subtitle\n",
    "text = \"Consommation d'énergie et consommation d'énergie par habitant, de 2011 à 2021, par région (or Corse et territoires d'outre-mer)\"\n",
    "ax_text(\n",
    "    0, -3.5,\n",
    "    text,\n",
    "    ha='left', va='center',\n",
    "    fontsize=14, \n",
    "    color='grey',\n",
    "    ax=axs[0]\n",
    ")\n",
    "\n",
    "# credit source\n",
    "text = \"\"\"\n",
    "<Design>: Joseph Barbier   <Données>: data.gouv.fr\n",
    "Consommation de gaz et d'électricité en France, mesuré en MWh. Les données sont agrégées par région et par année.\n",
    "La population moyenne de 2008 et 2024 est utilisée pour normaliser la consommation d'énergie par habitant.\n",
    "Améliorer ce graphique: <https://github.com/JosephBARBIERDARNAL/Small-Projects/tree/main/conso-france>\n",
    "\"\"\"\n",
    "ax_text(\n",
    "    0, 13.9,\n",
    "    text,\n",
    "    ha='left', va='center',\n",
    "    fontsize=10, fontweight='light',\n",
    "    color='black',\n",
    "    highlight_textprops=[\n",
    "        {\"fontweight\": \"bold\"},\n",
    "        {\"fontweight\": \"bold\"},\n",
    "        {\"fontweight\": \"bold\"}\n",
    "    ],\n",
    "    ax=axs[0]\n",
    ")\n",
    "\n",
    "# left explanation\n",
    "text = \"Dans l'absolu, <l'Île-de-France>\\nest la région avec la plus\\n<haute consommation d'énergie>\"\n",
    "ax_text(\n",
    "    4, -1.7,\n",
    "    text,\n",
    "    ha='left', va='center',\n",
    "    fontsize=15, fontweight='light',\n",
    "    color='black',\n",
    "    highlight_textprops=[\n",
    "        {\"fontweight\": \"bold\"},\n",
    "        {\"color\": \"#e76f51\", \"fontweight\": \"bold\"}\n",
    "    ],\n",
    "    ax=axs[0]\n",
    ")\n",
    "\n",
    "# right explanation\n",
    "text = \"Relativement à son nombre d'habitants,\\n<l'Île-de-France> est une région avec\\n<consommation d'énergie par habitant>\\n<plutôt faible>\"\n",
    "ax_text(\n",
    "    12, -1.7,\n",
    "    text,\n",
    "    ha='left', va='center',\n",
    "    fontsize=15, fontweight='light',\n",
    "    color='black',\n",
    "    highlight_textprops=[\n",
    "        {\"fontweight\": \"bold\"},\n",
    "        {\"color\": \"#2a9d8f\", \"fontweight\": \"bold\"},\n",
    "        {\"color\": \"#2a9d8f\", \"fontweight\": \"bold\"}\n",
    "    ],\n",
    "    ax=axs[0]\n",
    ")\n",
    "\n",
    "# arrows\n",
    "def draw_arrow(tail_position, head_position, invert=False):\n",
    "    kw = dict(arrowstyle=\"Simple, tail_width=0.5, head_width=4, head_length=8\", color=\"k\")\n",
    "    if invert:\n",
    "        connectionstyle = \"arc3,rad=-.5\"\n",
    "    else:\n",
    "        connectionstyle = \"arc3,rad=.5\"\n",
    "    a = FancyArrowPatch(tail_position, head_position,\n",
    "                                connectionstyle=connectionstyle,\n",
    "                                transform=fig.transFigure,\n",
    "                                **kw)\n",
    "    fig.patches.append(a)\n",
    "\n",
    "draw_arrow((0.25, 1.01), (0.17, 0.91))\n",
    "draw_arrow((0.77, 1.01), (0.82, 0.91), invert=True)\n",
    "\n",
    "# save and show\n",
    "plt.savefig('../../static/graph/web-heatmap-comparison-5.png', bbox_inches='tight')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![preview](https://github.com/holtzy/The-Python-Graph-Gallery/blob/master/static/graph/web-heatmap-comparison-5.png?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Going further\n",
    "\n",
    "This post shows how to create a **double heatmap** with **annotations** using [matplotlib](https://python-graph-gallery.com/matplotlib/) and [seaborn](https://python-graph-gallery.com/seaborn/).\n",
    "\n",
    "For more examples of **advanced customization**, check out [how to create a beautiful legend](https://python-graph-gallery.com/web-ridgeline-by-text/) and this [really nice heatmap for time series](https://python-graph-gallery.com/heatmap-for-timeseries-matplotlib/)."
   ]
  }
 ],
 "metadata": {
  "chartType": "heatmap",
  "description": "<a href='https://python-graph-gallery.com/heatmap/'>Heatmaps</a> can be a great way to visualize evolution through time. This post shows how to create a <b>double heatmap</b>, one normal and one normalized, with annotations using <a href='https://python-graph-gallery.com/matplotlib/'>matplotlib</a> and <a href='https://python-graph-gallery.com/seaborn/'>seaborn</a>.<br><br>We will create a <b>step-by-step</b> guide to build the plot, starting with a very simple heatmap and ending with a <b>fully customized one</b>.",
  "family": "correlation",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "keywords": "heatmap, seaborn, matplotlib, python, double heatmap, annotations, highlight_text",
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  },
  "seoDescription": "How to create a double heatmap with annotations using matplotlib and seaborn",
  "slug": "web-heatmap-comparison",
  "title": "Double heatmap with custom annotations"
 },
 "nbformat": 4,
 "nbformat_minor": 4
}