04-rdataframe-basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# ROOT RDataFrame\n",
    "\n",
    "[RDataFrame documentation](https://root.cern/doc/master/classROOT_1_1RDataFrame.html)\n",
    "\n",
    "- RDF is ROOT's high-level analysis interface. \n",
    "\n",
    "- Users define their analysis as a sequence of operations to be performed on the data-frame object; \n",
    "\n",
    "    - the framework takes care of the management of the loop over entries as well as low-level details such as I/O and parallelisation.\n",
    "\n",
    "- RDataFrame provides methods to perform most common operations required by ROOT analyses: \n",
    "\n",
    "    - at the same time, users can just as easily specify custom code that will be executed in the event loop.\n",
    "<img src=\"images/rdf_1.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# HEP data analysis with RDataFrame\n",
    "RDataFrame allows reading and writing trees, aiming at making HEP analysis easy to write and fast to perform."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import ROOT\n",
    "\n",
    "treename = \"dataset\"\n",
    "filename = \"data/example_file.root\"\n",
    "df = ROOT.RDataFrame(treename, filename)\n",
    "\n",
    "print(f\"Columns in the dataset: {df.GetColumnNames()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now we can `Define` new quantities, `Filter` rows based on custom expressions and retrieve some data aggregations such as a `Count` and a `Mean`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def1 = df.Define(\"c\", \"a+b\")\n",
    "\n",
    "fil1 = def1.Filter(\"c < 0.5\")\n",
    "\n",
    "count = fil1.Count()\n",
    "mean = fil1.Mean(\"c\")\n",
    "display = fil1.Display([\"a\",\"b\",\"c\"])\n",
    "\n",
    "print(f\"Number of rows after filter: {count.GetValue()}\")\n",
    "print(f\"Mean of column c after filter: {mean.GetValue()}\")\n",
    "print(\"Dataset contents:\")\n",
    "display.Print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Think about data-flow\n",
    "RDataFrame is built with a modular and flexible workflow in mind, summarised as follows:\n",
    "\n",
    "* build a data-frame object by specifying your data-set\n",
    "* apply a series of transformations to your data\n",
    "  * filter (e.g. apply some cuts) or\n",
    "  * define a new column (e.g. the result of an expensive computation on columns)\n",
    "* apply actions to the transformed data to produce results (e.g. fill a histogram)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Important Note!\n",
    "Make sure to **book all transformations and actions before** you access the contents of any of the results: this lets RDataFrame accumulate work and then produce all results at the same time, upon first access to any of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "df1 = df.Filter(\"a >= 0.2\")\n",
    "\n",
    "# Book in advance all Mean operations on the dataset columns\n",
    "cols = df.GetColumnNames()\n",
    "mean_ops = [df1.Mean(col) for col in cols]\n",
    "\n",
    "# Ask the result of one mean operation.\n",
    "# RDataFrame will process the whole computation graph\n",
    "print(f\"Number of RDataFrame runs so far: {df.GetNRuns()}\")\n",
    "print(f\"First mean result is: {mean_ops[0].GetValue()}\")\n",
    "print(f\"Number of RDataFrame runs so far: {df.GetNRuns()}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "# Print all results, the event loop won't be run another time\n",
    "for col, mean_op in zip(cols, mean_ops):\n",
    "    print(f\"Mean value of {col}: {mean_op.GetValue()}\")\n",
    "print(f\"Number of RDataFrame runs so far: {df.GetNRuns()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Histograms with RDataFrame\n",
    "RDataFrame helps you streamline the creation and filling of histogram objects from your data. \n",
    "\n",
    "For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "%jsroot on\n",
    "c = ROOT.TCanvas()\n",
    "h = df.Histo1D(\"vec1\")\n",
    "h.Draw()\n",
    "c.Draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- `Histo1D` will create a one-dimensional histogram holding `double` values. \n",
    "\n",
    "- `Histo{2,3}D` do the same in higher dimensions. \n",
    "\n",
    "- These operations also accept a tuple with the same arguments that would be passed to the equivalent histogram object constructors. \n",
    "\n",
    "- For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "histo_name = \"histo_name\"\n",
    "histo_title = \"histo_title\"\n",
    "nbinsx = 100\n",
    "xlow = -10\n",
    "xup = 10\n",
    "\n",
    "# The traditional TH1D constructor\n",
    "# ROOT.TH1D(histo_name, histo_title, nbinsx, xlow, xup)\n",
    "\n",
    "# With RDataFrame\n",
    "c = ROOT.TCanvas()\n",
    "h = df.Histo1D((histo_name, histo_title, nbinsx, xlow, xup), \"vec1\")\n",
    "h.Draw()\n",
    "c.Draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Operation categories in RDataFrame\n",
    "There are 3 main types of operations you can perform on RDataFrames:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "%%html\n",
    "<style>\n",
    "  th { font-size: 30px }\n",
    "  td { font-size: 30px }\n",
    "</style>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Transformations**: manipulate the dataset, return a modified RDataFrame for further processing.\n",
    "\n",
    "| Transformation    | Description                                                |\n",
    "|-------------------|------------------------------------------------------------|\n",
    "| Alias()           | Introduce an alias for a particular column name.           |\n",
    "| Define()          | Creates  a new column in the dataset.                      |\n",
    "| Filter()          | Filter rows based on user-defined conditions.              |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Actions**: aggregate (parts of) the dataset into a result.\n",
    "\n",
    "| Action                        | Description                                                                          |\n",
    "|------------------------------------|--------------------------------------------------------------------------------------|\n",
    "| Count()                            | Return the number of events processed.                                               |\n",
    "| Display()                          | Provides a printable object representing the dataset contents.                       |\n",
    "| Graph()                            | Fills a TGraph  with the two columns provided.                                       |\n",
    "| Histo1D(), Histo2D(), Histo3D()    | Fill a one-, two-, three-dimensional histogram with the processed column values.     |\n",
    "| Max(), Min()                       | Return the maximum(minimum) of processed column values.                              |\n",
    "| Snapshot()        | Writes processed data-set to a new TTree.              |\n",
    "| ...                                | ...  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Queries**: these methods  query information about your dataset and the RDataFrame status.\n",
    "\n",
    "| Operation           | Description                                                                              |\n",
    "|---------------------|------------------------------------------------------------------------------------------|\n",
    "| GetColumnNames()    | Get the names of all the available columns of the dataset.                               |\n",
    "| GetColumnType()     | Return the type of a given column as a string.                                           |\n",
    "| SaveGraph()         | Export the computation graph of an RDataFrame in graphviz format for easy inspection.     |\n",
    "| ...                 | ...                                                                                      |"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}