notebooks/Production Coverage Demo with Tablesaw and BeakerX.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Context\n",
    "John Doe remarked in [#AP1432](http://www.example.com/justalink) that there may be too much code in our application that isn't used at all. Before migrating the application to the new platform, we have to analyze which parts of the system are still in use and which are not."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Idea\n",
    "To understand how much code isn't used, we recorded the executed code in production with the coverage tool [JaCoCo](http://www.jacoco.org/). The measurement took place between 21st Oct 2017 and 27st Oct 2017. The results were exported into a CSV file using the JaCoCo command line tool with the following command:\n",
    "\n",
    "```bash\n",
    "java -jar jacococli.jar report \"C:\\Temp\\jacoco.exec\" --classfiles \\\n",
    "C:\\dev\\repos\\buschmais-spring-petclinic\\target\\classes --csv jacoco.csv\n",
    "```\n",
    "\n",
    "The CSV file contains all lines of code that were passed through during the measurement's time span."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "method": "display_data"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b26f03a4-ee15-4afb-8dc1-8d57e367f0c9",
       "version_major": 2,
       "version_minor": 0
      },
      "method": "display_data"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%classpath add mvn\n",
    "tech.tablesaw tablesaw-beakerx 0.24.5\n",
    "tech.tablesaw tablesaw-jsplot 0.24.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%import tech.tablesaw.api.*\n",
    "%import tech.tablesaw.columns.*\n",
    "%import static tech.tablesaw.aggregate.AggregateFunctions.*\n",
    "\n",
    "tech.tablesaw.beakerx.TablesawDisplayer.register()\n",
    "\n",
    "OutputCell.HIDDEN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "wget https://raw.githubusercontent.com/feststelltaste/software-analytics/master/demos/dataset/jacoco.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a1d16579-c1ff-4006-b0a2-8013fd7a7fca",
       "version_major": 2,
       "version_minor": 0
      },
      "method": "display_data"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "coverage = Table.read().csv(\"jacoco.csv\")\n",
    "coverage.first(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We just take the only relevant data `LINES_COVERED` as well as the information about the packages in `PACKAGE` as well as the `CLASS` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1f2975fc-8b82-4d73-81d5-6e4461c93f03",
       "version_major": 2,
       "version_minor": 0
      },
      "method": "display_data"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "coverage = coverage.retainColumns(\"PACKAGE\", \"CLASS\", \"LINE_COVERED\")\n",
    "coverage.first(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analysis\n",
    "It was stated that whole packages wouldn't be needed anymore and that they could be safely removed. Therefore, we sum up the coverage data per class for each package and calculate the coverage ratio for each package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a5f38416-3d1c-45ad-882a-efb432e5d514",
       "version_major": 2,
       "version_minor": 0
      },
      "method": "display_data"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "per_package = coverage.summarize(\"LINE_COVERED\", sum).by(\"PACKAGE\");\n",
    "per_package"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualisation\n",
    "We plot the data for the coverage ratio to get a brief overview of the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bce404fa-d1af-4ff9-b281-de3f6cff1f54",
       "version_major": 2,
       "version_minor": 0
      },
      "method": "display_data"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def bars = new CategoryBars(\n",
    "    value: per_package.column('Sum [LINE_COVERED]').asList())\n",
    "new CategoryPlot(\n",
    "    categoryNames: per_package.column('PACKAGE').asList(),\n",
    "    orientation: PlotOrientationType.HORIZONTAL) << bars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion \n",
    "The JDBC package `org.springframework.samples.petclinic.repository.jdbc` isn't used at all and can be left out safely when migrating to the new platform."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Groovy",
   "language": "groovy",
   "name": "groovy"
  },
  "language_info": {
   "codemirror_mode": "groovy",
   "file_extension": ".groovy",
   "mimetype": "",
   "name": "Groovy",
   "nbconverter_exporter": "",
   "version": "2.4.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": false,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}