GMD-manuscripts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Growth Maturity Decline for Manuscripts\n",
    "\n",
    "This Jupyter-lab notebook is about creating functions defining and showing the CHAOSS-GMD metrics. These functions will be incorporated into the Manuscripts project. We will also be testing visualizatons in this notebook.\n",
    "\n",
    "We'll be using the local elasticsearch instance and an already inserted index. This index has been created using the `p2o.py` script from the grimoirelab-toolset.\n",
    "\n",
    "We will start by importing the necessary libraries and initializing the necessary variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import elasticsearch\n",
    "\n",
    "from elasticsearch_dsl import Search\n",
    "from pprint import pprint\n",
    "\n",
    "import altair as alt\n",
    "import pandas as pd\n",
    "\n",
    "from datetime import date, timezone\n",
    "from dateutil import parser, relativedelta\n",
    "\n",
    "from manuscripts.manuscripts import esquery\n",
    "from manuscripts.manuscripts import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# address of the local elasticsearch instance\n",
    "es_url = \"http://localhost:9200\"\n",
    "\n",
    "# names of the git and github indices to be used\n",
    "git_index = \"perceval_git\"\n",
    "github_index = \"perceval_github\"\n",
    "\n",
    "# time interval in which the analysis has to be done\n",
    "end_date = parser.parse(date.today().strftime('%Y-%m-%d')).replace(tzinfo=timezone.utc)\n",
    "# start_date = end_date - relativedelta.relativedelta(months=18) \n",
    "start_date = date(2014, 1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The idea here is to divide the current reporting system into two parts: The metrics that are currently generated will have no change. \n",
    "Other than that, specific CHAOSS metrics can be generated using the `--chaoss` flag when calling the manuscripts command.\n",
    "\n",
    "We can start by adding a class method named `def get_chaoss_metrics(cls):` to each of the classes(git, github, gerrit, its) in the data sources. For example, in the github.py file, we can add the following section for CHAOSS metrics:\n",
    "```\n",
    "class GitHubIssues(its.ITS):\n",
    "    name = \"github_issues\"\n",
    "    \n",
    "    @classmethod\n",
    "    def get_chaoss_metrics(cls):\n",
    "        return {\n",
    "            \"issue_resolution\" : {\n",
    "                \"open\": [Open],\n",
    "                \"closed\": [Closed],\n",
    "                \"issue_resolution_efficiency\": [],\n",
    "                \"open_issue_age\": [],\n",
    "                \"first_response_to_issue_duration\": [],\n",
    "                \"closed_issue_resolution_Duration\": [],\n",
    "            }\n",
    "        }\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will go with the structure already defined in Manuscripts:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Issue Resolution\n",
    "\n",
    "Goal: Identify how effective the community is at addressing issues identified by community partcipants.\n",
    "\n",
    "Name | Question\n",
    "--- | ---\n",
    "[Open Issues](activity-metrics/open-issues.md) | What is the number of open issues?   DONE\n",
    "[Closed Issues](activity-metrics/closed-issues.md) | What is the number of closed issues?   DONE\n",
    "[Issue Resolution Efficiency](activity-metrics/issue-resolution-efficiency.md) | What is the number of closed issues/number of abandoned issues? \n",
    "[Open Issue Age](activity-metrics/open-issue-age.md) | What is the the age of open issues? \n",
    "[First Response to Issue Duration](activity-metrics/first-response-to-issue-duration.md) | What is the duration of time for a first response to an issue?\n",
    "[Closed Issue Resolution Duration](activity-metrics/closed-issue-resolution-duration.md) | What is the duration of time for issues to be resolved?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from manuscripts.manuscripts.metrics import github_issues, its, metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Open issues\n",
    "\n",
    "(Only create the classes that are not present and reuse code where possible)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this class goes into the its.py file in manuscripts/metrics folder\n",
    "# names of the classes will be changed according to the pattern used in that file\n",
    "\n",
    "class ITSOpen(its.ITSMetrics):\n",
    "    \"\"\" Tickets Open metric class for issue tracking systems \"\"\"\n",
    "    id = \"open\"\n",
    "    name = \"Open tickets\"\n",
    "    desc = \"Number of tickets currently open\"\n",
    "    FIELD_COUNT = \"id\"\n",
    "    FIELD_NAME = \"url\"\n",
    "    FIELD_DATE = \"created_at\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this class goes into github_issues.py file in manuscripts/metrics folder\n",
    "\n",
    "class Open(ITSOpen):\n",
    "    ds = github_issues.GitHubIssues\n",
    "    filters = {\"pull_request\": \"false\", \"state\": \"open\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Open class can be called inside report.py file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "open_issues = Open(es_url, github_index, start=start_date, end=end_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "22"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "open_issues.get_agg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Closed issues\n",
    "\n",
    "For this, we can used the already defined `Closed` class in github_issues.py file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "closed_issues = github_issues.Closed(es_url, github_index, start=start_date, end=end_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "113"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "closed_issues.get_agg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Issue Resolution Efficiency (What is the number of closed issues/number of abandoned issues?)\n",
    "\n",
    "How do we say that an issue has been abandoned?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Open issue age\n",
    "\n",
    "*This is not a mere metric dear mortal.*\n",
    "\n",
    "\n",
    "The documents are unclear if this only gives an average of the number of days since the open issues have been created or if individial days have to be looked at.\n",
    "I thought it will be a good idea to look at all the issues that have been created and where they stand now. We can visualise them and do other cool stuff.\n",
    "\n",
    "For this metric, we had to get data from 2 fields: `time_open_days` and `id_in_repo`. And both these values had to match each other, so simple aggregations wont work as the dicts are not ordered."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from manuscripts.manuscripts import esquery\n",
    "from manuscripts.manuscripts.metrics import github_issues, metrics\n",
    "\n",
    "class AgeOpenIssue(metrics.Metrics):\n",
    "    ds = github_issues.GitHubIssues\n",
    "    \n",
    "    id = \"age_open_issues\"\n",
    "    name = \"Age of open issues\"\n",
    "    desc = \"Number of days since the open issues were created\"\n",
    "    FIELD_COUNT = \"id_in_repo\"\n",
    "    FIELD_NAME = \"time_open_days\"\n",
    "    filters = {\"pull_request\": \"false\", \"state\": \"open\"}\n",
    "    FIELD_LIST = [\"time_open_days\", \"id_in_repo\"]\n",
    "\n",
    "    def get_agg(self):\n",
    "        agg = super(type(self), self).get_agg()\n",
    "        if agg is None:\n",
    "            agg = 0  # None is because NaN in ES. Let's convert to 0\n",
    "        return agg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "age_open_issues = AgeOpenIssue(es_url, github_index, start=start_date, end=end_date)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here the function `get_query_source` has been added to the `Metrics` class under `metrics.py`. This function gives returns the fields present in the FIELD_LIST varible and takes in a size parameter which tells elasticsearch how many entries to return."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "age_open_issues.get_query()\n",
    "q = age_open_issues.get_query_source(age_open_issues.get_agg())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "hits = age_open_issues.get_metrics_data(q)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'_id': 'fbe754f0a3cb220a57b1788852d8329873caa79d',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.9321957,\n",
       "  '_source': {'id_in_repo': '58', 'time_open_days': 606.51},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'f82ce08dde38d8924514df62aa2f73e2efb6d17c',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.9321957,\n",
       "  '_source': {'id_in_repo': '104', 'time_open_days': 496.61},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'a910a4e60fb632492c59414d71d3350cf0257c51',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.9321957,\n",
       "  '_source': {'id_in_repo': '319', 'time_open_days': 88.53},\n",
       "  '_type': 'items'},\n",
       " {'_id': '98b606ab48ab43a97c3508ea06082dfc8e0d6bd6',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.705124,\n",
       "  '_source': {'id_in_repo': '91', 'time_open_days': 546.59},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'c312200c337184e9bcfe14721ae9c9b3c81377d9',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.705124,\n",
       "  '_source': {'id_in_repo': '139', 'time_open_days': 363.42},\n",
       "  '_type': 'items'},\n",
       " {'_id': '2efa2af824ff33fe6973b1fc6c6d0b7a547438b5',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.705124,\n",
       "  '_source': {'id_in_repo': '217', 'time_open_days': 161.77},\n",
       "  '_type': 'items'},\n",
       " {'_id': '18f886632fa03f011b2136346f979269813f3aad',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.705124,\n",
       "  '_source': {'id_in_repo': '331', 'time_open_days': 76.96},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'd66c9891afbb5de3185805579e3ae16d958f9ae1',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3787837,\n",
       "  '_source': {'id_in_repo': '42', 'time_open_days': 699.53},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'c52fe7593316505dfb1c28d379889148bc3b6dc4',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3787837,\n",
       "  '_source': {'id_in_repo': '16', 'time_open_days': 803.36},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'f9cd314dc2838ae5faa8c6625779b47316099e80',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3787837,\n",
       "  '_source': {'id_in_repo': '229', 'time_open_days': 159.48},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'eced788b7cc2735a5b1adc80c50423357eaa19d8',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3787837,\n",
       "  '_source': {'id_in_repo': '59', 'time_open_days': 601.6},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'be9b217547e24c72bcd873ede98bbaab737c3934',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3787837,\n",
       "  '_source': {'id_in_repo': '367', 'time_open_days': 38.9},\n",
       "  '_type': 'items'},\n",
       " {'_id': '16a3b3891d400213588a3abea4233d2513895f95',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3492107,\n",
       "  '_source': {'id_in_repo': '18', 'time_open_days': 784.28},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'def7aa91516c87cb8bd96096aec84060ea0ec76a',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3492107,\n",
       "  '_source': {'id_in_repo': '20', 'time_open_days': 782.5},\n",
       "  '_type': 'items'},\n",
       " {'_id': '51b7f798df66c812f17a033155def4b7ad6fb11a',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3492107,\n",
       "  '_source': {'id_in_repo': '140', 'time_open_days': 361.7},\n",
       "  '_type': 'items'},\n",
       " {'_id': 'b737e0e8b0513af51bbef5794b69cd589edb9d31',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3492107,\n",
       "  '_source': {'id_in_repo': '234', 'time_open_days': 158.82},\n",
       "  '_type': 'items'},\n",
       " {'_id': '2fd7974b92e8e2a900a31d3c074247f03fcaac0b',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.3492107,\n",
       "  '_source': {'id_in_repo': '119', 'time_open_days': 440.31},\n",
       "  '_type': 'items'},\n",
       " {'_id': '22cd78994f2aa908f92013a83bb9aa360b44daf6',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.262173,\n",
       "  '_source': {'id_in_repo': '19', 'time_open_days': 783.85},\n",
       "  '_type': 'items'},\n",
       " {'_id': '390db8e46184564138c2c6dbcd2b54f6fb7564cf',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.262173,\n",
       "  '_source': {'id_in_repo': '28', 'time_open_days': 776.73},\n",
       "  '_type': 'items'},\n",
       " {'_id': '8567572c7b4cbc6ad188901c1cb13db766ad8cd8',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.262173,\n",
       "  '_source': {'id_in_repo': '74', 'time_open_days': 575.46},\n",
       "  '_type': 'items'},\n",
       " {'_id': '5a33627555c523b26e8ab6e0ceeceb4941303049',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.262173,\n",
       "  '_source': {'id_in_repo': '88', 'time_open_days': 557.79},\n",
       "  '_type': 'items'},\n",
       " {'_id': '1b1de72573fef54ef727c5e030ee070f6730da7c',\n",
       "  '_index': 'perceval_github',\n",
       "  '_score': 4.262173,\n",
       "  '_source': {'id_in_repo': '324', 'time_open_days': 83.84},\n",
       "  '_type': 'items'}]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hits['hits']['hits']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "open_issues_age = pd.DataFrame(i['_source'] for i in hits['hits']['hits']).sort_values('time_open_days')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   id_in_repo  time_open_days\n",
      "11        367           38.90\n",
      "6         331           76.96\n",
      "21        324           83.84\n",
      "2         319           88.53\n",
      "15        234          158.82\n",
      "9         229          159.48\n",
      "5         217          161.77\n",
      "14        140          361.70\n",
      "4         139          363.42\n",
      "16        119          440.31\n",
      "1         104          496.61\n",
      "3          91          546.59\n",
      "20         88          557.79\n",
      "19         74          575.46\n",
      "10         59          601.60\n",
      "0          58          606.51\n",
      "7          42          699.53\n",
      "18         28          776.73\n",
      "13         20          782.50\n",
      "17         19          783.85\n",
      "12         18          784.28\n",
      "8          16          803.36\n"
     ]
    }
   ],
   "source": [
    "print(open_issues_age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### First Response to Issue Duration\n",
    "\n",
    "\n",
    "### Closed Issue Resolution Duration\n",
    "\n",
    "For these metrics, the definitions are unclear and we might have to add code into Perceval."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code Development\n",
    "\n",
    "Goal: Identify how effective the community is at merging new code into the codebase.\n",
    "\n",
    "Name | Question\n",
    "--- | ---\n",
    "[Code Commits](activity-metrics/code-commits.md) | What is the number of code commits? \n",
    "[Lines of Code Changed](activity-metrics/lines-of-code-changed.md) | What is the number of lines of code changed?\n",
    "[Code Reviews](activity-metrics/code-reviews.md) | What is the number of code reviews?\n",
    "[Code Merge Duration](activity-metrics/code-merge-duration.md) | What is the duration of time between code merge request and code commit?\n",
    "[Code Review Efficiency](activity-metrics/code-review-efficiency.md) | What is the number of merged code changes/number of abandoned code change requests?\n",
    "[Maintainer Response to Merge Request Duration](activity-metrics/maintainer-response-to-merge-request-duration.md) | What is the duration of time for a maintainer to make a first response to a code merge request?\n",
    "[Code Review Iteration](activity-metrics/code-review-iteration.md) | What is the number of iterations that occur before a merge request is accepted or declined? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Code Commits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from manuscripts.manuscripts.metrics import git\n",
    "num_commits = git.Commits(es_url, git_index, start=start_date, end=end_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1176"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "num_commits.get_agg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lines of code changed\n",
    "\n",
    "This metric will require for a new aggregation `sum` to be added into ElasticQuery class. The `sum` aggregation is very similar to `avg` aggregation. We can just reuse the same code and rename the function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LinesChanged(git.GitMetrics):\n",
    "    \"\"\" Lines changed metric class for source code management systems \"\"\"\n",
    "\n",
    "    id = \"lineschanged\"\n",
    "    name = \"LinesChanged\"\n",
    "    desc = \"Number of lines changed\"\n",
    "    AGG_TYPE = \"sum\"\n",
    "    FIELD_COUNT = \"lines_changed\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "lines_changed = LinesChanged(es_url, git_index, start=start_date, end=end_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "177142.0"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lines_changed.get_agg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apart from LinesChanged, we can add LinesAdded and LinesRemoved to create a bargraph showing the distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "132512.0"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class LinesAdded(git.GitMetrics):\n",
    "    \"\"\" Lines added metric class for source code management systems \"\"\"\n",
    "\n",
    "    id = \"linesadded\"\n",
    "    name = \"LinesAdded\"\n",
    "    desc = \"Number of lines added\"\n",
    "    AGG_TYPE = \"sum\"\n",
    "    FIELD_COUNT = \"lines_added\"\n",
    "    \n",
    "lines_added = LinesAdded(es_url, git_index, start=start_date, end=end_date)\n",
    "\n",
    "lines_added.get_agg()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "44630.0"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class LinesRemoved(git.GitMetrics):\n",
    "    \"\"\" Lines removed metric class for source code management systems \"\"\"\n",
    "\n",
    "    id = \"linesremoved\"\n",
    "    name = \"LinesRemoved\"\n",
    "    desc = \"Number of lines removed\"\n",
    "    AGG_TYPE = \"sum\"\n",
    "    FIELD_COUNT = \"lines_removed\"\n",
    "    \n",
    "lines_removed = LinesRemoved(es_url, git_index, start=start_date, end=end_date)\n",
    "\n",
    "lines_removed.get_agg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Community Growth\n",
    "\n",
    "Goal: Identify the size of the project community and whether it's growing, shrinking, or staying the same.\n",
    "\n",
    "Name | Quesiton\n",
    "--- | ---\n",
    "[Contributors](activity-metrics/contributors.md) | What is the number of contributors?\n",
    "[New Contributors](activity-metrics/new-contributors.md) | What is the number of new contributors?\n",
    "[Contributing Organizations](activity-metrics/contributing-organizations.md) | What is the number of contributing organizations? \n",
    "[New Contributing Organizations](activity-metrics/new-contributing-organizations.md) | What is the number of new contributing organizations?\n",
    "[Sub-Projects](activity-metrics/sub-projects.md) | What is the number of sub-projects?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}