Notebook6_part3_webMiningAPIs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-6f2b19db531e74fd",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# Part 2: Mining the web: Web APIs\n",
    "\n",
    "We hope the preceding exercise was painful: even with tools to process HTML, it is rough downloading raw HTML and trying to extract information from it!\n",
    "\n",
    "> Can you think of any other reasons why scraping websites for data in this way is not a good idea?\n",
    "\n",
    "Luckily, many websites provide an application programming interface (API) for querying their data or otherwise accessing their services from your programs. For instance, Twitter provides a web API for gathering tweets, Flickr provides one for gathering image data, and Github for accessing information about repository histories.\n",
    "\n",
    "These kinds of web APIs are much easier to use than, for instance, the preceding technique which scrapes raw web pages and then has to parse the resulting HTML. Moreover, there are more scalable in the sense that the web servers can transmit structured data in a less verbose form than raw HTML.\n",
    "\n",
    "As a starting example, here is some code to look at the activity on Github related to the public version of our course's materials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e969cac2e271f5ec",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==> .headers: {'Date': 'Tue, 24 Oct 2017 03:48:26 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Server': 'GitHub.com', 'Status': '200 OK', 'X-RateLimit-Limit': '60', 'X-RateLimit-Remaining': '58', 'X-RateLimit-Reset': '1508820012', 'Cache-Control': 'public, max-age=60, s-maxage=60', 'Vary': 'Accept, Accept-Encoding', 'ETag': 'W/\"d53c46c2dfc971561af4fe0268891c8f\"', 'Last-Modified': 'Fri, 20 Oct 2017 04:47:45 GMT', 'X-Poll-Interval': '60', 'X-GitHub-Media-Type': 'github.v3; format=json', 'Access-Control-Expose-Headers': 'ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval', 'Access-Control-Allow-Origin': '*', 'Expect-CT': 'max-age=2592000, report-uri=\"https://api.github.com/_private/browser/errors\"', 'Content-Security-Policy': \"default-src 'none'\", 'Strict-Transport-Security': 'max-age=31536000; includeSubdomains; preload', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'deny', 'X-XSS-Protection': '1; mode=block', 'X-Runtime-rack': '0.055581', 'Content-Encoding': 'gzip', 'X-GitHub-Request-Id': 'CF20:2D5B3:B421B41:1033F2C1:59EEB80A'} \n",
      "\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "response = requests.get ('https://api.github.com/repos/cse6040/labs-fa17/events')\n",
    "\n",
    "print (\"==> .headers:\", response.headers, \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-2a08d680885303e5",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Note the `Content-Type` of the response:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-89ebf4242e6c1940",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "application/json; charset=utf-8\n"
     ]
    }
   ],
   "source": [
    "print (response.headers['Content-Type'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-752159137c70eb17",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "The response is in JSON format, which is an open format for exchanging semi-structured data. (JSON stands for **J**ava**S**cript **O**bject **N**otation.) JSON is designed to be human-readable and machine-readable, and maps especially well in Python to nested dictionaries. Let's take a look.\n",
    "\n",
    "> See also [this tutorial](http://www.w3schools.com/json/) for a JSON primer. JSON is among _the_ universal formats for sharing data on the web; see, for instance, https://www.sitepoint.com/10-example-json-files/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4bcf26ae80bab53b",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'list'>\n",
      "[\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-10-20T04:47:45Z\",\n",
      "    \"id\": \"6741980967\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"3aa6cf03bb6b44fc4a6ccd6aacef953365744042\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Lab 9\",\n",
      "          \"sha\": \"32b31bd35f906ef6d2121e961ce466b652d4d671\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/32b31bd35f906ef6d2121e961ce466b652d4d671\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"32b31bd35f906ef6d2121e961ce466b652d4d671\",\n",
      "      \"push_id\": 2064517171,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-10-18T16:44:57Z\",\n",
      "    \"id\": \"6734071518\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"48a01907828d16bd4574aa7c0a3aa49d52f71f1f\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added another sample solution\",\n",
      "          \"sha\": \"3aa6cf03bb6b44fc4a6ccd6aacef953365744042\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/3aa6cf03bb6b44fc4a6ccd6aacef953365744042\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"3aa6cf03bb6b44fc4a6ccd6aacef953365744042\",\n",
      "      \"push_id\": 2060438036,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-10-18T04:22:44Z\",\n",
      "    \"id\": \"6730929125\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"ff4669c6f64ef47d831ce5a90e5041d6f0c71c76\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Fixed CR/LF issue\",\n",
      "          \"sha\": \"48a01907828d16bd4574aa7c0a3aa49d52f71f1f\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/48a01907828d16bd4574aa7c0a3aa49d52f71f1f\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"48a01907828d16bd4574aa7c0a3aa49d52f71f1f\",\n",
      "      \"push_id\": 2058822826,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-10-18T04:04:25Z\",\n",
      "    \"id\": \"6730890086\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"d6af4899e299cf10799e2e6d1c3bf4e97c98c7d3\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added some solution data for Lab 9\",\n",
      "          \"sha\": \"ff4669c6f64ef47d831ce5a90e5041d6f0c71c76\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/ff4669c6f64ef47d831ce5a90e5041d6f0c71c76\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"ff4669c6f64ef47d831ce5a90e5041d6f0c71c76\",\n",
      "      \"push_id\": 2058802351,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-10-05T21:33:08Z\",\n",
      "    \"id\": \"6680506591\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"f0599fc53e36a98c42bdde19be5b190520cef9d0\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Kaggle Student Alcohol Consumption data [source: https://www.kaggle.com/uciml/student-alcohol-consumption/data]\",\n",
      "          \"sha\": \"d6af4899e299cf10799e2e6d1c3bf4e97c98c7d3\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/d6af4899e299cf10799e2e6d1c3bf4e97c98c7d3\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"d6af4899e299cf10799e2e6d1c3bf4e97c98c7d3\",\n",
      "      \"push_id\": 2032158267,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-23T15:54:04Z\",\n",
      "    \"id\": \"6628275422\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"c0b4d64ecb7957c09f655758a9fecc4c6f74850b\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Patch\",\n",
      "          \"sha\": \"f0599fc53e36a98c42bdde19be5b190520cef9d0\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/f0599fc53e36a98c42bdde19be5b190520cef9d0\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"f0599fc53e36a98c42bdde19be5b190520cef9d0\",\n",
      "      \"push_id\": 2004808906,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-23T05:51:58Z\",\n",
      "    \"id\": \"6627616393\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"7d90252951891a56090b551aadb67825dd8422b2\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Released Lab 8 (viz)\",\n",
      "          \"sha\": \"d68d64519032c5b685f77a184ac97df8396e6d8b\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/d68d64519032c5b685f77a184ac97df8396e6d8b\"\n",
      "        },\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Released Lab 7 (tidy data + pandas)\",\n",
      "          \"sha\": \"c0b4d64ecb7957c09f655758a9fecc4c6f74850b\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/c0b4d64ecb7957c09f655758a9fecc4c6f74850b\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 2,\n",
      "      \"head\": \"c0b4d64ecb7957c09f655758a9fecc4c6f74850b\",\n",
      "      \"push_id\": 2004424462,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 2\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-22T19:57:27Z\",\n",
      "    \"id\": \"6626527233\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"3234b2f0ad49a1eb34489ea1defe7ba6d06bdfe9\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added datasets for Lab 8 (viz), copied from seaborn datasets\",\n",
      "          \"sha\": \"7d90252951891a56090b551aadb67825dd8422b2\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/7d90252951891a56090b551aadb67825dd8422b2\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"7d90252951891a56090b551aadb67825dd8422b2\",\n",
      "      \"push_id\": 2003835772,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-22T04:42:37Z\",\n",
      "    \"id\": \"6623084591\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"3bdcdd8ca97073ee64b2feb4a04fc3c5830c7850\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Patched file download code\",\n",
      "          \"sha\": \"3234b2f0ad49a1eb34489ea1defe7ba6d06bdfe9\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/3234b2f0ad49a1eb34489ea1defe7ba6d06bdfe9\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"3234b2f0ad49a1eb34489ea1defe7ba6d06bdfe9\",\n",
      "      \"push_id\": 2002056671,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-21T02:13:42Z\",\n",
      "    \"id\": \"6617521983\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"aa9996e89a2efb29247b0c98eb04a433fb2f632a\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added datasets for tidy data (Pandas) notebook TBD\",\n",
      "          \"sha\": \"3bdcdd8ca97073ee64b2feb4a04fc3c5830c7850\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/3bdcdd8ca97073ee64b2feb4a04fc3c5830c7850\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"3bdcdd8ca97073ee64b2feb4a04fc3c5830c7850\",\n",
      "      \"push_id\": 1999184617,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/19946052?\",\n",
      "      \"display_login\": \"dthai91\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 19946052,\n",
      "      \"login\": \"dthai91\",\n",
      "      \"url\": \"https://api.github.com/users/dthai91\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-20T05:43:34Z\",\n",
      "    \"id\": \"6612682349\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"forkee\": {\n",
      "        \"archive_url\": \"https://api.github.com/repos/dthai91/labs-fa17/{archive_format}{/ref}\",\n",
      "        \"assignees_url\": \"https://api.github.com/repos/dthai91/labs-fa17/assignees{/user}\",\n",
      "        \"blobs_url\": \"https://api.github.com/repos/dthai91/labs-fa17/git/blobs{/sha}\",\n",
      "        \"branches_url\": \"https://api.github.com/repos/dthai91/labs-fa17/branches{/branch}\",\n",
      "        \"clone_url\": \"https://github.com/dthai91/labs-fa17.git\",\n",
      "        \"collaborators_url\": \"https://api.github.com/repos/dthai91/labs-fa17/collaborators{/collaborator}\",\n",
      "        \"comments_url\": \"https://api.github.com/repos/dthai91/labs-fa17/comments{/number}\",\n",
      "        \"commits_url\": \"https://api.github.com/repos/dthai91/labs-fa17/commits{/sha}\",\n",
      "        \"compare_url\": \"https://api.github.com/repos/dthai91/labs-fa17/compare/{base}...{head}\",\n",
      "        \"contents_url\": \"https://api.github.com/repos/dthai91/labs-fa17/contents/{+path}\",\n",
      "        \"contributors_url\": \"https://api.github.com/repos/dthai91/labs-fa17/contributors\",\n",
      "        \"created_at\": \"2017-09-20T05:43:33Z\",\n",
      "        \"default_branch\": \"master\",\n",
      "        \"deployments_url\": \"https://api.github.com/repos/dthai91/labs-fa17/deployments\",\n",
      "        \"description\": \"Lab notebooks for the Fall 2017 offering of Georgia Tech's CSE 6040\",\n",
      "        \"downloads_url\": \"https://api.github.com/repos/dthai91/labs-fa17/downloads\",\n",
      "        \"events_url\": \"https://api.github.com/repos/dthai91/labs-fa17/events\",\n",
      "        \"fork\": true,\n",
      "        \"forks\": 0,\n",
      "        \"forks_count\": 0,\n",
      "        \"forks_url\": \"https://api.github.com/repos/dthai91/labs-fa17/forks\",\n",
      "        \"full_name\": \"dthai91/labs-fa17\",\n",
      "        \"git_commits_url\": \"https://api.github.com/repos/dthai91/labs-fa17/git/commits{/sha}\",\n",
      "        \"git_refs_url\": \"https://api.github.com/repos/dthai91/labs-fa17/git/refs{/sha}\",\n",
      "        \"git_tags_url\": \"https://api.github.com/repos/dthai91/labs-fa17/git/tags{/sha}\",\n",
      "        \"git_url\": \"git://github.com/dthai91/labs-fa17.git\",\n",
      "        \"has_downloads\": true,\n",
      "        \"has_issues\": false,\n",
      "        \"has_pages\": false,\n",
      "        \"has_projects\": true,\n",
      "        \"has_wiki\": true,\n",
      "        \"homepage\": null,\n",
      "        \"hooks_url\": \"https://api.github.com/repos/dthai91/labs-fa17/hooks\",\n",
      "        \"html_url\": \"https://github.com/dthai91/labs-fa17\",\n",
      "        \"id\": 104171017,\n",
      "        \"issue_comment_url\": \"https://api.github.com/repos/dthai91/labs-fa17/issues/comments{/number}\",\n",
      "        \"issue_events_url\": \"https://api.github.com/repos/dthai91/labs-fa17/issues/events{/number}\",\n",
      "        \"issues_url\": \"https://api.github.com/repos/dthai91/labs-fa17/issues{/number}\",\n",
      "        \"keys_url\": \"https://api.github.com/repos/dthai91/labs-fa17/keys{/key_id}\",\n",
      "        \"labels_url\": \"https://api.github.com/repos/dthai91/labs-fa17/labels{/name}\",\n",
      "        \"language\": null,\n",
      "        \"languages_url\": \"https://api.github.com/repos/dthai91/labs-fa17/languages\",\n",
      "        \"merges_url\": \"https://api.github.com/repos/dthai91/labs-fa17/merges\",\n",
      "        \"milestones_url\": \"https://api.github.com/repos/dthai91/labs-fa17/milestones{/number}\",\n",
      "        \"mirror_url\": null,\n",
      "        \"name\": \"labs-fa17\",\n",
      "        \"notifications_url\": \"https://api.github.com/repos/dthai91/labs-fa17/notifications{?since,all,participating}\",\n",
      "        \"open_issues\": 0,\n",
      "        \"open_issues_count\": 0,\n",
      "        \"owner\": {\n",
      "          \"avatar_url\": \"https://avatars0.githubusercontent.com/u/19946052?v=4\",\n",
      "          \"events_url\": \"https://api.github.com/users/dthai91/events{/privacy}\",\n",
      "          \"followers_url\": \"https://api.github.com/users/dthai91/followers\",\n",
      "          \"following_url\": \"https://api.github.com/users/dthai91/following{/other_user}\",\n",
      "          \"gists_url\": \"https://api.github.com/users/dthai91/gists{/gist_id}\",\n",
      "          \"gravatar_id\": \"\",\n",
      "          \"html_url\": \"https://github.com/dthai91\",\n",
      "          \"id\": 19946052,\n",
      "          \"login\": \"dthai91\",\n",
      "          \"organizations_url\": \"https://api.github.com/users/dthai91/orgs\",\n",
      "          \"received_events_url\": \"https://api.github.com/users/dthai91/received_events\",\n",
      "          \"repos_url\": \"https://api.github.com/users/dthai91/repos\",\n",
      "          \"site_admin\": false,\n",
      "          \"starred_url\": \"https://api.github.com/users/dthai91/starred{/owner}{/repo}\",\n",
      "          \"subscriptions_url\": \"https://api.github.com/users/dthai91/subscriptions\",\n",
      "          \"type\": \"User\",\n",
      "          \"url\": \"https://api.github.com/users/dthai91\"\n",
      "        },\n",
      "        \"private\": false,\n",
      "        \"public\": true,\n",
      "        \"pulls_url\": \"https://api.github.com/repos/dthai91/labs-fa17/pulls{/number}\",\n",
      "        \"pushed_at\": \"2017-09-20T00:57:08Z\",\n",
      "        \"releases_url\": \"https://api.github.com/repos/dthai91/labs-fa17/releases{/id}\",\n",
      "        \"size\": 3541,\n",
      "        \"ssh_url\": \"git@github.com:dthai91/labs-fa17.git\",\n",
      "        \"stargazers_count\": 0,\n",
      "        \"stargazers_url\": \"https://api.github.com/repos/dthai91/labs-fa17/stargazers\",\n",
      "        \"statuses_url\": \"https://api.github.com/repos/dthai91/labs-fa17/statuses/{sha}\",\n",
      "        \"subscribers_url\": \"https://api.github.com/repos/dthai91/labs-fa17/subscribers\",\n",
      "        \"subscription_url\": \"https://api.github.com/repos/dthai91/labs-fa17/subscription\",\n",
      "        \"svn_url\": \"https://github.com/dthai91/labs-fa17\",\n",
      "        \"tags_url\": \"https://api.github.com/repos/dthai91/labs-fa17/tags\",\n",
      "        \"teams_url\": \"https://api.github.com/repos/dthai91/labs-fa17/teams\",\n",
      "        \"trees_url\": \"https://api.github.com/repos/dthai91/labs-fa17/git/trees{/sha}\",\n",
      "        \"updated_at\": \"2017-09-14T22:08:27Z\",\n",
      "        \"url\": \"https://api.github.com/repos/dthai91/labs-fa17\",\n",
      "        \"watchers\": 0,\n",
      "        \"watchers_count\": 0\n",
      "      }\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"ForkEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-20T00:57:08Z\",\n",
      "    \"id\": \"6612021233\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"a89442b9d643b069834ba30639c83eb5f1b37fb8\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Released Lab 6\",\n",
      "          \"sha\": \"aa9996e89a2efb29247b0c98eb04a433fb2f632a\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/aa9996e89a2efb29247b0c98eb04a433fb2f632a\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"aa9996e89a2efb29247b0c98eb04a433fb2f632a\",\n",
      "      \"push_id\": 1996362932,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-20T00:46:41Z\",\n",
      "    \"id\": \"6611997215\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"c138fd4aa94c56debe3e640dfd4a1ba53afcdb88\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Uploaded images for Lab 6\",\n",
      "          \"sha\": \"a89442b9d643b069834ba30639c83eb5f1b37fb8\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/a89442b9d643b069834ba30639c83eb5f1b37fb8\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"a89442b9d643b069834ba30639c83eb5f1b37fb8\",\n",
      "      \"push_id\": 1996350452,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-19T22:48:55Z\",\n",
      "    \"id\": \"6611692513\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"572167f7d65edd7a7aa12a5dd17d703656678d14\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Yelp! ATL universities dataset\",\n",
      "          \"sha\": \"fa1e95d4c350506b772e56bd41a01479ee3951f2\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/fa1e95d4c350506b772e56bd41a01479ee3951f2\"\n",
      "        },\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Merge branch 'master' of github.com:cse6040/labs-fa17\",\n",
      "          \"sha\": \"c138fd4aa94c56debe3e640dfd4a1ba53afcdb88\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/c138fd4aa94c56debe3e640dfd4a1ba53afcdb88\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 2,\n",
      "      \"head\": \"c138fd4aa94c56debe3e640dfd4a1ba53afcdb88\",\n",
      "      \"push_id\": 1996190687,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 2\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-19T19:38:11Z\",\n",
      "    \"id\": \"6610937299\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"917fffcca44b55042095d5be7de61b65414bad76\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Yelp\\\\! ATL universities raw search results\",\n",
      "          \"sha\": \"572167f7d65edd7a7aa12a5dd17d703656678d14\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/572167f7d65edd7a7aa12a5dd17d703656678d14\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"572167f7d65edd7a7aa12a5dd17d703656678d14\",\n",
      "      \"push_id\": 1995800949,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-14T23:28:12Z\",\n",
      "    \"id\": \"6593207015\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"4365a884df66895ad18e5f41a280cc81797e7923\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Lab 5 (regular expressions)\",\n",
      "          \"sha\": \"917fffcca44b55042095d5be7de61b65414bad76\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/917fffcca44b55042095d5be7de61b65414bad76\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"917fffcca44b55042095d5be7de61b65414bad76\",\n",
      "      \"push_id\": 1986327053,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-14T22:08:26Z\",\n",
      "    \"id\": \"6592980070\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"89628fe83572dbf96bdaa38a5048a407f51a4416\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Lab 2 and Lab 5 datasets\",\n",
      "          \"sha\": \"4365a884df66895ad18e5f41a280cc81797e7923\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/4365a884df66895ad18e5f41a280cc81797e7923\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"4365a884df66895ad18e5f41a280cc81797e7923\",\n",
      "      \"push_id\": 1986206900,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-07T07:40:57Z\",\n",
      "    \"id\": \"6560029268\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"9d248cb45764a59f2475c97d4667e21c8c3a7ad5\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Lab 4\",\n",
      "          \"sha\": \"89628fe83572dbf96bdaa38a5048a407f51a4416\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/89628fe83572dbf96bdaa38a5048a407f51a4416\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"89628fe83572dbf96bdaa38a5048a407f51a4416\",\n",
      "      \"push_id\": 1969155278,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31221497?\",\n",
      "      \"display_login\": \"Augus-Kong\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31221497,\n",
      "      \"login\": \"Augus-Kong\",\n",
      "      \"url\": \"https://api.github.com/users/Augus-Kong\"\n",
      "    },\n",
      "    \"created_at\": \"2017-09-06T19:13:54Z\",\n",
      "    \"id\": \"6557907651\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"action\": \"started\"\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"WatchEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-30T23:59:27Z\",\n",
      "    \"id\": \"6532575125\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"e7fca9c0dafa23031a7341e038316935957e3551\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Incorporated latest Lab 2 patches\",\n",
      "          \"sha\": \"9d248cb45764a59f2475c97d4667e21c8c3a7ad5\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/9d248cb45764a59f2475c97d4667e21c8c3a7ad5\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"9d248cb45764a59f2475c97d4667e21c8c3a7ad5\",\n",
      "      \"push_id\": 1954853950,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-30T23:14:01Z\",\n",
      "    \"id\": \"6532465042\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"1c4397fc9fff78665daab6a863b82f90be01a33c\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added the groceries.csv dataset\",\n",
      "          \"sha\": \"e7fca9c0dafa23031a7341e038316935957e3551\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/e7fca9c0dafa23031a7341e038316935957e3551\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"e7fca9c0dafa23031a7341e038316935957e3551\",\n",
      "      \"push_id\": 1954796136,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-26T20:54:15Z\",\n",
      "    \"id\": \"6515810341\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"5ec77836ff0f0f898211bda9dcf3c76d90f849b4\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added practice problems for the math prerequisites (probability, calculus, linear algebra)\",\n",
      "          \"sha\": \"3bd9c6e2d06177e948d85394356514d703f785e9\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/3bd9c6e2d06177e948d85394356514d703f785e9\"\n",
      "        },\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Merge branch 'master' of github.com:cse6040/labs-fa17\",\n",
      "          \"sha\": \"1c4397fc9fff78665daab6a863b82f90be01a33c\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/1c4397fc9fff78665daab6a863b82f90be01a33c\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 2,\n",
      "      \"head\": \"1c4397fc9fff78665daab6a863b82f90be01a33c\",\n",
      "      \"push_id\": 1946190317,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 2\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-21T19:37:53Z\",\n",
      "    \"id\": \"6493968437\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"03cdd7d672e496119479a6354fb26059fdf93804\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added 'bonus notebook' for on-campus students\",\n",
      "          \"sha\": \"5ec77836ff0f0f898211bda9dcf3c76d90f849b4\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/5ec77836ff0f0f898211bda9dcf3c76d90f849b4\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 1,\n",
      "      \"head\": \"5ec77836ff0f0f898211bda9dcf3c76d90f849b4\",\n",
      "      \"push_id\": 1934971731,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 1\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-18T05:12:12Z\",\n",
      "    \"id\": \"6477762298\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"ff3085c189b67f955b34df0634a097a4a026b756\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Lab 0: Sample assignment\",\n",
      "          \"sha\": \"a91a597ff4240a54fea042e0361f334501ee3cfa\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/a91a597ff4240a54fea042e0361f334501ee3cfa\"\n",
      "        },\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Lab 1: Python bootcamp review\",\n",
      "          \"sha\": \"7ef5ede23310fa733530d597c64143bd39db6e2e\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/7ef5ede23310fa733530d597c64143bd39db6e2e\"\n",
      "        },\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added Lab 2: Association rule mining\",\n",
      "          \"sha\": \"705547e199f0573b00c741ff032b9a8f0b9b0bd4\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/705547e199f0573b00c741ff032b9a8f0b9b0bd4\"\n",
      "        },\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Updated README with info on the first three labs (0-2)\",\n",
      "          \"sha\": \"03cdd7d672e496119479a6354fb26059fdf93804\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/03cdd7d672e496119479a6354fb26059fdf93804\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 4,\n",
      "      \"head\": \"03cdd7d672e496119479a6354fb26059fdf93804\",\n",
      "      \"push_id\": 1929289347,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 4\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-16T15:56:37Z\",\n",
      "    \"id\": \"6464863668\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"before\": \"c6458869f6f2085e4fe7ded71b5678605f7e41f2\",\n",
      "      \"commits\": [\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Added a test dataset\",\n",
      "          \"sha\": \"a25f6a00b56c7b0a4403e19f7142d2a4874d0b52\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/a25f6a00b56c7b0a4403e19f7142d2a4874d0b52\"\n",
      "        },\n",
      "        {\n",
      "          \"author\": {\n",
      "            \"email\": \"richie@cc.gatech.edu\",\n",
      "            \"name\": \"Richard (Rich) Vuduc\"\n",
      "          },\n",
      "          \"distinct\": true,\n",
      "          \"message\": \"Edited placeholder README\",\n",
      "          \"sha\": \"ff3085c189b67f955b34df0634a097a4a026b756\",\n",
      "          \"url\": \"https://api.github.com/repos/cse6040/labs-fa17/commits/ff3085c189b67f955b34df0634a097a4a026b756\"\n",
      "        }\n",
      "      ],\n",
      "      \"distinct_size\": 2,\n",
      "      \"head\": \"ff3085c189b67f955b34df0634a097a4a026b756\",\n",
      "      \"push_id\": 1925489793,\n",
      "      \"ref\": \"refs/heads/master\",\n",
      "      \"size\": 2\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"PushEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-16T15:51:50Z\",\n",
      "    \"id\": \"6464822952\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"description\": \"Lab notebooks for the Fall 2017 offering of Georgia Tech's CSE 6040\",\n",
      "      \"master_branch\": \"master\",\n",
      "      \"pusher_type\": \"user\",\n",
      "      \"ref\": \"master\",\n",
      "      \"ref_type\": \"branch\"\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"CreateEvent\"\n",
      "  },\n",
      "  {\n",
      "    \"actor\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/5316640?\",\n",
      "      \"display_login\": \"rvuduc\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 5316640,\n",
      "      \"login\": \"rvuduc\",\n",
      "      \"url\": \"https://api.github.com/users/rvuduc\"\n",
      "    },\n",
      "    \"created_at\": \"2017-08-16T15:51:49Z\",\n",
      "    \"id\": \"6464822819\",\n",
      "    \"org\": {\n",
      "      \"avatar_url\": \"https://avatars.githubusercontent.com/u/31073927?\",\n",
      "      \"gravatar_id\": \"\",\n",
      "      \"id\": 31073927,\n",
      "      \"login\": \"cse6040\",\n",
      "      \"url\": \"https://api.github.com/orgs/cse6040\"\n",
      "    },\n",
      "    \"payload\": {\n",
      "      \"description\": \"Lab notebooks for the Fall 2017 offering of Georgia Tech's CSE 6040\",\n",
      "      \"master_branch\": \"master\",\n",
      "      \"pusher_type\": \"user\",\n",
      "      \"ref\": null,\n",
      "      \"ref_type\": \"repository\"\n",
      "    },\n",
      "    \"public\": true,\n",
      "    \"repo\": {\n",
      "      \"id\": 100506580,\n",
      "      \"name\": \"cse6040/labs-fa17\",\n",
      "      \"url\": \"https://api.github.com/repos/cse6040/labs-fa17\"\n",
      "    },\n",
      "    \"type\": \"CreateEvent\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "print(type(response.json ()))\n",
    "print(json.dumps(response.json(), sort_keys=True, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-5be03b16bcb0c40f",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "**Exercise 0.** It should be self-evident that the JSON response above consists of a sequence of records, which we will refer to as _events_. Each event is associated with an _actor_. Write some code to extract a dictionary of all actors, where the key is the actor's login and the value is the actor's URL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "nbgrader": {
     "grade": false,
     "grade_id": "extract_actors",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def extract_actors (json_github_events):\n",
    "    \"\"\"Given JSON records for events in a GitHub repo,\n",
    "    returns a dictionary of the actors and their URLs.\n",
    "    \"\"\"\n",
    "# YOUR CODE HERE\n",
    "    actors_dict = {}\n",
    "    for i in range(len(json_github_events)):\n",
    "        key_login = json_github_events[i]['actor']['login']\n",
    "        value_URL = json_github_events[i]['actor']['url']\n",
    "        actors_dict[key_login] = value_URL\n",
    "\n",
    "    return actors_dict\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "extract_actors_test",
     "locked": true,
     "points": 0,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rvuduc: https://api.github.com/users/rvuduc\n",
      "dthai91: https://api.github.com/users/dthai91\n",
      "Augus-Kong: https://api.github.com/users/Augus-Kong\n"
     ]
    }
   ],
   "source": [
    "actor_urls = extract_actors(response.json ())\n",
    "\n",
    "for actor, url in actor_urls.items ():\n",
    "    print ('{}: {}'.format(actor, url))\n",
    "    assert url == \"https://api.github.com/users/{}\".format(actor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-23355ce56dd4dcdc",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "**Exercise 1.** Write some code that goes to each actor's URL and determines their name. If an actor URL is invalid, that actor should not appear in the output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true,
    "nbgrader": {
     "grade": false,
     "grade_id": "get_names",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def lookup_names (actor_urls):\n",
    "    \"\"\"Given a dictionary of (actor, url) pairs, looks up the JSON at\n",
    "    the URL and extracts the user's name (if any). Returns a new\n",
    "    dictionary of (actor, name) pairs.\n",
    "    \"\"\"\n",
    "# YOUR CODE HERE\n",
    "    actor_name_dict = {}\n",
    "    for actor, URL in actor_urls.items():\n",
    "        response = requests.get(URL).json()\n",
    "#         if response['message'] == 'Not Found':\n",
    "#             continue\n",
    "\n",
    "        name = response['name']\n",
    "        actor_name_dict[actor] = name\n",
    "        \n",
    "    return actor_name_dict\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "get_names_test",
     "locked": true,
     "points": 0,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "rvuduc: Rich Vuduc (personal account)\n",
      "dthai91: Duc Thai\n",
      "Augus-Kong: XIANGYU KONG\n"
     ]
    }
   ],
   "source": [
    "actor_names = lookup_names (actor_urls)\n",
    "\n",
    "for actor, name in actor_names.items ():\n",
    "    print (\"{}: {}\".format (actor, name))\n",
    "    \n",
    "assert actor_names['rvuduc'] == 'Rich Vuduc (personal account)'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ae7abfbce62b1cd6",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "That's the end of this notebook. Processing JSON is fairly straightforward, because it maps very naturally to nested dictionaries in Python. You might search the web for other sources of JSON data, including [this one](https://www.yelp.com/dataset/challenge), and do your own processing!"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Create Assignment",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}