Requests-lxml-Py3-Scraping.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Beginner's Guide to Scraping (Python 3 version)\n",
    "\n",
    "### Pre-requisite System Packages:\n",
    "- python 3\n",
    "- python3-pip\n",
    "\n",
    "* Most newer distributions already come with pip included. *\n",
    "\n",
    "### Required Python Packages:\n",
    "- requests\n",
    "- lxml\n",
    "\n",
    "### Optional Python Packages:\n",
    "- html5lib\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1. Import the packages to be used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from lxml import html, etree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2. Page retrieval (automated browse)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "page = requests.get(\"http://quotes.toscrape.com\")\n",
    "# establish connection and browse the page data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Content-Type': 'text/html; charset=utf-8', 'Server': 'nginx/1.10.1', 'Date': 'Tue, 22 Aug 2017 06:04:00 GMT', 'Connection': 'keep-alive', 'X-Upstream': 'spidyquotes-master_web', 'Transfer-Encoding': 'chunked', 'Content-Encoding': 'gzip'}\n"
     ]
    }
   ],
   "source": [
    "print(page.headers)\n",
    "# header info - useful for debugging the page retrieval process"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n"
     ]
    }
   ],
   "source": [
    "print(page.status_code)\n",
    "# another useful call to debug your scraper"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'<!DOCTYPE html>\\n<html lang=\"en\">\\n<head>\\n\\t<meta charset=\"UTF-8\">\\n\\t<title>Quotes to Scrape</title>\\n    <link rel=\"stylesheet\" href=\"/static/bootstrap.min.css\">\\n    <link rel=\"stylesheet\" href=\"/static/main.css\">\\n</head>\\n<body>\\n    <div class=\"container\">\\n        <div class=\"row header-box\">\\n            <div class=\"col-md-8\">\\n                <h1>\\n                    <a href=\"/\" style=\"text-decoration: none\">Quotes to Scrape</a>\\n                </h1>\\n            </div>\\n            <div class=\"col-md-4\">\\n                <p>\\n                \\n                    <a href=\"/login\">Login</a>\\n                \\n                </p>\\n            </div>\\n        </div>\\n    \\n\\n<div class=\"row\">\\n    <div class=\"col-md-8\">\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cThe world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Albert Einstein</small>\\n        <a href=\"/author/Albert-Einstein\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"change,deep-thoughts,thinking,world\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/change/page/1/\">change</a>\\n            \\n            <a class=\"tag\" href=\"/tag/deep-thoughts/page/1/\">deep-thoughts</a>\\n            \\n            <a class=\"tag\" href=\"/tag/thinking/page/1/\">thinking</a>\\n            \\n            <a class=\"tag\" href=\"/tag/world/page/1/\">world</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cIt is our choices, Harry, that show what we truly are, far more than our abilities.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">J.K. Rowling</small>\\n        <a href=\"/author/J-K-Rowling\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"abilities,choices\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/abilities/page/1/\">abilities</a>\\n            \\n            <a class=\"tag\" href=\"/tag/choices/page/1/\">choices</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cThere are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Albert Einstein</small>\\n        <a href=\"/author/Albert-Einstein\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"inspirational,life,live,miracle,miracles\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/inspirational/page/1/\">inspirational</a>\\n            \\n            <a class=\"tag\" href=\"/tag/life/page/1/\">life</a>\\n            \\n            <a class=\"tag\" href=\"/tag/live/page/1/\">live</a>\\n            \\n            <a class=\"tag\" href=\"/tag/miracle/page/1/\">miracle</a>\\n            \\n            <a class=\"tag\" href=\"/tag/miracles/page/1/\">miracles</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cThe person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Jane Austen</small>\\n        <a href=\"/author/Jane-Austen\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"aliteracy,books,classic,humor\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/aliteracy/page/1/\">aliteracy</a>\\n            \\n            <a class=\"tag\" href=\"/tag/books/page/1/\">books</a>\\n            \\n            <a class=\"tag\" href=\"/tag/classic/page/1/\">classic</a>\\n            \\n            <a class=\"tag\" href=\"/tag/humor/page/1/\">humor</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cImperfection is beauty, madness is genius and it&#39;s better to be absolutely ridiculous than absolutely boring.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Marilyn Monroe</small>\\n        <a href=\"/author/Marilyn-Monroe\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"be-yourself,inspirational\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/be-yourself/page/1/\">be-yourself</a>\\n            \\n            <a class=\"tag\" href=\"/tag/inspirational/page/1/\">inspirational</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cTry not to become a man of success. Rather become a man of value.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Albert Einstein</small>\\n        <a href=\"/author/Albert-Einstein\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"adulthood,success,value\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/adulthood/page/1/\">adulthood</a>\\n            \\n            <a class=\"tag\" href=\"/tag/success/page/1/\">success</a>\\n            \\n            <a class=\"tag\" href=\"/tag/value/page/1/\">value</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cIt is better to be hated for what you are than to be loved for what you are not.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Andr\\xc3\\xa9 Gide</small>\\n        <a href=\"/author/Andre-Gide\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"life,love\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/life/page/1/\">life</a>\\n            \\n            <a class=\"tag\" href=\"/tag/love/page/1/\">love</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cI have not failed. I&#39;ve just found 10,000 ways that won&#39;t work.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Thomas A. Edison</small>\\n        <a href=\"/author/Thomas-A-Edison\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"edison,failure,inspirational,paraphrased\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/edison/page/1/\">edison</a>\\n            \\n            <a class=\"tag\" href=\"/tag/failure/page/1/\">failure</a>\\n            \\n            <a class=\"tag\" href=\"/tag/inspirational/page/1/\">inspirational</a>\\n            \\n            <a class=\"tag\" href=\"/tag/paraphrased/page/1/\">paraphrased</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cA woman is like a tea bag; you never know how strong it is until it&#39;s in hot water.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Eleanor Roosevelt</small>\\n        <a href=\"/author/Eleanor-Roosevelt\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"misattributed-eleanor-roosevelt\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/misattributed-eleanor-roosevelt/page/1/\">misattributed-eleanor-roosevelt</a>\\n            \\n        </div>\\n    </div>\\n\\n    <div class=\"quote\" itemscope itemtype=\"http://schema.org/CreativeWork\">\\n        <span class=\"text\" itemprop=\"text\">\\xe2\\x80\\x9cA day without sunshine is like, you know, night.\\xe2\\x80\\x9d</span>\\n        <span>by <small class=\"author\" itemprop=\"author\">Steve Martin</small>\\n        <a href=\"/author/Steve-Martin\">(about)</a>\\n        </span>\\n        <div class=\"tags\">\\n            Tags:\\n            <meta class=\"keywords\" itemprop=\"keywords\" content=\"humor,obvious,simile\" /    > \\n            \\n            <a class=\"tag\" href=\"/tag/humor/page/1/\">humor</a>\\n            \\n            <a class=\"tag\" href=\"/tag/obvious/page/1/\">obvious</a>\\n            \\n            <a class=\"tag\" href=\"/tag/simile/page/1/\">simile</a>\\n            \\n        </div>\\n    </div>\\n\\n    <nav>\\n        <ul class=\"pager\">\\n            \\n            \\n            <li class=\"next\">\\n                <a href=\"/page/2/\">Next <span aria-hidden=\"true\">&rarr;</span></a>\\n            </li>\\n            \\n        </ul>\\n    </nav>\\n    </div>\\n    <div class=\"col-md-4 tags-box\">\\n        \\n            <h2>Top Ten tags</h2>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 28px\" href=\"/tag/love/\">love</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 26px\" href=\"/tag/inspirational/\">inspirational</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 26px\" href=\"/tag/life/\">life</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 24px\" href=\"/tag/humor/\">humor</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 22px\" href=\"/tag/books/\">books</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 14px\" href=\"/tag/reading/\">reading</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 10px\" href=\"/tag/friendship/\">friendship</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 8px\" href=\"/tag/friends/\">friends</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 8px\" href=\"/tag/truth/\">truth</a>\\n            </span>\\n            \\n            <span class=\"tag-item\">\\n            <a class=\"tag\" style=\"font-size: 6px\" href=\"/tag/simile/\">simile</a>\\n            </span>\\n            \\n        \\n    </div>\\n</div>\\n\\n    </div>\\n    <footer class=\"footer\">\\n        <div class=\"container\">\\n            <p class=\"text-muted\">\\n                Quotes by: <a href=\"https://www.goodreads.com/quotes\">GoodReads.com</a>\\n            </p>\\n            <p class=\"copyright\">\\n                Made with <span class=\\'sh-red\\'>\\xe2\\x9d\\xa4</span> by <a href=\"https://scrapinghub.com\">Scrapinghub</a>\\n            </p>\\n        </div>\\n    </footer>\\n</body>\\n</html>'\n"
     ]
    }
   ],
   "source": [
    "print(page.content)\n",
    "# get the page data and print it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3. Parse the page"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "tree = html.fromstring(page.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'<head>\\n\\t<meta charset=\"UTF-8\"/>\\n\\t<title>Quotes to Scrape</title>\\n    <link rel=\"stylesheet\" href=\"/static/bootstrap.min.css\"/>\\n    <link rel=\"stylesheet\" href=\"/static/main.css\"/>\\n</head>\\n'\n"
     ]
    }
   ],
   "source": [
    "page_head = tree.xpath('//head')\n",
    "from lxml import etree\n",
    "for item in page_head:\n",
    "    print(etree.tostring(item))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Quotes to Scrape']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree.xpath('//title/text()')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4. Crawling other pages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'<a href=\"/\" style=\"text-decoration: none\">Quotes to Scrape</a>\\n                '\n",
      "b'<a href=\"/login\">Login</a>\\n                \\n                '\n",
      "b'<a href=\"/author/Albert-Einstein\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/change/page/1/\">change</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/deep-thoughts/page/1/\">deep-thoughts</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/thinking/page/1/\">thinking</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/world/page/1/\">world</a>\\n            \\n        '\n",
      "b'<a href=\"/author/J-K-Rowling\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/abilities/page/1/\">abilities</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/choices/page/1/\">choices</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Albert-Einstein\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/inspirational/page/1/\">inspirational</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/life/page/1/\">life</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/live/page/1/\">live</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/miracle/page/1/\">miracle</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/miracles/page/1/\">miracles</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Jane-Austen\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/aliteracy/page/1/\">aliteracy</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/books/page/1/\">books</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/classic/page/1/\">classic</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/humor/page/1/\">humor</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Marilyn-Monroe\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/be-yourself/page/1/\">be-yourself</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/inspirational/page/1/\">inspirational</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Albert-Einstein\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/adulthood/page/1/\">adulthood</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/success/page/1/\">success</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/value/page/1/\">value</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Andre-Gide\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/life/page/1/\">life</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/love/page/1/\">love</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Thomas-A-Edison\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/edison/page/1/\">edison</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/failure/page/1/\">failure</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/inspirational/page/1/\">inspirational</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/paraphrased/page/1/\">paraphrased</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Eleanor-Roosevelt\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/misattributed-eleanor-roosevelt/page/1/\">misattributed-eleanor-roosevelt</a>\\n            \\n        '\n",
      "b'<a href=\"/author/Steve-Martin\">(about)</a>\\n        '\n",
      "b'<a class=\"tag\" href=\"/tag/humor/page/1/\">humor</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/obvious/page/1/\">obvious</a>\\n            \\n            '\n",
      "b'<a class=\"tag\" href=\"/tag/simile/page/1/\">simile</a>\\n            \\n        '\n",
      "b'<a href=\"/page/2/\">Next <span aria-hidden=\"true\">&#8594;</span></a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 28px\" href=\"/tag/love/\">love</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 26px\" href=\"/tag/inspirational/\">inspirational</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 26px\" href=\"/tag/life/\">life</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 24px\" href=\"/tag/humor/\">humor</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 22px\" href=\"/tag/books/\">books</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 14px\" href=\"/tag/reading/\">reading</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 10px\" href=\"/tag/friendship/\">friendship</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 8px\" href=\"/tag/friends/\">friends</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 8px\" href=\"/tag/truth/\">truth</a>\\n            '\n",
      "b'<a class=\"tag\" style=\"font-size: 6px\" href=\"/tag/simile/\">simile</a>\\n            '\n",
      "b'<a href=\"https://www.goodreads.com/quotes\">GoodReads.com</a>\\n            '\n",
      "b'<a href=\"https://scrapinghub.com\">Scrapinghub</a>\\n            '\n"
     ]
    }
   ],
   "source": [
    "page_links = tree.xpath('//a')\n",
    "for item in page_links:\n",
    "    print(etree.tostring(item))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### References:\n",
    "\n",
    "- http://docs.python-guide.org/en/latest/scenarios/scrape/\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}