Fixed url for espn scoreboard. Also deprecated nwhl usage due to pbp …

…not being avaialble anymore
HarryShomer · Sep 29, 2019 · 3d636ac · 3d636ac
1 parent d6b421f
commit 3d636ac
Show file tree

Hide file tree

Showing 19 changed files with 60 additions and 55 deletions.
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -4,11 +4,7 @@ v1.2.6
   * Added test coverage for most modules using pytest
   * Refactored large portion of 'html_pbp.py' and corrected minor parser fixes in regards to penalties
   * Added the module 'save_pages.py' which allows one to saves scraped files
-  * Added keyword arguments 'rescrape' and 'docs_dir' to the three main scraping functions. Specifying a valid directory
-    using 'docs_dir' will make us check if a file was already scraped and saved before getting it from the source. It will
-    also provide a location for us to save it if we don't have it yet. 'rescrape' only applies when a valid directory
-    is provided with 'docs_dir'. Setting 'rescrape' equal to True will have us scrape the file from the source even if
-    it's saved and save this new one.
+  * Added keyword arguments 'rescrape' and 'docs_dir' to the three main scraping functions. Specifying a valid directory using 'docs_dir' will make us check if a file was already scraped and saved before getting it from the source. It will also provide a location for us to save it if we don't have it yet. 'rescrape' only applies when a valid directory is provided with 'docs_dir'. Setting 'rescrape' equal to True will have us scrape the file from the source even if it's saved and save this new one.
 
 v1.2.7
 ------
@@ -28,7 +24,13 @@ v1.31
   * Added functionality to automatically create docs_dir
   * Added folder to store csv files
 
-v1.34
+v1.33
 -----
   * Fixed bug with nhl changing contents of eventTypeId
   * Updated ESPN scraping after they changed the layout of the pages
+
+
+v1.34
+-----
+  * Reflected change in url for ESPN scoreboard
+  * Deprecated NWHL usage due as pbp parser isn't applicable due to UI changes (new source unknown)
diff --git a/docs/build/doctrees/environment.pickle b/docs/build/doctrees/environment.pickle
diff --git a/docs/build/doctrees/index.doctree b/docs/build/doctrees/index.doctree
diff --git a/docs/build/doctrees/nhl_scrape_functions.doctree b/docs/build/doctrees/nhl_scrape_functions.doctree
diff --git a/docs/build/html/.buildinfo b/docs/build/html/.buildinfo
@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: fed6b95c7870305f50fcb3b94bddc01d
+config: 65e9aa145442b387247bed1e665e6065
 tags: 645f666f9bcd5a90fca523b33c5a78b7
diff --git a/docs/build/html/genindex.html b/docs/build/html/genindex.html
@@ -7,15 +7,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>Index &#8212; hockey_scraper 1.32 documentation</title>
+    <title>Index &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -38,7 +38,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="py-modindex.html" title="Python Module Index"
              >modules</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -536,11 +536,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="py-modindex.html" title="Python Module Index"
              >modules</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/index.html b/docs/build/html/index.html
@@ -6,15 +6,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>Hockey-Scraper &#8212; hockey_scraper 1.32 documentation</title>
+    <title>Hockey-Scraper &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -41,7 +41,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="nhl_scrape_functions.html" title="NHL Scraping Functions"
              accesskey="N">next</a> |</li>
-        <li class="nav-item nav-item-0"><a href="#">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="#">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -62,6 +62,7 @@ <h2>Contents<a class="headerlink" href="#contents" title="Permalink to this head
 <li class="toctree-l1"><a class="reference internal" href="license_link.html">License</a></li>
 </ul>
 </div>
+<p><strong>Note: Due to changes in the NHL API and ESPN&#8217;s website, coordinates are only scraped when using version 1.33 and higher.</strong></p>
 </div>
 <div class="section" id="purpose">
 <h2>Purpose<a class="headerlink" href="#purpose" title="Permalink to this headline">¶</a></h2>
@@ -308,11 +309,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="nhl_scrape_functions.html" title="NHL Scraping Functions"
              >next</a> |</li>
-        <li class="nav-item nav-item-0"><a href="#">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="#">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/license_link.html b/docs/build/html/license_link.html
@@ -6,15 +6,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>License &#8212; hockey_scraper 1.32 documentation</title>
+    <title>License &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -41,7 +41,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="live_scrape.html" title="Live Scraping"
              accesskey="P">previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -803,11 +803,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="live_scrape.html" title="Live Scraping"
              >previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/live_scrape.html b/docs/build/html/live_scrape.html
@@ -6,15 +6,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>Live Scraping &#8212; hockey_scraper 1.32 documentation</title>
+    <title>Live Scraping &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -45,7 +45,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="nwhl_scrape_functions.html" title="NWHL Scraping Functions"
              accesskey="P">previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -631,11 +631,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="nwhl_scrape_functions.html" title="NWHL Scraping Functions"
              >previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/nhl_scrape_functions.html b/docs/build/html/nhl_scrape_functions.html
@@ -6,15 +6,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>NHL Scraping Functions &#8212; hockey_scraper 1.32 documentation</title>
+    <title>NHL Scraping Functions &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -45,7 +45,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="index.html" title="Hockey-Scraper"
              accesskey="P">previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -1353,7 +1353,7 @@ <h2>Scraping<a class="headerlink" href="#scraping" title="Permalink to this head
 <dt id="hockey_scraper.nhl.pbp.espn_pbp.get_espn_game_id">
 <code class="descclassname">hockey_scraper.nhl.pbp.espn_pbp.</code><code class="descname">get_espn_game_id</code><span class="sig-paren">(</span><em>date</em>, <em>home_team</em>, <em>away_team</em><span class="sig-paren">)</span><a class="headerlink" href="#hockey_scraper.nhl.pbp.espn_pbp.get_espn_game_id" title="Permalink to this definition">¶</a></dt>
 <dd><p>Scrapes the day&#8217;s schedule and gets the id for the given game
-Ex: <a class="reference external" href="http://www.espn.com/nhl/scoreboard?date=20161024">http://www.espn.com/nhl/scoreboard?date=20161024</a></p>
+Ex: <a class="reference external" href="http://www.espn.com/nhl/scoreboard/_/date/20161024">http://www.espn.com/nhl/scoreboard/_/date/20161024</a></p>
 <table class="docutils field-list" frame="void" rules="none">
 <col class="field-name" />
 <col class="field-body" />
@@ -1392,6 +1392,8 @@ <h2>Scraping<a class="headerlink" href="#scraping" title="Permalink to this head
 <dt id="hockey_scraper.nhl.pbp.espn_pbp.get_teams">
 <code class="descclassname">hockey_scraper.nhl.pbp.espn_pbp.</code><code class="descname">get_teams</code><span class="sig-paren">(</span><em>response</em><span class="sig-paren">)</span><a class="headerlink" href="#hockey_scraper.nhl.pbp.espn_pbp.get_teams" title="Permalink to this definition">¶</a></dt>
 <dd><p>Extract Teams for date from doc</p>
+<p>ul-&gt; class = ScoreCell__Competitors</p>
+<p>div -&gt; class = ScoreCell__TeamName ScoreCell__TeamName&#8211;shortDisplayName truncate db</p>
 <table class="docutils field-list" frame="void" rules="none">
 <col class="field-name" />
 <col class="field-body" />
@@ -2219,11 +2221,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="index.html" title="Hockey-Scraper"
              >previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/nwhl_scrape_functions.html b/docs/build/html/nwhl_scrape_functions.html
@@ -6,15 +6,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>NWHL Scraping Functions &#8212; hockey_scraper 1.32 documentation</title>
+    <title>NWHL Scraping Functions &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -45,7 +45,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="nhl_scrape_functions.html" title="NHL Scraping Functions"
              accesskey="P">previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -504,11 +504,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="nhl_scrape_functions.html" title="NHL Scraping Functions"
              >previous</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/objects.inv b/docs/build/html/objects.inv
diff --git a/docs/build/html/py-modindex.html b/docs/build/html/py-modindex.html
@@ -6,15 +6,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>Python Module Index &#8212; hockey_scraper 1.32 documentation</title>
+    <title>Python Module Index &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -40,7 +40,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="#" title="Python Module Index"
              >modules</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -172,11 +172,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="#" title="Python Module Index"
              >modules</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/search.html b/docs/build/html/search.html
@@ -6,15 +6,15 @@
   <head>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 
-    <title>Search &#8212; hockey_scraper 1.32 documentation</title>
+    <title>Search &#8212; hockey_scraper 1.34 documentation</title>
 
     <link rel="stylesheet" href="_static/nature.css" type="text/css" />
     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
 
     <script type="text/javascript">
       var DOCUMENTATION_OPTIONS = {
         URL_ROOT:    './',
-        VERSION:     '1.32',
+        VERSION:     '1.34',
         COLLAPSE_INDEX: false,
         FILE_SUFFIX: '.html',
         HAS_SOURCE:  true,
@@ -45,7 +45,7 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="py-modindex.html" title="Python Module Index"
              >modules</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>  
 
@@ -96,11 +96,11 @@ <h3>Navigation</h3>
         <li class="right" >
           <a href="py-modindex.html" title="Python Module Index"
              >modules</a> |</li>
-        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.32 documentation</a> &#187;</li> 
+        <li class="nav-item nav-item-0"><a href="index.html">hockey_scraper 1.34 documentation</a> &#187;</li> 
       </ul>
     </div>
     <div class="footer" role="contentinfo">
-        &#169; Copyright 2018, Harry Shomer.
+        &#169; Copyright 2019, Harry Shomer.
       Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.5.1.
     </div>
   </body>

diff --git a/docs/build/html/searchindex.js b/docs/build/html/searchindex.js
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -52,9 +52,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.32'
+version = '1.34'
 # The full version, including alpha/beta/rc tags.
-release = '1.32'
+release = '1.34'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.