Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in cpi.update() #74

Closed
tusharsinghsoam opened this issue Apr 14, 2023 · 5 comments
Closed

Error in cpi.update() #74

tusharsinghsoam opened this issue Apr 14, 2023 · 5 comments

Comments

@tusharsinghsoam
Copy link

Following is the traceback for the issue I faced during cpi.update():

Version of cpi used: 1.0.17

AssertionError Traceback (most recent call last)
Cell In [40], line 1
----> 1 cpi.update()

File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi_init_.py:163, in update()
157 def update():
158 """
159 Updates the Consumer Price Index dataset at the core of this library.
160
161 Requires an Internet connection.
162 """
--> 163 Downloader().update()

File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi\download.py:75, in Downloader.update(self)
73 # Download the TSVs
74 logger.debug(f"Downloading {len(self.FILE_LIST)} files from the BLS")
---> 75 [self.get_tsv(file) for file in self.FILE_LIST]
77 # Insert the TSVs
78 logger.debug("Loading data into SQLite database")

File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi\download.py:75, in (.0)
73 # Download the TSVs
74 logger.debug(f"Downloading {len(self.FILE_LIST)} files from the BLS")
---> 75 [self.get_tsv(file) for file in self.FILE_LIST]
77 # Insert the TSVs
78 logger.debug("Loading data into SQLite database")

File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cpi\download.py:109, in Downloader.get_tsv(self, file)
107 tsv_path = self.get_data_dir() / f"{file}.tsv"
108 response = requests.get(url)
--> 109 assert response.ok
110 with open(tsv_path, "w") as fp:
111 fp.write(response.text)

AssertionError:
@palewire
Copy link
Owner

Hmm. If that response isn't "ok" I'm guessing something failed in the download, yes? Is it possible your internet went out? does this problem consistently persist? If so, maybe the BLS changed the URL we are downloading from?

@palewire
Copy link
Owner

Looks like we are getting blocked from downloading by bot detection added to the BLS website.

<!DOCTYPE HTML> <html lang="en-us">                      <head>  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Access Denied</title> </head>   <style type="text/css">     .centerDiv     {       width: 60%;       height:200px;       margin: 0 auto;       background-color:#FFFFFF ;     }   </style> <body> <div class="centerDiv">                <h1><a href=https://www.bls.gov><!--img src="/apology_objects/images/01.jpg" border="0"--></a><span style="font-family: Times, serif; color: #990000; font-size: 38px;">Bureau of Labor Statistics</span></h1>     <h2>Access Denied</h2>                               <p>The BLS is committed to providing data promptly and according to established schedules. Automated retrieval programs (commonly called "robots" or "bots") can cause delays and interfere with other customers' timely access to information. Therefore, bot activity that doesn't conform to BLS usage policy is prohibited.</p>                                 <p>We apologize for any inconvenience. If you believe we have made an error, please <a href=https://data.bls.gov/forms/opb.htm?akamai-0.36aa2c17.1681487650.919dabe>contact us</a>.</p>                               <p>Please contact your administrator with the error code: 0.36aa2c17.1681487650.919dabe</p> </div> </body> </html>

@palewire
Copy link
Owner

I've attempted to patch this bug, and better test for its future recurrence. Please upgrade to version 1.0.18 and try again. Assuming it's fixed, I'll close this ticket. If you still have the problem, please speak up.

@tusharsinghsoam
Copy link
Author

Hi Ben,

It has fixed the issue. I am able to run "cpi.update()" with the latest version (1.0.18). Thank you so much.

@palewire
Copy link
Owner

palewire commented Apr 14, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@palewire @tusharsinghsoam