Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated Scraper? #6

Open
rgomezp opened this issue Jul 8, 2020 · 1 comment
Open

Updated Scraper? #6

rgomezp opened this issue Jul 8, 2020 · 1 comment

Comments

@rgomezp
Copy link

rgomezp commented Jul 8, 2020

Howdy,
The scraper linked is outdated it seems. I got this for the output:

Could not scrape token unus-sed-leo.
Traceback (most recent call last):
  File "scrape.py", line 57, in <module>
    if scrapeMarketCap(token['slug'], token['name'], 'token'):
  File "scrape.py", line 46, in scrapeMarketCap
    result = coinmarketcap.parseMarketCap(jsonDump, slug)
  File "/Users/*****/Documents/Github/Personal/coinmarketcap-scraper/coinmarketcap.py", line 93, in parseMarketCap
    rawData = json.loads(jsonDump)
  File "/Users//*****/anaconda2/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/Users//*****/anaconda2/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users//*****/anaconda2/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

------------------------------------------------------------
07/08/2020 12:00:30 AM INFO: >> Could not scrape unus-sed-leo. Skipping.
07/08/2020 12:00:30 AM INFO: > Starting scrape of token usd-coin...
07/08/2020 12:00:30 AM INFO: Sleeping for 30.2140883146 seconds before request.
^CTraceback (most recent call last):
  File "scrape.py", line 57, in <module>
    if scrapeMarketCap(token['slug'], token['name'], 'token'):
  File "scrape.py", line 45, in scrapeMarketCap
    jsonDump = coinmarketcap.requestMarketCap(slug)
  File "/Users//*****/Documents/Github/Personal/coinmarketcap-scraper/coinmarketcap.py", line 51, in requestMarketCap
    graphBaseUrl, slug))
  File "/Users//*****/Documents/Github/Personal/coinmarketcap-scraper/coinmarketcap.py", line 26, in _request
    time.sleep(timeToSleep)

I'm wondering if you have either an updated version or if you could add a table with the format that this project expects the data to be in.

This looks so awesome by the way. Excited to get this working.

@prouast
Copy link
Owner

prouast commented Jul 21, 2020

The scraper breaks whenever coinmarketcap changes - I don't have time to fix this at the moment. Feel free to create a pull request at https://github.com/prouast/coinmarketcap-scraper though. Check https://github.com/prouast/coinmarketcap-scraper/blob/master/db.py for the database details. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants