json.decoder.JSONDecodeError

[mikan01]

Hi,

I keep getting this error after scraping few books (3 to 4 books). Please help

Traceback (most recent call last): File "main.py", line 62, in <module> audio_files = scraper.scrape_book_audio(driver, book_json) File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 223, in scrape_book_audio audio_url = audio_request.json()['url'] File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json return complexjson.loads( File "C:\Python38\lib\json\__init__.py", line 357, in loads return _default_decoder.decode(s) File "C:\Python38\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[leoncvlt]

I could not replicate that, but the latest commit 43aa262 should at least make it fail gracefully and not stop the scraping process.

I'll leave this open for now - let me know if you figure a way to consistently replicate that or if it fails with a specific book in particular.

[mikan01]

It's still failing. :\

I am able to download few books 1 or 2 at a time then process gets stopped with this error, then I have rerun the command to scrape more books.

Traceback (most recent call last):
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 229, in scrape_book_audio
audio_request_json = audio_request.json();
File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json
return complexjson.loads(
File "C:\Python38\lib\json_init_.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 66, in
audio_files = scraper.scrape_book_audio(driver, book_json, args.language)
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 239, in scrape_book_audio
print(f'[!] Received malformed json data: {audio_request_json.text}')
UnboundLocalError: local variable 'audio_request_json' referenced before assignment

[Wunderharke]

It's still failing. :\

I am able to download few books 1 or 2 at a time then process gets stopped with this error, then I have rerun the command to scrap more books.

Traceback (most recent call last):
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 229, in scrape_book_audio
audio_request_json = audio_request.json();
File "C:\Python38\lib\site-packages\requests\models.py", line 888, in json
return complexjson.loads(
File "C:\Python38\lib\json__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 66, in
audio_files = scraper.scrape_book_audio(driver, book_json, args.language)
File "C:\Users\Fang Yuan\Downloads\blinkist-scraper-master\scraper.py", line 239, in scrape_book_audio
print(f'[!] Received malformed json data: {audio_request_json.text}')
UnboundLocalError: local variable 'audio_request_json' referenced before assignment

I am getting the same error when scraping the second book on a new clone on ca0d913.

[Wunderharke]

I might found the cause:

The error occurs after accepting the cookies inside the ChromeDriver window.
It seems there is some interference, maybe with some json data on side-loaded sites, just guessing ...

When not interacting with the Chrome window, the program seems to work like expected.

[leoncvlt]

My bad, latest commit 426d5bd should fix the handling of the exception. Again, it will not fix the bug but at least it should handle it without stopping the program and provide some more information about what's failing.

@Wunderharke Interesting! I never touch the Chrome window while scraping anyway, since it's all automated - just let it do its job. @mikan01 can you confirm if that makes it better for you too?

[mikan01]

My bad, latest commit 426d5bd should fix the handling of the exception. Again, it will not fix the bug but at least it should handle it without stopping the program and provide some more information about what's failing.

@Wunderharke Interesting! I never touch the Chrome window while scraping anyway, since it's all automated - just let it do its job. @mikan01 can you confirm if that makes it better for you too?

Thank you, now the problem seems to be fixed. :)

[rocketinventor]

@mikan01,
If this issue has been resolved would you please consider closing it?

Thanks!