Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] JSONDecodeError when attempting to download the chat of any twitch VOD #186

Closed
chaosaudit opened this issue Dec 6, 2022 · 10 comments
Labels
bug Something isn't working

Comments

@chaosaudit
Copy link

Basic information

  • Program version: 0.2.2
  • Python version: 3.10
  • Operating system: Windows 11

Describe the bug

JSONDecodeError when attempting to download the chat of any twitch VOD

Command/Code used

from chat_downloader import ChatDownloader
chat = ChatDownloader().get_chat(vod_url)
for message in chat:
    print(message)
  1. Output, traceback or other information relating to the bug:
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\site-packages\requests\models.py", line 910, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Program Files\Python310\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\Python310\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\Python310\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\site-packages\chat_downloader\sites\twitch.py", line 1175, in _get_chat_messages_by_vod_id
    info = self._session_get_json(url)
  File "C:\Program Files\Python310\lib\site-packages\chat_downloader\sites\common.py", line 512, in _session_get_json
    return self._session_get(url, **kwargs).json()
  File "C:\Program Files\Python310\lib\site-packages\requests\models.py", line 917, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value] : 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\chaos\PycharmProjects\AdventOfCode\scratch.py", line 5, in <module>
    for message in chat:
  File "C:\Program Files\Python310\lib\site-packages\chat_downloader\sites\common.py", line 283, in __next__
    item = next(self.chat)
  File "C:\Program Files\Python310\lib\site-packages\chat_downloader\sites\twitch.py", line 1178, in _get_chat_messages_by_vod_id
    self.retry(attempt_number, error=e, **params)
  File "C:\Program Files\Python310\lib\site-packages\chat_downloader\sites\common.py", line 640, in retry
    page_title = get_title_of_webpage(error.doc)
AttributeError: 'JSONDecodeError' object has no attribute 'doc'

Process finished with exit code 1
@chaosaudit chaosaudit added the bug Something isn't working label Dec 6, 2022
@chaosaudit chaosaudit changed the title [BUG] Title goes here [BUG] JSONDecodeError when attempting to download the chat of any twitch VOD Dec 6, 2022
@xenova
Copy link
Owner

xenova commented Dec 6, 2022

It finally happened :') Looks like they discontinued the /v5/videos/.../comments endpoint. (Must have happened in the past 24 hours since the last unit tests succeeded).

I imagine every other tool that downloads Twitch chat is currently broken.

Thanks for the report! I will fix ASAP

@gadi-playstream-gg
Copy link

Thanks for the update! Please keep us posted :)

@xenova xenova closed this as completed in 6d4d990 Dec 7, 2022
@xenova
Copy link
Owner

xenova commented Dec 7, 2022

That should fix it 👍 Just running unit tests then will publish package.

@xenova
Copy link
Owner

xenova commented Dec 7, 2022

Published 👍 You can install using
pip install chat-downloader --upgrade

@wordlebot-uxy
Copy link

@xenova I see that you solved it by moving to gql endpoint. Is it pssible that twitch will close that endpoint as well, as they just closed the v5? And if so, would your package have another way of extracting the chat data? Wondering bc of future proof. Thanks for your work!

@xenova
Copy link
Owner

xenova commented Dec 19, 2022

@wordlebot-uxy While they may eventually decide to switch to another protocol, they have been slowly transitioning to GQL (GraphQL) over the past few years... so, I do not imagine that they will switch any time soon.

That being said, if any breaking changes do come, I will be ready to fix them :)

@wordlebot-uxy
Copy link

Got it, thanks! One more q: do they require you to use a token to communicate with that endpoint? If so, couldn't they just kill your token?

@xenova
Copy link
Owner

xenova commented Dec 19, 2022

Got it, thanks! One more q: do they require you to use a token to communicate with that endpoint? If so, couldn't they just kill your token?

The client id (token) that is used throughout the program (kimne78kx3ncx6brgo4mv6wki5h1ko) is public and is used for all unauthenticated users (you can test it out by opening an incognito tab and viewing the request that is sent). This token has been in use for many years, so, they shouldn't just kill it. If they do... it can just be replaced.

@wordlebot-uxy
Copy link

Thanks a lot! You got a patreon or Ko-Fi or similar?

@xenova
Copy link
Owner

xenova commented Dec 19, 2022

Thanks a lot! You got a patreon or Ko-Fi or similar?

No worries! 👍 I have a Ko-Fi: https://ko-fi.com/xenova 😊

irystocratanon pushed a commit to irystocratanon/chat-downloader that referenced this issue Aug 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants