Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

youtube-dl 2017.09.11 fails to rip Twitter URLs with "unable to extract guest token" #14197

Closed
jadrianjohnson-jukin opened this issue Sep 13, 2017 · 9 comments
Closed

youtube-dl 2017.09.11 fails to rip Twitter URLs with "unable to extract guest token" #14197

jadrianjohnson-jukin opened this issue Sep 13, 2017 · 9 comments
Labels
cant-reproduce

Comments

@jadrianjohnson-jukin
Copy link

@jadrianjohnson-jukin jadrianjohnson-jukin commented Sep 13, 2017

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like this: [x])
  • Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.09.11. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2017.09.11

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

root@rails-app1:~# youtube-dl --verbose -v -k -f mp4 -o '/tmp/twitter829114722234617858.mp4' 'https://twitter.com/Ryan_McCran/status/829114722234617858'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'-v', u'-k', u'-f', u'mp4', u'-o', u'/tmp/twitter829114722234617858.mp4', u'https://twitter.com/Ryan_McCran/status/829114722234617858']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.09.11
[debug] Python version 2.7.6 - Linux-3.13.0-129-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: ffmpeg 3.3.3, ffprobe 3.3.3
[debug] Proxy map: {}
[twitter] 829114722234617858: Downloading webpage
[twitter:card] 829114722234617858: Downloading webpage
[twitter:card] 829114722234617858: Downloading m3u8 information
[twitter:card] 829114722234617858: Downloading mobile webpage
[twitter:card] 829114722234617858: Downloading main script
ERROR: Unable to extract guest token; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 776, in extract_info
    ie_result = ie.extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 434, in extract
    ie_result = self._real_extract(url)
  File "/usr/bin/youtube-dl/youtube_dl/extractor/twitter.py", line 225, in _real_extract
    formats.extend(self._extract_mobile_formats(username, video_id))
  File "/usr/bin/youtube-dl/youtube_dl/extractor/twitter.py", line 155, in _extract_mobile_formats
    webpage, 'guest token')
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 797, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract guest token; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

root@rails-app1:~# youtube-dl -U
youtube-dl is up-to-date (2017.09.11)

youtube-dl 2017.09.11 reports ERROR extracting guest token for Twitter URLs

We are noticing that youtube-dl is failing to download and process videos from Twitter URLs such as the following:

https://twitter.com/Ryan_McCran/status/829114722234617858

The error returned is that youtube-del is unable to extract a guest token. Please advise.
@jadrianjohnson-jukin
Copy link
Author

@jadrianjohnson-jukin jadrianjohnson-jukin commented Sep 13, 2017

We attempted to try this same command with python 2.7.13 to affirm that it wasn't our older python version, but we still get this error:

(ripperupgrade)admins@rails-app2:~$ python /usr/local/bin/youtube-dl -v -k -f mp4 -o '/tmp/twitter829114722234617858.mp4' 'https://twitter.com/Ryan_McCran/status/829114722234617858'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-k', u'-f', u'mp4', u'-o', u'/tmp/twitter829114722234617858.mp4', u'https://twitter.com/Ryan_McCran/status/829114722234617858']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.09.11
[debug] Python version 2.7.13 - Linux-3.13.0-129-generic-x86_64-with-debian-jessie-sid
[debug] exe versions: ffmpeg 3.3.3, ffprobe 3.3.3
[debug] Proxy map: {}
[twitter] 829114722234617858: Downloading webpage
[twitter:card] 829114722234617858: Downloading webpage
[twitter:card] 829114722234617858: Downloading m3u8 information
[twitter:card] 829114722234617858: Downloading mobile webpage
[twitter:card] 829114722234617858: Downloading main script
ERROR: Unable to extract guest token; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/admins/venvs/ripperupgrade/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 776, in extract_info
    ie_result = ie.extract(url)
  File "/home/admins/venvs/ripperupgrade/lib/python2.7/site-packages/youtube_dl/extractor/common.py", line 434, in extract
    ie_result = self._real_extract(url)
  File "/home/admins/venvs/ripperupgrade/lib/python2.7/site-packages/youtube_dl/extractor/twitter.py", line 225, in _real_extract
    formats.extend(self._extract_mobile_formats(username, video_id))
  File "/home/admins/venvs/ripperupgrade/lib/python2.7/site-packages/youtube_dl/extractor/twitter.py", line 155, in _extract_mobile_formats
    webpage, 'guest token')
  File "/home/admins/venvs/ripperupgrade/lib/python2.7/site-packages/youtube_dl/extractor/common.py", line 797, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract guest token; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

(ripperupgrade)admins@rails-app2:~$ python --version
Python 2.7.13
(ripperupgrade)admins@rails-app2:~$ youtube-dl --version
2017.09.11
@dstftw
Copy link
Collaborator

@dstftw dstftw commented Sep 14, 2017

Post the output with --dump-pages.
@jadrianjohnson-jukin
Copy link
Author

@jadrianjohnson-jukin jadrianjohnson-jukin commented Sep 14, 2017

Attaching output from --dump-pages.

blah.txt
@dstftw
Copy link
Collaborator

@dstftw dstftw commented Sep 14, 2017

Can you watch it in browser from the same IP in the first place?
@jadrianjohnson-jukin
Copy link
Author

@jadrianjohnson-jukin jadrianjohnson-jukin commented Sep 14, 2017

Yes.
@jadrianjohnson-jukin
Copy link
Author

@jadrianjohnson-jukin jadrianjohnson-jukin commented Sep 18, 2017

Can I provide any other logs/output to help us figure this out?
@snarfed
Copy link

@snarfed snarfed commented Oct 25, 2017

we're also seeing this, on youtube-dl 2017-10-20, on e.g. https://twitter.com/Seahawks/status/922246436753936386 .
@snarfed snarfed mentioned this issue Oct 25, 2017
Closed
@Rudloff
Copy link
Contributor

@Rudloff Rudloff commented Nov 10, 2017

I'm getting a similar issue when running youtube-dl on Travis:

[debug] System config: []

[debug] User config: []

[debug] Custom config: []

[debug] Command-line args: [u'https://twitter.com/verge/status/813055465324056576/video/1', u'-v']

[debug] Encodings: locale UTF-8, fs UTF-8, out None, pref UTF-8

[debug] youtube-dl version 2017.11.06

[debug] Git HEAD: 6a7c07c

[debug] Python version 2.7.6 - Linux-4.9.6-040906-generic-x86_64-with-Ubuntu-14.04-trusty

[debug] exe versions: ffmpeg 3.4-static, phantomjs 2.0.0, rtmpdump 2.3

[debug] Proxy map: {}

[twitter] 813055465324056576: Downloading webpage

[twitter:card] 813055465324056576: Downloading webpage

[twitter:card] 813055465324056576: Downloading m3u8 information

[twitter:card] 813055465324056576: Downloading mobile webpage

[twitter:card] 813055465324056576: Downloading main script

ERROR: Unable to extract guest token; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Traceback (most recent call last):

  File "/home/travis/build/Rudloff/alltube/vendor/rg3/youtube-dl/youtube_dl/YoutubeDL.py", line 784, in extract_info

    ie_result = ie.extract(url)

  File "/home/travis/build/Rudloff/alltube/vendor/rg3/youtube-dl/youtube_dl/extractor/common.py", line 437, in extract

    ie_result = self._real_extract(url)

  File "/home/travis/build/Rudloff/alltube/vendor/rg3/youtube-dl/youtube_dl/extractor/twitter.py", line 225, in _real_extract

    formats.extend(self._extract_mobile_formats(username, video_id))

  File "/home/travis/build/Rudloff/alltube/vendor/rg3/youtube-dl/youtube_dl/extractor/twitter.py", line 155, in _extract_mobile_formats

    webpage, 'guest token')

  File "/home/travis/build/Rudloff/alltube/vendor/rg3/youtube-dl/youtube_dl/extractor/common.py", line 800, in _search_regex

    raise RegexNotFoundError('Unable to extract %s' % _name)

RegexNotFoundError: Unable to extract guest token; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

However I can't reproduce the issue on my local computer.
@Rudloff Rudloff mentioned this issue Nov 10, 2017
Closed
@Rudloff
Copy link
Contributor

@Rudloff Rudloff commented Dec 9, 2017

You can find my page dumps here: https://travis-ci.org/Rudloff/alltube/builds/314124111
I had a quick look at them and didn't see anything different from what I get with my computer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
cant-reproduce
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
@snarfed @Rudloff @dstftw @jadrianjohnson-jukin
You can’t perform that action at this time.