Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compat_urllib_parse_unquote breaks google storage signed objects #22704

Open
legraphista opened this issue Oct 14, 2019 · 0 comments · May be fixed by #22708
Open

compat_urllib_parse_unquote breaks google storage signed objects #22704

legraphista opened this issue Oct 14, 2019 · 0 comments · May be fixed by #22708

Comments

@legraphista
Copy link

@legraphista legraphista commented Oct 14, 2019

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dl version 2019.09.28
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

$ youtube-dl https://cdn.kamua.com/view/media/4a14378d-7b9c-43b9-aafe-f34e405d32c9 --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'https://cdn.kamua.com/view/media/4a14378d-7b9c-43b9-aafe-f34e405d32c9', u'--verbose']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.09.12.1
[debug] Python version 2.7.16 (CPython) - Linux-5.0.0-31-generic-x86_64-with-Ubuntu-19.04-disco
[debug] exe versions: ffmpeg N-94689-g3e93d7ce33, ffprobe N-94689-g3e93d7ce33
[debug] Proxy map: {}
[generic] 4a14378d-7b9c-43b9-aafe-f34e405d32c9: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 4a14378d-7b9c-43b9-aafe-f34e405d32c9: Downloading webpage
[generic] 4a14378d-7b9c-43b9-aafe-f34e405d32c9: Extracting information
[download] Downloading playlist: video
[generic] playlist video: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://storage.googleapis.com/kamua-media/4a14378d-7b9c-43b9-aafe-f34e405d32c9/720p.mp4?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=kamua-api-storage@kamua-238810.iam.gserviceaccount.com/20191014/auto/storage/goog4_request&X-Goog-Date=20191014T084411Z&X-Goog-Expires=86401&X-Goog-SignedHeaders=host&X-Goog-Signature=4d1beafbf85d9ba5567d941ee6bd0e6bcf083a64b0f03cb4355b537e1445812611da4fc2624d71a24d727c2b21d4328d061b23a2eb286c46fa53a9af15ceb82bdbd53a55c63269b24b617bff34f502098c10e321abbdcf5ee7e529b2b4c6f2dff5679a1d18e459f7afd6fc3bd055d5c5daa02aba52e8facc9d61bd7c16d7374c75c63bfc828f95b78b26504428cd81c81309dd9b1c01eeaae748a2c651217f91d6d62e07c67ca69860c074e9d0129c2fe219cfb49de8988846fc016f4889a4bc921708c61181b9acc2312d5e6d49d6ffb0db1571f679c62a6af29234c5217fedbf20c8276d85e7b858278d028de9828c61caaa6ec8bbdbe2aa82ddda62336e33'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1917, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1856, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 341, in real_download
    establish_connection()
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 109, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2229, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

Description

Page https://cdn.kamua.com/view/media/4a14378d-7b9c-43b9-aafe-f34e405d32c9 contains a URL link to a signed google cloud storage video.
The src of the <video> is https://storage.googleapis.com/kamua-media/4a14378d-7b9c-43b9-aafe-f34e405d32c9%2F720p.mp4?...[signature]... but after the webpage passes through compat_urllib_parse_unquote, the url transforms in https://storage.googleapis.com/kamua-media/4a14378d-7b9c-43b9-aafe-f34e405d32c9/720p.mp4?...[signature]... (%2F -> /) which breaks the signature.

I think the strings in quotes on the webpage should not have their characters escaped

@legraphista legraphista linked a pull request that will close this issue Oct 14, 2019
5 of 9 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

1 participant
You can’t perform that action at this time.