soundcloud: cannot download secret embedded playlist

[LaHaine]

$ youtube-dl --verbose "https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/playlists/49676035%3Fsecret_token%3Ds-phZhg"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/playlists/49676035%3Fsecret_token%3Ds-phZhg']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.09.06
[debug] Python version 2.6.6 - Linux-2.6.32-431.23.3.el6.x86_64-x86_64-with-redhat-6.5-Santiago
[debug] Proxy map: {}
[soundcloud:playlist] 49676035: Downloading playlist
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update.
File "/home/me/bin/youtube-dl/youtube_dl/extractor/common.py", line 211, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/home/me/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1244, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib64/python2.6/urllib2.py", line 397, in open
response = meth(req, response)
File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.6/urllib2.py", line 435, in error
return self._call_chain(_args)
File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
result = func(_args)
File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

[dequis]

Works for me?

[juancri]

Works for me too. Please update youtube-dl to the last version and try again.

[LaHaine]

That's because that playlist was changed on soundcloud from private to public since the bug report. But here's another one:
% youtube-dl --verbose "https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/167464311%3Fsecret_token%3Ds-IcRLW"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/167464311%3Fsecret_token%3Ds-IcRLW']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.09.16.1
[debug] Python version 2.6.6 - Linux-2.6.32-431.29.2.el6.x86_64-x86_64-with-redhat-6.5-Santiago
[debug] Proxy map: {}
[soundcloud] 167464311: Downloading info JSON
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update.
File "/afs/ifh.de/user/w/waschk/public/bin/youtube-dl/youtube_dl/extractor/common.py", line 211, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/afs/ifh.de/user/w/waschk/public/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1264, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib64/python2.6/urllib2.py", line 397, in open
response = meth(req, response)
File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.6/urllib2.py", line 435, in error
return self._call_chain(_args)
File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
result = func(_args)
File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

[LaHaine]

It is fixed now for a single track, but not for a secret playlist with more than one track, e.g. this one with a fresh checkout from git after the merge of that commit:

./youtube-dl --verbose "https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/playlists/51064053%3Fsecret_token%3Ds-OtAhG"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/playlists/51064053%3Fsecret_token%3Ds-OtAhG']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.09.16.1
[debug] Python version 2.6.6 - Linux-2.6.32-431.29.2.el6.x86_64-x86_64-with-redhat-6.5-Santiago
[debug] Proxy map: {}
[soundcloud:playlist] 51064053: Downloading playlist
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update.
File "./youtube-dl/youtube_dl/extractor/common.py", line 211, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 1264, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib64/python2.6/urllib2.py", line 397, in open
response = meth(req, response)
File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.6/urllib2.py", line 435, in error
return self._call_chain(_args)
File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
result = func(_args)
File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

[dequis]

On it.

Not sure what to do to create a test case for a private playlist...

[dequis]

Submitted the thing without tests, hope that's not a requirement.

[zommuter]

Got a new one, https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/playlists/948472102%3Fsecret_token%3Ds-16wpL currently opens fine

youtube-dl --verbose https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/playlists/948472102%3Fsecret_token%3Ds-16wpL
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/playlists/948472102%3Fsecret_token%3Ds-16wpL']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2020.01.15
[debug] Python version 3.6.0 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg N-91977-ge33b28cc79, ffprobe N-91977-ge33b28cc79
[debug] Proxy map: {}
[soundcloud:playlist] 948472102: Downloading playlist
[download] Downloading playlist: Hp 3 - Harry Potter und der Gefangene von Askaban
[soundcloud:playlist] playlist Hp 3 - Harry Potter und der Gefangene von Askaban: Collected 23 video ids (downloading 23 of them)
[download] Downloading video 1 of 23
[soundcloud] 730840669: Downloading info JSON
WARNING: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
[soundcloud] 730840669: Downloading info JSON
[soundcloud] 730840669: Downloading track url
[soundcloud] None: Downloading webpage
[soundcloud] None: Downloading webpage
[soundcloud] 730840669: Downloading track url
[soundcloud] None: Downloading webpage
[soundcloud] None: Downloading webpage
[soundcloud] 730840669: Downloading webpage
WARNING: Unable to download webpage: HTTP Error 401: Unauthorized
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "c:\python36\lib\site-packages\youtube_dl\YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "c:\python36\lib\site-packages\youtube_dl\extractor\common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "c:\python36\lib\site-packages\youtube_dl\extractor\soundcloud.py", line 519, in _real_extract
    return self._extract_info_dict(info, full_title, token, version)
  File "c:\python36\lib\site-packages\youtube_dl\extractor\soundcloud.py", line 441, in _extract_info_dict
    self._sort_formats(formats)
  File "c:\python36\lib\site-packages\youtube_dl\extractor\common.py", line 1327, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

[LaHaine]

I can confirm that the fix from commit 2a5c26c fixes the test case from @zommuter .