spankbang.com support is broken

[grill05]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2019.11.28
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dl -v https://spankbang.com/2q2ry/video/allura+5
[debug] System config: []
[debug] User config: [u'--external-downloader', u'aria2c', u'--hls-prefer-native']
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://spankbang.com/2q2ry/video/allura+5']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.11.28
[debug] Python version 2.7.13 (CPython) - Linux-4.19.72+-x86_64-with-debian-9.11
[debug] exe versions: ffprobe N-49006-gd81913e680-static, phantomjs 2.1.1
[debug] Proxy map: {}
[SpankBang] 2q2ry: Downloading webpage
ERROR: An extractor error has occurred. (caused by KeyError(u'sb_csrf_session',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/tinku6152112231/.python_scripts/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/home/tinku6152112231/.python_scripts/bin/youtube-dl/youtube_dl/extractor/spankbang.py", line 97, in _real_extract
    'https://spankbang.com')['sb_csrf_session'].value
KeyError: u'sb_csrf_session'
Traceback (most recent call last):
  File "/home/tinku6152112231/.python_scripts/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/home/tinku6152112231/.python_scripts/bin/youtube-dl/youtube_dl/extractor/common.py", line 543, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
ExtractorError: An extractor error has occurred. (caused by KeyError(u'sb_csrf_session',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

Spankbang.com support is broken since today. They have changed their site layout, so that might be the reason. youtube-dl cannot find/download any spankang.com urls now. For example, this url fails: https://spankbang.com/2q2ry/video/allura+5

[grill05]

I've investigated a bit more, and it seems its a simple renaming of variables.
sb_csrf_session needs to change to sb_session , and then the rest of the extractor works perfectly.

[vicm3]

Works for me.

diff --git a/youtube_dl/extractor/spankbang.py b/youtube_dl/extractor/spankbang.py
index e040ada29..48d4b350e 100644
--- a/youtube_dl/extractor/spankbang.py
+++ b/youtube_dl/extractor/spankbang.py
@@ -93,18 +93,18 @@ class SpankBangIE(InfoExtractor):
                 r'data-streamkey\s*=\s*(["\'])(?P<value>(?:(?!\1).)+)\1',
                 webpage, 'stream key', group='value')
 
-            sb_csrf_session = self._get_cookies(
-                'https://spankbang.com')['sb_csrf_session'].value
+            sb_session = self._get_cookies(
+                'https://spankbang.com')['sb_session'].value
 
             stream = self._download_json(
                 'https://spankbang.com/api/videos/stream', video_id,
                 'Downloading stream JSON', data=urlencode_postdata({
                     'id': stream_key,
                     'data': 0,
-                    'sb_csrf_session': sb_csrf_session,
+                    'sb_session': sb_session,
                 }), headers={
                     'Referer': url,
-                    'X-CSRFToken': sb_csrf_session,
+                    'X-CSRFToken': sb_session,
                 })
 
             for format_id, format_url in stream.items():

[Luppakorva]

sb_session is not enough. Spankbang seemed to have changed into competely different video playback method in the past few days.

[grill05]

they introduced new changes a couple of days after my first report. You need some more variable substitutions now

stream_url_720p -> 720p
stream_url_480p -> 480p
stream_url_320p -> 320p
stream_url_240p -> 240p

and then the extractor will work fine

[Luppakorva]

@grill05 any chance you could give a bit more detailed instructions / example about that? I can look into it myself but no idea what you mean by that.

[bobfoss]

This works for now..

--- a/youtube_dl/extractor/spankbang.py
+++ b/youtube_dl/extractor/spankbang.py
@@ -93,22 +93,22 @@
                 r'data-streamkey\s*=\s*(["\'])(?P<value>(?:(?!\1).)+)\1',
                 webpage, 'stream key', group='value')

-            sb_csrf_session = self._get_cookies(
-                'https://spankbang.com')['sb_csrf_session'].value
+            sb_session = self._get_cookies(
+                'https://spankbang.com')['sb_session'].value

             stream = self._download_json(
                 'https://spankbang.com/api/videos/stream', video_id,
                 'Downloading stream JSON', data=urlencode_postdata({
                     'id': stream_key,
                     'data': 0,
-                    'sb_csrf_session': sb_csrf_session,
+                    'sb_session': sb_session,
                 }), headers={
                     'Referer': url,
-                    'X-CSRFToken': sb_csrf_session,
+                    'X-CSRFToken': sb_session,
                 })

             for format_id, format_url in stream.items():
-                if format_id.startswith(STREAM_URL_PREFIX):
+                if format_id in ('240p','320p','480p','720p','1080p','4k'):
                     if format_url and isinstance(format_url, list):
                         format_url = format_url[0]
                     extract_format(

[Luppakorva]

@bobfoss thanks, works. Though it doesn't seem to really download 720p and higher qualities.

[AdzFM]

I'm using the standalone downloader (the last updated version is Nov. 28th) , and am unable to download. Is there a way I can apply this fix to the standalone version to work?

[charmingdingo]

Including this fix in the next release would be much appreciated! A big thank you to all youtube-dl contributors; your hard work is much appreciated! <3

[inversezebratransform]

Format_id no longer has stream_url prefix:

+++ b/youtube_dl/extractor/spankbang.py
@@ -112,7 +112,7 @@ class SpankBangIE(InfoExtractor):
                     if format_url and isinstance(format_url, list):
                         format_url = format_url[0]
                     extract_format(
-                        format_id[len(STREAM_URL_PREFIX):], format_url)
+                        format_id, format_url)

         self._sort_formats(formats)

This allows format detection to work, so download can automatically select highest 720p resolution instead of lowest 240p resolution.

[Luppakorva]

@inversezebratransform thank you! Works perfectly.
@vicm3 you should add this into the pull request.

[vicm3]

@inversezebratransform thank you! Works perfectly.
@vicm3 you should add this into the pull request.

Added to pull request. Thanks to all for sharing.

[TheBeekeepersApprentice]

Can someone slowly explain why do I get this error ?

Traceback (most recent call last):
File "main.py", line 19, in
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp243932
xz\build\youtube_dl_init_.py", line 474, in main
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp243932
xz\build\youtube_dl_init_.py", line 58, in _real_main
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp243932
xz\build\youtube_dl\options.py", line 904, in parseOpts
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp243932
xz\build\youtube_dl\options.py", line 87, in _readUserConf
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp243932
xz\build\youtube_dl\options.py", line 52, in _readOptions
File "C:\Python\Python34\lib\shlex.py", line 276, in split
File "C:\Python\Python34\lib\shlex.py", line 266, in next
File "C:\Python\Python34\lib\shlex.py", line 93, in get_token
File "C:\Python\Python34\lib\shlex.py", line 169, in read_token
ValueError: No closing quotation

That's using bobfoss' solution from above.

[Luppakorva]

@TheBeekeepersApprentice based to the valueError: No closing quotation:

You are missing ' or some other character somewhere in the code. You definitely have typo in it.

Go through it, compare it to the original / bobfoss solution and then try again.

[TheBeekeepersApprentice]

I really can't see any difference, I do a simple copy&paste in my config file (which is otherwise empty)... I thought maybe that line would be the issue :

r'data-streamkey\s*=\s*(["'])(?P(?:(?!\1).)+)\1'

with its three ' and single ", but clearly it's not the case for other people...

[Luppakorva]

Do you make copy paste of the full example? It will not work.

Pay attention to - and +

• means, replace that line with the following + line