add support to daisuki.net #4738

fabiopr2 · 2015-01-18T04:57:33Z

UnsupportedError: Unsupported URL: http://www.daisuki.net/anime/watch/ALDNOAHZERO/hTc

jaimeMF · 2015-01-18T10:19:16Z

This videos is not georestricted: http://www.daisuki.net/anime/watch/TheIdolMasterCG/RUc

fabiopr2 · 2015-01-18T13:50:41Z

[debug] Command-line args: ['--format', '480p', '--all-subs', '--sub-format', 'ass', '--username', u'PRIVATE', '--password', u'PRIVATE', '--verbose', '--rate-limit', '1m', 'http://www.daisuki.net/anime/watch/TheIdolMasterCG/RUc', '-o', 'test.mp4']
[debug] Encodings: locale ISO-8859-1, fs ISO-8859-1, out ISO-8859-1, pref ISO-8859-1
[debug] youtube-dl version 2015.01.16
[debug] Python version 2.7.3 - Linux-3.2.0-75-generic-x86_64-with-Ubuntu-12.04-precise
[debug] exe versions: avconv 0.8.16-4, avprobe 0.8.16-4, ffmpeg 0.8.16-4, ffprobe 0.8.16-4, rtmpdump 2.4
[debug] Proxy map: {}
[generic] RUc: Requesting header
WARNING: Falling back on generic information extractor.
[generic] RUc: Downloading webpage
[generic] RUc: Extracting information
ERROR: Unsupported URL: http://www.daisuki.net/anime/watch/TheIdolMasterCG/RUc
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 670, in _real_extract
doc = parse_xml(webpage)
File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 1504, in parse_xml
tree = xml.etree.ElementTree.XML(s.encode('utf-8'), **kwargs)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
parser.feed(text)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1643, in feed
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1507, in _raiseerror
raise err
ParseError: mismatched tag: line 44, column 2
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 600, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 260, in extract
return self._real_extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1075, in _real_extract
raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://www.daisuki.net/anime/watch/TheIdolMasterCG/RUc

i have try but get same error

phihag · 2015-01-18T14:15:26Z

Of course you do; there have been no changes. @jaimeMF just proposed a better test video for this bug report.

fb39ca4 · 2015-07-06T22:23:34Z

Here is a forum thread that has some useful information.

http://forum.videohelp.com/threads/369812-How-to-Rip-From-Daisuki-net

Basically, we want to get the URL for a manifest.f4m file, substitute a few strings in that URL, and this will get us the URL for a m3u8 playlist that can then be played.

Now, the trick is getting that URL. It appears to be computed by the site's Flash player, which has the following intiialization code inline in the webpage for an episode:

var flashvars = {
    's':"ServerControlled",
    "mv_id":"",
    "device_cd":"1",
    "ss1_prm":"59",
    "ss2_prm":"11472",
    "ss3_prm":"US",

    //?????
    'country':'/common2/pages/anime/swf/country.xml',
    'serverIP':'/common2/pages/anime/swf/serverip.xml',
    'init':'/fastAPI/bgn/init/',
};

var params = {
    allowScriptAccess:"always", 
    allowfullscreen :'true',
    wmode :"transparent"
};
var attrs = {};
swfobject.embedSWF('http://img.daisuki.net/common2/pages/anime/swf/bngn_player_001.swf', "bngn-player", "854", "480", "12.0.0", null, flashvars, params, attrs);

The interesting variables are ss1_prm and ss2_prm, which appear to determine the show and episode. I've been looking at the decompiled code from the flash player, but can't seem to follow what it is doing with them to get the manifest URL.

yan12125 · 2017-02-27T11:01:37Z

For those who may be interesteed: I have a WIP at https://github.com/yan12125/youtube-dl/tree/wip-daisuki. Remaining works:

~~Add DER parsing to replace the PyCrypto implementation~~ Fixed.
~~Playlist support~~ Done.
Subtitles support Partially done, see #12303

Used in daisuki.net (#4738)

Used in daisuki.net (#4738) Both are adapted from public domain PyCrypto: https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py

yan12125 · 2017-02-28T14:30:37Z

daisuki will be supported in the next version. Note that --convert-subs does not support such multilingual TTML yet.

starchivore · 2017-02-28T18:46:22Z

Awesome, many thanks for this one and it's working fine for the downloads via proxy so far.

Is there a way to tweak the code a little bit for --geo-bypass-country US to work successfully because the speed via proxy ain't good enough for streaming in real-time with the mpv+youtube-dl combo?

youtube-dl -v --geo-bypass-country US -F http://www.daisuki.net/us/en/anime/watch.DRAGONBALLSUPERUniverseSurvivalsaga.16324.html
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--geo-bypass-country', 'US', '-F', 'http://www.daisuki.net/us/en/anime/watch.DRAGONBALLSUPERUniverseSurvivalsaga.16324.html']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.28
[debug] Python version 3.4.4 - Windows-Vista-6.0.6002-SP2
[debug] exe versions: ffmpeg N-82759-g1f5630a-Sherpya, ffprobe N-82759-g1f5630a-Sherpya
[debug] Proxy map: {}
[debug] Using fake IP 3.188.183.82 (US) as X-Forwarded-For.
[Daisuki] 16324: Downloading webpage
[Daisuki] 16324: Downloading JSON metadata
[Daisuki] 16324: Downloading m3u8 information
ERROR: Failed to download m3u8 information: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you  are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp_o63hpm_\build\youtube_dl\extractor\common.py", line 496, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp_o63hpm_\build\youtube_dl\YoutubeDL.py", line 2093, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

youtube-dl -v --hls-prefer-native --geo-bypass-country US -F http://www.daisuki.net/us/en/anime/watch.DRAGONBALLSUPERUniverseSurvivalsaga.16324.html

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--hls-prefer-native', '--geo-bypass-country', 'US', '-F', 'http://www.daisuki.net/us/en/anime/watch.DRAGONBALLSUPERUniverseSurvivalsaga.16324.html']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.02.28
[debug] Python version 3.4.4 - Windows-Vista-6.0.6002-SP2
[debug] exe versions: ffmpeg N-82759-g1f5630a-Sherpya, ffprobe N-82759-g1f5630a-Sherpya
[debug] Proxy map: {}
[debug] Using fake IP 3.172.221.45 (US) as X-Forwarded-For.
[Daisuki] 16324: Downloading webpage
[Daisuki] 16324: Downloading JSON metadata
[Daisuki] 16324: Waiting for 5 seconds
[Daisuki] 16324: Downloading JSON metadata (try #2)
[Daisuki] 16324: Waiting for 5 seconds
[Daisuki] 16324: Downloading JSON metadata (try #3)
[Daisuki] 16324: Downloading m3u8 information
ERROR: Failed to download m3u8 information: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp_o63hpm_\build\youtube_dl\extractor\common.py", line 496, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp_o63hpm_\build\youtube_dl\YoutubeDL.py", line 2093, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

dstftw · 2017-02-28T18:54:16Z

You totally misunderstand how geo bypass work. --geo-bypass-country US itself is complete solution and there is nothing to tweak. If it does not work then it's technically impossible to bypass geo restriction this way for this site.

yan12125 · 2017-02-28T19:04:46Z

Is there a way to tweak the code a little bit

No. --geo-bypass-country is not a universal solution. It works for only a few websites.

jaimeMF added the site-support-request label Jan 18, 2015

jaimeMF mentioned this issue Dec 6, 2015

Site Support Request: Daisuke #7776

Closed

dstftw mentioned this issue Jul 10, 2016

Site support request: daisuki.net #10060

Closed

4 of 8 tasks complete

Kcchouette mentioned this issue Feb 26, 2017

[brightcove] Daisuki ripping #2486

Closed

yan12125 mentioned this issue Feb 27, 2017

XML parsing error on http://www.daisuki.net/anime/watch/MUSHISHI_Next_Passage/gNc #3186

Closed

yan12125 added a commit that referenced this issue Feb 28, 2017

[aes] Add aes_cbc_encrypt

c9619f0

Used in daisuki.net (#4738)

yan12125 added a commit that referenced this issue Feb 28, 2017

[utils] Add pkcs1pad

f48409c

Used in daisuki.net (#4738)

yan12125 added a commit that referenced this issue Feb 28, 2017

[utils] Add bytes_to_long() and long_to_bytes()

0a5445d

Used in daisuki.net (#4738) Both are adapted from public domain PyCrypto: https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py

yan12125 closed this in 9bd05b5 Feb 28, 2017

yan12125 added a commit that referenced this issue Feb 28, 2017

[daisuki] Add subtitles (#4738)

Unverified

This user has not uploaded their public key yet.

GPG key ID: 7F902A182457CA23 Learn about signing commits

Loading status checks…

7c4aa6f

yan12125 mentioned this issue Feb 28, 2017

Support converting multilingual TTML to srt #12303

Open

4 of 8 tasks complete

starchivore mentioned this issue Feb 28, 2017

Feature: Generic geo restriction bypass mechanism based on faking X-Forwarded-For streamlink/streamlink#615

Closed

2 of 4 tasks complete

dstftw mentioned this issue Oct 15, 2017

Is there a way to obtain any of the --get-x options and download the actual file as well? Question 2: how to bypass geo restrictions? #14508

Closed

0 of 7 tasks complete

ytdl-org / youtube-dl

add support to daisuki.net #4738

add support to daisuki.net #4738

fabiopr2 commented Jan 18, 2015

jaimeMF commented Jan 18, 2015

fabiopr2 commented Jan 18, 2015

phihag commented Jan 18, 2015

fb39ca4 commented Jul 6, 2015

yan12125 commented Feb 27, 2017 •

edited

yan12125 commented Feb 28, 2017

starchivore commented Feb 28, 2017

dstftw commented Feb 28, 2017

yan12125 commented Feb 28, 2017 •

edited

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

add support to daisuki.net #4738

add support to daisuki.net #4738

Comments

fabiopr2 commented Jan 18, 2015

jaimeMF commented Jan 18, 2015

fabiopr2 commented Jan 18, 2015

phihag commented Jan 18, 2015

fb39ca4 commented Jul 6, 2015

yan12125 commented Feb 27, 2017 • edited

yan12125 commented Feb 28, 2017

starchivore commented Feb 28, 2017

dstftw commented Feb 28, 2017

yan12125 commented Feb 28, 2017 • edited

Essential cookies

Always active

Analytics cookies

yan12125 commented Feb 27, 2017 •

edited

yan12125 commented Feb 28, 2017 •

edited