[disneynow.go.com] Add Support - move from watchdisney to new site

[RuneMagle]

• I've verified and I assure that I'm running youtube-dl 2018.04.25
• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones
• Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

C:\Users\Desktop\youtube-dl>youtube-dl.exe -F http://disneynow.go.com/shows/bizaardvark/season-02/episode-21-her-me-and-hermie/vdka4378311 -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'http://disneynow.go.com/shows/bizaardvark/season-02/episode-21-her-me-and-hermie/vdka4378311', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2018.04.25
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: ffmpeg N-86911-gb664d1f, ffprobe N-86911-gb664d1f, rtmpdump 2.4
[debug] Proxy map: {}
[generic] vdka4378311: Requesting header
WARNING: Falling back on generic information extractor.
[generic] vdka4378311: Downloading webpage
[generic] vdka4378311: Extracting information
ERROR: Unsupported URL: http://disneynow.go.com/shows/bizaardvark/season-02/episode-21-her-me-and-hermie/vdka4378311
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpad1hmn8r\build\youtube_dl\YoutubeDL.py", line 789, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpad1hmn8r\build\youtube_dl\extractor\common.py", line 440, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpad1hmn8r\build\youtube_dl\extractor\generic.py", line 3182, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: http://disneynow.go.com/shows/bizaardvark/season-02/episode-21-her-me-and-hermie/vdka4378311
...
<end of log>

---

• Single video: http://disneynow.go.com/shows/bizaardvark/season-02/episode-21-her-me-and-hermie/vdka4378311
• Single video: http://disneynow.go.com/shows/bunkd/season-02/episode-14-mud-fight/vdka3736279

---

Description of issue

watchdisneychannel.go.com, watchdisneyxd.go.com and watchdisneyjunior.go.com are now all part of disneynow.go.com. Is it possible to add support, so that it works with the new site?

[tv21]

I got it to work by modifying the go.py extractor, although I was basically just guessing, so this may not be the correct way to do it:

Under the section that begins

class GoIE(AdobePassIE):
    _SITE_INFO = {

I added

        'disneynow': {
            'brand': '004',
            'requestor_id': 'Disney',
        },

And under
_TESTS = [{
I scrolled down to where the 'only_matching': True, tests are and added a new one in the same format using a valid "http://disneynow.go.com/shows/ ..." URL. So, using one of the Single Video links given by the creator of this issue...

    }, {
        'url': 'http://disneynow.go.com/shows/bizaardvark/season-02/episode-21-her-me-and-hermie/vdka4378311',
        'only_matching': True,
    }, {

This was just a wild guess because I am not a programmer and I don't know Python, but surprisingly it worked when I ran /path/to/youtube-dl-master/youtube_dl/__main__.py followed by a space and the URL (substituting /path/to with the actual path).

[SkiTheSlicer]

@tv21 your fix probably only works for shows that use the 'watchdisneychannel' API endpoint, but not for Junior or XD.

'disneynow' doesn't appear to be its own separate API endpoint, but rather still proxies somehow to the 'watchdisneychannel', 'watchdisneyjunior', and 'watchdisneyxd' API endpoints.

If I edit go.py to just iterate through the list of endpoints until one is successful, that seems to work, but I'm not going to submit a pull request for that because it's just guesswork and not clean code. For anyone who wants to take this a step further, I added an (incorrect) _SITE_INFO entry for 'disneynow' and replaced video_data = self._extract_videos(brand, video_id)[0] with:

if sub_domain == 'disneynow':
    for sub_domain in ['watchdisneychannel', 'watchdisneyjunior', 'watchdisneyxd']:
        site_info = self._SITE_INFO[sub_domain]
        brand = site_info['brand']
        try:
            video_data = self._extract_videos(brand, video_id)[0]
        except IndexError:
            continue
        else:
            break
else:
    video_data = self._extract_videos(brand, video_id)[0]

Edit: I hope that didn't sound condescending @tv21 . Good job getting it that far. Trying it out is the best way to learn.

[fkick]

Thanks @SkiTheSlicer, your patch has allowed me to successfully pull from watchdisneyjunior and XD.

[jdown077]

How do I make these edits on a Mac? I installed youtube-dl with handbrew.

[RuneMagle]

Don't you have Python installed?

[jdown077]

Yes I do, but on Mac I have no idea where the go.py file is located.

[RuneMagle]

Just download the Source Code: https://github.com/rg3/youtube-dl/releases/tag/2018.05.01/

go.py is located in > where you have extracted the source code > youtube_dl >> youtube_dl >>> extractor - and you will find the file in there. You can edit it through Wordpad or similar.

[jdown077]

Thanks! I figured it out and got it working!

[mgicash]

This only works for some videos. I can't get it to work for other videos on the same site, for example:
http://disneynow.go.com/shows/pj-masks/season-02/episode-07-wacky-floats-romeos-disguise/vdka4332783

That video and many others give me this error message

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--ap-mso', u'DTV', u'--ap-username', u'PRIVATE', u'--ap-password', u'PRIVATE', u'--no-check-certificate', u'http://disneynow.go.com/shows/pj-masks/season-02/episode-07-wacky-floats-romeos-disguise/vdka4332783']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2018.05.01
[debug] Python version 2.7.0 (CPython) - Windows-post2008Server-6.2.9200
[debug] exe versions: none
[debug] Proxy map: {}
[Go] vdka4332783: Downloading JSON metadata
Traceback (most recent call last):
File "main.py", line 19, in
youtube_dl.main()
File "C:\youtube-dl\youtube_dl_init_.py", line 471, in main
real_main(argv)
File "C:\youtube-dl\youtube_dl_init.py", line 461, in _real_main
retcode = ydl.download(all_urls)
File "C:\youtube-dl\youtube_dl\YoutubeDL.py", line 1993, in download
url, force_generic_extractor=self.params.get('force_generic_extractor', False))
File "C:\youtube-dl\youtube_dl\YoutubeDL.py", line 789, in extract_info
ie_result = ie.extract(url)
File "C:\youtube-dl\youtube_dl\extractor\common.py", line 440, in extract
ie_result = self._real_extract(url)
File "C:\youtube-dl\youtube_dl\extractor\go.py", line 103, in _real_extract
video_data = self._extract_videos(brand, video_id)[0]
IndexError: list index out of range

[mgicash]

Interesting, and you only have SkiTheSlicer's change? Are you running main.py? Are your arguments any different from mine?

Anyway you could post your go.py?

[RuneMagle]

I think it's the first change by "tv21", that does so it only works with the watchdisneychannel API endpoints. But maybe try using SkiTheSlicer's change, and run youtube-dl straight from the source code, and use the command "python setup.py install", that worked for me.

[iamtheddrman]

I was able to use SkiTheSlicer's patch to get publicly-available videos to work, but it does not seem to work with TV-provider-restricted videos. Even after adding my account info, I get "ERROR: Unable to extract post url"

F:\VideosToConvert\youtube-dl-master>python F:\VideosToConvert\youtube-dl-master\youtube_dl\__main__.py --cookies=F:\VideosToConvert\cookies.txt --ap-mso DTV --ap-username [REDACTED] --ap-password [REDACTED] -o F:/VideosToConvert/staging/%(id)s.%(ext)s --verbose http://disneynow.go.com/shows/mickey-and-the-roadster-racers/season-01/episode-19-garage-alone-camp-happy-helpers/vdka4054792
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--cookies=F:\\VideosToConvert\\cookies.txt', u'--ap-mso', u'DTV', u'--ap-username', u'PRIVATE', u'--ap-password', u'PRIVATE', u'-o', u'F:/VideosToConvert/staging/%(id)s.%(ext)s', u'--verbose', u'http://disneynow.go.com/shows/mickey-and-the-roadster-racers/season-01/episode-19-garage-alone-camp-happy-helpers/vdka4054792']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2018.05.09
[debug] Python version 2.7.15 (CPython) - Windows-10-10.0.16299
[debug] exe versions: ffmpeg 4.0, ffprobe 4.0
[debug] Proxy map: {}
[Go] vdka4054792: Downloading JSON metadata
[Go] vdka4054792: Downloading JSON metadata
[Go] VDKA4054792: Downloading Provider Redirect Page
[Go] VDKA4054792: Downloading Provider Login Page
[Go] VDKA4054792: Logging in
ERROR: Unable to extract post url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\extractor\go.py", line 104, in _real_extract
    video_data = self._extract_videos(brand, video_id)[0]
IndexError: list index out of range
Traceback (most recent call last):
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\extractor\common.py", line 503, in extract
    ie_result = self._real_extract(url)
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\extractor\go.py", line 134, in _real_extract
    url, video_id, requestor_id, resource)
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\extractor\adobepass.py", line 1511, in _extract_mvpd_auth
    post_form(mvpd_confirm_page_res, 'Confirming Login')
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\extractor\adobepass.py", line 1360, in post_form
    post_url = self._html_search_regex(r'<form[^>]+action=(["\'])(?P<url>.+?)\1', form_page, 'post url', group='url')
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\extractor\common.py", line 892, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "F:\VideosToConvert\youtube-dl-master\youtube_dl\extractor\common.py", line 883, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract post url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

[Kramerika]

I have been having the same issue using the Windows executable. I tried to apply the fixes above but was unsuccessful. Any chance a fix will be worked into a future update?

[RuneMagle]

There is an easier way. A friend told me this. The old disney websites are still working. You can browse them by going to the websites, and add qa3 in front of the url. Like this: http://qa3.watchdisneychannel.go.com/

New episodes are added there too. It has the old layout and everything. When you find a video, just copy the link, remove the qa3 from it, and youtube-dl should be able to handle the link. :)

[beren12]

Could someone add http://qa3.watchdisneychannel.go.com/ as a supported site so we can continue to download the playlist from it?

[RuneMagle]

It shouldn't be added. It's just an archive site, not the official disneynow. But just find links from the site, and remove qa3 from it when entering it in youtube-dl, then it works fine.

[TheDaChicken]

I have updated to the least version and I am still getting the "generic", for both Disney websites and there not working. Does anyone have a fix for that?

[shirt-dev]

bump

[shirt-dev]

I have tested @SkiTheSlicer's patch and it works perfectly.

[shirt-dev]

@SkiTheSlicer, mind if I open a pull request with your patch? I have the file all ready.