Add support for Adobe Pass Auth TV Provider YouTubeTV

[aretwo]

---

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.12.14. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2017.12.14

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

---

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

you http://watchdisneyxd.go.com/atomic-puppet/video/vdka4074861/01/26-finale-a-finale-b --ap-mso YouTubeTV --ap-username PRIV --ap-password PRIV -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['http://watchdisneyxd.go.com/atomic-puppet/video/vdka4074861/01/26-finale-a-finale-b', '--ap-mso', 'YouTubeTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-v']
Usage: you [OPTIONS] URL [URL...]

you: error: Unsupported TV Provider, use --ap-list-mso to get a list of supported TV Providers

---

Would like to request support for YouTubeTV as an Adobe Pass MSO option. Running --ap-mso does not recognize YouTubeTV as an option that can be used. Going by the providers list https://sp.auth.adobe.com/adobe-services/config/ABC it should be able to use the service. If an account is needed, I do have an invite so access can be had.

[mfocht]

Want to chime in here. I'm not sure exactly how to do this, but I may be able to help a bit. The login page for youtube tv is the same as the login for the standard youtube login. Hopefully, a good deal of that code could be reused.

[thieverycorp]

Looking forward to this feature

[arghc]

Is a YoutubeTV account still needed? I've got an active account...

[joeynotjoe]

I am really looking forward to this, as well.

[aretwo]

I'm still hoping that this will be worked on. Keeping an account saved in case somebody wants to pick it up.

[arcanez]

#15926

[go2tom42]

#15926

Being able to download from tv.youtube.com and adding YouTube TV as a MSO is two separate issues

I blindly fiddled with adobepass.py and got nowhere. Maybe someone with skills will give it a shot

When you login manually in a browser it goes to this order

1. https://youtube.auth-gateway.net/saml/module.php/authbypass/lastbookend.php?AuthState
2. https://accounts.youtube.com/accounts/SetSID
3. https://accounts.youtube.com/accounts/SetSID?ssdc
4. https://accounts.google.com/signin/v2/challenge/pwd?client_id
5. https://accounts.google.com/signin/oauth/identifier?client_id
6. https://youtube.auth-gateway.net/saml/module.php/authbypass/firstbookend.php?AuthState
7. https://sp.auth.adobe.com/adobe-services/authenticate/saml
8. Video plays

This might be useless
https://youtu.be/C-g4Y1pUE24

[dellis87]

Is a YoutubeTV account still needed? I've got an active account...

I've got one as well.

[connorwalks]

Is there any potential update on this? Would love to use youtube-dl instead of browser plugins but unfortunately many sites won't simply accept a cookie and require MSO so I can't download from there. Haven't seen any update on this in months

[handcraftedbits]

Like others I've been unable to use YouTube TV as a TV Provider, but I've found a really simple solution that lets me use my login cookies, and hopefully it will point the youtube-dl devs (I'm not much of a Python person myself...) in the direction of a "cookie ap-mso provider" approach that will work for (hopefully) all unsupported providers:

1. Log in to the on demand video site (e.g., Disney Junior).
2. From the same tab, export all cookies, not just the ones for the current tab, using a cookie exporter plugin. I use this one in Chrome. Save it as e.g., cookies.txt. I find it helpful to do this in an Incognito/private browser window as it keeps the number of cookies down to the bare minimum ones (I've seen youtube-dl complain about the format of cookies.txt based on certain unrelated cookies).
3. Clone the youtube-dl repo.

This is where "your mileage may vary." I can detail the change I made for Disney (go) sites, and I presume it's very similar for other extractors. Later on I'll discuss what I think is the "real" solution -- this is just a hack that'll get you going until it's properly fixed.

4. Edit youtube_dl/extractor/go.py.
5. Search for self._extract_mvpd_auth. This is where youtube-dl is attempting to sign in to the Adobe Pass site. We're going to bypass that check because the cookies we're providing should be sufficient, it's merely that youtube-dl isn't considering that we can simply bypass the login if the cookies are available. Here's a snippet of what it looks like in go.py to give some context:

if video_data.get('accesslevel') == '1':
    requestor_id = site_info.get('requestor_id', 'DisneyChannels')
    resource = site_info.get('resource_id') or self._get_mvpd_resource(
        requestor_id, title, video_id, None)
    auth = self._extract_mvpd_auth(
        url, video_id, requestor_id, resource)
    data.update({
        'token': auth,
        'token_type': 'ap',
        'adobe_requestor_id': requestor_id,
    })
else:
    self._initialize_geo_bypass({'countries': ['US']})

What this block is doing, as best as I can tell, is seeing if a video is considered "protected" (accesslevel == '1') and starting the SSO process. Otherwise, it just continues on with the download. What gave me the idea to look here is that youtube-dl has no problems downloading videos from Disney if they aren't protected (and each show has a couple of them available at any time), so the approach here is to simply make it appear as if all videos are unprotected. To do so:

6. Change if video_data.get('accesslevel') == '1' to something that should never be true e.g., if video_data.get('accesslevel') == '1000'.
7. Delete youtube_dl/extractor/go.pyc (might not be necessary, just me being paranoid I guess).
8. Run make from the top of the source tree.
9. Make sure you don't have any credentials cached -- rm -rf ~/.cache/youtube-dl
10. (From the top of the source tree) run ./youtube-dl --cookies <path_to_cookies.txt> <url, etc.>
11. Success!

I can confirm that this works with YouTube TV as my provider and the Disney video sites (Disney, Disney Junior, etc.). This is obviously a hack, but it points us in the direction of...

A Proper Solution

What I "think" the proper solution for providers like YouTube TV that aren't supported (and apparently won't be any time soon) is to make a generic "cookie" MSO provider, e.g.:

youtube-dl --ap-mso cookie --cookies <path_to_cookies.txt> <url, etc.>`

This flag would be used to indicate that _extract_mvpd_auth etc. shouldn't be called because the provided cookies are sufficient to bypass authorization checks.

Alternately, perhaps _extract_mvpd_auth could be smart enough to see if the proper cookies are available and use them.

Thoughts? I can certainly try to make a PR for this, though Python isn't really my choice of language.

[harryt04]

@handcraftedbits I was very eager to see if your solution worked. I followed the steps exactly, triple checked, even. But when I try it, this happens. Can you see if you can download this URL from your machine after performing those steps? Perhaps there was another change in youtube-dl that broke your workaround?

https://tv.youtube.com/watch/yOGRebatQvg

./youtube-dl --cookies ./yttv.txt https://tv.youtube.com/watch/yOGRebatQvg
[generic] yOGRebatQvg: Requesting header
WARNING: Falling back on generic information extractor.
[generic] yOGRebatQvg: Downloading webpage
[generic] yOGRebatQvg: Extracting information
ERROR: Unsupported URL: https://tv.youtube.com/watch/yOGRebatQvg

[dellis87]

You will need to go to the individual provider website. It only works for me on disney and not abc but it’s something!

…

Sent from my iPhone

On Feb 6, 2020, at 7:11 PM, Harrison Thomas ***@***.***> wrote:  @handcraftedbits I was very eager to see if your solution worked. I followed the steps exactly, triple checked, even. But when I try it, this happens. Can you see if you can download this URL from your machine after performing those steps? Perhaps there was another change in youtube-dl that broke your workaround? https://tv.youtube.com/watch/yOGRebatQvg ./youtube-dl --cookies ./yttv.txt https://tv.youtube.com/watch/yOGRebatQvg [generic] yOGRebatQvg: Requesting header WARNING: Falling back on generic information extractor. [generic] yOGRebatQvg: Downloading webpage [generic] yOGRebatQvg: Extracting information ERROR: Unsupported URL: https://tv.youtube.com/watch/yOGRebatQvg — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[harryt04]

@dellis87 How do you determine the provider for a show? Like I'm trying to download the bachelor, for example. Is there a way to tell on YoutubeTV's website to see who the 'individual provider' is?

[dellis87]

It would be whatever channel that show is on. Like the bachelor is ABC I think.

…

Sent from my iPhone

On Feb 6, 2020, at 8:18 PM, Harrison Thomas ***@***.***> wrote:  @dellis87 How do you determine the provider for a show? Like I'm trying to download the bachelor, for example. Is there a way to tell on YoutubeTV's website to see who the 'individual provider' is? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[handcraftedbits]

@harryt04 , in my example I was referring to using YouTube TV credentials to download a Disney video stream. It's very likely that youtube-dl doesn't support video streams originating from YouTube TV itself, and that's why you're getting an error. The example I listed is only valid for the go extractor, which is supposed to handle the Disney family of sites, and I would think ABC as well, but apparently not. Like I mentioned in my workaround, there really needs to be a more generic solution (like a "cookie" MSO provider), but I'm not familiar enough with the youtube-dl internals to take a stab at that.

@dellis87 , just curious, but what extractor does youtube-dl say it's using when you download from an ABC site? Is it not go?

[dellis87]

It is go.py. It gives error about not being able to find the files required for download or something like that.