Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding Dish MSO #12000

Closed
wants to merge 2 commits into from
Closed

Adding Dish MSO #12000

wants to merge 2 commits into from

Conversation

gkoelln
Copy link
Contributor

@gkoelln gkoelln commented Feb 6, 2017

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])
  • Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

Adding Dish MSO to adobepass.py
Issue #10111
Issue #11886

gkoelln referenced this pull request in gkoelln/youtube-dl Feb 7, 2017
@gkoelln
[Go] Add support for Adobe Pass
b3d2b79 
In conjunctions with PR #12000, this fixes Issue #11468
@gkoelln
Copy link
Contributor Author

gkoelln commented Feb 20, 2017

If no one is willing to share Dish login info (including myself), is posting traffic for different tests sufficient?

@gkoelln gkoelln mentioned this pull request Mar 21, 2017
Open
4 tasks
@keybounce
Copy link

keybounce commented Mar 23, 2017 •
edited

Works for me on BBCA.
Works for Disney/go.com

Silly BBCA dispenses TT subtitles ("SMPTE-TT Captioning Default") when downloading latest Planet Earth 2 that neither vlc nor mplayer understand.

No help for Science Channel GO (uses a different scheme, based on cookies, and doesn't support Dish for some reason...)

Still fails on Adult Swim (already reported: Bootstraped data issue)

But that's 2 out of 2 where the issue is supporting Dish over adobe's signin system.

EDIT: Clarify:
From yt-dl commit 8a8cc33
git fetch https://github.com/gkoelln/youtube-dl.git Dish
git checkout FETCH_HEAD
git tag -a Dish
git checkout master
git checkout -b merged-dish
git merge Dish
make youtube-dl

worked for me with no problems.

@gkoelln
Copy link
Contributor Author

gkoelln commented Mar 23, 2017

I've tested it successfully on NBC and FOX as well as Disney/Go and BBCA. I'm relying on you (and anyone else who's willing and able) to test, since I recently cancelled Dish in favor of Playstation Vue. (Now if I can just figure out how to set up that MSO...)

@keybounce
Copy link

And it works on ABC/Go (had to update youtube-dl from the 20th to the 24th.)

Now installing it on my system as "tested and better than the default".

@gkoelln
Copy link
Contributor Author

gkoelln commented Mar 24, 2017 •
edited

If someone can try https://github.com/gkoelln/youtube-dl/raw/adobepass/youtube_dl/extractor/adobepass.py with a Dish login, I'm curious if it will work as well as it did for my PSVue login.

EDIT: Edited link.

@keybounce
Copy link

Can you clarify what you want tested there?

@gkoelln
Copy link
Contributor Author

gkoelln commented Mar 25, 2017

I haven't done a PR for it yet, but I'm hoping that this extractor works for all MSOs listed in the dictionary. Unfortunately, I don't have a way of testing it with a Dish login. There's one line in particular that may cause problems.

@gkoelln
Copy link
Contributor Author

gkoelln commented Mar 25, 2017

You might try:

git fetch https://github.com/gkoelln/youtube-dl.git adobepass
git checkout FETCH_HEAD
git tag -a adobepass
git checkout master
git checkout -b merged-adobepass
git merge adobepass
make youtube-dl

@gkoelln
Copy link
Contributor Author

gkoelln commented Mar 27, 2017

Changes have been made to this PR. Dish users, please test this, as I cannot.

@gkoelln
Copy link
Contributor Author

gkoelln commented Apr 3, 2017

More testing needed, please. I can't use the Dish MSO anymore, but I'd love to make this available for anyone who can.

@keybounce
Copy link

keybounce commented Apr 4, 2017 via email

@gkoelln
Copy link
Contributor Author

gkoelln commented Apr 4, 2017 via email

@casual-programmer
Copy link

casual-programmer commented Apr 5, 2017 •
edited

I'm brand new at this and would love to test the Dish MSO. I'm having some issues, though. Here are the commands I'm executing and their results:

git clone https://github.com/rg3/youtube-dl.git (result: successful)
cd youtube-dl
git fetch https://github.com/gkoelln/youtube-dl.git adobepass (result: * branch adobepass -> FETCH_HEAD)
git checkout FETCH_HEAD (result: "You are in 'detached HEAD' state...HEAD is now at deb24bc... [adobepass] Add function process_redirects")
git tag -a adobepass (result: Opens nano, haven't the slightest idea what to do here, so I simply exit)
git checkout master (result: "Warning: you are leaving 5 commits behind...Switched to branch 'master'")
git checkout -b merged-adobepass (result: "Switched to a new branch 'merged-adobepass'")
git merge adobepass (result: "merge: adobepass - not something we can merge")
make (result: successful)
./youtube-dl -v --ap-mso Dish --ap-username SECRET --ap-password SECRET --list-formats http://channel.nationalgeographic.com/water-and-power-a-california-heist/videos/water-power-a-california-heist/ (result: "youtube-dl: error: Unsupported TV Provider")

I know I'm doing something wrong here, but just don't know what.

@brbsix
Copy link

brbsix commented Apr 5, 2017

@casual-programmer

Try this:

git clone https://github.com/rg3/youtube-dl.git
cd youtube-dl
git fetch origin pull/12000/head:Dish
git checkout Dish
make youtube-dl

@casual-programmer
Copy link

Thanks, @brbsix It looks like the MSO parts worked, but the actual download did not. Here's the output:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--ap-mso', u'Dish', u'--ap-username', u'PRIVATE', u'--ap-password', u'PRIVATE', u'http://channel.nationalgeographic.com/parched/videos/global-water-wars1/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.01.10
[debug] Python version 2.7.12 - Linux-4.8.0-45-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: none
[debug] Proxy map: {}
[natgeo] global-water-wars1: Downloading webpage
[natgeo] BVAYhMKQ1yOR: Downloading Provider Redirect Page
[natgeo] BVAYhMKQ1yOR: Downloading Provider Login Page
[natgeo] BVAYhMKQ1yOR: Downloading Provider Login Page
[natgeo] BVAYhMKQ1yOR: Logging in
[natgeo] BVAYhMKQ1yOR: Confirming Login
[natgeo] BVAYhMKQ1yOR: Retrieving Session
ERROR: Unable to download webpage: HTTP Error 401: Unauthorized (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "./youtube-dl/youtube_dl/extractor/common.py", line 406, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 2001, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

@keybounce
Copy link

keybounce commented Apr 6, 2017 via email

@gkoelln
Copy link
Contributor Author

gkoelln commented Apr 6, 2017

I just reverted to the previous working version.

@keybounce
Copy link

That merge conflict: it's two different segments being added to the same place. They are just elseif / then / elseif things, and go in together.

I'm using the 4-17 youtube-dl with the dish MSO with that -- both pieces of code in there.

@keybounce
Copy link

OK, so some more info.

I had been using ce75bd2e2a3e9e6a09e82f07e1533313fc9bb47b, which is the Jan 13th version of Dish MSO. I did not install 01727b79adea089c56b3c938aa94f398c93a2bd1 (dated april 3rd), because on April 5th there was a "revert to last working version" bed2e998a17b8353be35b056bb67596df4e3b108

I assumed "revert to last working version" would be the same as what I had. Git diff shows significant changes, it was not just a revert commit.

Now updating to the april 5th version for another week's TV downloads.

@gkoelln
Copy link
Contributor Author

gkoelln commented Apr 24, 2017

@keybounce Please keep me updated. It wasn't an actual revert, but me trying to undo some changes, but I couldn't find my history, so I tried to recreate it.

@keybounce
Copy link

keybounce commented Apr 26, 2017 •
edited

Seems to be working. I did not get any errors getting another week's worth of downloads. (Not all of the sites I fetch from want a sub.)

@krg99
Copy link

krg99 commented May 5, 2017

Just switched over from Comcast_SSO and everything seems to be working great. I was able to pull some Viceland episodes I was missing with no problems. Anything specific you would like me to test?

@gkoelln
Copy link
Contributor Author

gkoelln commented May 5, 2017

Resolved merge conflict. I'm ready to squash this for merge, if there are no further changes needed.

@dellis87
Copy link

dellis87 commented Oct 3, 2019

Looks like they changed it again. I’m getting an error “unable to extract post url”

@keybounce
Copy link

keybounce commented Oct 4, 2019 via email

mascondante added a commit to mascondante/youtube-dl that referenced this pull request Nov 6, 2019
@mascondante
Update new format
fa5e535 
Trying ytdl-org/youtube-dl#12000 (comment)
@bobbintb
Copy link

bobbintb commented Dec 26, 2019 •
edited

I tried added the MSO stuff to the code like others have and I too am getting a post error. Anyone have a fix for this? I don't know enough about how the logins work to try and fix it myself it seems. I can test though.

@bobbintb
Copy link

Just to double check then -- shinji257/youtube-dl is the repository that has the current version of this patch?

Does anyone understand why travis reports that checks have failed, or has this been fixed (ATT restored) to the point that it would go back into the main Youtube-dl base?

I just tried that version. It seems to login just fine but says my subscription package doesn't include it, yet I am able to watch it in the browser using the same credentials. I tried a video on Cartoon Network.

@dellis87
Copy link

Mine says it can’t find the post_url. Trying on Disney, fx, free form, etc.

@bobbintb
Copy link

bobbintb commented Mar 2, 2020

Has anyone gotten this working recently?

@dellis87
Copy link

Unfortunately not. :(

@JacobHepworth
Copy link

JacobHepworth commented Jul 9, 2020 •
edited

I was able to get it to work just now using the txt file supplied by @JayBird15. Not all urls are working however. When I run it on the following url, it sucessfully downloads.
https://www.adultswim.com/videos/beef-house/crab-dip
When I try to run it with the url below, a KeyError is raised for 'media'. After a bunch of digging I feel I've come to a dead-end as I'm not familiar enough with youtube-dl's source. It's frustrating because when I go to debug it, the new MSO method for dish is never accessed.
https://www.adultswim.com/videos/my-hero-academia/japanese-hero-billboard-chart

I'm not seeing a media key in the _download_json tuple but I am seeing a 'MediaID' key.
Tuple
({'data': {'getShowBySlug': {'title': 'My Hero Academia', 'getVideoBySlug': {'_id': 'AXLIy4sxC8QgH-t19JtJ', 'auth': True, 'description': "It's time for the next Japanese Hero Billboard Chart rankings to be announced, and this time, all the top ten heroes make an appearance.", 'duration': 1342.5746, 'episodeNumber': 24, 'launchDate': '2020-06-21T07:30:00.000Z', 'mediaID': 'd91d4a9489e7631b558b8b38d405678542d3920a', 'seasonNumber': 4, 'poster': 'https://media.cdn.adultswim.com/uploads/20200618/thumbnails/2_206181457580-myheroacademia_087.jpg', 'title': 'Japanese Hero Billboard Chart', 'tvRating': 'tv-14'}}}}, <addinfourl at 2503417980720 whose fp = <_io.BytesIO object at 0x00000246DF59B6D0>>)

Error

ERROR: An extractor error has occurred. (caused by KeyError('media')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\youtube_dl\extractor\common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "c:\python38\lib\site-packages\youtube_dl\extractor\adultswim.py", line 156, in _real_extract
    info.update(self._extract_ngtv_info(media_id, {
  File "c:\python38\lib\site-packages\youtube_dl\extractor\turner.py", line 198, in _extract_ngtv_info
    streams_data = self._download_json(
KeyError: 'media'
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\youtube_dl\extractor\common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "c:\python38\lib\site-packages\youtube_dl\extractor\adultswim.py", line 156, in _real_extract
    info.update(self._extract_ngtv_info(media_id, {
  File "c:\python38\lib\site-packages\youtube_dl\extractor\turner.py", line 198, in _extract_ngtv_info
    streams_data = self._download_json(
KeyError: 'media'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python38\lib\site-packages\youtube_dl\YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "c:\python38\lib\site-packages\youtube_dl\extractor\common.py", line 543, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
youtube_dl.utils.ExtractorError: An extractor error has occurred. (caused by KeyError('media')); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

@bobbintb
Copy link

Hmmm. I wasn't able to get it to work when I last tried it but maybe I'll try again since I know more about the source code now. The adult swim issue should be an easy fix.

@dellis87
Copy link

dellis87 commented Jul 10, 2020 via email

@keybounce
Copy link

keybounce commented Jul 10, 2020 via email

@pukkandan
Copy link
Contributor

Does this patch still work?

@hansgerig
Copy link

I have a dish login. How can I test the code or contribute? I would really like this to be finished and merged with the main code. Thanks.

@keybounce
Copy link

This is a "me-too" moment. I don't know how to get started on this. I have coding experience and a dish login. But no knowledge of what I'm doing here.

@pukkandan
Copy link
Contributor

You can install https://github.com/gkoelln/youtube-dl/tree/Dish with pip to test. Note that this will be based on an older upstream version. To get the best of both, you'll need to use git to rebase/merge this on top of the current master/release

@hansgerig
Copy link

You can install https://github.com/gkoelln/youtube-dl/tree/Dish with pip to test. Note that this will be based on an older upstream version. To get the best of both, you'll need to use git to rebase/merge this on top of the current master/release

Are there step by step instructions somewhere on how to do this? I'm not experienced with using the pip and git commands.

@keybounce
Copy link

keybounce commented Jun 29, 2021 via email

@hansgerig
Copy link

You can install https://github.com/gkoelln/youtube-dl/tree/Dish with pip to test. Note that this will be based on an older upstream version. To get the best of both, you'll need to use git to rebase/merge this on top of the current master/release

Here we go with this nonsense again:
https://github.com/gkoelln/youtube-dl/tree/Dish: Repository unavailable due to DMCA takedown.

@shinji257
Copy link

shinji257 commented Jun 30, 2021 via email

@hansgerig
Copy link

So does that mean that since the code for Dish MSO is not in the master branch or main repository or whatever it's called, the code is no longer available? If so, then does that mean someone will have to start from scratch to add support for Dish MSO?

@keybounce
Copy link

keybounce commented Jun 30, 2021 via email

@liamengland1
Copy link

Clone the main repo and make the changes yourself, then test... ?

https://github.com/ytdl-org/youtube-dl/pull/12000/files

@shinji257
Copy link

So does that mean that since the code for Dish MSO is not in the master branch or main repository or whatever it's called, the code is no longer available? If so, then does that mean someone will have to start from scratch to add support for Dish MSO?

I've addressed the question already. The code is available from the pull request still and you can try to merge it into the current base code if you wish.

@JeffreyBytes JeffreyBytes mentioned this pull request Jul 13, 2021
Open
5 tasks
JacobHepworth added a commit to JacobHepworth/yt-dlp that referenced this pull request Feb 12, 2022
@JacobHepworth
Added Dish MSO
e693e52 
Additions were borrowed from the following comment.
ytdl-org/youtube-dl#12000 (comment)
@dirkf dirkf closed this Aug 1, 2023
@dirkf dirkf added the defunct PR source branch is not accessible label Oct 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yan12125 yan12125 yan12125 left review comments

Assignees
No one assigned
Labels
defunct PR source branch is not accessible tv-provider-account-needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet