Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please add Charter_Direct to your supported MSO list. #11465

Closed
tv21 opened this issue Dec 16, 2016 · 19 comments
Closed

Please add Charter_Direct to your supported MSO list. #11465

tv21 opened this issue Dec 16, 2016 · 19 comments

Comments

@tv21
Copy link

@tv21 tv21 commented Dec 16, 2016

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like that [x])
  • Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.12.15. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2016.12.15

Before submitting an issue make sure you have:

  • At least skimmed through README and most notably FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue


If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

$ sudo youtube-dl -v --ap-mso Charter_Direct --ap-username *****@charter.net --ap-password ***** http://www.*****.com/videos/*****.html
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'--ap-mso', u'Charter_Direct', u'--ap-username', u'PRIVATE', u'--ap-password', u'PRIVATE', u'http://www.*****.com/videos/*****.html']
Usage: youtube-dl [OPTIONS] URL [URL...]

youtube-dl: error: Unsupported TV Provider, use --ap-list-mso to get a list of supported TV Providers
<end of log>

Description of your issue, suggested solution and other information

This is not a site support request, per se, since the site (which I don't particularly want to reveal) is already supported. The problem is that the --ap-mso option does not recognize Charter_Direct as a valid option, even though it is among the providers shown at https://sp.auth.adobe.com/adobe-services/config/ABC which I assume is Adobe's list of supported MSO's. Can you please add Charter_Direct as a valid MSO?

@tv21
Copy link
Author

@tv21 tv21 commented Jan 12, 2017

Is there any information you can use in lieu of a provider account? The problem is I'm trying to get this working for a relative that has cable, but I'm at a different location and they don't want to provide their login info. Is there anything that I could run that would give you the information you need, and if so, is there a way to send it to you without posting it publicly?

@raleeper
Copy link
Contributor

@raleeper raleeper commented Jan 13, 2017

@tv21 I am not sure if Charter works the same way, but when I was working with the devs to get Comcast support to work, I was able to create a secondary Comcast username (under my main account) that had no rights other than TV access. I gave that username and its password to the devs for testing. When it was all done, I suspended the secondary username. If Charter offers multiple usernames per account, the same approach may work for you.

Failing that, if you know any python, you can run a simple test to see if just adding the Charter name to the list of MSO's will work for you. The MSO list is in a file called youtube_dl/extractor/adobepass.py.

@gkoelln
Copy link
Contributor

@gkoelln gkoelln commented Jan 13, 2017

I started working on the adobepass.py file to updated MSOs and I included Charter, but I have no way of testing it. (I realized that I was trying to commit a file that wasn't fully tested yet.) For whatever it's worth to the devs:
mso_id = Charter_Direct
username_field = IDToken1
password_field = IDToken2

@tv21
Copy link
Author

@tv21 tv21 commented Jan 13, 2017

raleeper, I am running it on Linux, having obtained it using the install instructions on this page:

https://rg3.github.io/youtube-dl/download.html

I cannot seem to find a youtube_dl/extractor/adobepass.py file - is that something that's compiled into the executable file?

gkoelln, do you perhaps have a Linux download of your modified version available? It would be running under Debian, if that matters. Might be able to give it a test run and see if it works, as long as I only have to copy the executable to the /usr/local/bin/ directory.

@raleeper
Copy link
Contributor

@raleeper raleeper commented Jan 14, 2017

@tv21 Yes, adobepass.py is one of the source files that is built into the executable you downloaded. It's only helpful if you want to do a little tinkering on your own. It would be more trouble than it's worth if you aren't already set up to do python coding.

@tv21
Copy link
Author

@tv21 tv21 commented Jan 14, 2017

EDIT: I finally got this to work, just not with the provider I'd originally tried, however it seems to work with other sources just fine. See my followup post further down.

I tried an experiment today. Bear in mind I have not a clue what I am doing since I am not a programmer, but I did download the source code zip file to a Linux system and expanded it into a directory. I then found the youtube_dl/extractor/adobepass.py file, and going off of gkoelln's post I added these lines in the MSO_INFO section:

    'Charter_Direct': {
        'name': 'Charter Spectrum',
        'username_field': 'IDToken1',
        'password_field': 'IDToken2',
    },

I am not sure if that was correct, but it was my best guess. Anyway, after many tries at things that didn't work I realized I could run it directly by running the __main__.py file with appropriate options, for example:

./__main__.py --ap-mso Charter_Direct --ap-username user --ap-password pw http://...

So I tried a couple of test runs, and for a while it actually looked like it might work, but in one case it errored out with a couple of "HTTP error 403 Forbidden" errors, and in another with an error from the source that said "Unable to retrieve required files for playback. Please try again in a few minutes." Since this is a an open forum I don't want to be too specific about exactly what I was trying to retrieve for the test, but it got further along in the process than it has before. There are so many variables in this, including the main one, that I DON'T KNOW WHAT I'M DOING, but it didn't immediately shut me down for invalid login information. Does what I have done so far seem valid, or is there no hope of it working right by just running the main.py file? The system has Python 2.7.9 installed, if that makes any difference.

Here is the output of a run with all the information regarding the attempted download and other potentially sensitive information redacted, I include it only to show how far it got before failing:

:~/Temp$ ~/Temp/youtube-dl-master/youtube_dl/__main__.py  --ap-mso Charter_Direct --ap-username user@charter.net --ap-password pw http://www.xxxxx.com/xxxxx/videos/xxxxx
[SourceName] video-title: Downloading webpage
[SourceName] xxxxxxx: Retrieving Media Token
[ThePlatform] xxxxxxx: Downloading SMIL data
[ThePlatform] xxxxxxx: Downloading m3u8 information
[ThePlatform] xxxxxxx: Downloading JSON metadata
[download] Destination: Video Title-xxxxx.mp4
avconv version 11.8-6:11.8-1~deb8u1, Copyright (c) 2000-2016 the Libav developers
  built on Oct  1 2016 07:51:06 with gcc 4.9.2 (Debian 4.9.2-10)
[https @ 0xxxxxxxx] HTTP error 403 Forbidden
Unable to open key file https://...
[http @ 0xxxxxxxx] HTTP error 403 Forbidden
[crypto @ 0xxxxxxxx] Unable to open input
https://...: Input/output error


ERROR: ffmpeg exited with code 1
@raleeper
Copy link
Contributor

@raleeper raleeper commented Jan 15, 2017

@tv21 The MSO login information isn't implemented for all extractors that might benefit from it. I am having the same problem with individual TVEverywhere locked videos from some sources. When the MSO login is actually working, you should see messages in the log similar to:

...Downloading Provider Redirect Page
...Confirming login
...Retrieving Session
...Retrieving Authorization Token
...Retrieving Media Token

It's that "Retrieving Media Token" part that gets you past the 403's. So, from the log fragment you provided, you may not be able to get the video you want even if Charter is added because the extractor for the particular source you want isn't making use of the MSO information.

@tv21
Copy link
Author

@tv21 tv21 commented Feb 28, 2017

Well I had some more time to experiment today and tried it again, and it now DOES work if in youtube_dl/extractor/adobepass.py these lines are added:

    'Charter_Direct': {
        'name': 'Charter Spectrum',
        'username_field': 'IDToken1',
        'password_field': 'IDToken2',
    },

You then use the options --ap-mso Charter_Direct --ap-username username@charter.net --ap-password password

It simply hadn't worked with the first services I tried it with so I had assumed it did not work. Don't know if anything else changed in the software that might have affected the results.

So if the above lines could be added in the distribution version of youtube-dl it would be much appreciated!

@tv21
Copy link
Author

@tv21 tv21 commented Mar 1, 2017

For anyone else waiting for Charter support, here is how to make it work until it's added to the official release:

cd ~
mkdir ytdl
cd ytdl
wget https://github.com/rg3/youtube-dl/archive/master.zip
unzip master.zip

OMIT THE FOLLOWING (UP TO "END OF OMITTED SECTION") IF DOING THIS AFTER ON OR AFTER MARCH 1, 2017. THE FOLLOWING CHANGE IS ALREADY IN MASTER.ZIP AND JUST NEEDS TESTING BY OTHER USERS:

nano ~/ytdl/youtube-dl-master/youtube_dl/extractor/adobepass.py (or use your favorite text editor)

Now inside the adobepass.py file look for the line:
MSO_INFO = {
Just below that you can add:

    'Charter_Direct': {
        'name': 'Charter Spectrum',
        'username_field': 'IDToken1',
        'password_field': 'IDToken2',
    },

use Ctrl-X in nano to exit the file (be sure to save it when prompted), or the equivalent commands in your favorite text editor.

END OF OMITTED SECTION

To check that it is working do:

~/ytdl/youtube-dl-master/youtube_dl/__main__.py --help

It should print the youtube-dl help file. You can create a symlink to run the script without typing the path:

ln -s ~/ytdl/youtube-dl-master/youtube_dl/__main__.py ~/youtube-dl

Then to run it:

~/youtube-dl <...options...>

The options should include
--ap-mso Charter_Direct --ap-username username@charter.net --ap-password password
when you need to provide authentication for a link from a supported site that requires a cable company login. Not all sites will work.

If you ever do an upgrade (by re-downloading the master.zip file and expanding it) don't forget to re-add the lines in adobepass.py (NOT NECESSARY AFTER MARCH 1, 2017).

Thanks to previous commenters for providing the necessary info.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Mar 1, 2017

just not with the provider I'd originally tried

I'd like to commit it, but this sentence hesitates me. Could you describe more about what's broken with the approach listed above?

@tv21
Copy link
Author

@tv21 tv21 commented Mar 1, 2017

Originally tried it with a program from a particular specific site and got the errors in my Jan 14 post. At the time there was nothing else I wanted to d/l that required a cable company login so I assumed it didn't work. But now I have tried it with a couple other sources that require a cable company login and they do work. So, I assume it was just a problem with that one source. (EDITED to remove name of site, which now works).

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Mar 1, 2017

Thanks for the answer. Do you want to open a pull request?

@tv21
Copy link
Author

@tv21 tv21 commented Mar 1, 2017

Sorry, but I don't know what you mean by "pull request."

yan12125 added a commit that referenced this issue Mar 1, 2017
Thanks @tv21 for the fix!
@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Mar 1, 2017

Well, it can be too long to explain what a pull request is.

By the way, I just commited your change. If the fix is confirmed by another person, this issue can be closed.

@yan12125 yan12125 added the test-needed label Mar 1, 2017
@tv21
Copy link
Author

@tv21 tv21 commented Mar 1, 2017

Thank you. I hope another Charter user will confirm that this works.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Mar 2, 2017

The patch by @tv21 is included in 2017.03.02. Please check it out, thanks.

@tv21
Copy link
Author

@tv21 tv21 commented Mar 4, 2017

EDIT: After a new install of Ubuntu 16.04, updating youtube-dl to version 2017.03.06 (the regular compiled version), and installing ffmpeg (rather than avconf as had been installed on the previous system) everything I've tried seems to work now, including the one source that didn't work previously. Since no other Charter users seem to want to report and since it's now working for me I will close this issue so the developers can focus on other issues. Thank you again for the adding this support.

@tv21 tv21 closed this Mar 7, 2017
@braskie
Copy link

@braskie braskie commented May 11, 2017

I can confirm that Charter is working in version 2017.05.09. Thank you!

@Tatsh
Copy link
Contributor

@Tatsh Tatsh commented Aug 5, 2017

This does not seem to work for me with a Spectrum account (specified with the --ap-* options) at https://watch.spectrum.net/livetv . Anyone have the live TV working? Or the on-demand content?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.