Support for NYU videos

[swaglaw]

My university uses Mediasite player to host video lectures. The videos can be manually downloaded by pulling a link from a javascript file that is referenced in the HTML in script tags.

The system is set up exactly as with the existing Sandia extractor, with the exception that one must login via Shibboleth before accessing the videos, and some minor case stuff (lowercase v uppercase of certain strings).

I modified the Sandia extractor to match the NYU videos, and tried various methods of authentication, including: -u/-pw; adding in the AUTH string from the cookie to the extractor file, and; importing the cookie with --cookies.

Interestingly enough, the error I get is with regexp and not authentication.

Here's the output I get:

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', '--verbose', 'http
://nyulaw.mediasite.com/mediasite/Play/197180ea8f5c448e9ccf3290c5d855411d']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2015.04.09
[debug] Python version 3.4.3 - Windows-8-6.2.9200
[debug] exe versions: none
[debug] Proxy map: {}
[Sandia] 197180ea8f5c448e9ccf3290c5d855411d: Downloading webpage
ERROR: Unable to extract JS code URL; please report this issue on https://yt-dl.
org/bug . Make sure you are using the latest version; see  https://yt-dl.org/upd
ate  on how to update. Be sure to call youtube-dl with the --verbose flag and in
clude its complete output.
Traceback (most recent call last):
  File "D:\Knowledge\NYU Lectures\youtube-dl-master\youtube_dl\YoutubeDL.py", li
ne 651, in extract_info
    ie_result = ie.extract(url)
  File "D:\Knowledge\NYU Lectures\youtube-dl-master\youtube_dl\extractor\common.
py", line 275, in extract
    return self._real_extract(url)
  File "D:\Knowledge\NYU Lectures\youtube-dl-master\youtube_dl\extractor\sandia.
py", line 33, in _real_extract
    r'<script type="text/javascript" src="(/mediasite/FileServer/Presentation/[^
"]+)"', webpage, 'JS code URL')
  File "D:\Knowledge\NYU Lectures\youtube-dl-master\youtube_dl\extractor\common.
py", line 549, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract JS code URL; please repor
t this issue on https://yt-dl.org/bug . Make sure you are using the latest versi
on; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl
with the --verbose flag and include its complete output.'

[beddari]

We should be able to come up with a more generic Mediasite extractor somehow? There are ... millions ... of students that would be able to use it.

[beddari]

Just to have another example, here's an URL with a presentation http://mediasite.uib.no/Mediasite/Play/90bb363295d945d6b548c867d01181361d?catalog=a452b7df-9ae1-46b7-a3ba-aceeb285f3eb ... this one isn't behind SAML2 login, but most others are.

[beddari]

I think auth would somehow need to be two-step, e.g http://stackoverflow.com/questions/20990571/how-to-call-rest-api-using-curl-saml-token-auth

[andrewgra]

I'm an NYU student, and I found this thread (via google) because I'm looking for a way to download these lecture videos. I don't have any technical skills; can you explain to me, did you have any success? What's the method?

[woojoo666]

UCLA also uses Shibboleth login, and forces us to use Silverlight to view videos, but I want to download them for offline viewing. I tried to give youtube-dl the manifest file, but it keeps redirecting to the shibboleth login page, even when I use a cookies file. I have no idea what's wrong, because I can import the cookies to a fresh browser and access the manifest file no problem. Were any of you guys able to get it working?

[woojoo666]

@swaglaw my (uneducated) guess for the reason why you're getting errors with regex is because you're still getting redirected, and it can't parse the shibboleth url. At least for me, when I try to download a video using the cookies option, I have tried everything but still get redirected to the login page every time, after which youtube-dl gives me a "error: unsupported url". Full log below:

>youtube-dl -o 'test.%(ext)s' --cookies cookies.txt http://bruinmedia.ucla.edu/arc2017w/ee115b-1/ee115b-1-20170308/ee115b-1-20170308.ism/Manifest

[generic] Manifest: Requesting header
[redirect] Following redirect to https://shb.ais.ucla.edu/shibboleth-idp/profile/SAML2/Redirect/SSO?execution=e1s1
[generic] SSO?execution=e1s1: Requesting header
WARNING: Falling back on generic information extractor.
[generic] SSO?execution=e1s1: Downloading webpage
[generic] SSO?execution=e1s1: Extracting information
ERROR: Unsupported URL: https://shb.ais.ucla.edu/shibboleth-idp/profile/SAML2/Redirect/SSO?execution=e1s1