AOL extractor broken

[SeanPM5]

• I've verified and I assure that I'm running youtube-dl 2016.04.06
• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)

youtube-dl -v http://on.aol.com/video/meredith-amelia-are-sisters-again-5707f7f6e4b09a6ef926105a
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://on.aol.com/video/meredith-amelia-are-sisters-again-5707f7f6e4b09a6ef926105a']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.04.06
[debug] Python version 2.7.11 - Darwin-15.3.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] meredith-amelia-are-sisters-again-5707f7f6e4b09a6ef926105a: Requesting header
WARNING: Falling back on generic information extractor.
[generic] meredith-amelia-are-sisters-again-5707f7f6e4b09a6ef926105a: Downloading webpage
[generic] meredith-amelia-are-sisters-again-5707f7f6e4b09a6ef926105a: Extracting information
[5min] 5707: Downloading embed page
[5min] 5707: Downloading JSON metadata
ERROR: 5min said: We're sorry, but the video you are trying to watch does not exist.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 669, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 323, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/fivemin.py", line 125, in _real_extract
    expected=True)
ExtractorError: 5min said: We're sorry, but the video you are trying to watch does not exist.

...
<end of log>

Description of your issue, suggested solution and other information

A few days ago the AOL On website (which always worked great in youtube-dl) was refreshed which changed their URL's including video URL's. It appears to have switched from "5min" to "vidible" but the actual video formats and resolutions seem to have remained identical. So I think it's mostly just a name change / URL update?

Example URL: http://on.aol.com/video/meredith-amelia-are-sisters-again-5707f7f6e4b09a6ef926105a

The direct video link should be like this (got it from the website "savevideo.me" ):
http://cdn.vidible.tv/prod/2016-04/08/5707f7f6e4b09a6ef926105a_1920x1080_v1.mp4?hdnea=exp=1460269904~acl=/*~hmac=cc5efbd5cb9288ab1b7b4583c9028407cf6d367cfd7184c856e9f106479e1e01

AOL is one of the larger video sites, they host/syndicate videos for many television networks (NBC,ABC,MSNBC, etc) and whatnot. It doesn't require an account and isn't geo-blocked as far as I know.

Hoping someone can update the extractor to get it working again. Thanks!

[remitamine]

will be fixed in the next version.

[SeanPM5]

Hi @remitamine thank you very much for the fast fix. There is still one issue however, it does not work on all of the same URL's that it did before.

For example I usually access site from links like this ("channels"):
http://on.aol.com/partner/abc-518421766
http://on.aol.com/partners/nbc-518886504
http://on.aol.com/partners/access-hollywood-519021709
http://on.aol.com/nbcu

This will be a common scenario since that is the type of url's you most often get when navigating their site (on AOL On homepage, in the main navigation bar at top, goto Channels -> NBCU).

Better explanation:

This URL does not work:
http://on.aol.com/partners/abc-551438d309eab105804dbfe8/sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944
This URL does work (same video, same website, just different URL):
http://on.aol.com/video/sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944

Here is output of the failed one, it is still attempting 5min:

iMac:Desktop Sean$ youtube-dl -v http://on.aol.com/partners/abc-551438d309eab105804dbfe8/sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://on.aol.com/partners/abc-551438d309eab105804dbfe8/sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.04.13
[debug] Python version 2.7.11 - Darwin-15.3.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944: Requesting header
WARNING: Falling back on generic information extractor.
[generic] sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944: Downloading webpage
[generic] sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944: Extracting information
[5min] 570: Downloading embed page
[5min] 570: Downloading JSON metadata
ERROR: 5min said: We're sorry, but the video you are trying to watch does not exist.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 671, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 341, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/fivemin.py", line 125, in _real_extract
    expected=True)
ExtractorError: 5min said: We're sorry, but the video you are trying to watch does not exist.

This URL does not work:
http://on.aol.com/partners/nbc-518886504/ge-fallonventions-project-maverick-570d0f31e4b00f7a95994bc4
This URL does work (same video, same website, just different URL):
http://on.aol.com/video/ge-fallonventions-project-maverick-570d0f31e4b00f7a95994bc4

Cause and Solution

This is the part of the URL that is causing issues:
/partners/abc-551438d309eab105804dbfe8/
/partners/nbc-518886504/

replace with this:
/video/

and it will work. So basically the problem is it works with /video/ but not the other URL's I mentioned. I can work around this for now by renaming the URL but it would be nice if that wasn't necessary.

Hopefully you can add support for these type of URL as well? And once again thank you :)

[remitamine]

the extractor will match video only urls in the next version, like:
http://on.aol.com/partners/abc-551438d309eab105804dbfe8/sneak-peek-was-haley-really-framed-570eaebee4b0448640a5c944
http://on.aol.com/shows/park-bench-shw518173474-559a1b9be4b0c3bfad3357a7?context=SH:SHW518173474:PL4327:1460619712763

channel and show pages should be handled differently.

some of the urls you have posted didn't work for me in the browser(may be georestricted):
http://on.aol.com/partners/nbc-518886504/ge-fallonventions-project-maverick-570d0f31e4b00f7a95994bc4
http://on.aol.com/partners/nbc-518886504
http://on.aol.com/partners/access-hollywood-519021709
http://on.aol.com/nbcu

[SeanPM5]

edit:

Thank you, that is great!

Yes, from testing with my VPN's just now it appears that the NBC examples are georestricted to US. But if it works on the ABC link you gave then it should work for the others too since its the same format urls. i will test this stuff when the next version gets pushed out and if there's any issues I will let you know.

thanks again @remitamine :)