Brightcove player not being detected

[SeanPM5]

• I've verified and I assure that I'm running youtube-dl 2016.04.06
• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)

Description of your issue, suggested solution and other information

Brightcove Player is not being detected on people.com (a major website, ranked 620 on alexa) and ew.com. I'm not sure if this falls under a site support request or bug, but I believe it's a bug because of what I mention at the bottom (and the fact that this player is detected fine on most other sites).

Example page(s):
http://www.people.com/people/videos/0,,20999362,00.html
http://www.people.com/people/videos/0,,20991602,00.html
http://www.people.com/article/greys-anatomy-exclusive-is-meredith-moving-on-mcdreamy
Many more video links: http://www.people.com/people/videos/
http://www.ew.com/article/2016/02/02/supernatural-sneak-peek-winchesters-jody-mills
http://www.ew.com/article/2016/03/11/bates-motel-sneak-peek-norman-threatens-norma

youtube-dl output:

iMac:~ Sean$ youtube-dl -v http://www.people.com/people/videos/0,,20991602,00.html
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.people.com/people/videos/0,,20991602,00.html']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.04.06
[debug] Python version 2.7.11 - Darwin-15.3.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] 0,,20991602,00: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 0,,20991602,00: Downloading webpage
[generic] 0,,20991602,00: Extracting information
ERROR: Unsupported URL: http://www.people.com/people/videos/0,,20991602,00.html
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1348, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 279, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=etree.TreeBuilder(element_factory=_element_factory)))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 268, in _XML
    parser.feed(text)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: not well-formed (invalid token): line 36, column 161
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 669, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 323, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 2012, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://www.people.com/people/videos/0,,20991602,00.html

Using Firefox extensions/web tools I was able to get this link from that page which works with youtube-dl (when wrapped in quotes):
http://admin.brightcove.com/viewer/us20151216.1310/BrightcoveBootloader.swf?playerID=3476534560001&playerKey=AQ~~%2CAAAAABjSC6Q~%2CpGevSATpV8E-Iy5Qd4HGKkonZVLcEf1Y&purl=http%3A%2F%2Fwww.people.com%2Farticle%2Fgreys-anatomy-exclusive-is-meredith-moving-on-mcdreamy&%40videoPlayer=4812242623001&adServerURL=http%3A%2F%2Fpubads.g.doubleclick.net%2Fgampad%2Fads%3Fenv%3Dvp%26gdfp_req%3D1%26impl%3Ds%26output%3Dvast%26iu%3D%2F8484%2Fpeo%2Fvideo_bc%2Fnewsembed_bc%26sz%3D1000x1%26unviewed_position_start%3D1%26url%3D%5Breferrer_url%5D%26correlator%3D%5Btimestamp%5D&additionalAdTargetingParams=%3Bchann%3DBrightcove&autoStart=&bgcolor=%23FFFFFF&debuggerID=&dynamicStreaming=true&flashID=myExperience4812242623001&height=338&includeAPI=true&isUI=true&isVid=true&originalTemplateReadyHandler=PVonTemplateReady&startTime=1460416356415&templateLoadHandler=PVonTemplateLoad&templateReadyHandler=brightcove%5B%22templateReadyHandlermyExperience4812242623001%22%5D&width=600&wmode=transparent

I am guessing something in there is breaking the way youtube-dl detects for this player and perhaps by fixing this it could improve Brightcove detection on other sites too...

[RuneMagle]

Will you add this soon? Still don't work. But IDM can download.

C:\Users\Desktop\youtube-dl>youtube-dl.exe -f best --all-subs http://ew.com/tv/2017/03/08/guardians-galaxy-cartoon-exclusive-clip/ -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'best', '--all-subs', 'http://ew.com/tv/2017/03/08/guardians-galaxy-cartoon-exclusive-clip/', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2017.03.07
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-83034-gf48b6b8, ffprobe N-83096-g6596b34
[debug] Proxy map: {}
[generic] guardians-galaxy-cartoon-exclusive-clip: Requesting header
WARNING: Falling back on generic information extractor.
[generic] guardians-galaxy-cartoon-exclusive-clip: Downloading webpage
[generic] guardians-galaxy-cartoon-exclusive-clip: Extracting information
ERROR: Unsupported URL: http://ew.com/tv/2017/03/08/guardians-galaxy-cartoon-exclusive-clip/
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpvin3ute2\build\youtube_dl\YoutubeDL.py", line 761, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpvin3ute2\build\youtube_dl\extractor\common.py", line 427, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpvin3ute2\build\youtube_dl\extractor\generic.py", line 2628, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: http://ew.com/tv/2017/03/08/guardians-galaxy-cartoon-exclusive-clip/

[dstftw]

Stop mentioning IDM here. It uses browser to resolve links to the media.

[RuneMagle]

Sorry. I was just saying it since you never fix it.

[RuneMagle]

Nearly*

[johnhawkinson]

This is fixed in PR #12178.

[RuneMagle]

But how did he get it to work with that link? When I try to take the video from http://ew.com/tv/2017/03/08/guardians-galaxy-cartoon-exclusive-clip/

I can get this link: c.brightcove.com/services/mobile/streaming/index/rendition.m3u8?assetId=5352122725001&pubId=219646971&videoId=5352121610001

I can't get it to work with the link yan put up :(

[johnhawkinson]

@RuneMagle, I'm afraid your pronouns are confusing me.

But how did he get it to work with that link?

Can't get which "it" to work? And which "he"?

I can't get it to work with the link yan put up :(

Which link?

In any case, #12178 hasn't yet been merged, so this won't work in the mainline. I'm sorry if I was not clear about that. But here's that in action:

pb3:youtube-dl jhawk$ git checkout brightcove-12005
Switched to branch 'brightcove-12005'
Your branch is up-to-date with 'origin/brightcove-12005'.
pb3:youtube-dl jhawk$ python -m youtube_dl -vs 'http://ew.com/tv/2017/03/08/guardians-galaxy-cartoon-exclusive-clip/'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-vs', u'http://ew.com/tv/2017/03/08/guardians-galaxy-cartoon-exclusive-clip/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.03.07
[debug] Git HEAD: 34da9a4df
[debug] Python version 2.7.10 - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg git-2017-02-28-7f62368, ffprobe git-2017-02-28-7f62368, rtmpdump 2.4
[debug] Proxy map: {}
[generic] guardians-galaxy-cartoon-exclusive-clip: Requesting header
WARNING: Falling back on generic information extractor.
[generic] guardians-galaxy-cartoon-exclusive-clip: Downloading webpage
[generic] guardians-galaxy-cartoon-exclusive-clip: Extracting information
[generic] Brightcove (new) video detected.
[download] Downloading playlist: Superheroes Clash in Exclusive 'Guardians of the Galaxy' Cartoon Clip
[generic] playlist Superheroes Clash in Exclusive 'Guardians of the Galaxy' Cartoon Clip: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[brightcove:new] 5352121610001: Downloading webpage
[brightcove:new] 5352121610001: Downloading JSON metadata
[brightcove:new] 5352121610001: Downloading m3u8 information
[brightcove:new] 5352121610001: Downloading m3u8 information
[download] Finished downloading playlist: Superheroes Clash in Exclusive 'Guardians of the Galaxy' Cartoon Clip
pb3:youtube-dl jhawk$ 

[johnhawkinson]

Well, I though the link yan put up,

I still don't know what link you are referring to, @RuneMagle. Perhaps I missed it, but it would be a lot easier to understand you if you just included the link you were talking about.

How do you get the video?

Well, you can wait for PR #12178 to be merged.

Or you could go to my branch at https://github.com/johnhawkinson/youtube-dl/tree/brightcove-12005 and download it: https://github.com/johnhawkinson/youtube-dl/archive/brightcove-12005.zip

[RuneMagle]

Don't matter what I thought. Clearly it makes no sense. I tried downloading your thing from the link, but I have no idea how to make it work... :(

[RuneMagle]

Can you send the video you got instead, with subs please? I just really want that video. :)

[exander77]

I am having similar problem, try this URL: https://www.matrix-2001.cz/gaia-video/zprava-pro-lidstvo-192#

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'--cookies=cookies.txt', u'https://www.matrix-2001.cz/gaia-video/zprava-pro-lidstvo-192'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2018.05.09 [debug] Python version 2.7.14 (CPython) - Linux-4.13.0-39-generic-x86_64-with-Ubuntu-17.10-artful [debug] exe versions: avconv 3.3.4-2, avprobe 3.3.4-2, ffmpeg 3.3.4-2, ffprobe 3.3.4-2, phantomjs 2.1.1, rtmpdump 2.4 [debug] Proxy map: {} [generic] zprava-pro-lidstvo-192: Requesting header WARNING: Falling back on generic information extractor. [generic] zprava-pro-lidstvo-192: Downloading webpage [generic] zprava-pro-lidstvo-192: Extracting information ERROR: Unsupported URL: https://www.matrix-2001.cz/gaia-video/zprava-pro-lidstvo-192 Traceback (most recent call last): File "/home/xxx/repos/youtube-dl/youtube-dl/youtube_dl/extractor/generic.py", line 2261, in _real_extract doc = compat_etree_fromstring(webpage.encode('utf-8')) File "/home/xxx/repos/youtube-dl/youtube-dl/youtube_dl/compat.py", line 2542, in compat_etree_fromstring doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory))) File "/home/xxx/repos/youtube-dl/youtube-dl/youtube_dl/compat.py", line 2531, in _XML parser.feed(text) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1659, in feed self._raiseerror(v) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1523, in _raiseerror raise err ParseError: not well-formed (invalid token): line 42, column 22 Traceback (most recent call last): File "/home/xxx/repos/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info ie_result = ie.extract(url) File "/home/xxx/repos/youtube-dl/youtube-dl/youtube_dl/extractor/common.py", line 503, in extract ie_result = self._real_extract(url) File "/home/xxx/repos/youtube-dl/youtube-dl/youtube_dl/extractor/generic.py", line 3201, in _real_extract raise UnsupportedError(url) UnsupportedError: Unsupported URL: https://www.matrix-2001.cz/gaia-video/zprava-pro-lidstvo-192