Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugins.bloomberg: fix vods and live streams #2825

Merged
merged 1 commit into from Mar 14, 2020

Conversation

@bastimeyer
Copy link
Member

bastimeyer commented Mar 11, 2020

Fixes #2823

Fixes live streams and vods (tested US+EU live streams and most recent VOD).

Couple of notes:

  • The JSON object for the live stream CDN data has to be extracted from a webpack-bundled and minified JS file, just like before. The reason why the extraction failed was a change in the structure of this app file. Extracting this data will always be problematic, because the regex requires anchors for the JS module-IDs of the webpack bundle, and just like it is with all webpack production builds, all useful data (for us) gets stripped off and we're left with IDs. This means that as soon as they'll add another module and rebuild the whole thing, the regex will most likely break again due to an ID shift. Even though I've added a second regex for getting the module ID from another module's dependency list, this will break as soon as the "config/livestreams" module gets renamed or moved on their side.
  • I'm a bit unsure about the VOD list schema and the ID extraction afterwards (which depends on the input URL). I think this can be done better, but I don't know the validation API that well.
$ streamlink 'https://www.bloomberg.com/live/us' -l debug
[cli][debug] OS:         Linux-5.6.0-rc5-1-git-x86_64-with-glibc2.2.5
[cli][debug] Python:     3.8.2
[cli][debug] Streamlink: 1.3.1+55.g6141075.dirty
[cli][debug] Requests(2.23.0), Socks(1.7.1), Websocket(0.56.0)
[cli][info] Found matching plugin bloomberg for URL https://www.bloomberg.com/live/us
[plugin.bloomberg][debug] Found liveId: PHOENIX_US
[plugin.bloomberg][debug] Player URL: https://cdn.gotraffic.net/projector/v0.11.182/app.js
[plugin.bloomberg][debug] Webpack module ID: 213
[plugin.bloomberg][debug] Found stream: https://www.bloomberg.com/media-manifest/streams/phoenix-us.m3u8
[utils.l10n][debug] Language code: en_US
[plugin.bloomberg][debug] Found stream: https://www.bloomberg.com/media-manifest/streams/phoenix-us.m3u8?group=backup
[utils.l10n][debug] Language code: en_US
Available streams: 272p_alt (worst), 272p, 360p_alt, 360p, 720p_alt2, 720p_alt, 720p, 1080p_alt, 1080p (best)
$ streamlink 'https://www.bloomberg.com/live/europe' -l debug
[cli][debug] OS:         Linux-5.6.0-rc5-1-git-x86_64-with-glibc2.2.5
[cli][debug] Python:     3.8.2
[cli][debug] Streamlink: 1.3.1+55.g6141075.dirty
[cli][debug] Requests(2.23.0), Socks(1.7.1), Websocket(0.56.0)
[cli][info] Found matching plugin bloomberg for URL https://www.bloomberg.com/live/europe
[plugin.bloomberg][debug] Found liveId: EU
[plugin.bloomberg][debug] Player URL: https://cdn.gotraffic.net/projector/v0.11.182/app.js
[plugin.bloomberg][debug] Webpack module ID: 213
[plugin.bloomberg][debug] Found stream: https://www.bloomberg.com/media-manifest/streams/eu.m3u8
[utils.l10n][debug] Language code: en_US
[plugin.bloomberg][debug] Found stream: https://www.bloomberg.com/media-manifest/streams/eu.m3u8?group=backup
[utils.l10n][debug] Language code: en_US
Available streams: 180p_alt (worst), 180p, 270p_alt, 270p, 360p_alt, 360p, 720p_alt, 720p (best)
$ streamlink 'https://www.bloomberg.com/news/videos/2020-03-11/-bloomberg-daybreak-americas-full-show-03-11-20-video' -l debug
[cli][debug] OS:         Linux-5.6.0-rc5-1-git-x86_64-with-glibc2.2.5
[cli][debug] Python:     3.8.2
[cli][debug] Streamlink: 1.3.1+55.ga5002a1
[cli][debug] Requests(2.23.0), Socks(1.7.1), Websocket(0.56.0)
[cli][info] Found matching plugin bloomberg for URL https://www.bloomberg.com/news/videos/2020-03-11/-bloomberg-daybreak-americas-full-show-03-11-20-video
[plugin.bloomberg][debug] Found videoId: yzOs1G3_R8q5PKO2bso4zQ
[plugin.bloomberg][debug] Found stream: https://cdn-mobapi.bloomberg.com/wssmobile/v1/video/cc/android/WiFi/vodpkg-Fastly--Azure-US-East1-Zenko/yzOs1G3_R8q5PKO2bso4zQ.m3u8
[utils.l10n][debug] Language code: en_US
[plugin.bloomberg][debug] Found stream: https://bbgvod-azure-us-east1-zenko.global.ssl.fastly.net/vod/vod/m/ODE0NjIxNA/MjcwMDM4MQ/daybreakam031120_150.mp4
Available streams: 100k (worst), 150k, 176p, 272p, 360p, 540p, 720p_alt, 720p, 1080p (best)
@codecov

This comment has been minimized.

Copy link

codecov bot commented Mar 11, 2020

Codecov Report

Merging #2825 into master will decrease coverage by 0.01%.
The diff coverage is 11.36%.

@@            Coverage Diff             @@
##           master    #2825      +/-   ##
==========================================
- Coverage   52.93%   52.91%   -0.02%     
==========================================
  Files         248      248              
  Lines       15537    15556      +19     
==========================================
+ Hits         8225     8232       +7     
- Misses       7312     7324      +12     
@bastimeyer bastimeyer force-pushed the bastimeyer:plugins/bloomberg/2823 branch 2 times, most recently from 23936ca to 6141075 Mar 11, 2020
@bastimeyer bastimeyer force-pushed the bastimeyer:plugins/bloomberg/2823 branch from 6141075 to a5002a1 Mar 14, 2020
@bastimeyer

This comment has been minimized.

Copy link
Member Author

bastimeyer commented Mar 14, 2020

the streams seems to have static URLs, might need to use them instead of this.

It's possible, but I'm not sure. I've changed it for now so that it gets the module ID for the regex anchors from another module's dependency list (a simple plain JS object). This will keep working until they change the name of the "config/livestreams" module or if the structure of that module changes. But that was the case previously anyway.

If we'll see them changing this, we can think about adding the streams data to the plugin directly.

@back-to back-to merged commit 3d85e51 into streamlink:master Mar 14, 2020
3 checks passed
3 checks passed
codecov/project 52.91% (target 30.00%)
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants
You can’t perform that action at this time.