[CBS.com] Fetching video asset codes isn't good enough

[spookyahell]

• I've verified and I assure that I'm running youtube-dl 2016.11.08.1

Before submitting an issue make sure you have:

• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

Description of your issue, suggested solution and other information

Issue is that it seams that fetching video ids isn't good enough http://can.cbs.com/thunder/player/videoPlayerService.php (as currently called in youtube-dl) doesn't always return all available formats.
I mostly use CBS in combination with -f "best[protocol=m3u8]" but lately that won't give me 1280x720 streams anymore, (Just 940x480 or something and that returns a 403 Unauthorized) So I looked into it and it seems that is neccesary to always check if http://link.theplatform.com/s/dJ5BDC/media/guid/2198311517/%CONTENTCODE%?assetTypes=OnceURL&format=SMIL&mbr=true&formats=MPEG4%2CM3U returns a URL like http://once.unicornmedia.com/now/od/auto/[...]/content.once. Because I tried it on the newer videos that wouldn't work on youtube-dl directly and was able to get them manually. So checking http://can.cbs.com/thunder/player/videoPlayerService.php is a good thing, but isn't sufficient in more recent videos on the CBS.com website.

[yan12125]

Any example?

[Forcer78]

This sounds similar to my open issue #10918.

[spookyahell]

Any example? (yan12125)

http://www.cbs.com/shows/the-odd-couple/video/B21681C5-A393-155C-D690-1BCADE40DB83/the-odd-couple-you-re-not-evan-/ would be an example that doesn't even get 1280x720 hls even though it exists!

[spookyahell]

I by the way meanwhile have made my own tool for CBS 4400k stream
No offense, but I used to work with youtube-dl daily and that's why I for myself had to come up with a better solution than purely manually downloading.
The purpose of my app: Get the stream link in order to downloaf it with livestreamer.
It's free to use but please don't publish it anywhere else yet. (I plan to do this myself)
It will be here: (no software without source codes here)
Programming language: Delphi, only Windows binary possible (for me)
This solution also for me is only temporary, meaning until youtube-dl collabs can find some of their valueable time and resolve the error.
Could also be used afterwards, if someone wanted to download source stream.
For now for you guys it is here to use. You will need "livestreamer" and your own Webserver (nginx works well)!
Maybe some of will make some use out of this! I know I did and will.

Windows binary:
[Link removed, will replace soon]

PS: I haven't found any 5000k Streams as Mr-Jake says.
Do you have an example link and how do you know it's 5000k?

[remitamine]

http://www.cbs.com/shows/the-odd-couple/video/B21681C5-A393-155C-D690-1BCADE40DB83/the-odd-couple-you-re-not-evan-/ would be an example that doesn't even get 1280x720 hls even though it exists!

youtube-dl -F http://www.cbs.com/shows/the-odd-couple/video/B21681C5-A393-155C-D690-1BCADE40DB83/the-odd-couple-you-re-not-evan-/
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading XML
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading 3G SMIL data
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Checking video URL
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Checking video URL
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading HLS SMIL data
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading RTMP SMIL data
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading Downloadable SMIL data
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Checking video URL
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: video URL is invalid, skipping
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading StreamPack SMIL data
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading m3u8 information
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading WIFI SMIL data
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Checking video URL
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Checking video URL
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading OnceURL SMIL data
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Checking video URL
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Downloading m3u8 information
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Checking http-1200 video format URL
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Checking http-4400 video format URL
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Checking http-2000 video format URL
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Checking http-764 video format URL
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Checking http-512 video format URL
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Checking http-264 video format URL
[CBS] 38501e23-4721-4188-8677-19252f353ae5: Checking http-60 video format URL
[CBS] B21681C5-A393-155C-D690-1BCADE40DB83: Downloading JSON metadata
[info] Available formats for B21681C5-A393-155C-D690-1BCADE40DB83:
format code  extension  resolution note
hls-meta-0   mp4        multiple   Quality selection URL 
hls-meta-1   mp4        multiple   Quality selection URL 
hls-58       mp4        audio only   58k , mp4a.40.2
hls-60       mp4        audio only   60k , mp4a.40.5
http-60      mp4        audio only   60k , mp4a.40.5
hls-1        mp4        400x224       1k 
hls-110      mp4        400x224     110k 
http-150     mp4        400x224     150k 
hls-183      mp4        400x224     183k , avc1.66.30, mp4a.40.2
http-240     mp4        400x224     240k 
hls-250      mp4        400x224     250k 
hls-264      mp4        256x144     264k , avc1.42001e, mp4a.40.5
http-264     mp4        256x144     264k , avc1.42001e, mp4a.40.5
hls-327      mp4        400x224     327k , avc1.66.30, mp4a.40.2
rtmp-364     flv        384x216     364k 
http-440     mp4        400x224     440k 
rtmp-496     flv        640x360     496k 
hls-500      mp4        512x288     500k 
hls-512      mp4        384x216     512k , avc1.42001e, mp4a.40.5
http-512     mp4        384x216     512k , avc1.42001e, mp4a.40.5
hls-573      mp4        512x288     573k , avc1.66.30, mp4a.40.2
http-740     mp4        640x360     740k 
hls-764      mp4        480x270     764k , avc1.42001e, mp4a.40.2
http-764     mp4        480x270     764k , avc1.42001e, mp4a.40.2
rtmp-796     flv        640x360     796k 
hls-800      mp4        640x360     800k 
hls-884      mp4        640x360     884k , avc1.66.30, mp4a.40.2
hls-1200-0   mp4        640x360    1200k , avc1.42001f, mp4a.40.2
http-1200    mp4        640x360    1200k , avc1.42001f, mp4a.40.2
hls-1200-1   mp4        768x432    1200k 
rtmp-1296    flv        640x360    1296k 
hls-1300     mp4        768x432    1300k , avc1.77.30, mp4a.40.2
hls-1800     mp4        960x540    1800k 
hls-1911     mp4        960x540    1911k , avc1.77.30, mp4a.40.2
hls-2000     mp4        960x540    2000k , avc1.4d001f, mp4a.40.2
http-2000    mp4        960x540    2000k , avc1.4d001f, mp4a.40.2
hls-2666     mp4        1024x576   2666k , avc1.77.30, mp4a.40.2
hls-3674     mp4        1280x720   3674k , avc1.77.30, mp4a.40.2
hls-4400     mp4        1280x720   4400k , avc1.640028, mp4a.40.2
http-4400    mp4        1280x720   4400k , avc1.640028, mp4a.40.2 (best)

[yan12125]

@spookyahell could you paste a log where some formats are missing?

[spookyahell]

When issue happens again I will be sure to post the log (verbose) here immediately...

[yan12125]

Thanks for the feedback. The Once approach you've proposed is exactly what youtube-dl does. I guess some step fixing other sites also fixes the problem here. If there are still problems, feel free to open a new issue.