Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CBS.com] Failed to download m3u8 information: HTTP Error 404: Not Found (4400k stream) #12788

Closed
Mr-Jake opened this issue Apr 19, 2017 · 7 comments

Comments

@Mr-Jake
Copy link

@Mr-Jake Mr-Jake commented Apr 19, 2017

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like that [x])
  • Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.04.17. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2017.04.17

Before submitting an issue make sure you have:

  • At least skimmed through README and most notably FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue


If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

Log for 4400k stream NOT found:

C:\youtube-dl\youtube-dl.exe "http://www.cbs.com/shows/amazing_race/video/2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj/the-amazing-race-scared-spitless/" -F -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['http://www.cbs.com/shows/amazing_race/video/2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj/the-amazing-race-scared-spitless/', '-F', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.04.17
[debug] Python version 3.4.4 - Windows-XP-5.1.2600-SP3
[debug] exe versions: ffmpeg n3.3, ffprobe n3.3, rtmpdump 2.4
[debug] Proxy map: {}
[CBS] 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj: Downloading XML
[CBS] 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj: Downloading RTMP SMIL data
[CBS] 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj: Downloading StreamPack SMIL data
[CBS] 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj: Downloading m3u8 information
[CBS] 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj: Downloading OnceURL SMIL data
[CBS] 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj: Checking video URL
[CBS] 3eb2e20f-e4be-403e-847f-16e6fa15d0b6: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 404: Not Found
[CBS] 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj: Downloading JSON metadata
[info] Available formats for 2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj:
format code  extension  resolution note
hls-meta     mp4        multiple   Quality selection URL
hls-48       mp4        audio only   48k , mp4a.40.2
hls-156      mp4        400x224     156k , avc1.66.30, mp4a.40.2
hls-295      mp4        400x224     295k , avc1.66.30, mp4a.40.2
rtmp-364     flv        384x216     364k
rtmp-496     flv        640x360     496k
hls-543      mp4        512x288     543k , avc1.66.30, mp4a.40.2
rtmp-796     flv        640x360     796k
hls-839      mp4        640x360     839k , avc1.66.30, mp4a.40.2
hls-1232     mp4        768x432    1232k , avc1.77.30, mp4a.40.2
rtmp-1296    flv        640x360    1296k
hls-1823     mp4        960x540    1823k , avc1.77.30, mp4a.40.2
rtmp-1928    flv        848x480    1928k
hls-2561     mp4        1024x576   2561k , avc1.77.30, mp4a.40.2
hls-3545     mp4        1280x720   3545k , avc1.77.30, mp4a.40.2 (best)


Log for 4400k stream found:

C:\youtube-dl\youtube-dl.exe  "http://www.cbs.com/shows/amazing_race/video/IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz/the-amazing-race-bucket-list-type-stuff/" -F -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['http://www.cbs.com/shows/amazing_race/video/IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz/the-amazing-race-bucket-list-type-stuff/', '-F', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.04.17
[debug] Python version 3.4.4 - Windows-XP-5.1.2600-SP3
[debug] exe versions: ffmpeg n3.3, ffprobe n3.3, rtmpdump 2.4
[debug] Proxy map: {}
[CBS] IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz: Downloading XML
[CBS] IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz: Downloading StreamPack SMIL data
[CBS] IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz: Downloading m3u8 information
[CBS] IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz: Downloading RTMP SMIL data
[CBS] IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz: Downloading OnceURL SMIL data
[CBS] IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz: Checking video URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Downloading m3u8 information
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Checking http-1200 video format URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: http-1200 video format URL is invalid, skipping
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Checking http-4400 video format URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: http-4400 video format URL is invalid, skipping
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Checking http-2000 video format URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: http-2000 video format URL is invalid, skipping
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Checking http-764 video format URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: http-764 video format URL is invalid, skipping
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Checking http-512 video format URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: http-512 video format URL is invalid, skipping
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Checking http-264 video format URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: http-264 video format URL is invalid, skipping
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: Checking http-60 video format URL
[CBS] d406b9c9-6a8d-416c-861c-f9e79b333b7d: http-60 video format URL is invalid, skipping
[CBS] IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz: Downloading JSON metadata
[info] Available formats for IWjq_IBfTKOTnZgTkxllDWFVTiHxWbbz:
format code  extension  resolution note
hls-meta-0   mp4        multiple   Quality selection URL
hls-meta-1   mp4        multiple   Quality selection URL
hls-48       mp4        audio only   48k , mp4a.40.2
hls-60       mp4        audio only   60k , mp4a.40.5
hls-156      mp4        400x224     156k , avc1.66.30, mp4a.40.2
hls-264      mp4        256x144     264k , avc1.42001e, mp4a.40.5
hls-293      mp4        400x224     293k , avc1.66.30, mp4a.40.2
rtmp-364     flv        384x216     364k
rtmp-496     flv        640x360     496k
hls-512      mp4        384x216     512k , avc1.42001e, mp4a.40.5
hls-540      mp4        512x288     540k , avc1.66.30, mp4a.40.2
hls-764      mp4        480x270     764k , avc1.42001e, mp4a.40.2
rtmp-796     flv        640x360     796k
hls-833      mp4        640x360     833k , avc1.66.30, mp4a.40.2
hls-1200     mp4        640x360    1200k , avc1.42001f, mp4a.40.2
hls-1223     mp4        768x432    1223k , avc1.77.30, mp4a.40.2
rtmp-1296    flv        640x360    1296k
hls-1810     mp4        960x540    1810k , avc1.77.30, mp4a.40.2
rtmp-1928    flv        848x480    1928k
hls-2000     mp4        960x540    2000k , avc1.4d001f, mp4a.40.2
hls-2542     mp4        1024x576   2542k , avc1.77.30, mp4a.40.2
hls-3519     mp4        1280x720   3519k , avc1.77.30, mp4a.40.2
hls-4400     mp4        1280x720   4400k , avc1.640028, mp4a.40.2 (best)




If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

(Free links, Login not needed)


Description of your issue, suggested solution and other information

youtube-dl fails to retrieve the 4400k hls stream for some videos on CBS.com.

I have posted two logs above. The first log shows youtube-dl can't find the 4400k stream. It displays the error "Failed to download m3u8 information".

The second log displays info when youtube-dl finds the 4400k stream.

When youtube-dl can't retrieve the 4400k, if you wait some time (up to a couple days) and try multiple attempts, youtube-dl will eventually retrieve the 4400k stream. Could this be fixed so youtube-dl retrives the 4400k stream everytime without needing to run it mutiple times?

@Mr-Jake Mr-Jake changed the title [CBS.com] Failed to download m3u8 information: HTTP Error 404: Not Found [CBS.com] Failed to download m3u8 information: HTTP Error 404: Not Found (4400k stream) Apr 19, 2017
@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Apr 28, 2017

if you wait some time (up to a couple days) and try multiple attempts, youtube-dl will eventually retrieve the 4400k stream

I tried several times and get no 4400k streams. Could you add a print in common.py and see what's the correct m3u8 URL for this video?

    def _extract_m3u8_formats(self, m3u8_url, video_id, ext=None,
                              entry_protocol='m3u8', preference=None,
                              m3u8_id=None, note=None, errnote=None,
                              fatal=True, live=False):
        print(m3u8_url)
        res = self._download_webpage_handle(
            m3u8_url, video_id,
            note=note or 'Downloading m3u8 information',
            errnote=errnote or 'Failed to download m3u8 information',
            fatal=fatal)
@Mr-Jake
Copy link
Author

@Mr-Jake Mr-Jake commented Apr 28, 2017

@yan12125

When youtube-dl detects the 4400k stream, the m3u8 is from once-aws-us-west-2-lb.unicornmedia.com domain. I believe it uses once.py

When youtube-dl does not detect the 4400k stream, the m3u8 is from cbsios-vh.akamaihd.net domain.

I searched the issues and found that other users reported similar issues with CBS and m3u8's from the unicornmedia domain. #11177 The user reports being able to get the 4400k stream manually but youtube-dl does not detect it.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Apr 29, 2017

I believe it uses once.py

I know. I asked you to add a print so that we can know what is the correct m3u8 URL.

#11177 is different. youtube-dl can fetch the 4400k stream in this case.

@Mr-Jake
Copy link
Author

@Mr-Jake Mr-Jake commented Apr 29, 2017

@yan12125

Here are the m3u8 URLs for both the 4400k and 3500k streams for the same video.

4400k stream m3u8 URL:

http://once-aws-us-east-1-lb.unicornmedia.com/now/media/playlist/bb0b18ba-64f5-4b1b-a29f-0ac252f06b68/77a785f3-5188-4806-b78-0893a61634ed/468fb310-a585-11e4-bfdb-005056837bc7/8444d878-f536-45b1-9aa9-55fb0256073f/0/0/2610/content.m3u8?visitguid=c3b2453c-df45-4e6d-9896-28fbfe895da&segmentlength=10&adsegmentlength=0&protocolversion=3

3500k stream m3u8 URL:

https://cbsios-vh.akamaihd.net/i/temp_hd_gallery_video/CBS_Production_Outlet_VMS/video_robot/CBS_Production_Entertainment/2017/04/25/929252419830/CBS_AMAZING_RACE_2906_CONTENT_CIAN_vr_20M_1113049_,2596,3596,1848,1248,848,548,298,158,000.mp4.csmil/index_1_av.m3u8?null=0&id=AgAYANI5FyulLS+aBFmvmwyKAH9gty6yOHmIK0oiiJh80iHTeZanu%2f%2fMS6sAeaJ1Ow9rs7MYJhQ%3d%3d

From this video:

http://www.cbs.com/shows/amazing_race/video/8484FF1D-845E-A8C6-F845-A6694556D012/the-amazing-race-double-u-turn-ahead/
@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Apr 29, 2017

For http://www.cbs.com/shows/amazing_race/video/8484FF1D-845E-A8C6-F845-A6694556D012/the-amazing-race-double-u-turn-ahead/, youtube-dl can already identify Once m3u8 URLs. Let's focus on those break youtube-dl.

Let me check: do you mean for http://www.cbs.com/shows/amazing_race/video/2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj/the-amazing-race-scared-spitless/, you can get 4400k videos sooner or later if you try the same URL again and again?

@Mr-Jake
Copy link
Author

@Mr-Jake Mr-Jake commented Apr 29, 2017

@yan12125

I tried multiple attempts and can't get the 4400k stream for:

http://www.cbs.com/shows/amazing_race/video/2NuEwUzxMyZrfEJiClHYhBP4WOSJJynj/the-amazing-race-scared-spitless/

But 4400k was available when it was first posted on CBS.

I know from prior CBS videos if I try again in a couple days, or a week, or 2 weeks it seems I will eventually get the 4400k stream with youtube-dl. But I do not know if it is the same URL since CBS uses the random characters in the URL. Perhaps CBS changes the URL making the 4400k stream available and then unavailable?

I can keep trying and keep track of the URLs to see if they change. It could be a CBS problem?

EDIT: I tried other CBS shows and youtube-dl gets the 4400k stream for the most recent episodes (2 to 3 episodes). But it does not detect the 4400k stream for shows posted previous to that. A solution would be to grab the 4400k streams as soon as they are posted to CBS. But occasionally the 4400k stream do become available once again for those episodes.

@yan12125
Copy link
Collaborator

@yan12125 yan12125 commented Apr 29, 2017

Perhaps CBS changes the URL making the 4400k stream available and then unavailable?

That's possible.

Closing as it's most likely a CBS issue. Feel free to post new findings below if any.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.