Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make possible to ignore or remove ad segment from hls stream which causes bad/malformed video streams #15748

Closed
fenosa opened this issue Mar 3, 2018 · 0 comments

Comments

@fenosa
Copy link

@fenosa fenosa commented Mar 3, 2018

  • I've verified and I assure that I'm running youtube-dl 2018.03.03

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

This is a real pain with downloads from video.vice.com
Vice extractor is not working right now anyway (#13778) but we can still download by providing m3u8 url to youtube-dl

Problem is some videos contain an ad segment and it causes downloaded video file to have bad resolutions, bad framerates, out of sync issues, compatibility issues to make it unable to play in a lot of players etc..
There is actually a closed issue opened before about this here: #14992
It was closed because dev couldn't reproduce it with another video. But these ad segments may not present in every video.

When I was searching a solution for this I came across a lot of forum threads about this issue where people were complaining about these downloaded videos from vice with bad resolutions and looking for a solution. So it's an annoying problem.

Anyway I found a solution and I will tell you how it's so much struggle to do that and would be nice if youtube-dl handles these manual steps.

Take this for example: https://video.vice.com/en_us/video/bleu-blanc-satan-full-length/58e3e28982d5b9350c82dbe5

Now I can't directly give you a m3u8 url because they contain tokens so you need to inspect the traffic and get it yourself.
Now when you get the m3u8 url from that page and pass it to youtube-dl, the video file downloaded will have incorrect/bad resolution, bad framerate and will fail to play in a lot of players and will have out of sync audio issues.
That's all because of an "ad" segment at the start of the hls stream. Copied this from start of the 1080p m3u8 file and highlighted that relevant part:

#EXTM3U
#EXT-X-VERSION:5
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:5
#UPLYNK-SEGMENT: [somevariables],00000000,ad
#UPLYNK-KEY:https://content-aeui1.uplynk.com/check?b=[somevariables]&v=[somevariables]&pbs=[somevariables]
#EXT-X-PROGRAM-DATE-TIME:2018-01-01T00:00:00.000000+00:00
#EXT-X-KEY:METHOD=AES-128,URI="https://content-aeui1.uplynk.com/check2?b=[somevariables]&v=[somevariables]&r=h&pbs=[somevariables]",IV=0x00000000000000000000000000000000
#EXTINF:1.2800,
https://stgec-ausw-tmp.uplynk.com/[somevariables]/ausw/slices/d43/[somevariables]/[somevariables]/H00000000.ts?pbs=[somevariables]&si=0

#UPLYNK-SEGMENT: [somevariables],00000000,segment
#UPLYNK-KEY:https://content-aeui1.uplynk.com/check?b=[somevariables]&v=[somevariables]&pbs=[somevariables]
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2018-01-01T00:00:00.000000+00:00
#EXT-X-KEY:METHOD=AES-128,URI="https://content-aeui1.uplynk.com/check2?b=[somevariables]&v=[somevariables]&r=h&pbs=[somevariables]",IV=0x00000000000000000000000000000000
#EXTINF:4.0960,
https://stgec-ausw-tmp.uplynk.com/[somevariables]/ausw/slices/344/[somevariables]/[somevariables]/H00000000.ts?pbs=[somevariables]&si=1
#EXT-X-KEY:METHOD=AES-128,URI="https://content-aeui1.uplynk.com/check2?b=[somevariables]&v=[somevariables]&r=h&pbs=[somevariables]",IV=0x00000000000000000000000000000001
#EXTINF:4.0960,
https://stgec-ausw-tmp.uplynk.com/[somevariables]/ausw/slices/344/[somevariables]/[somevariables]/H00000001.ts?pbs=[somevariables]&si=1

See that segment I highlighted? That's the source of all problems. Now when I save this m3u8 file to my computer and modify it by removing these lines I highlighted and save it. And then provide that modified m3u8 file to youtube-dl, resulting downloaded video file is perfect. It will have correct resolutions, correct framerate and will open in all players without a problem.
And btw I have to run a webserver at localhost to provide that modified m3u8 file to youtube-dl because it says file:// protocol is disabled for security reasons. So that's another pain in the process. You can see how much struggle it is to just remove these lines to download the video correctly.

So what I'm asking is it would be awesome if youtube-dl itself can handle these steps and remove or ignore that harmful "ad" segment so downloaded video files won't be malformed. I mean before sending the m3u8 data to ffmpeg, delete that segment I highlighted from it and send that modified m3u8 data to ffmpeg. So we won't have to deal with modifying m3u8 file, going through troubles to provide modified m3u8 to youtube-dl etc..

Please consider adding this fix to youtube-dl. Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.