Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about format selection: DASH video only #8017

Closed
zmwangx opened this issue Dec 27, 2015 · 10 comments
Closed

Question about format selection: DASH video only #8017

zmwangx opened this issue Dec 27, 2015 · 10 comments

Comments

@zmwangx
Copy link

@zmwangx zmwangx commented Dec 27, 2015

I was considering a one-stop configuration for downloading YouTube videos for native playback on iOS, which means I need the highest quality H.264+AAC, so I need to select the highest quality mp4 video (usually DASH) plus the highest quality m4a audio. bestvideo is no good since it could easily be VP9 in WebM these days. mp4 also doesn't work in most cases, since 720p non-DASH all-in-one stream is preferred (see example at the bottom).

Currently I'm hard coding format codes

-f "((299/138/266/264/137/298/136)+(141/140))/best"

which works but is certainly not ideal.

Therefore, my question is: Is it possible to filter out non-DASH streams? I glanced at how selectors and especially filters are built in YoutubeDl.py, but there seems to be nothing about DASH. Even filtering out certain format ids doesn't seem to be possible. If I'm right that this is indeed not possible right now, I'll convert this to a feature request: either add an id filter (so that we can possibly write [id > 22]), or better yet, a possibly boolean filter specifically for DASH.


Example of mp4 selecting 720p non-DASH (22):

> youtube-dl -vF q7o7R5BgWDY
[debug] System config: []
[debug] User config: [u'--format=((315/308/299/303/138/266/313/264/271/137/248/136/247/bestvideo)+(141/251/171/140/bestaudio))/best', u'--output=%(upload_date)s-%(title)s-%(id)s.%(ext)s', u'--write-sub', u'--sub-format=(ass,srt)/best', u'--sub-lang=en,zh-Hans,zh-Hant']
[debug] Command-line args: [u'-vF', u'q7o7R5BgWDY']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2015.12.23
[debug] Python version 2.7.11 - Darwin-15.2.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 2.8.4, ffprobe 2.8.4, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] q7o7R5BgWDY: Downloading webpage
[youtube] q7o7R5BgWDY: Downloading video info webpage
[youtube] q7o7R5BgWDY: Extracting video information
[youtube] q7o7R5BgWDY: Downloading DASH manifest
[youtube] q7o7R5BgWDY: Downloading DASH manifest
[info] Available formats for q7o7R5BgWDY:
format code  extension  resolution note
249          webm       audio only DASH audio   57k , opus @ 50k, 732.68KiB
250          webm       audio only DASH audio   80k , opus @ 70k, 966.71KiB
140          m4a        audio only DASH audio  127k , m4a_dash container, aac  @128k (44100Hz), 1.82MiB
171          webm       audio only DASH audio  132k , vorbis@128k (44100Hz), 1.70MiB
251          webm       audio only DASH audio  163k , opus @160k, 1.87MiB
160          mp4        256x144    DASH video  109k , avc1.42c00c, 12fps, video only, 1.51MiB
278          webm       256x144    DASH video  125k , webm container, vp9, 12fps, video only, 1.37MiB
133          mp4        426x240    DASH video  248k , avc1.4d4015, 24fps, video only, 3.41MiB
242          webm       426x240    DASH video  258k , vp9, 24fps, video only, 2.92MiB
243          webm       640x360    DASH video  490k , vp9, 24fps, video only, 5.39MiB
134          mp4        640x360    DASH video  609k , avc1.4d401e, 24fps, video only, 5.98MiB
244          webm       854x480    DASH video  881k , vp9, 24fps, video only, 9.63MiB
135          mp4        854x480    DASH video 1134k , avc1.4d401e, 24fps, video only, 11.22MiB
247          webm       1280x720   DASH video 1704k , vp9, 24fps, video only, 18.48MiB
136          mp4        1280x720   DASH video 2245k , avc1.4d401f, 24fps, video only, 21.40MiB
248          webm       1920x1080  DASH video 3042k , vp9, 24fps, video only, 32.88MiB
137          mp4        1920x1080  DASH video 4188k , avc1.640028, 24fps, video only, 38.99MiB
17           3gp        176x144    small ,  mp4a.40.2, mp4v.20.3
36           3gp        320x240    small ,  mp4a.40.2, mp4v.20.3
5            flv        400x240    small
43           webm       640x360    medium ,  vorbis, vp8.0
18           mp4        640x360    medium ,  mp4a.40.2, avc1.42001E
22           mp4        1280x720   hd720 ,  mp4a.40.2, avc1.64001F (best)
> youtube-dl -vf mp4+m4a q7o7R5BgWDY
[debug] System config: []
[debug] User config: [u'--format=((315/308/299/303/138/266/313/264/271/137/248/136/247/bestvideo)+(141/251/171/140/bestaudio))/best', u'--output=%(upload_date)s-%(title)s-%(id)s.%(ext)s', u'--write-sub', u'--sub-format=(ass,srt)/best', u'--sub-lang=en,zh-Hans,zh-Hant']
[debug] Command-line args: [u'-vf', u'mp4+m4a', u'q7o7R5BgWDY']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2015.12.23
[debug] Python version 2.7.11 - Darwin-15.2.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 2.8.4, ffprobe 2.8.4, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] q7o7R5BgWDY: Downloading webpage
[youtube] q7o7R5BgWDY: Downloading video info webpage
[youtube] q7o7R5BgWDY: Extracting video information
[youtube] q7o7R5BgWDY: Downloading DASH manifest
[youtube] q7o7R5BgWDY: Downloading DASH manifest
[info] Writing video subtitles to: 20151215-Google - Year In Search 2015-q7o7R5BgWDY.zh-Hans.srt
[info] Writing video subtitles to: 20151215-Google - Year In Search 2015-q7o7R5BgWDY.en.srt
[info] Writing video subtitles to: 20151215-Google - Year In Search 2015-q7o7R5BgWDY.zh-Hant.srt
[debug] Invoking downloader on u'https://r5---sn-o097znek.googlevideo.com/videoplayback?itag=22&signature=BA4BFB0E369C5747E5A7B8606F11C01B45A6AA7B.497201D6EE7951D3B7376DA14BE3CCC24558C55C&ipbits=0&mime=video%2Fmp4&pl=16&dur=120.186&expire=1451212686&sver=3&nh=IgpwcjAxLnBhbzAzKgkxMjcuMC4wLjE&ratebypass=yes&initcwndbps=1927500&id=o-AO0G8nTC7chlNEeDszf76kzV52VabV5V0ncAouK8QepW&upn=K-WSih51ovM&lmt=1450365890973199&ip=73.231.187.11&sparams=dur%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cnh%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&fexp=9408941%2C9415556%2C9416126%2C9417055%2C9419452%2C9419458%2C9420452%2C9421249%2C9422596%2C9422779%2C9423038%2C9423662%2C9423923%2C9423939%2C9424712%2C9425308%2C9425346%2C9425620%2C9426035%2C9426471%2C9426492&mt=1451191014&mv=m&ms=au&source=youtube&key=yt6&requiressl=yes&mm=31&mn=sn-o097znek'
[download] Destination: 20151215-Google - Year In Search 2015-q7o7R5BgWDY.f22.mp4
[download] 100% of 35.53MiB in 00:05
[debug] Invoking downloader on u'https://r5---sn-o097znek.googlevideo.com/videoplayback?id=abba3b4790605836&itag=140&source=youtube&requiressl=yes&mm=31&mn=sn-o097znek&mv=m&ms=au&nh=IgpwcjAxLnBhbzAzKgkxMjcuMC4wLjE&pl=16&ratebypass=yes&mime=audio/mp4&gir=yes&clen=1909551&lmt=1450365955162166&dur=120.186&signature=9211F3F71F613D44FD1AAEAFCF643E9793129B5D.30CF073773CECB3BD57B508113AD4E93E39A8B07&key=dg_yt0&sver=3&mt=1451191014&fexp=9408941,9415556,9416126,9417055,9419452,9419458,9420452,9421249,9422596,9422779,9423038,9423662,9423923,9423939,9424712,9425308,9425346,9425620,9426035,9426471,9426492&upn=bszWGfvPIhk&ip=73.231.187.11&ipbits=0&expire=1451212686&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,mv,ms,nh,pl,ratebypass,mime,gir,clen,lmt,dur'
[download] Destination: 20151215-Google - Year In Search 2015-q7o7R5BgWDY.f140.m4a
[download] 100% of 1.82MiB in 00:01
[ffmpeg] Merging formats into "20151215-Google - Year In Search 2015-q7o7R5BgWDY.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:20151215-Google - Year In Search 2015-q7o7R5BgWDY.f22.mp4' -i 'file:20151215-Google - Year In Search 2015-q7o7R5BgWDY.f140.m4a' -c copy -map 0:v:0 -map 1:a:0 'file:20151215-Google - Year In Search 2015-q7o7R5BgWDY.temp.mp4'
Deleting original file 20151215-Google - Year In Search 2015-q7o7R5BgWDY.f22.mp4 (pass -k to keep)
Deleting original file 20151215-Google - Year In Search 2015-q7o7R5BgWDY.f140.m4a (pass -k to keep)
@dstftw
Copy link
Collaborator

@dstftw dstftw commented Dec 27, 2015

-f bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best.

@dstftw dstftw closed this Dec 27, 2015
@zmwangx
Copy link
Author

@zmwangx zmwangx commented Dec 27, 2015

I didn't realize filters work with bestvideo too. Thanks.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Dec 27, 2015

This is all covered in format selection.

@zmwangx
Copy link
Author

@zmwangx zmwangx commented Dec 27, 2015

I read that, but somehow missed the bestvideo[height<=?1080] example. Sorry.

@porg
Copy link

@porg porg commented Jan 2, 2016

@zmwangx : I had the very same objective, I had stated it to myself like this: The result must be a MP4 container -- for optimal Apple device compatibility plus for the usage of --embed-subs and --add-metadata -- and in the container I want the best available MP4 compatible video stream with no further limitations and the best MP4 compatible audio stream with a bitrate up to 128Kbps as I consider this satisfying.

@dstftw : Thanks for your command line! This is real insider knowledge, thanks. The documentation section on https://github.com/rg3/youtube-dl/#format-selection currently does not mention the possibility to use the extension filter. I had attempted a lot of trial and error. Lucky to now know of this possibility.

Nevertheless your command line can still not achieve to automatically get the maximum possible quality according to my criteria as stated in the beginning. In some examples the combined A/V file contains a better MP4 stream (higher resolution and bitrate) than the best MP4 DASH file, but nevertheless the DASH variation is chosen over it, or vice versa.

I have conducted intense testing and preliminary (possibly confusing) protocols, which I can provide if you are interested (maybe also contains false assumptions as I don't know the inner workings of youtube-dl), but I simply hand you over my "troublemaker test set" (a subset of PL5117944B5ABF7038 ) and you will realize yourself most likely:

youtube-dl -o "%(id)s -- Best MP4 video with best MP4 audio up to 128k -- %(format)s.%(ext)s" -i -f "bestvideo[ext=mp4]+bestaudio[ext=m4a][abr<=128]/best[ext=mp4]/best" b5GnVVX7Ceg swoxY3dChSo 4P1j7AMM3G4 3ZKMBEo8xZE fRO_Z38KhyY Lfp-5Wmlw2M A4qgxzqG40Q DVsvdGWv1Lk f9eSDdIp68w PpCglDxddZs

Then try to lookup --list-formats for them, state some other or manual format selections, and you will realize that in some cases you can achieve higher quality with manual tweaking.

@zmwangx
Copy link
Author

@zmwangx zmwangx commented Jan 2, 2016

In some examples the combined A/V file contains a better MP4 stream (higher resolution and bitrate) than the best MP4 DASH file.

The most important question is... Example (link/id please)? I've never seen such a video.

Sorry, nevermind, I didn't see ids on the command line (because it's long).

@porg
Copy link

@porg porg commented Jan 2, 2016

Example: A4qgxzqG40Q The A/V file has the same resolution but a higher bitrate, nevertheless DASH is chosen.

@porg
Copy link

@porg porg commented Jan 2, 2016

Example: fRO_Z38KhyY The MP4 A/V file has a higher resolution AND a higher bitrate, nevertheless MP4 DASH is chosen.
Not only is the stated A/V resolution higher (640x360) but also the actual resolution as determined with VideoSpec is higher (352x288=101.376px) as opposed to the VideoSpec determined resolution of (294x240=70.560px) for the DASH variation.

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented Jan 2, 2016

@porg

Nevertheless your command line can still not achieve to automatically get the maximum possible quality according to my criteria as stated in the beginning. In some examples the combined A/V file contains a better MP4 stream (higher resolution and bitrate) than the best MP4 DASH file, but nevertheless the DASH variation is chosen over it, or vice versa.

This is working as "expected" (bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best is designed to do exactly that), but hopefully some day we will add support for choosing the best of them (see #7034).

@porg
Copy link

@porg porg commented Jan 5, 2016

@jaimeMF I hope that one day youtube-dl will be able to really choose the best video stream or audio stream regardless wether contained in a single stream (DASH file) or multi stream container (muxed A/V file).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.