--prefer-free-formats does not prefer webm to mp4 #6018
Comments
|
This is expected behavior. |
|
I think the option documentation should mention that. Right now it reads: |
|
The bitrate will never be exactly the same, it will always differ slightly. But higher bit rate doesn't imply higher quality, it depends on the codec. So that assumption is somewhat off. But as @rg3 says, that option is not about quality, at least as it is currently documented. It says "pick the free format, if it is available". It does not speak about quality at all. So maybe the current implementation is expected by the developers, but certainly not by the users. And, if only quality is concerned, what would be the point of such an option? If mp4 has slightly better bitrate, it is picked. If webm has slightly better bitrate, it is picked. Such option would have no effect at all, because it is already the default behavior! So really, this option should mean "I always want free formats, if they are available, otherwise fall back to proprietary formats". That's how I want to configure youtube-dl. Please make it possible. Thank you. |
I can use this, but I guess that will fail for videos which don't have a webm version. So I would have to do this: However, when I download videos for my tablet (not PC), I want to avoid 4K videos. So I need height filters. And then the filter gets really complicated. I would much rather have |
|
Opening as I think |
|
@kparal |
|
What I'm saying is that there's no easy way to prefer free formats (regardless of quality, literally prefer) without writing an explosive set of command line arguments. That's also the case of aforementioned #2196 - it gives a workaround, but it's neither simple nor clean, if you want to cover all the cases. (And I don't think anything was rejected - "you are welcome to contribute a patch that adds one"). @dstftw Can you please clarify what is the purpose of In my understanding "require" has different meaning than "prefer". When you require a free format, you pick the best free format, and if it is not available, you fail with an error ("free format required, but none available"). When you prefer a free format, you pick the best free format, and if it is not available, you pick the best non-free format. |
|
One further argument - some Linux distributions like Fedora add The reason is that h264/aac/other codecs are neither installed by default nor offered in the repositories, because of patent issues. So the wish of the package maintainers of youtube-dl is to allow all Fedora/other distro users to make as many videos playable by default as possible - that means they want to configure youtube-dl to download free formats every time it is possible to do so. So that it is really useful for all users, and not just for those who live outside of USA + other countries where software patents apply and installed proprietary codecs. They don't want to require it, because downloading something is better than downloading nothing, but they want to prefer it as much as possible. |
|
I agree with @kparal, I think that with |
|
@kparal I've already stated the purpose of it - downloading free format when both formats are equal according to metadata describing quality. youtube-dl supports not only YouTube so the probability is much higher especially if some metadata is not available.
Huh? Not a random but best quality format according to metadata. |
|
I believe (so basically I'm inventing stuff as I write, haha) people using this option are probably interested in downloading free formats if available, just like the current option documentation suggests. So if a given website (YouTube or other) offers videos in either MP4 or Webm, using --prefer-free-formats should pick a Webm one, and should only download MP4 if no free formats are available. Once that "format filter" has been applied, it should choose the highest quality one, or obey other format restrictions the user may have specified. But then we can raise other questions. For example, someone may want to have --prefer-free-formats in the config file, but in a specific case (s)he may want to specify a format filter that wouldn't work with that, so they need a second option to override the config file setting in a given program call (as in --no-prefer-free-format). Just my two cents. You know I don't want to have any voting weight in the development process and I don't even use that option, so it's irrelevant to me. :P |
So IIUIC And it doesn't help at all to those users who don't have proprietary codecs installed, and want to download free fomats without hand-picking them every time. @dstftw For you, quality might be the most important factor. But this belief is not universal and other groups of users might have different priorities. Like open formats. I'm one of them, I believe that patented formats need to be purged from this world and I'll gladly sacrifice some quality because of it. It would be great to have an option which would allow such groups of users to use youtube-dl easily. It can be a new option, if needed (even though in my personal opinion it would make most sense to just use the current one, especially since it currently doesn't seem to have any effect on the largest and most widely used video site). |
|
@dstftw I think we should only take into account the resolution for |
|
@jaimeMF if some hoster serves formats of the same resolution but really bad bitrate for free format it will end up selecting free format for
So, with your suggestion the general semantics of |
Yes, that's true, but my point is that deciding if two formats have the same quality (for It prefers 720 mp4 over 1080 webm based just on the bitrate, for me it's more sensible to prefer resolution. But I understand it's not so simple (and my knowledge is quite limited, so I may be completely wrong).
I wouldn't say it changes, it would "fix" it, since that's how it was supposed to work (at least that's my impression when looking into the original implementation, it seem that it picked the mp4 file if it was the one with the highest resolution). Additionally with that behaviour you could use formats filters like Anyways, since this is probably a bit subjective and I don't use that option, I don't really mind what it does as long as we clearly document what it does and add tests to verify we don't break it. |
|
Thanks @jaimeMF my interpretation of Prioritizing resolutions than bitrates is reasonable. Technically free formats often have lower bitrates than non-free formats. Take VP9 for example, it aims to "Reduce video bitrate by 50% with image quality comparable to VP" ( VP9 requirements ). In other words, bitrates is not a synonym of quality. In real cases, it's uncommon for VP9 to have higher bitrates than H264. As a result, relying on bitrates is problematic. |
…sorting formats (Closes #6018, closes #8001)
…sorting formats (Closes #6018, closes #8001)
|
I always thought that If I'm specifying this option, odds are that I either:
So I think it's fair to choose a lower-quality free format over a higher-quality fair proprietary format when this option is specified. |
|
That should probably be a separate flag, such as "--only-free-formats". The main issue was that while getting only free formats was relatively easy ("-f bestvideo[ext=webm]+bestaudio[ext=webm]/best[ext=webm]"), it got really complicated if you wanted to get the highest resolution video, and then prefer the free format if it had one which would lead to such horrors as: with entries for every possible video size you could expect. This was further complicated by the fact that youtube periodically introduces higher resolutions (such as 1440), and things like 3D videos, which mean you have to continually retune the script to add new resolutions to a massive list. |
|
This seems like a conflict of interests. If you make this behavior a separate flag, then There's no inherent reason to prefer free formats over proprietary formats other than ideological ones. The way I see it, the gigantic format string you are describing is born from a need to work-around a different youtube-dl limitation: The fact that it doesn't understand that VP9 is higher quality than AVC even at lower bitrates. But why are we spending so much time focusing on a work-around instead of improving the core problem here? Note: I use a similar such work-around, namely: But I consider this a pure work-around simply because I really don't care if my video stream is VP9 or AVC. I'm just special-casing VP9 because of aforementioned youtube-dl bug. |
|
This should work: |
|
Thanks a lot! |
|
In the case I posted above it will only select opus when a video is found I should mention, for pure audio streaming sites it falls back to "best" |
|
Thanks again, got it working. :) |
|
@utack How is it going to look for WebM container with Vorbis video and Opus audio? |
|
To those who suggest that it's simple to replicate the preference / requirement for free formats with To make everyone happy, perhaps we could keep And while it's not entirely on-topic here, I have also noticed that youtube-dl incorrectly prefers H.264 over VP9 due to the slightly higher bitrate. I don't know what's the best solution to that problem though, as you can't necessarily just say VP9 is always better than AAC, because there's a possibility a site will make a worse-quality VP9 encode than H.264 for the same resolution. |
The approach that always pops in my head is introducing “bitrate coefficients”, perhaps based on some commonly available codec benchmark. (Basically, the idea is that if VP9 encodes ~20% smaller files than H.264 at the same visual quality, as assessed by an objective quality metric such as MS-SSIM, then you can multiply 1.2 into the VP9 bitrate) |
|
@haasn Has anyone found a VP9 with same resolution and FPS that looked worse than the H264 counterpart yet? |
|
@utack I remember seeing some pathological VP9 encodes of video game streams a year or two ago, specifically Dwarf Fortress. May have changed in the meantime, and I may be misremembering, but afaik you'll still find weird clips like that where x264 is still hard to beat, in some of the areas it's specifically tuned for. |
Well, at least not on YouTube. |
For lack of anything better, we could use netflix's results (which are normalized to the VMAF metric). This seems to suggest that e.g. at 1080p, a coefficient of “1.4 * bitrate” for VP9 and “1.5 * bitrate” for HEVC would be good candidates. (This scale would presumably normalized such that x264 represents the reference) |
|
Even if we decided the "coefficient" for each format, how we're going to deal with the fact that YouTube sometimes straightly reports wrong bit-rate? |
|
Probably by ignoring the “bitrate” and using the filesize, or something. Actually, looking at e.g. Comparing formats 298 and 302 specifically, if we assume 3348k is the true bitrate for the 683.20MiB H.264 stream, then |
|
Well it doesn't always return file-size. |
|
@nyuszika7h
Add --require-free-formats was declined but with your vote on the issue they might change their mind. |
Example? |
|
Which is also an example I mentioned before that reports wrong bitrate (original report: #11451), but it seems YouTube already corrected the bitrate issue for this particular one (format 136 now correctly shows lower bitrate than 137).
This still shows no file-size information and wrong bitrate info It reports actual bit rates: 135: 112kbps 133: 32kbps I know those are not "free formats", but this topic has already became the place to discuss "should we choose best video/audio merely based on bitrate (current behavior)", and the fact YouTube sometimes returns totally erroneous bit rate info should be taken into consideration before we start theorizing things like "weighted bitrate". I'm still baffled why we don't use resolution first as quality indicator, which has virtually no drawbacks. |
@nyuszika7h : the problem is that this is not how it behaves as-is. If that was the case, everyone would be happy indeed. |
|
I agree with the comments made by @fireattack above here and here that resolution should be used instead of bitrate for the Rather than continue to hijack the One unqualified demerit of using bitrate--or for that matter, anything-rate--during the Given a "most pristine" master copy which must exist for every video, entropy can be dissipated in several dimensions, but duration time is not one of them. Either the compression algorithms are more/less efficient, or the frame dimensions or frame rate are different. And it seems to me that given the special constraint discussed above, the latter two will swamp the former in all cases, making some sensible combination of dimension and fps a much more stable indicator of "best." I think that, possibly, people have gotten in the mindset of elevating bitrate because it is quite likely a better indicator of quality when comparing dissimilar source material, but this habit may be obscuring better judgment when it comes to the specialized task of comparing all-alike material. Irrefutably, really, bitrate is just a needlessly complex proxy for the total entropy each available format ultimately makes available. Add in the fact that the YouTube reported bitrate values are often wildly wrong (in fact far more so than this thread has reported; see #14143) and it's clear that by the nature of this type of selection task, bitrate is not the way to go. So as I've noted, the |
|
I use this setting in mpv to always get VP9 + Opus/Vorbis, as suggested by utack (thanks again!): |
|
@aufkrawall Yeah, it gets identifies as vp9.2 I use this: |
|
Great, just added vp9.2 and it works like a charm. |
Specify the format (ex. best[ext=webm]) to use open-source codecs. ytdl-org/youtube-dl#6018
|
When people visit youtube from chrome, and click on a video, and hand pick the highest resolution, (for instance 1080 60fps), is there a good chance youtube would arbitrarily decide to serve webm? If yes, maybe we could have an flag Let's face it, newer codecs are expected to have lower bitrates. That's the goal when they were developed. |
and others
Please note that
[debug] System config: [u'--prefer-free-formats']shows that I have preference to free formats enabled, butyoutube-dlwants to downloadmp4instead (266+140).The text was updated successfully, but these errors were encountered: