Prefer WebM when available

[jankratochvil]

$subj says it all.
WebM is high quality Free codec with widely available decoding support so when the same resolution is available in multiple formats incl. WebM choose WebM.

[jankratochvil]

You need to review it anyway so here is just a patch:

• _available_formats = ['38', '37', '22', '45', '35', '44', '34', '18', '43', '6', '5', '17', '13']
• _available_formats = ['38', '37', '45', '22', '44', '35', '43', '34', '18', '6', '5', '17', '13']

[phihag]

We already tried that and did prefer webm for a time (to be precise, from 2011.09.06 to 2011.09.18b). Users immediately complained (see #165 and #168). You can prefer webm with --format 38/37/45/22/44/35/34/18/6/5/17/13.

I think we should have a configuration file so that everyone can set his or her preference (and debian would probably serve it with webm preference). However, you can easily configure a shell alias or install a wrapper script to the same effect.

[jankratochvil]

I do not see it patched in Debian unstable.
Anyway filed it for Free Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=757577
Thanks for the reply.

[phihag]

Any idea on how this could be solved without having to patch youtube-dl downstream? Does Fedora allow to install a default configuration (maintained by the package maintainer) into /etc/youtube-dl or so?

[jankratochvil]

I do not see any existing Fedora patches downstream for some custom configurations: http://pkgs.fedoraproject.org/gitweb/?p=youtube-dl.git;a=tree;h=refs/heads/master;hb=master
I have proposed there a normal patch. There is quilt and even GIT to maintain the Fedora patches up to date.
In fact it could be also "reconfigured" from /etc/profile.d/ like I do it in my ~/.bashrc: http://git.jankratochvil.net/?p=nethome.git;a=commitdiff;h=31ce090f436babfc7f4819096cfff9cc34bd667c
But I find the patch the normal way, for example I maintain 117 Fedora patches for GDB myself, some pretty heavy ones: http://pkgs.fedoraproject.org/gitweb/?p=gdb.git;a=tree;h=refs/heads/master;hb=master

[phihag]

Sure, downstream maintainers are certainly free to maintain patches. However, that only complicates their life, and I personally do prefer WebM as well, so why not make life easy for everyone?

I'd love to have Till Maas or @rbrito chip in. Could you ship a configuration file /etc/youtube-dl.conf (or similar, but should be the same on both distributions) with the content

--prefer-free-formats

? If not, is there any other way upstream can make your life easier?

[jankratochvil]

Sure an upstream implementation of both a config file read-in and some generic Free-formats preferring option would be ... preferred. Distro can then just ship a distro-specific config file, without maintenance of a code patch, thanks.

[rbrito]

@jankratochvil, I use to patch youtube-dl in the packages that I provide to Debian, but having to patch it all the time that I uploaded something new was killing me (I already orphaned some packages in Debian that I used to maintain, but I'm not really giving up on youtube-dl, especially because I use it way too frequently), and I am really short on time these few weeks.

Things will become saner for me starting with the 15th of December and I will have free time during the holidays to hack on things, have fun playing with experimental stuff... For instance, I hope to integrate the ability to grab subtitles in the free time that I will have. If possible, implement the selection of quality of videos for other information extractors and generalize that in all the classes...

The idea of @phihag of having a configuration file like /etc/youtube-dl.conf (that is gracefully ignored if not present) is very nice, IMVHO, but having a system-wide configuration brings us another point: we should have per-user configuration files (say, .youtube-dl.rc or whatever is the preferred configuration location of the day), so that users can override the preferences stipulated by the sysadmin...

OK, so this was much longer than you probably expected, but I'm also giving you some details of the next steps... :)

[phihag]

@rbrito Configuration files /etc/youtube-dl.conf and ~/.youtube-dl.conf are now read. I'll probably have to document them and add a --config file option. Adding a --prefer-free-formats switch doesn't seem trivial, so it may take some time on my end as well.

[tyll]

How about using $XDG_CONFIG_HOME/youtube-dl.conf (or $HOME/.conf/youtube-dl.conf) by default for the per user config file to follow current standard processes.

[phihag]

Reopening until we have --prefer-free-formats.

[non7top]

Possible solution could be to add option similar to '--formats', i.e. '--video-formats webm/mp4/flv', though naming of options becomes a bit confusing.

[rbrito]

Hi there.

2011/11/30 Vladimir Berezhnoy
reply@reply.github.com:

Possible solution could be to add option similar to '--formats', i.e. '--video-formats webm/mp4/flv', though naming of options becomes a bit confusing.

Why not make --prefer-free-formats (or whatever) to be an alias to
--format 45/44/43/(...) and be done with it for the moment, and note
that this only works for youtube because only youtube offers (now)
free video formats?

Regards.

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://rb.doesntexist.org : Packages for LaTeX : algorithms.berlios.de
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[non7top]

After a comment from rbrito I wonder if there is any sense in adding one more option, while the same behaviour can be achieved by using --format 45/44/43/(...). This will be more confusing for user experience.

[rbrito]

On Dec 01 2011, Vladimir Berezhnoy wrote:

After a comment from rbrito I wonder if there is any sense in adding one
more option, while the same behaviour can be achieved by using --format 45/44/43/(...). This will be more confusing for user experience.

Agreed. If this is going to a configuration file, anyway, we can simply put
that in there and move along.

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://rb.doesntexist.org : Packages for LaTeX : algorithms.berlios.de
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[jankratochvil]

The problem is after YouTube adds new dimensions/extensions=formats such extra free-preferring configuration file (such as my current alias in ~/.bashrc) will become stale. The new dimensions/extensions will get disabled by using such configuration file. --prefer-free-formats would still stay working.

[non7top]

jankratochvil> --prefer-free-formats would still stay working.
and how would that happen? someone will still have to add new format in youtube-dl code and define it as a free format

A better option would be to add alias '--format webm/mp4/flv'

[jankratochvil]

The new format to youtube-dl gets added by upstream. The new format will not be noticed by distro maintainers who just rebase to a new upstream release but the distro still would have --format 45/44/43/(...) in their distro config.

[rbrito]

Just for the record, I'm not disagreeing here with anybody.
I am agreeing and giving some of my background on this.

On Dec 04 2011, Jan Kratochvil wrote:

The new format to youtube-dl gets added by upstream. The new format will
not be noticed by distro maintainers who just rebase to a new upstream
release but the distro still would have --format 45/44/43/(...) in their
distro config.

Geee... Besides contributing some parts to youtube-dl, I'm a distro
maintainer (I package this for Debian---and Ubuntu as a side effect).

I'm not rebasing anything on anything, as that was too much work, especially
with (sometimes really needed) rapid-fire releases. And, of course, I would
like to be as faithful to the upstream releases as possible.

From a distro point-of-view, the best is to just ship an unmodified
youtube-dl script, ship an adequate configuration file and be done with it,
almost never updating anything.

I already have divergence on my own tree, where I maintain a branch called
use-other-downloaders to use aria2c to really speed up things when
downloading stuff (and, honestly, this is the version that I use on my own,
not the versions that I package myself).

Regards,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://rb.doesntexist.org : Packages for LaTeX : algorithms.berlios.de
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

[jankratochvil]

FYI found now that:
$ youtube-dl -F http://www.youtube.com/watch?v=DTIqyurSTr0
Available formats:
22 : mp4 [720x1280]
45 : webm [720x1280]
...
$ youtube-dl -f 45 http://www.youtube.com/watch?v=DTIqyurSTr0
...
[youtube] DTIqyurSTr0: Extracting video information
ERROR: Did not get any data blocks
$ youtube-dl -f 22 http://www.youtube.com/watch?v=DTIqyurSTr0
...
[youtube] DTIqyurSTr0: Extracting video information
[download] Destination: DTIqyurSTr0.mp4
[download] 23.7% of 39.00M at 71.42k/s ETA 07:06

Sure it may be just some server-side problem but preferring WebM apparently breaks the functionality in this case.

[phihag]

@jankratochvil That's just a server-side issue which has been cleaned up by now. This happens from time to time with mp4 as well.