Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow merging formats #1612

Closed
phihag opened this issue Oct 17, 2013 · 21 comments
Closed

Allow merging formats #1612

phihag opened this issue Oct 17, 2013 · 21 comments

Comments

@phihag
Copy link
Contributor

@phihag phihag commented Oct 17, 2013

Allow individual formats in -f to consist of multiple format IDs, for example

-f 137+141/137+160/37/mp4+mp3

A format is picked only if all the IDs it consists of are present (and ffmpeg is available?). If this options is set, the format name is automatically appended to the video filename while downloading, i.e. we'll get title-id.ext.f137 and title-id.ext.f1141 by default. If a format with a plus is picked, the extracted information contains a _postprocessing: {type:"ffmpeg-merge", "files": ["title-id.ext.f137", "title-id.ext.f1141"]} hint, and ffmpeg is automatically invoked to create a single file.

@Elite
Copy link

@Elite Elite commented Oct 18, 2013

Great, this will take care of missing 1080p. Pls. do this with priority.

@KlaasOberlin
Copy link

@KlaasOberlin KlaasOberlin commented Oct 18, 2013

We also need -f 135 as 480p is now only available as dash video.

@xanadu
Copy link
Contributor

@xanadu xanadu commented Oct 18, 2013

This need to be put as default if other 1080p is not available.

@rbrito
Copy link
Contributor

@rbrito rbrito commented Oct 18, 2013

Perfect!

I just came here to file an issue asking for much less (saving audio from dash streams with an extension different from those of the videos---namely, saving AAC audio with extension m4a).

Otherwise, to download 480p videos (which I do for lectures and so on with other projects of mine, like edx-dl) I have to call youtube-dl twice: once for format 135 and another for format 140, since the old (?) format 35 files are much smaller than the lower resolution 360p files (due to the former being encoded in High profile vs. the latter being encoded in Constrained Baseline profile).

The way we are right now, calling youtube-dl once for format 135 results in a file ending with extension mp4 and doing a successive download call to format 140 gives me the message that the file was already downloaded.

I can obviously workaround this, renaming the files, taking care of the muxing of both the audio and the video etc., but that would be a pain.

Great that other people also felt the same way that I did. :)

Regards,

Rogério.

@rbrito
Copy link
Contributor

@rbrito rbrito commented Oct 18, 2013

Another thing: the templates for file naming don't seem to have (after a quick look) a string for the format of the video. This would be a very welcome addition, IMVHO.

It would also fix a bug that I received in Debian quite some time ago: bugs.debian.org/cgi-bin/bugreport.cgi?bug=643831

Should I open a separate issue for the bug above?

Thanks.

@yasoob
Copy link
Contributor

@yasoob yasoob commented Oct 18, 2013

That is entirely a separate bug. I think you should open a separate ticket.

On Sat, Oct 19, 2013 at 1:27 AM, Rogério Brito notifications@github.comwrote:

Another thing: the templates for file naming don't seem to have (after a
quick look) a string for the format of the video. This would be a very
welcome addition, IMVHO.

It would also fix a bug that I received in Debian quite some time ago:
bugs.debian.org/cgi-bin/bugreport.cgi?bug=643831

Should I open a separate issue for the bug above?

Thanks.


Reply to this email directly or view it on GitHubhttps://github.com//issues/1612#issuecomment-26627161
.

@xanadu
Copy link
Contributor

@xanadu xanadu commented Oct 18, 2013

@rbrito

$ youtube-dl b6vSf0cA9qY -o "%(format)s.%(ext)s"
[youtube] Setting language
[youtube] b6vSf0cA9qY: Downloading video webpage
[youtube] b6vSf0cA9qY: Downloading video info webpage
[youtube] b6vSf0cA9qY: Extracting video information
[download] Destination: 22 - 720x1280.mp4
[download] 100% of 46.49MiB in 00:14
phihag added a commit that referenced this issue Oct 18, 2013
@phihag
Copy link
Contributor Author

@phihag phihag commented Oct 18, 2013

@rbrito I have documented the format name in the filename template in fdefe96.

@rbrito
Copy link
Contributor

@rbrito rbrito commented Oct 18, 2013

@phihag, thanks. I missed that, but I also sent a micro-pull request with another way of working around the output naming.

@ghost
Copy link

@ghost ghost commented Oct 28, 2013


@phihag
Copy link
Contributor Author

@phihag phihag commented Oct 28, 2013

@hdevelop Thank you, we are well aware of basic ffmpeg muxing. If you post, please don't include the names of malicious software. Thanks!

@rbrito
Copy link
Contributor

@rbrito rbrito commented Oct 28, 2013

Just for the record, when muxing we may want to partially get rid of
youtube-only metadata via -map_metadata -1.

With this option turned on, two muxes of the same audio and video are
(at least in my tests) bit-identical (provided you use the same
versions of ffmpeg/avconv, since they include a tag along the lines
"encoded by libavfoo ").

Nice way to help deduplicating filesystems if you happen to download
the same file twice (which I frequently do, from playlists of
lectures, like those from edX).

Regards,

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

@ghost
Copy link

@ghost ghost commented Oct 28, 2013


@ericzutter
Copy link

@ericzutter ericzutter commented Oct 29, 2013

With Windows Media Player Classic when you play a DASH-video (mp4), it automatically detects the DASH-audio (m4a). To make it work, I changed the following setting :
enable menu view+options+player+formats+MPEG-4 Audio+choose DirectShow instead of QuickTime

I don't know if the other video players also automatically detects m4a-files when playing mp4-files. I would prefer that in youtube-dl, there is an option to enable/disable the automatically merge of DASH-video+DASH-audio to a single file with ffmpeg.

@shantiq
Copy link

@shantiq shantiq commented Nov 19, 2013

ok thanx to info by jaimeFM managed a 4k here which plays on my ubuntu perfectly with mpv https://launchpad.net/~mc3man/+archive/mpv-tests

youtube-dl -t -f 138 k_okcNVZqqI
[youtube] Setting language
[youtube] k_okcNVZqqI: Downloading video webpage
[youtube] k_okcNVZqqI: Downloading video info webpage
[youtube] k_okcNVZqqI: Extracting video information
[download] Destination: 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.mp4
[download] 100% of 376.18MiB in 12:50

mediainfo '00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.mp4'
General
Complete name : 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.mp4
Format : dash
Codec ID : dash
File size : 376 MiB
Duration : 3mn 4s
Overall bit rate : 17.1 Mbps
Encoded date : UTC 2013-09-16 07:46:45
Tagged date : UTC 2013-09-16 07:46:45

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : No
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3mn 4s
Bit rate mode : Constant
Bit rate : 17.1 Mbps
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 23.976 fps
Minimum frame rate : 23.974 fps
Maximum frame rate : 2 195.122 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.086
Stream size : 376 MiB (100%)
Encoded date : UTC 2013-09-16 07:46:45
Tagged date : UTC 2013-09-16 07:46:45

4kdrops.mp4
General
Complete name : 4kdrops.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 382 MiB
Duration : 3mn 4s
Overall bit rate : 17.4 Mbps
Writing application : Lavf55.19.103

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : No
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3mn 4s
Bit rate : 17.1 Mbps
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.086
Stream size : 376 MiB (99%)

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 3mn 4s
Bit rate mode : Constant
Bit rate : 254 Kbps
Channel count : 2 channels
Channel positions : Front: L R
Sampling rate : 44.1 KHz
Compression mode : Lossy
Delay relative to video : 42ms
Stream size : 5.58 MiB (1%)

then audio

youtube-dl -t --extract-audio -f 141 k_okcNVZqqI
[youtube] Setting language
[youtube] k_okcNVZqqI: Downloading video webpage
[youtube] k_okcNVZqqI: Downloading video info webpage4kdrops.mp4
General4kdrops.mp4
General
Complete name : 4kdrops.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 382 MiB
Duration : 3mn 4s
Overall bit rate : 17.4 Mbps
Writing application : Lavf55.19.103

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : No
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3mn 4s
Bit rate : 17.1 Mbps
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.086
Stream size : 376 MiB (99%)

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 3mn 4s
Bit rate mode : Constant
Bit rate : 254 Kbps
Channel count : 2 channels
Channel positions : Front: L R
Sampling rate : 44.1 KHz
Compression mode : Lossy
Delay relative to video : 42ms
Stream size : 5.58 MiB (1%)

Complete name : 4kdrops.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 382 MiB
Duration : 3mn 4s
Overall bit rate : 17.4 Mbps
Writing application : Lavf55.19.103

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : No
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3mn 4s
Bit rate : 17.1 Mbps
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.086
Stream size : 376 MiB (99%)

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 3mn 4s
Bit rate mode : Constant
Bit rate : 254 Kbps
Channel count : 2 channels
Channel positions : Front: L R
Sampling rate : 44.1 KHz
Compression mode : Lossy
Delay relative to video : 42ms
Stream size : 5.58 MiB (1%)

[youtube] k_okcNVZqqI: Extracting video information
[download] Destination: 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.m4a
[download] 100% of 5.61MiB in 00:11
[youtube] Post-process file 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.m4a exists, skipping

to bind mute mp4 and sound m4a 256k

ffmpeg -i INK.mp4 -i INK.m4a -c copy 4kdrops.mp4

which gives us

4kdrops.mp4
General
Complete name : 4kdrops.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 382 MiB
Duration : 3mn 4s
Overall bit rate : 17.4 Mbps
Writing application : Lavf55.19.103

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : No
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 3mn 4s
Bit rate : 17.1 Mbps
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.086
Stream size : 376 MiB (99%)

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 3mn 4s
Bit rate mode : Constant
Bit rate : 254 Kbps
Channel count : 2 channels
Channel positions : Front: L R
Sampling rate : 44.1 KHz
Compression mode : Lossy
Delay relative to video : 42ms
Stream size : 5.58 MiB (1%)

@lxyu
Copy link

@lxyu lxyu commented Jan 16, 2014

Good to see it solved.

While I think it would be better to auto delete the orig files after they were successful merged.

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented Jan 16, 2014

@lxyu see #2108, that's a reasonable idea, we'll implement it some day (contributions welcommed!)

@Glideh
Copy link

@Glideh Glideh commented Feb 28, 2014

Is the doc updated somewhere ? (I'm using 2014.02.27.1)
From my understanding, + is used for specifying merged formats with -f.
Example:

$ youtube-dl -f 137+140 http://www.youtube.com/watch?v=yPeH888D9Hg

Would download both 137 & 140 formats and merge them into a new files (3 files created)
Is that correct ?

@jaimeMF
Copy link
Collaborator

@jaimeMF jaimeMF commented Feb 28, 2014

@Glideh No, the feature is not documented because there are some things that need to be improved.
You're right, using -f 137+140 will download formats 137 and 140 and then merge them into a single file (you'll need to have ffmpeg or avconv installed).

@Glideh
Copy link

@Glideh Glideh commented Feb 28, 2014

Ho ok thanks. It works here

@seed3r
Copy link

@seed3r seed3r commented Jun 27, 2017

what is the combination of '18' format id?
?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.