Merged format fails with %(format_id)s in output template path

[glenn-slayden]

An output template such as:

--output C:/yt%(format_id)s/%(id)s.%(ext)s

...crashes when the format_id refers to a merged format such as 136+140, because the template resolves to different directories during different stages of the download.

youtube_dl --ignore-config 
          --verbose
          --write-info-json
           --ffmpeg-location "C:\Program Files\ffmpeg\bin" 
           -f 136+140 
           --output "C:/yt%(format_id)s/%(id)s.%(ext)s" 
           "https://www.youtube.com/watch?v=BaW_jenozKc"

This command fails with [Errno 2] No such file or directory: 'C:\yt136\BaW_jenozKc.f136.mp4.part'. The full output is below; note that the Json file was correctly downloaded to C:\yt136+140\..., but the video download fails, possibly because it's targeting a directory C:\yt136\... which does not exist.

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-config', '--verbose', '--write-info-json', '--ffmpeg-location', 'C:\\Program Files\\ffmpeg\\bin', '-f', '136+140', '--output', 'C:/yt%(format_id)s/%(id)s.%(ext)s', 'https://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2017.09.02
[debug] Python version 3.6.0b3 - Windows-10-10.0.15063-SP0
[debug] exe versions: ffmpeg 3.1.5, ffprobe 3.1.5
[debug] Proxy map: {}
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading video info webpage
[youtube] BaW_jenozKc: Extracting video information
[youtube] BaW_jenozKc: Downloading MPD manifest
[info] Writing video description metadata as JSON to: C:\yt136+140\BaW_jenozKc.info.json
[debug] Invoking downloader on 'https://r3---sn-nx5e6nez.googlevideo.com/videoplayback?id=05a5bf8de9e8cca7&itag=136&source=youtube&requiressl=yes&initcwndbps=1697500&mm=31&ei=Bd6tWaTlFczK-gPw2rDIDw&ms=au&mn=sn-nx5e6nez&mv=m&pl=16&ratebypass=yes&mime=video/mp4&gir=yes&clen=1673012&lmt=1387961826998447&dur=9.800&key=dg_yt0&signature=3CD2D31737CCD4655CA6DF6E8F8A868654D545C4.5FF6A98DC4D6FAE33FA26774B9C2244E92994706&mt=1504566668&ip=71.231.140.159&ipbits=0&expire=1504588389&sparams=ip,ipbits,expire,id,itag,source,requiressl,initcwndbps,mm,ei,ms,mn,mv,pl,ratebypass,mime,gir,clen,lmt,dur'
ERROR: unable to open for writing: [Errno 2] No such file or directory: 'C:\\yt136\\BaW_jenozKc.f136.mp4.part'
Traceback (most recent call last):
  File "D:\programming-others\youtube-dl-master\youtube_dl\downloader\http.py", line 200, in download
    ctx.tmpfilename, ctx.open_mode)
  File "D:\programming-others\youtube-dl-master\youtube_dl\utils.py", line 454, in sanitize_open
    stream = open(encodeFilename(filename), open_mode)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\yt136\\BaW_jenozKc.f136.mp4.part'

[glenn-slayden]

Additional information from my examination of the code:
The problem is not related to the "plus" + character being (incorrectly) split out during "sanitizing" or encoding, etc. of the filename path. Rather, the filename that crashes is "correctly" constructed from the prevailing format_id value in the template_dict at the time the filename is built from the template.

What's incorrect, then, is the merge code, which seems to be assuming that the directory locations for the temporary audio+video files either necessarily all indicate the same location, or are pre-existing, or both.

This analysis is further supported by the following output, where the format_id does not appear in the "path" portion of the template. In this case the merged file C:\yt\BaW_jenozKc.137+140.mp4 is successfully created from temporary files C:\yt\BaW_jenozKc.137.f137.mp4 and C:\yt\BaW_jenozKc.140.f140.m4a

[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading video info webpage
[youtube] BaW_jenozKc: Extracting video information
[youtube] BaW_jenozKc: Downloading MPD manifest
[info] Writing video description metadata as JSON to: C:\yt\BaW_jenozKc.137+140.info.json
[download] Destination: C:\yt\BaW_jenozKc.137.f137.mp4
[download] 100% of 2.11MiB in 00:00
[download] Destination: C:\yt\BaW_jenozKc.140.f140.m4a
[download] 100% of 154.06KiB in 00:00
[ffmpeg] Merging formats into "C:\yt\BaW_jenozKc.137+140.mp4"
Deleting original file C:\yt\BaW_jenozKc.137.f137.mp4 (pass -k to keep)
Deleting original file C:\yt\BaW_jenozKc.140.f140.m4a (pass -k to keep)
[ffmpeg] Adding metadata to 'C:\yt\BaW_jenozKc.137+140.mp4'

[glenn-slayden]

@dstftw Thanks for the fix, it works pretty well with one exception... After the merge, when the temporary .f136.mp4 and .f140.m4a files are deleted, the (now-empty) directories (i.e. ...\136\..., ...\140\... remain.

Would there be a way to store a flag value that indicates whether each of those directories had been created during a particular download operation, and if so, delete the directory when the single temporary file it contains is deleted? It could skip this step if it contains any files other than the one it expects to be there.

If this is worth fixing, would you want me to open a new bug for this?