Videos are added to download archive even if they weren't downloaded due to no space left on device

[snipem]

I've implemented a python script that invokes youtube-dl in a loop. I recognized that files are added to the download archive even if they weren't successfully downloaded because of no space left on the target device.

I'm using the latest version of youtube-dl from pypi youtube_dl-2014.10.05.2-py2.7.egg.

2014-10-15 22:42:36,982 [DEBUG] [YoutubeDL   ]  [download] Downloading video #5 of 87
2014-10-15 22:42:36,989 [DEBUG] [YoutubeDL   ]  [youtube] REuouBWHJRE: Downloading webpage
2014-10-15 22:42:37,456 [DEBUG] [YoutubeDL   ]  [youtube] REuouBWHJRE: Downloading video info webpage
2014-10-15 22:42:37,863 [DEBUG] [YoutubeDL   ]  [youtube] REuouBWHJRE: Extracting video information
2014-10-15 22:42:38,396 [DEBUG] [YoutubeDL   ]  [download] Destination: /mnt/commuteUSB/Download/NextGenGamingBlog-PES 2015 Preview - Menus-REuouBWHJRE.mp4
2014-10-15 22:42:38,398 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 74.08MiB at 282.64KiB/s ETA 04:32
2014-10-15 22:42:38,410 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 74.08MiB at 194.79KiB/s ETA 06:31
2014-10-15 22:42:38,412 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 74.08MiB at 402.39KiB/s ETA 03:08
2014-10-15 22:42:38,415 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 74.08MiB at 737.24KiB/s ETA 01:43
2014-10-15 22:42:38,418 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 74.08MiB at  1.28MiB/s ETA 00:58
2014-10-15 22:42:38,433 [DEBUG] [YoutubeDL   ]  

2014-10-15 22:42:38,435 [ERROR] [YoutubeDL   ]  ERROR: unable to write data: [Errno 28] No space left on device
2014-10-15 22:42:38,438 [DEBUG] [YoutubeDL   ]  [download] Downloading video #6 of 87
2014-10-15 22:42:38,444 [DEBUG] [YoutubeDL   ]  [youtube] HRjGfPZ3hP8: Downloading webpage
2014-10-15 22:42:38,909 [DEBUG] [YoutubeDL   ]  [youtube] HRjGfPZ3hP8: Downloading video info webpage
2014-10-15 22:42:39,331 [DEBUG] [YoutubeDL   ]  [youtube] HRjGfPZ3hP8: Extracting video information
2014-10-15 22:42:39,858 [DEBUG] [YoutubeDL   ]  [download] Destination: /mnt/commuteUSB/Download/NextGenGamingBlog-PES 2015 Preview - Master League-HRjGfPZ3hP8.mp4
2014-10-15 22:42:39,861 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 164.99MiB at 267.24KiB/s ETA 10:41
2014-10-15 22:42:39,872 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 164.99MiB at 191.52KiB/s ETA 14:43
2014-10-15 22:42:39,875 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 164.99MiB at 397.32KiB/s ETA 07:05
2014-10-15 22:42:39,877 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 164.99MiB at 738.44KiB/s ETA 03:49
2014-10-15 22:42:39,884 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 164.99MiB at  1.12MiB/s ETA 02:27
2014-10-15 22:42:39,894 [DEBUG] [YoutubeDL   ]  

2014-10-15 22:42:39,895 [ERROR] [YoutubeDL   ]  ERROR: unable to write data: [Errno 28] No space left on device

Download Archive:

...
youtube REuouBWHJRE
youtube HRjGfPZ3hP8
...

Edit 2:

On the other hand, other error-nous downloads do not show up in the download archive:

2014-10-17 01:18:45,163 [DEBUG] [YoutubeDL   ]  [download] Downloading video #47 of 90
2014-10-17 01:18:45,169 [DEBUG] [YoutubeDL   ]  [youtube] B9XdGlnLuhY: Downloading webpage
2014-10-17 01:18:45,593 [DEBUG] [YoutubeDL   ]  [youtube] B9XdGlnLuhY: Downloading video info webpage
2014-10-17 01:18:45,971 [DEBUG] [YoutubeDL   ]  [youtube] B9XdGlnLuhY: Extracting video information
2014-10-17 01:18:47,077 [DEBUG] [YoutubeDL   ]  [download] Destination: /mnt/commuteUSB/Download/GameTrailers-Pach-Attack! - How Does PlayStation Plus Make Money-B9XdGlnLuhY.mp4
2014-10-17 01:18:47,078 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 131.27MiB at 558.64KiB/s ETA 04:06
2014-10-17 01:18:47,095 [DEBUG] [YoutubeDL   ]  
[download]   0.0% of 131.27MiB at 157.70KiB/s ETA 14:13
...
[download]   6.1% of 131.27MiB at  5.23MiB/s ETA 00:23
2014-10-17 01:18:48,611 [DEBUG] [YoutubeDL   ]  
[download]   6.1% of 131.27MiB at  5.21MiB/s ETA 00:23
2014-10-17 01:18:48,613 [DEBUG] [YoutubeDL   ]  
[download] 100% of 131.27MiB in 00:01
2014-10-17 01:18:48,615 [ERROR] [YoutubeDL   ]  ERROR: content too short (expected 137645455 bytes and served 8389292)

The Youtube ID "B9XdGlnLuhY" isn't registered in the download archive.

Edit 1:

In my oppinion the reason for the behavior can be found here: https://github.com/rg3/youtube-dl/blob/c8e390c2b050e5fbe0a145ad4b621b2496020bab/youtube_dl/YoutubeDL.py#L1054

            if success:
                try:
                    self.post_process(filename, info_dict)
                except (PostProcessingError) as err:
                    self.report_error('postprocessing: %s' % str(err))
                    return

        self.record_download_archive(info_dict)

Youtube-Dl adds the download to the download archive either way. Regardless of whether it has been downloaded or not.

The name "download archive" implicates that only downloads that have been downloaded are in there. There should be at least an option to add only files that have been successfully downloaded.

Edit 3:

I have problemes reproducing this issue. I'll have a look at it.

[dstftw]

This has been fixed long time ago in cd58dc3.