Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Part Files Linger after Downloading When there are no errors in the final attempt to download. #1879

Closed
sirinath opened this issue Dec 3, 2013 · 8 comments

Comments

@sirinath
Copy link

@sirinath sirinath commented Dec 3, 2013

When downloading the play lists of user: http://www.youtube.com/user/DrMarkCLewis

I has many lingering part files when the file was finished.

@phihag
Copy link
Contributor

@phihag phihag commented Dec 3, 2013

I'm guessing you passed in -i. If that's the case, download errors are ignored. Having the .part files there is actually very useful - you can simply re-run youtube-dl when your connection gets better and then it will resume downloads from these part files.

Therefore, leaving the part files laying around is quite a sensible default. If we were to delete them silently, you would only get the videos that happened to be downloaded in time. Is that really what you want?

@sirinath
Copy link
Author

@sirinath sirinath commented Dec 3, 2013

The download completed successfully. I have the full video and the part file. I was thinking the part files should be deleted when completed. This does not happen always through.

@phihag
Copy link
Contributor

@phihag phihag commented Dec 3, 2013

Woah, that should indeed never happen. Can you post a command-line session that shows the error? Please call youtube-dl with the -v option in it. It works fine for me:

$ ls
$ youtube-dl --id BaW_jenozKc -v
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--id', 'BaW_jenozKc', '-v']
[debug] youtube-dl version 2013.12.02
[debug] Git HEAD: 36a826a
[debug] Python version 2.7.6 - Linux-3.4-trunk-amd64-x86_64-with-debian-jessie-sid
[debug] Proxy map: {}
[youtube] Setting language
[youtube] BaW_jenozKc: Downloading video webpage
[youtube] BaW_jenozKc: Downloading video info webpage
[youtube] BaW_jenozKc: Extracting video information
[download] Destination: BaW_jenozKc.mp4
[download] 100% of 1.90MiB in 00:02
$ ls
BaW_jenozKc.mp4
@sirinath
Copy link
Author

@sirinath sirinath commented Dec 3, 2013

I ended up with:
Project Implementation - Polymorphism 1-Bo-FmTsY0lc.mp4.part and Project Implementation - Polymorphism 1-Bo-FmTsY0lc.mp4
Project Implementation - Polymorphism 2-t1RUNUxBgCg.mp4.part and Project Implementation - Polymorphism 2-t1RUNUxBgCg.mp4

This does not happen always. May be less than one in couple of hundred downloads. I suspect manly when there was an error and re started. When I noticed the part files I downloaded where it kept saying the file already there and skip to the next. Then only I noticed the full download also being there. I never saved the original logs. But my version info is here:

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--id', '-v']
[debug] youtube-dl version 2013.11.29
[debug] Python version 2.7.5 - Windows-8-6.2.9200
[debug] Proxy map: {}
Usage: youtube-dl [options] url [url...]

youtube-dl: error: you must provide at least one URL

@sirinath
Copy link
Author

@sirinath sirinath commented Dec 3, 2013

The subject should read as "Part Files Linger after Downloading When there are no errors." not "Part Files Linger after Downloading When there are errors."

@phihag
Copy link
Contributor

@phihag phihag commented Dec 3, 2013

I'm very sorry, but without a log or a description of the exact circumstances, there is no chance that we can reproduce this issue. Our process is fairly simply: Once we start the download, we look whether there is a completed file. If there is, we're finished. If not, we look whether a part file is present, and if yes, we append to it. Once the download finishes successfully, we rename the file. The rename is atomic on most OSs, although not necessarily on Windows.

If the problem occurs again, please post a log and we'll reopen this issue. It may also be interesting to know how large the part files are. In the meantime, you can simply remove all of them with del *part or so.

@phihag phihag closed this Dec 3, 2013
@sirinath
Copy link
Author

@sirinath sirinath commented Dec 3, 2013

It is smaller. It is 75 mb but the completed file is 78 for 1 of the files.

@sirinath
Copy link
Author

@sirinath sirinath commented Dec 3, 2013

del *part will not work as I need to be sure the part corresponds to a finished file. If it is not finished it is best that it is left. I want just the files which have indeed completed but with part files not been deleted. When downloading a large playlist how I monitor if there were errors is looking if there are part files. How a noticed it is when I kept trying to redownload the play list and there were not errors and the part files remained.

May be an external process locks the file for index / thumbnail / preview creation. (just a thought)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.