Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync/Copy fails completely if even one file is corrupted or cannot be read. #856

Closed
ionFish opened this issue Jul 30, 2014 · 6 comments
Closed
Assignees
Labels
bug This issue is a bug. s3sync s3

Comments

@ionFish
Copy link

ionFish commented Jul 30, 2014

Problem: When one or more files is corrupted (and cannot be read either by the OS itself or of course AWS-CLI), the entire sync or cp process fails (fatally) with the following error:

[Error 2] The system cannot find the file specified: u'\\\\server\\path\\to\\file '

Note: I have only tested this on Windows. I tried to delete the files manually, and it returned a "no such file or directory" error.

Not sure how we would fix that, but sync and cp should not fail completely if one file is bad; it should make note of the error(s) and carry on.

@jamesls
Copy link
Member

jamesls commented Jul 30, 2014

I agree, we should just be displaying an error about the specific file and moving on to the remaining files. We shouldn't be stop the entire sync/cp process.

@ionFish
Copy link
Author

ionFish commented Jul 30, 2014

@jamesls Unrelated, but I'm curious: why is this written in Python and not C or something?

@ionFish
Copy link
Author

ionFish commented Aug 1, 2014

@jamesls - This also happens when (for example on a Windows network share) one of the files/directories has permissions set that do not allow read access. This is [Error 5] Access is denied: ...

@blackstar257
Copy link

Where does the code live that kills the process when it runs across one of these file system errors?

@jamesls
Copy link
Member

jamesls commented Aug 1, 2014

In awscli.s3.customization.filegenerator I believe. Note that we are currently working on a fix for this such that any errors that occur when trying to access a file will not stop the entire transfer process. I will update this issue when a fix is available.

@kyleknap
Copy link
Contributor

Closing, issue is fixed via #881 and is merged into the develop branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. s3sync s3
Projects
None yet
Development

No branches or pull requests

4 participants