Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entire backup fails when one path is missing #2919

Open
davegold24 opened this Issue Dec 5, 2017 · 6 comments

Comments

Projects
None yet
5 participants
@davegold24
Copy link
Contributor

davegold24 commented Dec 5, 2017

  • [ X] I have searched open and closed issues for duplicates.

Environment info

  • Duplicati version: 2.0.2.13_canary_2017-11-22
  • Operating system: Windows 7 32-bit
  • Backend: Minio S3 hosted at a DR site

Description

I had c:\gedhtree_280 selected as one of my backup sources, and deleted it. I received the following error in my email home:
Failed: The source folder C:\GedHTree280\ does not exist, aborting
backup
Details: System.IO.IOException: The source folder C:\GedHTree280\ does not
exist, aborting backup
at Duplicati.Library.Main.Controller.ExpandInputSources(String[]
inputsources, IFilter filter, ILogWriter log)
at
Duplicati.Library.Main.Controller.<>c__DisplayClass17_0.b__0(Backu
pResults result)
at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]&
paths, IFilter& filter, Action`1 method)

Steps to reproduce

  1. Create c:\gedhtree280
  2. Add to protected directory list
  3. Run backup a few times
  4. Remove c:\gedhtree280
  • Actual result:
    Backup fails
  • Expected result:
    Backup succeeds with warning about missing directory

Screenshots

Debug log

I can rerun this with profile logging enabled if that is helpful.

@kees-z

This comment has been minimized.

Copy link

kees-z commented Dec 6, 2017

You can use advanced option --allow-missing-source=true to continue the backup, even if source entries are missing. The missing source files will be treated as deleted.

@davegold24

This comment has been minimized.

Copy link
Contributor Author

davegold24 commented Dec 6, 2017

I appreciate the info. This probably should be a default.

@kees-z

This comment has been minimized.

Copy link

kees-z commented Dec 6, 2017

This is a duplicate of #2760 .
@kenkendk motivated the default behaviour Sep 29:

I think the problem is that Duplicati does not know why the source folder is missing or if that is acceptable.
The intention with the current default behaviour is that you would be really sad if you have been running backups regularly (configured, tested, and verified), but then suddenly find out that the folder you want to restore has not been backed up. By stopping the backup, Duplicati calls for "attention" saying "things are not working the way you set them up".

@Pectojin noted that missing files can flow out of all your backup versions if you have some form of retention, which is a good point.

However, I agree that raising PARSEDRESULT to Error should be enough to catch the user's attention, without having to abort the backup operation.

@hausmonarch

This comment has been minimized.

Copy link

hausmonarch commented Dec 14, 2017

I would rather have a backup continue to run (with some missing data) than stop running because one path is missing. While it is true that deleted data would eventually disappear from backups, I view it as my job (as backup manager) to keep an eye on logs and errors. If I set 30, 60, 90+ days as backups, then I have plenty of time to go back and recover missing data. But if I edited files today and my computer dies tomorrow morning, then under the current scheme, everything I accomplished today would be lost if I did not set the non-default allow-missing-source field, because the backup would have completely given up.

Erring on the side of caution is the safer bet when it comes to backups. We all maintain multiple days (or weeks, or months) worth of backups. The risk of a situation requiring recovery within one day is much greater than the risk that over some longer period we will fail to notice a big red "error" sign and an email that a path is missing.

The default behavior in my mind should be "back up as much as you can all the time" and "huge error message to admin if something is missing or goes wrong". Stopping the backup should never be part of the plan unless there is literally no retention plan, and then we're talking about snapshots, which are not backups (and should never be treated as such). --allow-missing-source=true should be the default behavior.

@redge76

This comment has been minimized.

Copy link

redge76 commented Dec 18, 2017

I'm not checking the logs every day on my personal server. I did not expect duplicati to stop working if I delete a directory.

@Germs2004

This comment has been minimized.

Copy link

Germs2004 commented Jun 4, 2018

I have this error too. I highly agree that the default should be backups continue to run if a source folder gets deleted. If someone only does weekly backups, you could set them back an entire extra week by aborting the job. And everyone else would lose those revision-versions that they'll never be able to get back.

It is a great idea to record that event as a warning so that it is clearly visible in the job log, a popup in the web GUI, and most importantly, it gets emailed immediately to the admin. That lets the admin investigate the issue when convenient while still protecting the company's files. I think an "error" is too serious as it implies that something has malfunctioned in a bad way.

I suggest changing "--allow-missing-source" to "--abort-if-source-missing" as allow is probably the expected behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.