New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error / DB corruption: Unexpected difference in fileset version X: found Y entries, but expected Z #3644

Open
SamSirry opened this Issue Feb 4, 2019 · 2 comments

Comments

Projects
None yet
3 participants
@SamSirry
Copy link

SamSirry commented Feb 4, 2019

  • I have searched open and closed issues for duplicates.

Environment info

  • Duplicati version: 2.0.4.12 (Canary)
  • Operating system: Windows 7 Pro x64 (8 CPU cores, 8 GB RAM)
  • Backend: SFTP (SSH)
    P.s. Testing with a local drive destination/backend did not generate the error.

General Settings:

--allow-sleep=true
--usn-policy=Auto
--tempdir=E:\Temp\SYSTEM\Duplicati
--retry-delay=20s
--number-of-retries=10
--dont-compress-restore-paths=true
--concurrency-block-hashers=6
--concurrency-max-threads=8
--concurrency-compressors=3

Job-specific settings:

--ssh-fingerprint=ssh-rsa 2048 {xxx}
Remote volume size: 10MB
Retention: Keep all backups

Description

The error "Unexpected difference in fileset version X: found Y entries, but expected Z" gets generated in certain cases when the backup operation gets interrupted before completion.
It seems to be a corruption in the local job database, and fixing it requires either deleting the backup set by running a manual Duplicati command line (tried it only once and didn't manage to delete successfully), or a full rebuild of the database.

Steps to reproduce

  1. Create a backup job with the destination set to SSH/SFTP.
  2. Start the job, and let it upload a couple of volumes.
  3. Attempt to stop it by requesting to "Stop after upload". It is a known bug that this will not actually stop the job, and it will go on.
  4. Now stop the job by selecting the option "Stop now".
  5. Attempt to start the job again. The error Unexpected difference in fileset version X: found Y entries, but expected Z will be generated.
  • Actual result:
    The backup job terminates apparently normally in step 4, but in step 5 an error message is displayed the the backup job terminates before processing any data (to be backed up) files.
  • Expected result:
    In step 5, starting the backup job should proceed normally, just like any job that gets started manually, first checking the remote data, scanning the local files for changes, uploading new blocks etc...

Screenshots

none

Debug log

Error from the internal log:

Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 1: 2019-01-30 16:19:11 (database id: 1), found 76 entries, but expected 417 at 
Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction) at 
Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass32_0.<VerifyConsistencyAsync>b__0() at 
Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.<RunOnMain>b__0() at 
Duplicati.Library.Main.Operation.Common.SingleRunner.<DoRunOnMain>d__2`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at 
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at 
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at 
Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at 
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at 
CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at 
Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result) at 
Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method) at 
Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter) at 
Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
@mcouillard

This comment has been minimized.

Copy link

mcouillard commented Feb 9, 2019

I had the same problem crop up one day after months of successful daily backups. Using canary builds.

Solution was actually easy: job > database > recreate from the web UI. Next backup ran without issue. Impressive software!

@ts678

This comment has been minimized.

Copy link

ts678 commented Feb 13, 2019

“Unexpected difference in fileset” errors [2.0.3.9] provides several workarounds for those who don't want to recreate the database (which can be very slow). Basically, one deletes the troubled version. 2.0.4.5 directions are here. The prior 2.0.3.3 beta didn't identify the version, so needed awkward help, but that's described too.

In spite of fortune that workarounds that generally work are known, reproducible DB corruption should be a call to action IMO. My backup got corrupted when my Internet connection glitched briefly. Test case is here. The interesting thing is that the network issue or stop silently corrupts, then the next backup trips over that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment