Extremely slow when restoring files, Fetching path information ...

[Athep]

Referring to issue #2198.

Both when the backup set was on google drive and now on local storage, it is extremely slow working with.
Example: When restoring files, "Fetching path information ..." takes about 11 minutes. And after that listing directory's is almost impossible. It takes forever.

Something must be wrong. What can I do to correct this ?

[agrajaghh]

can you please give some more information about the backup and your machine? How many files? What size? Blocksize? Volume Size? Used advanced options?

[Athep]

Windows server 2012 R2 64 bit.
Intel Xeon E3 3,2 GHz, 4 GB DDR3

Total size of files backed up: 275 GB. Approx 119 000 files in 16 000 folders.
Mixed file types; office documents, stored program files etc.

Upload volume size: 200 Mbyte

No advanced options except:
snapshot-policy: On
vss-exclude-writers
sendmail.

Encryption AES-256.

[Athep]

I did some more tests:

I tried to run a "restore backup" without the local database. That was working fine. No problems with browsing all dirs etc. Restored a file and a local temporary database was created. All working fine. So i decided to delete the database and rebuild. After approx 15 hours it failed with the following error:

9 jan 2017 21:26: Failed while executing "Repair" with id: 1

System.Data.SQLite.SQLiteException (0x80004005): constraint failed UNIQUE constraint failed: BlocksetEntry.BlocksetID, BlocksetEntry.Index at System.Data.SQLite.SQLite3.Reset(SQLiteStatement stmt) at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt) at System.Data.SQLite.SQLiteDataReader.NextResult() at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave) at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior) at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery(CommandBehavior behavior) at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteNonQuery(IDbCommand self, String cmd, Object[] values) at Duplicati.Library.Main.Database.LocalRecreateDatabase.FindMissingBlocklistHashes(Int64 hashsize, Int64 blocksize, IDbTransaction transaction) at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor) at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.Run(String path, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor) at Duplicati.Library.Main.Operation.RepairHandler.RunRepairLocal(IFilter filter) at Duplicati.Library.Main.Operation.RepairHandler.Run(IFilter filter) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)
at Duplicati.Library.Main.Controller.Repair(IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)`

[kenkendk]

This is the same as #1699 #2140 and #2178

[kenkendk]

If you can try with the latest canary, it has some extra information that can help trace the cause of this issuse: https://github.com/duplicati/duplicati/releases/tag/v2.0.1.39-2.0.1.39_canary_2017-01-08

[Athep]

Duplicati - 2.0.1.39_canary_2017-01-08

Blockset insert failed, this is likely issue #2140, attempting to locate the culprit System.Data.SQLite.SQLiteException (0x80004005): constraint failed UNIQUE constraint failed: BlocksetEntry.BlocksetID, BlocksetEntry.Index at System.Data.SQLite.SQLite3.Reset(SQLiteStatement stmt) at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt) at System.Data.SQLite.SQLiteDataReader.NextResult() at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave) at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior) at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery(CommandBehavior behavior) at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteNonQuery(IDbCommand self, String cmd, Object[] values) at Duplicati.Library.Main.Database.LocalRecreateDatabase.FindMissingBlocklistHashes(Int64 hashsize, Int64 blocksize, IDbTransaction transaction)

[kenkendk]

@Athep Do you see any other messages in the log?

[Athep]

The only entries in the log before the error is:
9 jan 2017 22:20: Message
Rebuild database started, downloading 504 filelists

10 jan 2017 12:20: Message
Filelists restored, downloading 1395 index files

10 jan 2017 15:02: Message
Blockset insert failed, this is likely issue #2140, attempting to locate the culprit
System.Data.SQLite.SQLiteException (0x80004005): constraint failed
..

[ormanya]

This is still an issue for me. Very slow resolving each step as I browse down through directory structure in restore dialog