Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account DB bug report code keeps original FileLookup Path intact (privacy regression) #3790

ts678 opened this issue Jun 3, 2019 · 2 comments


None yet
2 participants
Copy link

commented Jun 3, 2019

  • I have searched open and closed issues for duplicates.

Environment info

  • Duplicati version:
  • Operating system: Windows 10 Version 1809
  • Backend: local folder


The DB bug report has long scrubbed sensitive information such as source pathnames. Various forum posts and documentation describe this. canary code for the new path storage scheme (maybe inadvertently) leaves the FileLookup table intact including its Path. This could not be seen until canary fixed "Create bug report ..." results in syntax error #3725. I'm not sure which tables are most helpful. The new code creates one it calls FixedFile, and drops others. Should it drop or fix FileLookup?

cmd.ExecuteNonQuery(string.Format(@"CREATE TABLE ""FixedFile"" AS SELECT ""B"".""ID"" AS ""ID"", ""A"".""Obfuscated"" AS ""Path"", ""B"".""BlocksetID"" AS ""BlocksetID"", ""B"".""MetadataID"" AS ""MetadataID"" FROM ""{0}"" ""A"", ""File"" ""B"" WHERE ""A"".""RealPath"" = ""B"".""Path"" ", tablename));
cmd.ExecuteNonQuery(@"DROP VIEW ""File"" ");
cmd.ExecuteNonQuery(@"DROP TABLE ""PathPrefix"" ");

Steps to reproduce

  1. Backup
  2. "Create bug report"
  3. Look at the FileLookup table with a viewer such as DB Browser for SQLite
  • Actual result:
    Path is sitting there readable.
  • Expected result:
    Obfuscated path or no table.



Debug log


This comment has been minimized.

Copy link
Contributor Author

commented Jun 3, 2019

Feature/fix path storage2 #3468 might be the pull request. There are some side issues we could get into, such as the future of DB bug reports. People haven't been very willing to post them (although some have), and this makes it hard to dig into field failures for the problems that developers can't reproduce themselves. Regarding deletion, a VACUUM would be more thorough, otherwise deleted information remains. Tools can dig data out.


This comment has been minimized.

Copy link
Contributor Author

commented Jun 27, 2019

Rather than open a different issue on VACUUM (which is probably an issue dating back to the earliest code):

Re: [sqlite] Parsing sqlitedb file. describes the problem, and the SQLite leader's advice, which is linked below:

PRAGMA schema.secure_delete; which says VACUUM should work too. That fits the bug report usage better.

Although web search can find many tools that claim to recover deleted data, the sqlite mailing list discussion (and other resources) claim a proper recovery is iffy. From a privacy point of view though, the scrubbing that Duplicati does leaves things like pathnames behind, easily seen by looking into the file, e.g. with a hex editor. does expand the privacy leak to be obvious with the tools one normally uses, e.g. an SQLite browser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.