Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expected there to be a temporary fileset for synthetic filelist #2506

Open
RubbeeDuck opened this issue May 27, 2017 · 8 comments

Comments

Projects
None yet
5 participants
@RubbeeDuck
Copy link

commented May 27, 2017

I have:

  • searched open and closed issues for duplicates

Version info

Duplicati Version: 2.0.1.56
Operating System: QNAP - Linux
Backend: Linux

Bug Description

I got this error when I run an Backup.

Steps to reproduce

  • Starr a Backup

Actual result: Describe here what happens after you run the steps above (i.e. the buggy behaviour)
Expected result: Describe here what should happen after you run the steps above (i.e. what would be the correct behaviour)

Screenshots

debug log

DeletedFiles: 0
DeletedFolders: 0
ModifiedFiles: 1321
ExaminedFiles: 2792
OpenedFiles: 1395
AddedFiles: 74
SizeOfModifiedFiles: 102369729
SizeOfAddedFiles: 267267633
SizeOfExaminedFiles: 15278576024
SizeOfOpenedFiles: 8032101584
NotProcessedFiles: 0
AddedFolders: 0
TooLargeFiles: 0
FilesWithError: 0
ModifiedFolders: 0
ModifiedSymlinks: 0
AddedSymlinks: 0
DeletedSymlinks: 0
PartialBackup: False
Dryrun: False
MainOperation: Backup
CompactResults: null
DeleteResults: null
RepairResults: null
TestResults:
MainOperation: Test
Verifications: [
Key: duplicati-20170104T135943Z.dlist.zip.aes
Value: [],
Key: duplicati-i376bbec0de2d48b1a635bc1e92272b1f.dindex.zip.aes
Value: [],
Key: duplicati-bba40cac5e8e04492b714bdd38f512ba1.dblock.zip.aes
Value: []
]
ParsedResult: Success
EndTime: 5/26/2017 11:12:49 PM
BeginTime: 5/26/2017 11:12:09 PM
Duration: 00:00:40.0580100
BackendStatistics:
RemoteCalls: 10
BytesUploaded: 146015489
BytesDownloaded: 105765895
FilesUploaded: 5
FilesDownloaded: 3
FilesDeleted: 0
FoldersCreated: 0
RetryAttempts: 0
UnknownFileSize: 0
UnknownFileCount: 0
KnownFileCount: 311
KnownFileSize: 14579977579
LastBackupDate: 5/27/2017 1:00:00 AM
BackupListCount: 15
TotalQuotaSpace: 0
FreeQuotaSpace: 0
AssignedQuotaSpace: -1
ParsedResult: Success
ParsedResult: Warning
EndTime: 5/26/2017 11:12:49 PM
BeginTime: 5/26/2017 11:00:00 PM
Duration: 00:12:49.5599190
Messages: [
scheduling missing file for deletion, currently listed as Uploading: duplicati-20170526T071003Z.dlist.zip.aes,
promoting uploaded complete file from Uploading to Uploaded: duplicati-ida79c3f2d3c4449c9b36cde7b6aec699.dindex.zip.aes,
scheduling missing file for deletion, currently listed as Uploading: duplicati-b8b56137049844b79a9c31a43110bdc04.dblock.zip.aes,
scheduling missing file for deletion, currently listed as Uploading: duplicati-ie7e0ca0276104444967b0ea933eabd57.dindex.zip.aes,
removing file listed as Deleting: duplicati-bff73977655174231b9554d1d520df1f3.dblock.zip.aes,
...
]
Warnings: [
Expected there to be a temporary fileset for synthetic filelist (15, duplicati-20170108T031458Z.dlist.zip.aes), but none was found?
]
Errors: []

@kenkendk

This comment has been minimized.

Copy link
Member

commented May 29, 2017

The "synthetic filelist" is created after a backup has been interrupted.
It is synthetic as it uses the previous (successful) backup, and then just adds whatever new/updated files it managed to upload.

In a special case, this new synthetic filelist is somehow marked as not being synthetic, causing the warning you see.

@RubbeeDuck

This comment has been minimized.

Copy link
Author

commented May 29, 2017

Okay. Thanks for your reply.

But this means that the Backup is clean and works!?

Or I must do something!?

Thanks for your help

@kenkendk

This comment has been minimized.

Copy link
Member

commented May 30, 2017

Yes, the backup is clean and works.

You can run the "Verify" command to see if it works correctly.

@RubbeeDuck

This comment has been minimized.

Copy link
Author

commented May 30, 2017

I run Now Verify and get this result:

MainOperation: Test
Verifications: [
Key: duplicati-i95f595736800472a97df6bc25117658d.dindex.zip.aes
Value: [
Key: Extra
Value: duplicati-b087207026a31496094bb9cbeddaf618e.dblock.zip.aes
],
Key: duplicati-iba27544c705645aaa9e26668fde9d3d6.dindex.zip.aes
Value: [
Key: Extra
Value: duplicati-bc011dc4684684ea8a0edae8c682d59dd.dblock.zip.aes
],
Key: duplicati-20170211T040000Z.dlist.zip.aes
Value: [],
Key: duplicati-ia6546b05255941e4a64cfabd2af78c71.dindex.zip.aes
Value: [],
Key: duplicati-bbc69fabf7181487e934327b8449b6433.dblock.zip.aes
Value: []
]
ParsedResult: Success
EndTime: 5/30/2017 9:49:14 AM
BeginTime: 5/30/2017 9:05:12 AM
Duration: 00:44:01.8690780
Messages: []
Warnings: []
Errors: []
BackendStatistics:
RemoteCalls: 6
BytesUploaded: 0
BytesDownloaded: 120622321
FilesUploaded: 0
FilesDownloaded: 5
FilesDeleted: 0
FoldersCreated: 0
RetryAttempts: 0
UnknownFileSize: 0
UnknownFileCount: 0
KnownFileCount: 22758
KnownFileSize: 1192105753422
LastBackupDate: 2/11/2017 5:00:00 AM
BackupListCount: 14
TotalQuotaSpace: 0
FreeQuotaSpace: 0
AssignedQuotaSpace: -1
ParsedResult: Success

This means all is okay!?

Sorry that i ask But these are My timemachine Backup :-)

@kenkendk

This comment has been minimized.

Copy link
Member

commented May 31, 2017

Yes, that looks like everything is in order.

@jpschewe

This comment has been minimized.

Copy link

commented Jun 30, 2018

This warning causes the tray icon to go red like there was an error. Should this condition alert the user if Duplicati fixes everything itself on the next backup?

@cy2k

This comment has been minimized.

Copy link

commented Oct 10, 2018

How do we clear this error message? I did a Verify and I think everything checks out (see below)... but still I get this error message every time Duplicati does a backup.

MainOperation: Test
Verifications: [
Key: duplicati-20181005T012338Z.dlist.zip.aes
Value: [],
Key: duplicati-id23d3d4d7ed44b2f82b010a1b991243e.dindex.zip.aes
Value: [],
Key: duplicati-bdfb37a3b69214ca9bcab1c074a3a6f5f.dblock.zip.aes
Value: []
]
ParsedResult: Success
EndTime: 10/10/2018 1:28:56 AM (1539149336)
BeginTime: 10/10/2018 1:27:29 AM (1539149249)
Duration: 00:01:26.3042239
Messages: [
removing file listed as Temporary: duplicati-20181010T052338Z.dlist.zip.aes,
scheduling missing file for deletion, currently listed as Uploading: duplicati-b26f930dd44e04bff899db344a44630ca.dblock.zip.aes,
scheduling missing file for deletion, currently listed as Uploading: duplicati-i28d4337e98814fe393fbe4d7f43d7f5e.dindex.zip.aes,
scheduling missing file for deletion, currently listed as Uploading: duplicati-b45f33b59d1eb48a4b1390d2fd064fba0.dblock.zip.aes,
scheduling missing file for deletion, currently listed as Uploading: duplicati-i01426ebab489438ba48d13be62226ec7.dindex.zip.aes,
...
]
Warnings: []
Errors: []
BackendStatistics:
RemoteCalls: 4
BytesUploaded: 0
BytesDownloaded: 65858951
FilesUploaded: 0
FilesDownloaded: 3
FilesDeleted: 0
FoldersCreated: 0
RetryAttempts: 0
UnknownFileSize: 0
UnknownFileCount: 0
KnownFileCount: 11992
KnownFileSize: 301608446376
LastBackupDate: 10/5/2018 1:23:38 PM (1538760218)
BackupListCount: 698
TotalQuotaSpace: 3000249008128
FreeQuotaSpace: 426405670912
AssignedQuotaSpace: -1
ReportedQuotaError: False
ReportedQuotaWarning: False
ParsedResult: Success

@ts678

This comment has been minimized.

Copy link

commented Feb 14, 2019

In a special case, this new synthetic filelist is somehow marked as not being synthetic, causing the warning you see.

I suspect OP problem arises from passing a fileset ID into a routine expecting a volume ID, thereby making a synthetic filelist warning about a January dlist when it's actually May then. Fileset ID probably aims low when applied to the larger Remotevolume table. Its semi-random select also finds non-dlist files, as explained here.

Fixing the code might just need passing in the right type of ID, however I don't know how much actual usage the synthetic filelist code has gotten (given this bug which often prevents its run), so it requires some testing.

lasttempfileid = candidates.Last().Key;
lasttempfilelist = m_database.GetRemoteVolumeFromID(lasttempfileid).Name;

// Check that we still need to process this after the cleanup has performed its duties
var syntbase = await database.GetRemoteVolumeFromIDAsync(lasttempfileid);
// If we do not have a valid entry, warn and quit
if (syntbase.Name == null || syntbase.State != RemoteVolumeState.Uploaded)
{
// TODO: If the repair succeeds, this could give a false warning?
Logging.Log.WriteWarningMessage(LOGTAG, "MissingTemporaryFilelist", null, "Expected there to be a temporary fileset for synthetic filelist ({0}, {1}), but none was found?", lasttempfileid, lasttempfilelist);
return;
}

Also see forum Warning Message after Backup replying to "One weirdness is that the filename is not a filelist".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.