Buckup failes due to "Document {DocumentId} seems to have a attachment hash: {Hash}, but no correlating hash was found in the storage."

[oiBio]

I have several Databases where i cant performe my backup anymore.
i get this error seen below.

I seems like the attachment of the revision is not available anymore. I can access the current attachment via code and studio, but cannot access the attachment from the revision.

This problem occurs on several databases, but not on all. Is there a known bug for this or how can this happen?
Im using 5.4.104 on the server.

[11:11:53 INFO] Started task: 'Server Wide Backup, Backup Local' [11:11:54 INFO] Read 6,000 documents. Read 289 attachments. [11:11:55 INFO] Read 21,000 documents. Read 289 attachments. [11:11:56 INFO] Read 23,000 documents. Read 289 attachments. [11:11:56 INFO] Read 24,000 documents. Read 289 attachments. [11:11:58 INFO] Read 33,000 documents. Read 289 attachments. [11:11:58 INFO] Read 42,000 documents. Read 289 attachments. Document RavenDbFile/2528-B seems to have a attachment hash: wVniTqaesp4btKF8vBJaqsORkbvphFarrGDfjTCQiKk=, but no correlating hash was found in the storage. System.ArgumentException: Document RavenDbFile/2528-B seems to have a attachment hash: wVniTqaesp4btKF8vBJaqsORkbvphFarrGDfjTCQiKk=, but no correlating hash was found in the storage. at Raven.Server.Smuggler.Documents.StreamDestination.StreamDocumentActions.WriteUniqueAttachmentStreamsAsync(Document document, CountsWithLastEtagAndAttachments progress) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\StreamDestination.cs:line 1328 at Raven.Server.Smuggler.Documents.StreamDestination.StreamDocumentActions.WriteUniqueAttachmentStreamsAsync(Document document, CountsWithLastEtagAndAttachments progress) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\StreamDestination.cs:line 1328 at Raven.Server.Smuggler.Documents.StreamDestination.StreamDocumentActions.<>c__DisplayClass6_0.<<WriteDocumentAsync>g__AsyncWork|0>d.MoveNext() in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\StreamDestination.cs:line 1220 --- End of stack trace from previous location --- at Raven.Server.Smuggler.Documents.DatabaseSmuggler.ProcessRevisionDocumentsAsync(SmugglerResult result) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\DatabaseSmuggler.cs:line 672 at Raven.Server.Smuggler.Documents.DatabaseSmuggler.ProcessRevisionDocumentsAsync(SmugglerResult result) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\DatabaseSmuggler.cs:line 672 at Raven.Server.Smuggler.Documents.DatabaseSmuggler.ProcessRevisionDocumentsAsync(SmugglerResult result) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\DatabaseSmuggler.cs:line 672 at Raven.Server.Smuggler.Documents.DatabaseSmuggler.ProcessTypeAsync(DatabaseItemType type, SmugglerResult result, BuildVersionType buildType, Boolean ensureStepsProcessed) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\DatabaseSmuggler.cs:line 291 at Raven.Server.Smuggler.Documents.DatabaseSmuggler.ExecuteAsync(Boolean ensureStepsProcessed, Boolean isLastFile) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\DatabaseSmuggler.cs:line 110 at Raven.Server.Smuggler.Documents.DatabaseSmuggler.ExecuteAsync(Boolean ensureStepsProcessed, Boolean isLastFile) in C:\Builds\RavenDB-Stable-5.4\54049\src\Raven.Server\Smuggler\Documents\DatabaseSmuggler.cs:line 110

[shiranshalom]

Hi Tobias Bartsch,
I see that you had a similar issue before (#16624); we have fixed it in 5.4.105 (thanks to you :)). In addition, we recently found and handled (available in 5.4.111) another edge case related (https://issues.hibernatingrhinos.com/issue/RavenDB-20608/Excessive-number-of-revisions-after-replication-from-sharded-to-non-sharded).
To resume your Backup task, you can do the same as suggested here: #16624 (comment)

Thanks!

[oiBio]

Ah perfect :-) i knew i already wrote about the missing revisions but could not find my post.
So its time for upgrad. Thanks for your reply