Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Orphaned objects check skipped (needs all archives checked). #7752

Closed
xLogiiCx opened this issue Aug 1, 2023 · 7 comments
Closed

Orphaned objects check skipped (needs all archives checked). #7752

xLogiiCx opened this issue Aug 1, 2023 · 7 comments
Milestone

Comments

@xLogiiCx
Copy link

xLogiiCx commented Aug 1, 2023

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

Your borg version (borg -V).

borg 1.2.4

Operating system (distribution) and version.

EndeavourOS Linux x86_64 6.4.7-zen1-1-zen

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg check --repair --archives-only --glob-archives * --lock-wait 5 --debug --show-rc --progress --save-space ssh://x@x.com:22/./repo.borg

Describe the problem you're observing.

I originally opened the issue in the Borgmatic repo, but realized that it's probably more of a problem with borg itself.
If you use the parameter --glob-archives the "Orphaned objects check" is skipped, even if the filter includes all archives.

Include any warning/errors/backtraces from the system logs

borg check --repair --archives-only --glob-archives * --lock-wait 5 --debug --show-rc --progress --save-space ssh://x@x.com:22/./repo.borg > 
using builtin fallback logging configuration
33 self tests completed in 0.05 seconds
This is a potentially dangerous function.
check --repair might lead to data loss (for kinds of corruption it is not
capable of dealing with). BE VERY CAREFUL!

Type 'YES' if you understand this and want to continue: YES (from BORG_CHECK_I_KNOW_WHAT_I_AM_DOING)
Starting archive consistency check...
TAM-verified manifest
Analyzing archive 2021-12-31T23:00:07 (1/42)                                                                                                                                                                                                
Analyzing archive 2022-06-29T21:00:04 (2/42)                                                                                                                                                                                                
Analyzing archive 2022-08-24T00:25:05 (3/42)                                                                                                                                                                                                
Analyzing archive 2022-09-30T22:25:09 (4/42)                                                                                                                                                                                                
Analyzing archive 2022-10-31T22:26:10 (5/42)                                                                                                                                                                                                
Analyzing archive 2022-11-30T21:25:13 (6/42)                                                                                                                                                                                                
Analyzing archive 2022-12-30T21:25:12 (7/42)                                                                                                                                                                                                
Analyzing archive 2023-01-31T21:25:12 (8/42)                                                                                                                                                                                                
Analyzing archive 2023-02-28T21:25:13 (9/42)                                                                                                                                                                                                
Analyzing archive 2023-03-31T23:25:14 (10/42)                                                                                                                                                                                               
Analyzing archive 2023-04-29T17:59:32 (11/42)                                                                                                                                                                                               
Analyzing archive 2023-05-31T20:05:21 (12/42)                                                                                                                                                                                               
Analyzing archive 2023-06-18T21:25:10 (13/42)                                                                                                                                                                                               
Analyzing archive 2023-06-24T23:25:09 (14/42)                                                                                                                                                                                               
Analyzing archive 2023-06-28T17:25:03 (15/42)                                                                                                                                                                                               
Analyzing archive 2023-07-02T19:25:10 (16/42)                                                                                                                                                                                               
Analyzing archive 2023-07-07T15:25:13 (17/42)                                                                                                                                                                                               
Analyzing archive 2023-07-12T15:25:11 (18/42)                                                                                                                                                                                               
Analyzing archive 2023-07-13T15:25:23 (19/42)                                                                                                                                                                                               
Analyzing archive 2023-07-16T17:25:10 (20/42)                                                                                                                                                                                               
Analyzing archive 2023-07-17T17:25:21 (21/42)                                                                                                                                                                                               
Analyzing archive 2023-07-19T15:22:24 (22/42)                                                                                                                                                                                               
Analyzing archive 2023-07-20T13:57:23 (23/42)                                                                                                                                                                                               
Analyzing archive 2023-07-21T09:26:08 (24/42)                                                                                                                                                                                               
Analyzing archive 2023-07-22T21:26:01 (25/42)                                                                                                                                                                                               
Analyzing archive 2023-07-22T23:26:10 (26/42)                                                                                                                                                                                               
Analyzing archive 2023-07-23T11:26:18 (27/42)                                                                                                                                                                                               
Analyzing archive 2023-07-23T13:41:51 (28/42)                                                                                                                                                                                               
Analyzing archive 2023-07-24T11:28:22 (29/42)                                                                                                                                                                                               
Analyzing archive 2023-07-24T13:26:08 (30/42)                                                                                                                                                                                               
Analyzing archive 2023-07-24T17:31:45 (31/42)                                                                                                                                                                                               
Analyzing archive 2023-07-25T11:29:11 (32/42)                                                                                                                                                                                               
Analyzing archive 2023-07-25T13:28:48 (33/42)                                                                                                                                                                                               
Analyzing archive 2023-07-25T17:17:49 (34/42)                                                                                                                                                                                               
Analyzing archive 2023-07-26T10:15:19 (35/42)                                                                                                                                                                                               
Analyzing archive 2023-07-26T11:26:06 (36/42)                                                                                                                                                                                               
Analyzing archive 2023-07-30T22:08:44 (37/42)                                                                                                                                                                                               
Analyzing archive 2023-07-31T11:48:05 (38/42)                                                                                                                                                                                               
Analyzing archive 2023-07-31T12:00:01 (39/42)                                                                                                                                                                                               
Analyzing archive 2023-07-31T14:00:02 (40/42)                                                                                                                                                                                               
Analyzing archive 2023-07-31T18:00:01 (41/42)                                                                                                                                                                                               
Analyzing archive 2023-07-31T20:00:02 (42/42)                                                                                                                                                                                               
Orphaned objects check skipped (needs all archives checked).

You can see here all 42 archives get Analyzed but still the Orphaned objects check is skipped.

@ThomasWaldmann
Copy link
Member

Well, if you want all archives check, why do you use --glob-archives?

The borg code just checks for presence of limiting options and then assumes that this means not all archives are checked.

But maybe it is a good idea to change that code to rather check the set difference between the to-be-checked archives and the all-archives set.

@ThomasWaldmann ThomasWaldmann added this to the 1.2.5 milestone Aug 1, 2023
@xLogiiCx
Copy link
Author

xLogiiCx commented Aug 1, 2023

Well, if you want all archives check, why do you use --glob-archives?

It's not called by "me" its called like this by borgmatic because of my archive_name_format: '{now}' option, which borgmatic is likely replacing with *

Ref: https://projects.torsion.org/borgmatic-collective/borgmatic/issues/734#issuecomment-6822

@ThomasWaldmann
Copy link
Member

OK. Well, guess this could be fixed by both borg (as I described above) and borgmatic (checking for *).

@xLogiiCx
Copy link
Author

xLogiiCx commented Aug 1, 2023

I am not 100% sure if this is the right way but I would find it cleaner if borg would check the difference between the to-be-checked archives and the all-archives set.

@ThomasWaldmann
Copy link
Member

ThomasWaldmann commented Aug 1, 2023

Just checked the code: it's not as easy as I thought - there are also checkpoint archives (which are not considered if any of the first, last, glob/match, older, newer, newest, oldest options is given, but are considered in absence of these).

@witten
Copy link
Contributor

witten commented Aug 1, 2023

FWIW, borgmatic will make the fix on the borgmatic side as part of https://projects.torsion.org/borgmatic-collective/borgmatic/issues/734

@ThomasWaldmann
Copy link
Member

So, guess for borg 1.2, this works as designed / as expected.

Also not sure if we can do this better for borg 2, because most borg commands ignore checkpoint archives, except when --consider-checkpoints is given.

OTOH, borg check (without options) does consider checkpoints by default (so it really checks all archives by default), but it would be unexpected if it would consider them also when using matching, first/last etc..

So guess there is no better solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants