Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add modifyTimeAgo to volume.fsck #5133

Merged
merged 2 commits into from Dec 23, 2023
Merged

Conversation

SmsS4
Copy link
Contributor

@SmsS4 SmsS4 commented Dec 23, 2023

What problem are we solving?

Adding modifyTimeAgo to volume.fsck like fs.verify

How are we solving the problem?

Only collect volume needles and filer entries modified after modifyTimeAgo flag

How is the PR tested?

I removed some files from filer and some other files from volume server and checked if volume.fsck can find missing chunks and files.

> volume.fsck -cutoffTimeAgo 0s -modifyTimeAgo 10m
total 2 directories, 162307 files
dataNode:192.168.91.169:8080    volume:2        entries:27196   orphan:1        0.00%   1059B
dataNode:192.168.91.169:8080    volume:3        entries:26898   orphan:1        0.00%   1059B
dataNode:192.168.91.169:8080    volume:4        entries:26662   orphan:1        0.00%   1059B

Total           entries:162310  orphan:3        0.00%   3177B
> volume.fsck -cutoffTimeAgo 0s -modifyTimeAgo 1m
total 2 directories, 162307 files
Total           entries:162307  orphan:0        0.00%   0B
> volume.fsck -cutoffTimeAgo 0s -modifyTimeAgo 10m -findMissingChunksInFiler
total 2 directories, 162307 files
/buckets/test/536.khUqYqATfyptTbGs.rnd
/buckets/test/540.ghkJxoRx1Hio6fDw.rnd
/buckets/test/542.0fQh2YxHwO436VMI.rnd
/buckets/test/546.XeE4fG3JSLdQCXRh.rnd
/buckets/test/547.YBf33q6wCl0Otyor.rnd
> volume.fsck -cutoffTimeAgo 0s -modifyTimeAgo 1m -findMissingChunksInFiler
total 2 directories, 162307 files

Checks

  • I have added unit tests if possible.
  • I will add related wiki document changes and link to this PR after merging.

@SmsS4
Copy link
Contributor Author

SmsS4 commented Dec 23, 2023

Is it ok to check cutoffFrom and modifyFrom with AppendAtNs instead of LastModified?

(modifyFrom == 0 || modifyFrom <= resp.AppendAtNs) && (resp.AppendAtNs <= cutoffFrom)

@SmsS4
Copy link
Contributor Author

SmsS4 commented Dec 23, 2023

Looks like E2E tests fails buy I have no idea why.
Can you give me a hint what is wrong?:)
Thanks

@chrislusf chrislusf merged commit 9723638 into seaweedfs:master Dec 23, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants