Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix potential metadata corrupt in Redis caused by gc #1110

Merged
merged 1 commit into from Dec 9, 2021

Conversation

davies
Copy link
Contributor

@davies davies commented Dec 9, 2021

The gc command will scan all the entries and inodes to find out dangling (not used by any entry) inodes. The scanning of entries is not atomic, so we ignore all the inodes (checking the atime) created in last hour. There is still a chance that a inode was renamed and both the old name and new name are missed from the scanning, then the inode will be identify as dangling and deleted.

This PR change it to check the Ctime of inode, ignore the inodes that was created or renamed within a hour.

@davies davies requested a review from SandyXSD December 9, 2021 03:50
@codecov-commenter
Copy link

Codecov Report

Merging #1110 (1c0ccce) into main (1eaa406) will increase coverage by 0.42%.
The diff coverage is 84.61%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #1110      +/-   ##
============================================
+ Coverage     53.24%   53.66%   +0.42%     
- Complexity      149      167      +18     
============================================
  Files           146      146              
  Lines         23050    22989      -61     
  Branches        422      410      -12     
============================================
+ Hits          12272    12337      +65     
+ Misses         9253     9129     -124     
+ Partials       1525     1523       -2     
Impacted Files Coverage Δ
pkg/meta/redis.go 71.15% <0.00%> (ø)
.../src/main/java/io/juicefs/JuiceFileSystemImpl.java 65.16% <83.33%> (+7.96%) ⬆️
pkg/meta/base.go 75.65% <100.00%> (ø)
pkg/meta/sql.go 67.73% <100.00%> (+0.27%) ⬆️
pkg/vfs/writer.go 78.64% <0.00%> (-1.02%) ⬇️
pkg/vfs/reader.go 79.56% <0.00%> (+2.91%) ⬆️
...src/main/java/io/juicefs/utils/ConsistentHash.java 76.66% <0.00%> (+3.33%) ⬆️
...main/java/io/juicefs/utils/RedefineClassAgent.java 53.57% <0.00%> (+7.14%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffb3186...1c0ccce. Read the comment docs.

@SandyXSD SandyXSD merged commit 6f6396f into main Dec 9, 2021
@SandyXSD SandyXSD deleted the fix_corrupt_in_gc branch December 9, 2021 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants