Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib/fs: Handle deduplicated files on NTFS (fixes #1845) #4622

Closed
wants to merge 1 commit into from

Conversation

calmh
Copy link
Member

@calmh calmh commented Dec 29, 2017

These files always have the symlink bit set, because they are reparse points. Nonetheless they are not symlinks, and Lstat reports a size for them. We use this fact to disambiguate, and hope fervently that nothing else matches this description so it comes back to bite us...

Testing

@niphlod tested:

screen shot 2017-12-29 at 20 42 40


func (e fsFileInfo) Mode() FileMode {
m := e.FileInfo.Mode()
if e.Size() > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking if symlink bit it set first is cheaper, only then check for size.

if e.Size() > 0 {
return false
}
return e.FileInfo.Mode()&os.ModeSymlink == os.ModeSymlink
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you even need this? Can't we just call e.Mode() instead of e.FileInfo.Mode()?

if e.Size() > 0 {
return true
}
return e.FileInfo.Mode().IsRegular()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you even need this? Can't we just call e.Mode() instead of e.FileInfo.Mode()?


func (e fsFileInfo) IsSymlink() bool {
// N.B. this uses *our* Mode() which appropriately masks this bit.
return e.Mode()&FileMode(os.ModeSymlink) != 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just ModeSymlink, which is already defined in fs.go.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually isn't but maybe it should be

@calmh
Copy link
Member Author

calmh commented Dec 29, 2017

There, now I actually read your comments Audrius.

m := e.FileInfo.Mode() &^ os.ModeSymlink
return m.IsRegular()
}
return e.FileInfo.Mode().IsRegular()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps:

m := e.FileInfo.Mode()
if m&.ModeSymlink && .. .{
    m ^= ModeSymlink
}

return m.IsRegular()

@calmh calmh changed the title WIP: Dedup debugging Handle deduplicated files on NTFS (fixes #1845) Dec 29, 2017
These files always have the symlink bit set, because they are reparse
points. Nonetheless they are not symlinks, and Lstat reports a size for
them. We use this fact to disambiguate, and hope fervently that nothing
else matches this description so it comes back to bite us...
@calmh calmh changed the title Handle deduplicated files on NTFS (fixes #1845) lib/fs: Handle deduplicated files on NTFS (fixes #1845) Dec 29, 2017
@calmh
Copy link
Member Author

calmh commented Dec 29, 2017

Good to go

@AudriusButkevicius
Copy link
Member

@st-review merge

@st-review
Copy link

👌 Merged as a9f0659. Thanks, @calmh!

@st-review st-review closed this Dec 29, 2017
st-review pushed a commit that referenced this pull request Dec 29, 2017
These files always have the symlink bit set, because they are reparse
points. Nonetheless they are not symlinks, and Lstat reports a size for
them. We use this fact to disambiguate, and hope fervently that nothing
else matches this description so it comes back to bite us...

GitHub-Pull-Request: #4622
@calmh calmh deleted the dedup branch December 30, 2017 09:18
@st-review st-review added the pr-merged Legacy label used in the past for pull requests merged to the main tree label Jan 15, 2018
@calmh calmh restored the dedup branch November 1, 2018 06:07
@st-review st-review added the frozen-due-to-age Issues closed and untouched for a long time, together with being locked for discussion label Apr 30, 2019
@syncthing syncthing locked and limited conversation to collaborators Apr 30, 2019
@calmh calmh deleted the dedup branch March 23, 2020 07:13
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
frozen-due-to-age Issues closed and untouched for a long time, together with being locked for discussion pr-merged Legacy label used in the past for pull requests merged to the main tree
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants