Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue where lfs_fs_deorphan may run more than needed #811

Merged
merged 1 commit into from
May 1, 2023

Conversation

geky
Copy link
Member

@geky geky commented Apr 27, 2023

The underlying issue is that lfs_fs_deorphan did not updating gstate correctly. The way it determined if there are any orphans remaining in the filesystem was by subtracting the number of found orphans from an internal counter.

This internal counter is a leftover from a previous implementation that allowed leaving the lfs_fs_deorphan loop early if we know the number of expected orphans. This can happen during recursive mdir relocations, but with only a single bit in the gstate, can't happen during mount. If we detect orphans during mount, we set this internal counter to 1, assuming we will find at least one orphan.

But this presents a problem, what if we find no orphans? If this happens we never decrement the internal counter of orphans, so we would never clear the bit in the gstate. This leads to a running lfs_fs_deorphan on more-or-less every mutable operation in the filesystem, resulting in an extreme performance hit.

The solution here is to not subtract the number of found orphans, but assume that when our lfs_fs_deorphan loop finishes, we will have no orphans, because that's the whole point of lfs_fs_deorphan.

Note that the early termination of lfs_fs_deorphan was dropped because it would not actually change the runtime complexity of lfs_fs_deorphan, adds code cost, and risks fragile corner cases such as this one.


Also added tests to assert we run lfs_fs_deorphan at most once.

Found by @kasper0 and @Ldd309
See #604 for more info

The underlying issue is that lfs_fs_deorphan did not updating gstate
correctly. The way it determined if there are any orphans remaining in
the filesystem was by subtracting the number of found orphans from an
internal counter.

This internal counter is a leftover from a previous implementation that
allowed leaving the lfs_fs_deorphan loop early if we know the number of
expected orphans. This can happen during recursive mdir relocations, but
with only a single bit in the gstate, can't happen during mount. If we
detect orphans during mount, we set this internal counter to 1, assuming
we will find at least one orphan.

But this presents a problem, what if we find _no_ orphans? If this happens
we never decrement the internal counter of orphans, so we would never
clear the bit in the gstate. This leads to a running lfs_fs_deorphan
on more-or-less every mutable operation in the filesystem, resulting in
an extreme performance hit.

The solution here is to not subtract the number of found orphans, but assume
that when our lfs_fs_deorphan loop finishes, we will have no orphans, because
that's the whole point of lfs_fs_deorphan.

Note that the early termination of lfs_fs_deorphan was dropped because
it would not actually change the runtime complexity of lfs_fs_deorphan,
adds code cost, and risks fragile corner cases such as this one.

---

Also added tests to assert we run lfs_fs_deorphan at most once.

Found by kasper0 and Ldd309
@geky
Copy link
Member Author

geky commented May 1, 2023

Going ahead and bringing this in so it makes it into the next minor release

@geky geky merged commit 3dca029 into devel May 1, 2023
93 checks passed
@geky geky mentioned this pull request May 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant