Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Want non-memory-mapped persistence for StreamExistenceFilter #3423

Closed
timothycoleman opened this issue Feb 11, 2022 · 0 comments
Closed

Want non-memory-mapped persistence for StreamExistenceFilter #3423

timothycoleman opened this issue Feb 11, 2022 · 0 comments
Assignees
Labels
cherry-pick:master Cherry picks PR into master branch

Comments

@timothycoleman
Copy link
Contributor

timothycoleman commented Feb 11, 2022

While the stream existence filter is flushing (msync) there appears to be a small chance that ordinary writes can be delayed several seconds (highest observed was 25s on ZFS, GCP).

This may be ZFS specific, possibly related (though not identical) to openzfs/zfs#12662 for which reason we recently switched away from memory mapped checkpoints on zfs (#3332)

Plan is to create a non memory mapped version so that (1) we are no longer calling msync, avoiding the problem if it is ZFS related and (2) we take control of flushing directly so that we can control the rate and ensure it has a low impact on other disk operations regardless

@timothycoleman timothycoleman added cherry-pick:release/oss-v21.10 Cherry picks PR into v21.10 release branch cherry-pick:master Cherry picks PR into master branch and removed cherry-pick:release/oss-v21.10 Cherry picks PR into v21.10 release branch labels Feb 11, 2022
@timothycoleman timothycoleman self-assigned this Feb 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick:master Cherry picks PR into master branch
Projects
None yet
Development

No branches or pull requests

2 participants