Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic Join of Cache Space and Disk Space Accounting for pre-emptive fallbacks on inserts #9

Open
jwuensche opened this issue Sep 9, 2022 · 0 comments

Comments

@jwuensche
Copy link
Owner

jwuensche commented Sep 9, 2022

For some things like objects it would be nice to know before hand if the data we are writing will fit on to the device, this has been exempted from the space accounting implemented as this does only reflect the actual on-disk state of the filesystem, which might not be the current actual state. For this reason we might want to keep an estimation of the future space requirements of data in the cache to know if storage will be available in the future.

For now a "fix" could be the assumption that all data of the cache will be written on whatever storage one wants to write data to, or faster. This will keep some space unusable and be overly careful but includes all cases in which errors might occur.

The determination of how much space will be required when writing to disk is easy as this is already part of the cache, but how this is distributed in the then storage is not trivial and also depends on the order of these objects in the tree. Determining this beforehand requires modification of the fallback semantics which leads to many more implications not fully explored yet.

@jwuensche jwuensche changed the title Preliminary Pre-emptive Space Accounting Sep 9, 2022
@jwuensche jwuensche changed the title Pre-emptive Space Accounting Semantic Join of Cache Space and Disk Space Accounting for pre-emptive fallbacks on inserts Sep 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

1 participant