Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Free page management #106

Closed
Scooletz opened this issue Jun 26, 2023 · 0 comments · Fixed by #142
Closed

Free page management #106

Scooletz opened this issue Jun 26, 2023 · 0 comments · Fixed by #142
Labels
bug Something isn't working 💾 disk size A change makes the disk size smaller 🐌 performance Perofrmance related issue

Comments

@Scooletz
Copy link
Contributor

Scooletz commented Jun 26, 2023

When running tests, we found that there's an unhealthy amount of pages that are not reused. It could be that they are not updated so therefore they are there forever. At least some of them are probably wrongly not reused, taking into consideration the algorithm that is used for storing the abandoned pages. Currently, the algorithm promotes the pages that were not used in the given batch so that they are not reused in the next one, but are considered to be reused after N blocks.

To test it one could:

  1. set history depth to 2
  2. write one block with 1 million entries (an allocation of new pages)
  3. update all the entries from the previous block in another one (all are COWed, so should be considered for a reuse)
  4. start writing small blocks so that the previously COWed pages are reused properly without updating them to the last batch

The comment in the code about the page reuse:

// The current approach is to squash sets of _abandoned and _unused into one set.
// The disadvantage is that the _unused will get their AbandonedAt bumped to the recent one.
// This meant that that they will not be reused sooner.
// The advantage is that usually, there number of abandoned pages create is lower and
// the book keeping is simpler.
// The pages are put in a linked list, when first -> ... -> last -> NULL

Results of the run with percentiles

Image

@Scooletz Scooletz added 🐌 performance Perofrmance related issue 💾 disk size A change makes the disk size smaller bug Something isn't working labels Jun 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working 💾 disk size A change makes the disk size smaller 🐌 performance Perofrmance related issue
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant