Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make collStats calculate collection size accurately after deletions of documents #3483

Closed
Tracked by #75
chilagrow opened this issue Oct 3, 2023 · 1 comment · Fixed by #3513
Closed
Tracked by #75
Assignees
Labels
area/diag Issues about diagnostic commands backend/pg Issues about PostgreSQL backend backend/sqlite Issues about SQLite backend code/chore Code maintenance improvements not ready Issues that are not ready to be worked on; PRs that should skip CI
Milestone

Comments

@chilagrow
Copy link
Contributor

chilagrow commented Oct 3, 2023

What should be done?

Capped collections are fixed size collections.

Currently in PostgreSQL backend, collStats calculates collection size using pg_table_size and pg_indexes_size.

When we introduce capped collections, assuming running vacuum full is unlikely, the capped collection after many deletions and insertions will only increase in size with the current way of calculating collection size. This is inaccurate reflection of capped collection.

This issue is about finding way to report actual used size disregarding deleted items in the table.

This applies to both PostgreSQL and SQLite.

Where?

https://github.com/FerretDB/FerretDB/blob/main/internal/backends/postgresql/collection.go
https://github.com/FerretDB/FerretDB/blob/main/internal/backends/sqlite/collection.go

Definition of Done

  • all handlers updated;
  • unit tests added/updated;
  • spot refactorings done;
@chilagrow chilagrow added code/chore Code maintenance improvements not ready Issues that are not ready to be worked on; PRs that should skip CI labels Oct 3, 2023
@AlekSi AlekSi added backend/sqlite Issues about SQLite backend backend/pg Issues about PostgreSQL backend labels Oct 3, 2023
@AlekSi AlekSi added the area/diag Issues about diagnostic commands label Oct 3, 2023
@AlekSi
Copy link
Member

AlekSi commented Oct 5, 2023

I think as a part of this issue we should make a unified backend test, not two different sets of tests for two backends

@AlekSi AlekSi changed the title collStats calculates collection size accurately after deletions of documents Make collStats calculate collection size accurately after deletions of documents Oct 10, 2023
chilagrow added a commit that referenced this issue Oct 11, 2023
@AlekSi AlekSi added this to the Next milestone Oct 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/diag Issues about diagnostic commands backend/pg Issues about PostgreSQL backend backend/sqlite Issues about SQLite backend code/chore Code maintenance improvements not ready Issues that are not ready to be worked on; PRs that should skip CI
Projects
Archived in project
3 participants