Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sqlite cleanup functionality #4799

Merged
merged 7 commits into from
Dec 26, 2023
Merged

Add sqlite cleanup functionality #4799

merged 7 commits into from
Dec 26, 2023

Commits on Dec 23, 2023

  1. sqlite: use variadic function for serialize_run

    This makes the usage of the function a bit nicer
    brunnre8 committed Dec 23, 2023
    Configuration menu
    Copy the full SHA
    60ddf17 View commit details
    Browse the repository at this point in the history
  2. sqlite: accept db connection string

    This allows us to inject a memory db during testing
    brunnre8 committed Dec 23, 2023
    Configuration menu
    Copy the full SHA
    aec8d0b View commit details
    Browse the repository at this point in the history
  3. sqlite: implement deleteMessages

    This is laying the foundation to build a cleaning task that's
    sort of database agnostic.
    All calls are done by acting on a "DeletionRequest" so interpretation
    of the config will go through a single point
    brunnre8 committed Dec 23, 2023
    Configuration menu
    Copy the full SHA
    14d9ff2 View commit details
    Browse the repository at this point in the history

Commits on Dec 24, 2023

  1. introduce storage cleaner

    Once this is getting hooked up, it'll periodically delete old
    messages.
    
    The StoragePolicy can be chosen by the user, currently there's
    two versions, delete everything based on age is the obvious.
    
    The other is for the data hoarders among us. It'll only delete
    message types which can be considered low value... Types with
    a time aspect like away / back... joins / parts etc.
    
    It tries to do that in a sensible way, so that we don't block
    all other db writers that are ongoing.
    The "periodically" interval is by design not exposed to the user.
    brunnre8 committed Dec 24, 2023
    Configuration menu
    Copy the full SHA
    74aff7e View commit details
    Browse the repository at this point in the history
  2. cleaner: expose cli task to do cleaning + vacuum

    Make the cleaner available to users by exposing it as a subcommand
    to thelounge storage.
    
    This is recommended to be run whenever the storage policy significantly
    changes in a way that makes many messages eligible for deletion.
    The cleaner would cope, but it'll be inefficient and can take many hours.
    Due to how storage works in sqlite, the space would not actually be
    given back to the OS, just marked for future writes.
    Hence this also runs a vacuum to compact the DB as much as it can.
    brunnre8 committed Dec 24, 2023
    Configuration menu
    Copy the full SHA
    21b1152 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b0ca8e5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    edb1226 View commit details
    Browse the repository at this point in the history