🧹 Clean-Up File Versions

[tbsbdr]

Description

User Stories

• As a user I want old file versions to be cleaned up so that the quota is used efficiently while still keeping important versions available.

Value

• save storage space

Acceptance Criteria

• 50% rule Versions must never use more than 50% of the user’s storage quota. If the stored versions exceed this limit, the oldest file versions get deleted until it meets the disk space limit again.
• Default versions clean-up pattern
  1. Last second one version is kept
  2. Last 10 seconds one version is kept every 2 seconds
  3. Last minute one version is kept every 10 seconds
  4. Last hour one version is kept every minute
  5. Last 24 hours one version is kept every hour
  6. Last 30 days one version is kept every day
  7. Older than 30 days one version is kept every week
• Config: The default cleanup-pattern can be changed via config
  • auto Default, default clean-up pattern applies
  • D, auto Keeps versions at least for D days, apply default clean-up pattern to all versions that are older than D days
  • auto, D Delete all versions that are older than D days automatically, delete other versions according to expiration rules
  • D1, D2 Keep versions for at least D1 days and delete when they exceed D2 days.
  • disabled Disable version retention; no files will be deleted.

Definition of ready

• Everybody needs to understand the value written in the user story
• Acceptance criteria have to be defined
• All dependencies of the user story need to be identified
• Feature should be seen from an end user perspective
• Story has to be estimated
• Story points need to be less than 20

Definition of done

• Functional requirements
  • Functionality described in the user story works
  • Acceptance criteria are fulfilled
• Quality
  • Code review happened
  • CI is green (that includes new and existing automated tests)
  • Critical code received unit tests by the developer
• Non-functional requirements
  • No sonar cloud issues
• Configuration changes
  • The next branch of the OpenCloud charts is compatible

[RoroTitiFR]

This implementation is nice, it should be implemented.
This is really a must feature.

[bernhardkaindl]

@tbsbdr, the list of implemented features describes the basic minimum of this feature as if it would already be available:

https://docs.opencloud.eu/docs/admin/welcome/features#versioning-and-recovery:

Versioning and Recovery

Every change is automatically saved as a version and can be restored at any time. Administrators can define how many versions to keep and the rules for deleting older versions.

But, as this bug shows, this is apparently not the case.

If this is correct, the only way to recover the space occupied by older versions may be:

1. Copy files that could have versions to another space
2. Delete those files
3. Empty the trash bins of the affected spaces
4. Copy the files back
5. Delete the copied files in the temporary location

I suggest as a first immediate remediation to implement only (of course to be extended as planned later):

• A configuration of a simple count of versions to keep (per space)
• Delete the versions that should no longer be kept when the value is reduced.

That would allow users to make room for new data to be stored when there is not enough space for older versions.

It would also be useful to have:

At or below the admin menu for deleted files,
there should be a one-shot option for cleaning up older versions

• up to a given minimum age in terms of time (most helpful, this is what e.g. Seafile implements)
  (older than x days, hours, minutes)
• up to a given minimum age in terms of generations
  (older than x versions)
• (possibly limited until a given amount of space is reclaimed, but that is probably overkill)

Also to consider:

When configuring a quota for a space:
there should be the option like this:

When the quota is exceeded when storing new data (radio-boxed or so):

• keep old versions and fail as long as the quota is exceeded
• clean-up older versions as needed to store the new data within the limits of the quota.