Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Also cleanup version metadata if expiring #39786

Merged
merged 1 commit into from Aug 14, 2023

Conversation

juliushaertl
Copy link
Member

@juliushaertl juliushaertl commented Aug 10, 2023

Found while writing e2e tests for #39140

Steps to reproduce:

  • Upload some file multiple times with close timestamps
  • Run cron which expires versions (ref
    private static $max_versions_per_interval = [
    //first 10sec, one version every 2sec
    1 => ['intervalEndsAfter' => 10, 'step' => 2],
    //next minute, one version every 10sec
    2 => ['intervalEndsAfter' => 60, 'step' => 10],
    //next hour, one version every minute
    3 => ['intervalEndsAfter' => 3600, 'step' => 60],
    //next 24h, one version every hour
    4 => ['intervalEndsAfter' => 86400, 'step' => 3600],
    //next 30days, one version per day
    5 => ['intervalEndsAfter' => 2592000, 'step' => 86400],
    //until the end one version per week
    6 => ['intervalEndsAfter' => -1, 'step' => 604800],
    ];
    )

This leads to the version files being deleted while the oc_files_versions table still contains references, meaning that they show up as broken files in the versions sidebar.

Ideally expiration should also be handled by the version manager, but this requires some larger efforts and is not backportable.

Quick script to reproduce:

#!/bin/bash
NC_URL=https://admin:admin@nextcloud.local/remote.php/webdav/

filename=$(date | md5sum | cut -d " " -f 1)
echo $filename

echo a1 > 1/$filename.md
sleep 2
echo a2 > 2/$filename.md
sleep 2
echo a3 > 3/$filename.md
sleep 2
echo a4 > 4/$filename.md
sleep 2
echo a5 > 5/$filename.md

curl -T "1/$filename.md" "$NC_URL/$filename.md"
curl -T "2/$filename.md" "$NC_URL/$filename.md"
curl -T "3/$filename.md" "$NC_URL/$filename.md"
curl -T "4/$filename.md" "$NC_URL/$filename.md"
curl -T "5/$filename.md" "$NC_URL/$filename.md"

Afterwards you can compare the following sql queries before and after cron for validation:

# with the echoed filename
select * from oc_filecache where name like '3882259fffd1ef36e300511488e561be%';
# with the file id
select * from oc_files_versions where file_id = 8740;

Checklist

@juliushaertl juliushaertl requested review from a team, ArtificialOwl, icewind1991, Fenn-CS and artonge and removed request for a team August 10, 2023 07:29
@juliushaertl juliushaertl added bug 3. to review Waiting for reviews labels Aug 10, 2023
@juliushaertl juliushaertl added this to the Nextcloud 28 milestone Aug 10, 2023
@juliushaertl
Copy link
Member Author

/backport to stable27

@juliushaertl
Copy link
Member Author

/backport to stable26

Signed-off-by: Julius Härtl <jus@bitgrid.net>
@juliushaertl juliushaertl merged commit 2b7d037 into master Aug 14, 2023
40 checks passed
@juliushaertl juliushaertl deleted the bugfix/version-expire-cleanup branch August 14, 2023 20:49
@solracsf
Copy link
Member

@juliushaertl no 25 ?

@juliushaertl
Copy link
Member Author

Only 26+ is affected where the new table for metadata was introduced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants