files_versions taking up a lot of space

[bhcopeland]

I am running linux and have a real issue with file_versions filling up space.

My disk quota is set to 100G, but I still get stupid sized folders with file versions

693G /home/owncloud/data/bhcopeland/files_versions/config/hexchat/logs/

ls | wc -l 61782

[MorrisJobke]

cc @schiesbn

[enoch85]

Shouldn't cron.php take care of that?

[PVince81]

In OC < 8.0 the expiration of versions for a file X should happen on the fly as soon as file X is overwritten. Not sure why it wouldn't do so here.
In OC >= 8.1 instead of expiring directly, a job is scheduled from the cron queue to do the expiration for file X. So the next cron run should do the cleanup for that specific file.

[PVince81]

also the expiration algorithm is supposed to take the quota into account when deciding which versions to expire.

[xn--nding-jua]

Hey,

the problem may be regarded to this problem: #8531

calling cron.php will only execute one single job. On my ownCloud-instance more than 3000 jobs (most of them regarded to file_version) have been collected. My cron.php is called every 15 Minutes, but this was not enought for this huge amount of jobs (3000*0,25h=750h). But new jobs will be added to the jobs-list. So in my view here is a design-problem. A workaround may be:

in cron.php replace

$jobList = \OC::$server->getJobList();
$job = $jobList->getNext();
if ($job != null) {
    $job->execute($jobList, $logger);
    $jobList->setLastJob($job);
}

with

$jobList = \OC::$server->getJobList();
$jobs = $jobList->getAll();
foreach ($jobs as $job) {
    if ($job != null) {
        $job->execute($jobList, $logger);
    }
}

kind regards,
Chris

[PVince81]

@xn--nding-jua are you on 8.1 ?
Too many oc_jobs => see "Debounce expiration jobs" #16826

[xn--nding-jua]

Yepp, using 8.1.0 at the moment.

[ljsjl]

My $0.02. We're looking at deploying owncloud in a research college, and in testing I ran into this oc_jobs issue with cron.php and files_version/files_trashbin (via some aggressive test scripts). I can't see us running with those apps enabled as cron.php would never catch up, and the oc_jobs table would get out of hand.

On our system cron.php seems to manage about 2-3 trashbin tasks a minute which seems really slow. I killed my test script this morning and there's about three day's worth of cron tasks queued up. A better queuing system as discussed in the issue linked above, which allows more than one worker would go some ways to helping.

[PVince81]

@MorrisJobke I think this should be fixed/improved already, was it in 8.2.2 or 9.0 ?

[MorrisJobke]

@MorrisJobke I think this should be fixed/improved already, was it in 8.2.2 or 9.0 ?

8.2.3 and 8.1.6 (so the upcoming releases)

[MorrisJobke]

8.2.3 and 8.1.6 (so the upcoming releases)

This is only the deduplication. For parallel cron.php runs we have #22000

[MorrisJobke]

In the end the files_versions should be fixed now. We will improve this further in the future with faster/parallel execution, but it should work.

If there are still too many files in the versions, feel free to reopen this case. Thanks

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.