Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(caldav): automatically delete outdated scheduling objects #45235

Merged
merged 1 commit into from
Jun 3, 2024

Conversation

miaulalala
Copy link
Contributor

@miaulalala miaulalala commented May 8, 2024

Summary

oc_schedulingobjects currently grows without ever deleting outdated objects. This PR a repair step that is declared as expensive so admins can decide to run them at their convenience for the initial delete.

The delete is chunked to 50k rows on each transaction so the database isn't locked for a long time (especially in clustered setups this could cause issues). MySQL needs special treatment as it doesn't support LIMITs on DELETE queries, so it does a SELECT on the ids to delete, and then runs the delete on those.

After the repair step has run, a regular cron job is added to the Jobs List that runs every hour to get rid of scheduling objects that are older than an hour. We don't really need them and could theoretically delete them as soon as they're processed by the ITip\Broker but as rooms and resources are also run in a cron job, keeping them until the principal room and resources are added is probably a good idea as I can't exclude unwanted side effects. I also updated the runtime for rooms and resources to run every half hour for that reason.

Checklist

@miaulalala miaulalala self-assigned this May 8, 2024
@miaulalala miaulalala added 2. developing Work in progress performance 🚀 feature: caldav Related to CalDAV internals labels May 8, 2024
@miaulalala miaulalala added this to the Nextcloud 30 milestone May 8, 2024
miaulalala

This comment was marked as outdated.

@miaulalala

This comment was marked as outdated.

@miaulalala

This comment was marked as outdated.

apps/dav/lib/CalDAV/CalDavBackend.php Fixed Show fixed Hide fixed
lib/private/Log.php Fixed Show fixed Hide fixed
@miaulalala miaulalala force-pushed the fix/remove-old-scheduling-objects branch from cbcae93 to f2d7e9f Compare May 16, 2024 18:22
@miaulalala miaulalala marked this pull request as ready for review May 16, 2024 18:23
@miaulalala miaulalala requested review from nickvergessen, Altahrim, a team, yemkareems and sorbaugh and removed request for a team May 16, 2024 18:23
@miaulalala miaulalala marked this pull request as draft May 16, 2024 18:34
Copy link
Member

@ChristophWurst ChristophWurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get rid of the hourly full table scan

👍 otherwise

apps/dav/lib/CalDAV/CalDavBackend.php Outdated Show resolved Hide resolved
apps/dav/lib/CalDAV/CalDavBackend.php Outdated Show resolved Hide resolved
apps/dav/lib/CalDAV/CalDavBackend.php Outdated Show resolved Hide resolved
apps/dav/lib/CalDAV/CalDavBackend.php Dismissed Show dismissed Hide dismissed
@miaulalala miaulalala marked this pull request as ready for review May 28, 2024 15:55
@miaulalala miaulalala added 3. to review Waiting for reviews and removed 2. developing Work in progress labels May 28, 2024
Copy link
Member

@ChristophWurst ChristophWurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good

ITimeFactory $timeFactory,
) {
parent::__construct($timeFactory);
$this->setInterval(23 * 60 * 60);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not 24? 😆

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nickvergessen can explain 😉

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would move back all the time, as the run time is not 0 seconds and it can be started with up to 14 minutes delay.
That would mean it would fall further back and back in the "maintenance time window" and eventually skip some day as current time + 24h is outside of maintenance window.
With 23h it will just be saver to run every day.

@miaulalala miaulalala force-pushed the fix/remove-old-scheduling-objects branch from 1a53766 to 6ba43b3 Compare May 28, 2024 19:18
@miaulalala
Copy link
Contributor Author

/backport to stable29

@miaulalala
Copy link
Contributor Author

/backport to stable28

@miaulalala
Copy link
Contributor Author

/backport to stable27

Signed-off-by: Anna Larch <anna@nextcloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews feature: caldav Related to CalDAV internals performance 🚀
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove old scheduling objects from INBOX and oc_schedulingobjects via cron
5 participants