Scalability issues when deleting pages with multiple tasks

[aubincleme]

Related internal issue : https://git.xwikisas.com/xwiki-sas-cpt/ulo/-/issues/1

When deleting a document containing many tasks, the deletion job will take a lot of time, and will also significantly increase the memory consumption of XWiki.

My understanding comes from the fact that, when deleting a task document, the task manager app tries to update the XDOM of every document referring to this task. This will lead to creating a lot of new new versions of this document (which may be in the list of pages to delete), which will slow down the process and trigger other listeners.

Marking as blocker because currently, this causes out of memory exceptions on XWiki instances.

[mflorea]

A task page has only a sigle task object (i.e. it defines a single task). A wiki page can have multiple task macro calls, each call referencing a single task page. Thus it's not clear what you mean by this:

When deleting a document containing many tasks

• you're deleting a nested page that has child / descendant task pages?
• or are you deleting a page that has multiple task macro calls?

The macro call has a reference to the task page, and the task page has a field to store the task owner (where the task macro call is). So I don't see what doesn't scale:

• when you delete a page with multiple task macro calls, we know the target task page for each of those calls, so we delete those task pages
• when you delete a task page, we know the owner page (that has the task macro call), so we update only that page

There is a mechanism in place to ensure we avoid a loop when deleting a task macro call or page. Maybe there is a bug in that code.

[aubincleme]

Thanks for the reply @mflorea !

• you're deleting a nested page that has child / descendant task pages?
• or are you deleting a page that has multiple task macro calls?

My understanding is that, in most cases, these two use-cases will be the same : Adding a task macros within a page A.WebHome will create A.Task_0, A.Task_1, … and so on. Deleting A.WebHome with children will both lead to the deletion of a page with multiple macro calls and to the deletion of the nested page.

I have not reviewed how the delete refactoring job works but it seems that the job starts by removing tasks before removing A.WebHome (maybe pages are ordered by fullname desc, or there is dedicated logic to remove nested pages before the top level page).

[trrenty]

@mflorea @aubincleme I did some investigation and explored some possible solutions.

• when you delete a page with multiple task macro calls, we know the target task page for each of those calls, so we delete those task pages

This is correct. The problem is that the other listener will trigger and update the content of the owner page, deleting the task macro call. This will happen for all the task macros inside the page. This issue will happen when deleting only the owner page, without checking Affect child pages

The solution for this would be, with few words, to check for the presence of the TASK_UPDATE_FLAG when handling the deletion of pages. While testing the possible solution, it turned out that it is a little bit more complex than that. But this is the general idea.

However, this does not fix the next issue: deleting the page while also checking the Affect child pages. If the deletion job started with the owner page, this was not a problem. Unfortunately, it deletes the pages in a seemingly alphabetic order which causes the same problem that was mentioned above.

The problem is that the other listener will trigger and update the content of the owner page

To fix this issue, I was thinking of the following solution: when deleting a task page, also check if we are inside a DeleteJob and if the deletion request is for an entire space. If true, we should not update the content of the owner page. This would align well with the fact that we don't run the listeners while in FoldEvents. WDYT?