Creating and deleting models can lead to a broken tree in the database

[Meteor0id]

Issue Summary

On my development server I see my site has records which don't match up. It first occured after deleting a number of pages in Wagtail, thus it is likely a bug within Wagtail.

Looking at the table "wagtailcore_page" , the table does NOT match what is shown in wagtail.

In the attachment, notice that "digitale rechten" is a child of "achtergrond & verdieping", yet numchild indicates 0, where it should be 1. The path does confirm this page is a child of "achtergrond & verdieping" There might be more errors in other tables.

Steps to Reproduce

Leading up to all this, I was working on the models. Initially a model "Article" was allowed as subpage_type of "PublicationIndex" ("achtergrond & verdieping" is a publication index). I modified the models by adding two new page models in between. Now the subpage_type for PublicaitonIndex would allow for a type "Theme". The subpage_type for "Theme" would allow a type "Subject", and "Subject" would accept a supage_type "Article". I then ran makemigrations, followed by migrate, which ran succesfully. Now I wanted to get rid of the leftover Articles which were still a direct child of "PublicationIndex".

1. In the CMS I deleted a bunch of pages, some had children some did not. The goal was to delete all sub-pages of "PublicationIndex". 2. All subpages seemed to be deleted, I create a new page of type "Theme". 3. The new page created succesfully, but now the list showed my newly created page AND a page which was meant to be deleted, and did not show up before! 4. I deleted the Article page whihc I had meant to delete. 5. If I try to add another subpage to PublicationIndex, I get
   ValidationError at /cms/pages/add/ffs_article/theme/5/{'path': ['Page with this Path already exists.']}
   Besides that, the earlier added page "DigitaleRechten" (of type "Theme") does not show up anywhere in the CMS, while it does show in the database!

• I have confirmed that this issue can be reproduced as described on a fresh Wagtail project:
  No. But this issue should certainly not occur anyway. A currupted datatable is severe.

Technical details

• Python version: 3.7.0
• Django version: 2.0.8
• Wagtail version: 2.1.2
• Browser version: Firefox/61.0.2

[Meteor0id]

[Meteor0id]

[tomdyson]

No. But this issue should certainly not occur anyway. A currupted datatable is severe.

The requirement to reproduce the issue on a fresh Wagtail project isn't about severity, it's about our ability to fix it. If you can't reproduce it, it's very hard for us to help.

[Meteor0id]

By no, I meant I have not tried it yet, because I just havent had the time. I could just write the issue report in a month or so, but by than I would have forgotten de details.
Better to write this up right away, and come back to it when I have time to do more testing, or when someone else encounteres the same issue.

[Paulmolin]

I encountered the exact same error.
I removed a page that was a child of Root. Then I couldn't add new pages to Root, and, while there was still a page that was a child of Root, it did not appear in the admin pages index page.
It appears that parent_page.get_children() was returning an empty queryset. This is due to is_leaf checking numchild that was set to 0 although there was a child.

I thinks this is also what caused the error ['Page with this Path already exists.'].
I couldn't understand though why the numchild property was set to 0.

I wonder if the bug is caused by code in wagtail or in treebeard.

[gasman]

@Paulmolin FYI, running ./manage.py fixtreeshould be able to fix errors of this type.

[Paulmolin]

@gasman Thank you for the tip, I will try this!

I just found out that there was a find_problems method in mp_tree. The documentation explains that problems can occur when:

1. your code breaks and you get incomplete transactions (always use transactions!)
2. changing the steplen value in a model (you must :meth:dump_bulk first, change steplen and then :meth:load_bulk

I keep investigating this problem.

[lb-]

As there is a documented solution to this and no further feedback on this issue being a problem I will close this.

Feel free to comment if anyone thinks we should re-open.