AttributeError: 'NoneType' object has no attribute 'is_descendant_of'

[misli]

I know, that similar issue was marked as solved few months ago, but I am experiencing this issue now again when using postgres.

Summary

I always get unhandled exeption when creating subpage or when moving some page under another one. It only happens with postgres.

Steps to reproduce

1. create basic django-cms project following the how to
2. set up the database connection to use postgresql database (Fedora 25, Postgresql 9.5)
3. create page "Home" in the admin (BTW, the wizzard, which appears, if there is no page, doesn't work, as I can't select the "Create page")
4. try to create subpage of the page created in previous step

Internal Server Error: /admin/cms/page/add/
Traceback (most recent call last):
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/core/handlers/base.py", line 149, in get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/core/handlers/base.py", line 147, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/contrib/admin/options.py", line 541, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/utils/decorators.py", line 149, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/views/decorators/cache.py", line 57, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 244, in inner
    return view(request, *args, **kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/cms/admin/pageadmin.py", line 374, in add_view
    return super(PageAdmin, self).add_view(request, form_url, extra_context=extra_context)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1437, in add_view
    return self.changeform_view(request, None, form_url, extra_context)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
    return bound_func(*args, **kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/utils/decorators.py", line 149, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
    return func.__get__(self, type(self))(*args2, **kwargs2)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/utils/decorators.py", line 184, in inner
    return func(*args, **kwargs)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1378, in changeform_view
    self.save_model(request, new_object, form, not add)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/cms/admin/pageadmin.py", line 167, in save_model
    obj = obj.move(target, pos=position)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/cms/models/pagemodel.py", line 1450, in move
    super(Page, self).move(target, pos)
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/treebeard/mp_tree.py", line 1046, in move
    return MP_MoveHandler(self, target, pos).process()
  File "/home/misli/testcms/django-cms-site/lib/python2.7/site-packages/treebeard/mp_tree.py", line 444, in process
    if self.target.is_descendant_of(self.node):
AttributeError: 'NoneType' object has no attribute 'is_descendant_of'

Environment

• Python version: 2.7.13
• Django version: 1.9.12
• django CMS version: 3.4.2

appdirs==1.4.2
Django==1.9.12
django-classy-tags==0.8.0
django-cms==3.4.2
django-formtools==2.0
django-sekizai==0.10.0
django-treebeard==4.1.0
djangocms-admin-style==1.2.6.2
packaging==16.8
psycopg2==2.6.2
pyparsing==2.1.10
six==1.10.0

[czpython]

Hello,
Thanks for reporting.
Just to make sure, you created a new project using the manual instructions and not the installer?
If so, are you able to reproduce this also on the installer?

[misli]

Unfortunately, the result is the same even using the installer.
Both creating a subpage or moving a page under another one fails.
I'll try to investigate, why it is happening.

[misli]

I have found that the problem is with postgres ordering of varchars.
The query SELECT '00010001' BETWEEN '00010000' AND '0001ZZZZ' returns true in sqlite, but false in postgres ('Z' < '0' in postgres). It means that it is probably treebeard's bug. But I can't believe that nobody uses django-cms with postgres. So the question is: how to set it up to work?

[misli]

So the solution was following:
ALTER TABLE public.cms_page ALTER COLUMN path TYPE character varying(255) COLLATE pg_catalog."C";
It would be great if treebeard took care about collation on columns, for which the collation is critical, but until that, there should at least be some note in django-cms documentation.

[czpython]

Hello @misli,
This is quite strange as we (Divio) use Postgres internally for all projects.
Would you be able to supply us with a dummy project where you can reproduce this consistently?

[misli]

Well, it took me some time, but I have finally found it. As I have written in the previous comment, the collation matters. You haven't encountered this issue, as You probably use en_US.UTF-8 as default collation. But for different locales the default collation is also different.
On system with system default locale for example (my case) cs_CZ.UTF-8 the command su - postgres -c "createdb test" creates database as following:

CREATE DATABASE test
  WITH OWNER = postgres
       ENCODING = 'UTF8'
       TABLESPACE = pg_default
       LC_COLLATE = 'cs_CZ.UTF-8'
       LC_CTYPE = 'cs_CZ.UTF-8'
       CONNECTION LIMIT = -1;

It is not wrong. I want all the names, titles, etc. to be sorted correctly (č between c and d, ch between h and i, etc.), but in case of public.cms_page.path the collation should always ensure that 0 is lower than A. Therefore i'd suggest to always use "C".
I'll create an issue for treebeard, but some note in django-cms documentation would also be nice.

[evildmp]

@czpython @misli What exactly do you suggest the documentation should say?

It needs to be as generic as possible in order to be useful.

[czpython]

@evildmp We need to add a note that says the cms should be used with no collation. On postgres this is COLLATE "C".

[jrief]

@misli Just checked django-treebeard's issue tracker, but couldn't find any issue on that. Could you please explain your issue there again.

[kwassholm-work]

This "incorrect" collation can also result in "IntegrityError: duplicate key value violates unique constraint "cms_cmsplugin_path_7692c19a7d5df9d5_uniq"" errors because the django-treebeard's get_root_nodes() returns the nodes in wrong order. This error happened when trying to publish pages.

Fixed it by setting the collation to "C" on the "path" column. Not sure if it would be necessary to use "C" collation in all cms tables?

Weirdly this error was only experienced in Azure PostgreSQL but not in my local dev (MacOS), even though in both cases the database was using seemingly the same collation (fi_FI with UTF-8 encoding).

Maybe django-treebeard/django-cms should use this PR #django-treebeard/django-treebeard#143 ?

[flutterq]

[Solved] PgAdmin4 - 'NoneType' object has no attribute 'value'

[fdobrovolny]

I have run into this issue and changed the command from @misli to fix the issue. Followingly:

ALTER TABLE public.cms_treenode ALTER COLUMN path TYPE character varying(255) COLLATE pg_catalog."C";
ALTER TABLE public.cms_cmsplugin ALTER COLUMN path TYPE character varying(255) COLLATE pg_catalog."C";