Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrading to Django 2.2.x gives No module named 'app_resolver' #6717

Closed
sounak98 opened this issue Aug 18, 2019 · 6 comments · Fixed by #6719

Comments

@sounak98
Copy link

commented Aug 18, 2019

Summary

After upgrading from Django 2.1.11 to 2.2.4, any django management command fails to run when there are cms apphooks in the app. This behavior can reproduced with the versions 3.6.x and 3.7.0rc1.

Traceback

Watching for file changes with StatReloader
Performing system checks...

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 409, in _check_custom_error_handlers
    handler, param_dict = self.resolve_error_handler(status_code)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 592, in resolve_error_handler
    callback = getattr(self.urlconf_module, 'handler%s' % view_type, None)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/utils/functional.py", line 80, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 572, in urlconf_module
    return import_module(self.urlconf_name)
  File "/Users/sounak/.pyenv/versions/3.6.5/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'app_resolver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sounak/.pyenv/versions/3.6.5/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/Users/sounak/.pyenv/versions/3.6.5/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/utils/autoreload.py", line 54, in wrapper
    fn(*args, **kwargs)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
    self.check(display_num_errors=True)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/management/base.py", line 390, in check
    include_deployment_checks=include_deployment_checks,
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/management/base.py", line 377, in _run_checks
    return checks.run_checks(**kwargs)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/checks/registry.py", line 72, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 399, in check
    messages.extend(check_resolver(pattern))
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 399, in check
    messages.extend(check_resolver(pattern))
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 399, in check
    messages.extend(check_resolver(pattern))
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 400, in check
    messages.extend(self._check_custom_error_handlers())
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 411, in _check_custom_error_handlers
    path = getattr(self.urlconf_module, 'handler%s' % status_code)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/utils/functional.py", line 80, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/Users/sounak/.pyenv/versions/python-test/lib/python3.6/site-packages/django/urls/resolvers.py", line 572, in urlconf_module
    return import_module(self.urlconf_name)
  File "/Users/sounak/.pyenv/versions/3.6.5/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'app_resolver'

Environment

  • Python version: 3.6.5
  • Django version: 2.1.11
  • django CMS version: 3.6.x, 3.7.0rc1

@sounak98 sounak98 changed the title Cannot upgrade to Django 2.2 Upgrading to Django 2.2.x gives No module named 'app_resolver' Aug 18, 2019

@FinalAngel

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

Thank you for the issue @sounak98
Might this be related to the problem you are experiencing:
#5797 ?

@stefanw

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2019

I have the same problem with 3.7rc1, but only when also upgrading Django to 2.2.4. On Django 2.1.11 with 3.7rc1 I do not get the error. I don't use the referenced "django-decorator-include". Removing the Django CMS URL include prevents the error (but also disables the CMS).

@lddubeau

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2019

I'm seeing the same problem sounak98 is seeing. I'm testing an application with Django CMS 3.7.0rc1. It works fine with Django 2.1.11 but not with Django 2.2.4. The error I'm seeing is the same as sounak98's.

@FinalAngel It is related to #5797 in the sense that the source of the problem here and the problem there is the same (a bug in appresolver.py). However, #5797 does not help solve the problem faced by those upgrading to Django 2.2. In #5797 guhuajun decided that the bug was not in Django CMS. I completely disagree with that assessment. guhuajun also produced a hack that allowed working around the bug but that hack is only useful to those using django-decorator-include, which I'm not using. (Though I may be wrong, I suspect sounak98 is also not using it because otherwise the issue would happen irrespective of whether Django 2.2 or 2.1 is used.)

If I grep though my entire Python virtual environment for \bapp_resolver\b, the only hit I get is in Django CMS' appresolver.py file:

                resolver = AppRegexURLResolver(
                    regex_pattern, 'app_resolver', app_name=app_ns, namespace=inst_ns)

AppRegexURLResolver is a class derived from Django's stock URLResolver class, and it is initialized like this:

class AppRegexURLResolver(URLResolver):
    def __init__(self, *args, **kwargs):
        self.page_id = None
        self.url_patterns_dict = {}
        super(AppRegexURLResolver, self).__init__(*args, **kwargs)

It just passes all arguments up to URLResolver's initialization method. I was not able to find documentation for URLResolver, but here is relevant part of the source:

class URLResolver:
    def __init__(self, pattern, urlconf_name, default_kwargs=None, app_name=None, namespace=None):
        self.pattern = pattern
        # urlconf_name is the dotted Python path to the module defining
        # urlpatterns. It may also be an object with an urlpatterns attribute
        # or urlpatterns itself.
        self.urlconf_name = urlconf_name

The comment explains that the 2nd argument (urlconf_name) is the dotted path of a module. So going back to AppRegexURLResolver(regex_pattern, 'app_resolver', app_name=app_ns, namespace=inst_ns), the 'app_resolver' argument is meant to be ultimately interpreted by URLResolver as a module name.

Ok, so where is this app_resolver module?? Searching through the entirety of my Python virtual environment, I'm not finding any module named app_resolver anywhere in Django CMS, Django, or any other package. This looks to me like Django CMS is requiring a module that does not exist, anywhere, and thus this is a bug in Django CMS, and not anywhere else.

Ok, but why is it working in Django versions prior to 2.2 but not with not Django 2.2?

That's because Django 2.2 is more aggressive with its system checks. In Django 2.1, the check method of URLResolver goes:

    def check(self):
        warnings = []
        for pattern in self.url_patterns:
            warnings.extend(check_resolver(pattern))
        return warnings or self.pattern.check()

Django 2.2 has this instead:

    def check(self):
        messages = []
        for pattern in self.url_patterns:
            messages.extend(check_resolver(pattern))
        messages.extend(self._check_custom_error_handlers())
        return messages or self.pattern.check()

Note the additional call to self._check_custom_error_handlers(). This method eventually accesses self.urlconf_module, which triggers the module resolution, and fails.

In Django versions prior to 2.2, it was possible to have a deployment of Django CMS that would never trigger the problematic code, but from 2.2 onward, by default, the system checks trigger the errant code.


By the way, I believe that Django CMS installations that don't use apphooks won't run into the bug.


ETA: Out of curiosity I've modified AppRegexURLResolver so that it overrides _check_custom_error_handlers with a method that returns an empty list. When I do this, I no longer get the error. This is a hack, not a solution but at least I can see whether I can run my test suite without running into other problems.

@stefanw

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2019

lddubeau seems to be spot on. I bisected this down in Django to this commit 245c36d.

Also it's worth mentioning that anything that doesn't use Django's system check will run fine. Default runserver breaks, but running from WSGI with gunicorn works.

lddubeau added a commit to lddubeau/django-cms that referenced this issue Aug 20, 2019
lddubeau added a commit to lddubeau/django-cms that referenced this issue Aug 20, 2019
Override urlconf_module so that Django system checks don't crash.
Without this fix, Django system checks in Django 2.2 will try to resolve the
"app_resolver" module, which does not exist. Overriding ``urlconf_module``
prevents the issue.

Fixes divio#6717
@lddubeau

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2019

I'm proposing a fix in #6719

@FinalAngel

This comment has been minimized.

Copy link
Member

commented Aug 21, 2019

Thank you @lddubeau and @sounak98, I'll check with the team for the review and potentially releasing in an RC2

lddubeau added a commit to lddubeau/django-cms that referenced this issue Aug 21, 2019
Override urlconf_module so that Django system checks don't crash.
Without this fix, Django system checks in Django 2.2 will try to resolve the
"app_resolver" module, which does not exist. Overriding ``urlconf_module``
prevents the issue.

Fixes divio#6717
yakky added a commit to yakky/django-cms that referenced this issue Aug 22, 2019
yakky added a commit to yakky/django-cms that referenced this issue Aug 22, 2019
Override urlconf_module so that Django system checks don't crash.
Without this fix, Django system checks in Django 2.2 will try to resolve the
"app_resolver" module, which does not exist. Overriding ``urlconf_module``
prevents the issue.

Fixes divio#6717
FinalAngel added a commit that referenced this issue Aug 27, 2019
Override urlconf_module so that Django system checks don't crash. (#…
…6719)

* Add a test case for #6717.

* Override urlconf_module so that Django system checks don't crash.

Without this fix, Django system checks in Django 2.2 will try to resolve the
"app_resolver" module, which does not exist. Overriding ``urlconf_module``
prevents the issue.

Fixes #6717
FinalAngel added a commit to FinalAngel/django-cms that referenced this issue Aug 27, 2019
Override urlconf_module so that Django system checks don't crash. (d…
…ivio#6719)

* Add a test case for divio#6717.

* Override urlconf_module so that Django system checks don't crash.

Without this fix, Django system checks in Django 2.2 will try to resolve the
"app_resolver" module, which does not exist. Overriding ``urlconf_module``
prevents the issue.

Fixes divio#6717
FinalAngel added a commit that referenced this issue Aug 27, 2019
Override urlconf_module so that Django system checks don't crash. (#…
…6719) (#6723)

* Add a test case for #6717.

* Override urlconf_module so that Django system checks don't crash.

Without this fix, Django system checks in Django 2.2 will try to resolve the
"app_resolver" module, which does not exist. Overriding ``urlconf_module``
prevents the issue.

Fixes #6717
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.