Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent "circular import" when starting Django app #71

Closed
perlun opened this issue Apr 20, 2021 · 10 comments
Closed

Intermittent "circular import" when starting Django app #71

perlun opened this issue Apr 20, 2021 · 10 comments

Comments

@perlun
Copy link

perlun commented Apr 20, 2021

Hi,

Thanks for a nice Python/Django package. It works fine, but sometimes I get this error on startup with this Django app installed (as described in the installation details):

pipenv run ./manage.py runserver
Watching for file changes with StatReloader
2021-04-20 13:37:09,008 INFO [django.utils.autoreload] Watching for file changes with StatReloader
Performing system checks...

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 121, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/templatetags/django_simple_bulma.py", line 11, in <module>
    from ..utils import (
ImportError: cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/per/git/centraln/./manage.py", line 22, in <module>
    main()
  File "/home/per/git/centraln/./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 61, in execute
    super().execute(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 96, in handle
    self.run(**options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 103, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 637, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 622, in start_django
    reloader.run(django_main_thread)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 327, in run
    autoreload_started.send(sender=self)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 180, in send
    return [
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 41, in watch_for_template_changes
    for directory in get_template_directories():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 14, in get_template_directories
    for backend in engines.all():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in all
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in <listcomp>
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 81, in __getitem__
    engine = engine_cls(params)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 25, in __init__
    options['libraries'] = self.get_templatetag_libraries(libraries)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries
    for name in get_package_libraries(pkg):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 123, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'django_simple_bulma.templatetags.django_simple_bulma': cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)
make: *** [Makefile:24: serve] Error 1

Note that this only happens sometimes, which is both puzzling and annoying. It can also happen when making a change to any .py file in the project, i.e. when the apps are reloaded. Any ideas? Thanks in advance.

@perlun
Copy link
Author

perlun commented Apr 20, 2021

One idea I had was that the load_django_simple_bulma from our base.html HTML template:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    {% load static %}
    {% load django_simple_bulma %}
    {% bulma %}
    <link rel="shortcut icon" href="{% static 'favicon.ico' %}"/>
    <title>{% block title %}{% endblock %}</title>
</head>
<body>

...would trigger the loading of the django_simple_bulma app. But, in our settings.py file, this app was actually listed after our "main" apps (index and hibox_db). So, my theory was that it would basically create a race condition between "starting up the index app" and "starting up the django_simple_bulma" app.

I thus tried rearranging my apps like this:

diff --git centraln/settings.py centraln/settings.py
index d94ee61..710e438 100644
--- centraln/settings.py
+++ centraln/settings.py
@@ -33,8 +33,6 @@ ALLOWED_HOSTS = ['127.0.0.1']
 # Application definition
 
 INSTALLED_APPS = [
-    'index',
-    'hibox_db',
     'simple_history',
     'django_simple_bulma',
     'django.contrib.admin',
@@ -43,6 +41,9 @@ INSTALLED_APPS = [
     'django.contrib.sessions',
     'django.contrib.messages',
     'django.contrib.staticfiles',
+
+    'index',
+    'hibox_db',
 ]
 
 MIDDLEWARE = [

...but unfortunately it still didn't work. I'm still seeing this problem intermittently. 😢

For reference, our django-simple-bulma version is 2.2.0.

@lemonsaurus
Copy link
Owner

I'm not sure exactly why this is happening, but I suspect something to do with INSTALLED_APPS load order.

Here's the load order we've been using on pages like https://pythondiscord.org for years now without incident:

INSTALLED_APPS = [
    'index',  # put all your own local apps at the start here.

    'django.contrib.admin',  # all the built-in django stuff here.
    
    'django-simple-bulma',  # Third party django apps go in the end.
]

Can you try this and report back if it still happens?

@perlun
Copy link
Author

perlun commented Apr 27, 2021

Thanks. 👍 🙇 I tried this now and re-ran pipenv run ./manage.py runserver about 5-6 times. No errors thus far. I'll keep you in the loop if I happen to see this again.

(The important part of the change: moving django_simple_bulma after django.contrib.admin in INSTALLED_APPS, that might have done the trick actually.)

@lemonsaurus
Copy link
Owner

Great to hear! I'll close this issue for now then, but feel free to re-open it if you have any more issues!

@perlun
Copy link
Author

perlun commented Apr 27, 2021

Tack så mycket, jag gör så. 😊👍🏻

@perlun
Copy link
Author

perlun commented Apr 28, 2021

Hmm, maybe we jumped to conclusions too early. Still seeing this sometimes, at least on reloads:

/home/per/git/centraln/index/migrations/0001_initial.py changed, reloading.
2021-04-28 14:14:09,267 INFO [django.utils.autoreload] /home/per/git/centraln/index/migrations/0001_initial.py changed, reloading.
Watching for file changes with StatReloader
2021-04-28 14:14:09,597 INFO [django.utils.autoreload] Watching for file changes with StatReloader
Performing system checks...

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 121, in get_package_libraries
    module = import_module(entry[1])
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/templatetags/django_simple_bulma.py", line 11, in <module>
    from ..utils import (
ImportError: cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/per/git/centraln/./manage.py", line 22, in <module>
    main()
  File "/home/per/git/centraln/./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 61, in execute
    super().execute(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 96, in handle
    self.run(**options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 103, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 637, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 622, in start_django
    reloader.run(django_main_thread)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 327, in run
    autoreload_started.send(sender=self)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 180, in send
    return [
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 41, in watch_for_template_changes
    for directory in get_template_directories():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/autoreload.py", line 14, in get_template_directories
    for backend in engines.all():
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in all
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 90, in <listcomp>
    return [self[alias] for alias in self]
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/utils.py", line 81, in __getitem__
    engine = engine_cls(params)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 25, in __init__
    options['libraries'] = self.get_templatetag_libraries(libraries)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries
    libraries = get_installed_libraries()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries
    for name in get_package_libraries(pkg):
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/template/backends/django.py", line 123, in get_package_libraries
    raise InvalidTemplateLibrary(
django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'django_simple_bulma.templatetags.django_simple_bulma': cannot import name 'fontawesome_token' from partially initialized module 'django_simple_bulma.utils' (most likely due to a circular import) (/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django_simple_bulma/utils.py)
make: *** [Makefile:26: serve] Error 1

@lemonsaurus
Copy link
Owner

Hopefully resolved by #72, which was released with version 2.3.1.

@perlun
Copy link
Author

perlun commented Jun 10, 2021

Haven't upgraded, but using the locally patched version gave me an error like this today. Honestly don't know if django-simple-bulma was involved in this at all. 😁 It happened as the app was reloaded when I saved some changes in my IDE (IntelliJ Ultimate):

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/autoreload.py", line 64, in wrapper
    fn(*args, **kwargs)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 118, in inner_run
    self.check(display_num_errors=True)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/management/base.py", line 419, in check
    all_issues = checks.run_checks(
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/checks/registry.py", line 76, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/resolvers.py", line 412, in check
    for pattern in self.url_patterns:
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/resolvers.py", line 598, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/resolvers.py", line 591, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/git/centraln/centraln/urls.py", line 23, in <module>
    path('', include('index.urls'), name="index"),
  File "/home/per/.virtualenvs/centraln-czG2gyNf/lib/python3.9/site-packages/django/urls/conf.py", line 34, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/per/git/centraln/index/urls.py", line 4, in <module>
    from .customer import views as customer_views
  File "/home/per/git/centraln/index/customer/views.py", line 4, in <module>
    from index.filters import AutoSubmittingChoiceFilter, AutoSubmittingBooleanFilter, AutoSubmittingModelChoiceFilter
  File "/home/per/git/centraln/index/filters.py", line 5, in <module>
    from index.server.views import NameField
  File "/home/per/git/centraln/index/server/views.py", line 7, in <module>
    from index.filters import AutoSubmittingChoiceFilter, AutoSubmittingBooleanFilter
ImportError: cannot import name 'AutoSubmittingChoiceFilter' from partially initialized module 'index.filters' (most likely due to a circular import) (/home/per/git/centraln/index/filters.py)

@perlun
Copy link
Author

perlun commented Jun 10, 2021

Heh. Please disregard that one. It was a genuine circular dependency in my own app... 🙈 🙈 🙈 😁 I found and fixed it now.

@lemonsaurus
Copy link
Owner

Haha, no problem. I appreciate your willingness to let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants