Import dependencies lazily

[pickfire]

Describe the bug
Import huge dependencies like openpyxl from tablib is causing a huge import time taken on startup unnecessarily.

To Reproduce
Steps to reproduce the behavior:

1. In sample project, run python -X importtime ./manage.py check 2> import.log
2. View it with tuna https://pypi.org/project/tuna/
3. Django import export is importing openpyxl (we used tablib[all]) which causes a huge chunk of time spent

Versions (please complete the following information):

• Django Import Export: 3.2.0
• Python 3.11.5
• Django 4.1.10

Expected behavior

Importing formats should be done lazily and not during application startup.

Screenshots

Taken from our project, django-import-export is taking up the most time during startup.

Additional context

[andrewgy8]

Hey! Any idea how to solve this?

Also, Im not sure this is an issue with Django import export. It seems to be an openpyxl issue since the other dependencies are not taking that much time. If the other dependencies were equally slow, that would be another story.

Having said that, it might be beneficial to allow users to import the depdendencies themselves by overriding the admin class where it is used.

[matthewhegarty]

Thanks for raising. I'm interested to know the context where this is an issue. It seems from the graph that the time to load these dependencies is <200ms. Do you have a requirement to shorten this startup time?

Do you have any suggestion as to how we could solve this, or what would be the best outcome for you? Would it work if you could select dependencies as described in #1459?

See also #1459 #1069

[bufke]

I'm interested to know the context where this is an issue

Wow odd timing, I was just looking at why my celery beat instance uses so much ram and came here. Celery calls django.setup() which eventually runs this and adds 12mb. These little things add up.

A workaround for non web apps loading django might be to set a environment variable and conditionally add INSTALLED_APPS like import-export. Maybe there is a way to change imports here to make it just never happen. If I come up with a reasonable solution, I will post again. Right now I'm learning towards just tweaking INSTALLED_APPS as needed for the service type.

[matthewhegarty]

Thanks for adding this. It helps us understand how folks are using this and the source of the issue. If it helps, I added this PR for our v4 release. It means that only basic dependencies are loaded by default (e.g. csv), and that others have to be explicitly declared during installation.

[matthewhegarty]

After #1647, dependencies can be selected at library installation