sub-packages/sub-modules not (lazy) imported

[livecat-ai]

I get AttributeError: module 'matplotlib' has no attribute 'dates', when trying to access date functions.
eg. mpl.dates.num2date(date + epoch_adjustment)

version: matplotlib-3.5.1-cp39-cp39-macosx_11_0_arm64.whl

[timhoffm]

This looks like a messed up installation, but without further context we can't tell:

• Are you using an environment? e.g. conda/virtualenv
• What does import matplotlib; print(matplotlib.__file__) say?
• Can you import other modules, e.g. import matplotlib.artist

[jklymak]

I think the installation is probably fine, just import matplotlib as mpl does not import matplotlib.dates, or colors or colorbar or many other things. I'm actually not clear why it imports some things and not others (i.e. mpl.ticker works fine). I'm not sure we can or should fix this, but it is a bit inconsistent.

[tacaswell]

If you just import the top level module this is what gets pulled in:

✔ 16:04:41 $ ipython
Python 3.10.1 (main, Dec 18 2021, 23:53:45) [GCC 11.1.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.0.0.dev -- An enhanced Interactive Python. Type '?' for help.

In [1]: import matplotlib

In [2]: import sys

In [3]: [k for k in sys.modules if k.startswith("matplotlib")]
Out[3]: 
['matplotlib._api.deprecation',
 'matplotlib._api',
 'matplotlib._version',
 'matplotlib._c_internal_utils',
 'matplotlib.cbook',
 'matplotlib.docstring',
 'matplotlib._path',
 'matplotlib.bezier',
 'matplotlib.path',
 'matplotlib.transforms',
 'matplotlib.ticker',
 'matplotlib.scale',
 'matplotlib._color_data',
 'matplotlib.colors',
 'matplotlib.fontconfig_pattern',
 'matplotlib._enums',
 'matplotlib.rcsetup',
 'matplotlib.ft2font',
 'matplotlib._cm',
 'matplotlib._cm_listed',
 'matplotlib.cm',
 'matplotlib']

Depending on the exact imports you have done @livecat-ai that attribute error may be the "correct" behavior.

Some of these come from imports in rcsetup.py (which we probably could bury a bit more of that in functions calls) and the rest are from the

matplotlib/lib/matplotlib/__init__.py

Lines 1449 to 1451 in 76012ae

	# workaround: we must defer colormaps import to after loading rcParams, because
	# colormap creation depends on rcParams
	from matplotlib.cm import _colormaps as colormaps

at the bottom of __init__.py.

I think we should do two things here:

1. put that import behind our caching module-level attr tool
2. go down the route of putting in lazy imports for all of our second-level sub-modules (following https://discuss.scientific-python.org/t/spec-1-lazy-loading-for-submodules/25)

[tacaswell]

On a bit more inspection, we unavoidably must import matplotlib.colors as part of validating the default rcparams which in turn imports everything else. There is not a short-cut to reducing the number of un-avoidable imports unfortunately.

We should still think about putting in lazy imports for all/a subset of the sub packages though.

[tacaswell]

@livecat-ai I took the liberty of changing the description to something that is clearer.

[livecat-ai]

@tacaswell Thanks and thanks everyone for the clarification. I might be mistaken but lazy imports seemed to be the default behaviour in previous versions. If so, re-instating it would seem like a reasonable option in terms of compatibility across versions.

[tacaswell]

We have not had lazy imports in the past, but it is possible that we did import more subpackages/modules on top level import in the past. If you import pyplot you get many more of the sub modules imported (94!) .

I do not think we want to promote the exact set of modules imported as side-effects to a stable API. It is a very fiddly thing to track, there is a clear work-around on the user side (import the sub-module you want (which also leads to more readable user code)), and would constrain us if we wanted to improve import time by further reducing / re-arranging imports.