[Bug]: Access of top-level matplotlib modules raises AttributeError

[WillTarran]

Bug summary

Direct access to top level modules of matplotlib raises AttributeError for most of the documented top-level modules. This was discovered when attempting to refer to module contents for type-hints that are evaluated when a function is defined. This occurs reproducibly where no previous matplotlib module imports have occurred prior to evaluation of the code referencing a module namespaced by matplotlib

Code for reproduction

import matplotlib

def do_something_with_figure(fig: matplotlib.figure.Figure):
    pass

print('doing some actual runtime code...')

Actual outcome

Traceback (most recent call last):
  File "typehint-mpl.py", line 3, in <module>
    def do_something_with_figure(fig: matplotlib.figure.Figure):
  File "/usr/local/lib/python3.8/site-packages/matplotlib/_api/__init__.py", line 226, in __getattr__
    raise AttributeError(
AttributeError: module 'matplotlib' has no attribute 'figure'

Expected outcome

For this example script, we would obviously expect to reach the print() statement.

Additional information

This appears to be a general issue with most top-level modules. The following attempts to access all top-level modules named in the current help(matplotlib):

$ cat test_modules.py 
import matplotlib


MODULES = ['axes', 'figure', 'artist', 'lines', 'patches', 'text', 'image', 'collections',
           'colors', 'cm', 'ticker', 'backends']

for module in MODULES:
    try:
        _ = getattr(matplotlib, module)
        print(f'got {module}')
    except AttributeError:
        print(f'failed to access {module}')
$ python test_modules.py
failed to access axes
failed to access figure
failed to access artist
failed to access lines
failed to access patches
failed to access text
failed to access image
failed to access collections
got colors
got cm
got ticker
failed to access backends

Note - I think this issue is due to some sort of lazy-loading behaviour, there are various ways by which the AttributeError can be avoided. If previous imports such as import matplotlb.pyplot as plt have already occurred before references to matplotlib.xxxx occur this appears ok, and also, while investigating interactively, it appears that simply showing the help() in the python shell unblocks this access:

Python 3.8.16 (default, May 23 2023, 14:26:40) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> matplotlib.figure
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/matplotlib/_api/__init__.py", line 226, in __getattr__
    raise AttributeError(
AttributeError: module 'matplotlib' has no attribute 'figure'
>>> help(matplotlib)
# q to leave help
>>> matplotlib.figure
<module 'matplotlib.figure' from '/usr/local/lib/python3.8/site-packages/matplotlib/figure.py'>

The above interactive behaviour can be replicated by invoking pydoc.render_doc(); the following version of the code example gives no error:

import pydoc
import matplotlib

_ = pydoc.render_doc(matplotlib)

def do_something_with_figure(fig: matplotlib.figure.Figure):
    pass

All examples in this report are with matplotlib installed in python3.8 bullseye docker image via pip as follows:

FROM python:3.8-bullseye

RUN pip install --upgrade pip \
    && pip install matplotlib==3.7.1

ENTRYPOINT ["python"]

Operating system

Observed on Debian buster & bullseye and Ubuntu 18.04, 20.04 & 22.04

Matplotlib Version

Various. Code examples using 3.7.1, with behaviour definitely occurring with 3.5.2, 3.6.2 & 3.6.3

Matplotlib Backend

'agg'

Python version

Various 3.8.x, 3.9.x

Jupyter version

No response

Installation

pip

[tacaswell]

This is standard Python behavior, importing a package does not implicitly import all sub-packages. This is good both from an import-time point of view (no need to import things you will never use), modules can do significant work at import time, not all sub-packages are garunteed to be importable (e.g. you do not have an optional dependency installed and we fail at import rather than use time), or the sub-packages are mutually exclusive.

If you want to access objects in the matplotlib.figure sub-package, the correct fix is to do

import matplotlib.figure

def do_something_with_figure(fig: matplotlib.figure.Figure):
    pass

print('doing some actual runtime code...')

Due to the way that the Python module import system works, if anything else does import matplotilb.figure then it will be available in the parent packages namespace globally. Importing pyplot (and apparently help?!) does this import so it appears to work.

---

There was a PEP to add lazy importing to the language (https://peps.python.org/pep-0690/) however it was rejected. There is on-going work to add lazy loading (so we can make all of the top level modules available without paying the import costs) across the scientific Python ecosystem (https://scientific-python.org/specs/spec-0001/ ) but we have not adopted that yet.

---

We now have a __all__ in the top level `init.py``

matplotlib/lib/matplotlib/__init__.py

Lines 104 to 133 in d464cbf

	__all__ = [
	"__bibtex__",
	"__version__",
	"__version_info__",
	"set_loglevel",
	"ExecutableNotFoundError",
	"get_configdir",
	"get_cachedir",
	"get_data_path",
	"matplotlib_fname",
	"MatplotlibDeprecationWarning",
	"RcParams",
	"rc_params",
	"rc_params_from_file",
	"rcParamsDefault",
	"rcParams",
	"rcParamsOrig",
	"defaultParams",
	"rc",
	"rcdefaults",
	"rc_file_defaults",
	"rc_file",
	"rc_context",
	"use",
	"get_backend",
	"interactive",
	"is_interactive",
	"colormaps",
	"color_sequences",
	]

and that is the set of things you can count on being there due to:

import matplotlib

matplotlib.XYZ

If you want any other sub-packages you must import them explicitly to be sure. I would not even rely on the three that are there are available as an implementation detail and may change in the future.

---

As a technical note, we need rcsetup to be imported in __init__.py and that in turn imports colors which in turn imports ticker. I suspect we could get a speed up on import if we trimmed things a bit more....

---

I am going to close this with no-action as it is a duplicate of #22247 . I am contemplating closing both, but lets keep further discussion on that issue which has more discussion already.

[WillTarran]

Thanks for the response @tacaswell - I probably should have been more familiar with python's import behaviour! I'll update any usage to explicitly import the required sub-packages.

[tacaswell]

Python's import behavior is fiendishly complicated (see https://docs.python.org/3/reference/import.html ) and for most use cases you can get away with not thinking about it too much (the behind the curtain complication makes the user facing side look simple)!