home-brew python3.9 with pip install matplotlib

[leej3]

Bug report

A recent transition to python3.9 on homebrew seems to be causing problems on OSX.

Code for reproduction

I believe the following would reproduce it on a fresh MacOS system:

brew update
brew install python
pip install matplotlib numpy>=1.14.5

I encountered this as part of a pip install of a package that has just these two python dependencies:
https://github.com/afni/afni/blob/master/src/python_scripts/setup.py

Failed build and log (not sure if these are accessible though:
log
build

The relevant error message is:

click here

BUILDING MATPLOTLIB
      matplotlib: yes [3.3.2]
          python: yes [3.9.0 (default, Oct 27 2020, 14:15:17)  [Clang 12.0.0
                      (clang-1200.0.32.21)]]
        platform: yes [darwin]
     sample_data: yes [installing]
           tests: no  [skipping due to configuration]
          macosx: yes [installing]
    
    running egg_info
    creating /private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-pip-egg-info-8wops8wy/matplotlib.egg-info
    writing /private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-pip-egg-info-8wops8wy/matplotlib.egg-info/PKG-INFO
    writing dependency_links to /private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-pip-egg-info-8wops8wy/matplotlib.egg-info/dependency_links.txt
    writing namespace_packages to /private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-pip-egg-info-8wops8wy/matplotlib.egg-info/namespace_packages.txt
    writing requirements to /private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-pip-egg-info-8wops8wy/matplotlib.egg-info/requires.txt
    writing top-level names to /private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-pip-egg-info-8wops8wy/matplotlib.egg-info/top_level.txt
    writing manifest file '/private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-pip-egg-info-8wops8wy/matplotlib.egg-info/SOURCES.txt'
    init_dgelsd failed init
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-install-s9dsdpao/matplotlib/setup.py", line 242, in <module>
        setup(  # Finally, pass this all along to distutils to do the heavy lifting.
      File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 298, in run
        self.find_sources()
      File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 305, in find_sources
        mm.run()
      File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 536, in run
        self.add_defaults()
      File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 572, in add_defaults
        sdist.add_defaults(self)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/sdist.py", line 228, in add_defaults
        self._add_defaults_ext()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/sdist.py", line 311, in _add_defaults_ext
        build_ext = self.get_finalized_command('build_ext')
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 299, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "/private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-install-s9dsdpao/matplotlib/setup.py", line 88, in finalize_options
        self.distribution.ext_modules[:] = [
      File "/private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-install-s9dsdpao/matplotlib/setup.py", line 91, in <listcomp>
        for ext in package.get_extensions()
      File "/private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-install-s9dsdpao/matplotlib/setupext.py", line 345, in get_extensions
        add_numpy_flags(ext)
      File "/private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-install-s9dsdpao/matplotlib/setupext.py", line 469, in add_numpy_flags
        import numpy as np
      File "/private/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/pip-install-s9dsdpao/matplotlib/.eggs/numpy-1.19.2-py3.9-macosx-10.15-x86_64.egg/numpy/__init__.py", line 286, in <module>
        raise RuntimeError(msg)
    RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, see site.cfg.example for information. Otherwise report this to the vendor that provided NumPy.
    RankWarning: Polyfit may be poorly conditioned

Switching back to python 3.8 fixes the problem:
afni/afni@f2e608f

[tacaswell]

RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. If you compiled yourself, see site.cfg.example for information. Otherwise report this to the vendor that provided NumPy.

This is an issue is that numpy does not have py39 wheels up so you are building them from source, which in turn (I think) links against Accelerate (which seems to be a broken Apple library), which in turn means numpy will not import (see numpy/numpy#15759) and our install fails.

You will either need to make sure you build numpy against anything that is not Accelerate, install numpy from brew, or wait for the py39 wheels for numpy to be posted.

Closing as there is nothing we can do on the Matplotlib side.

[leej3]

Sounds good. Thanks for the explanation.