In macOS defaults to installing python in /usr/local/usr/local

[kjmeagher]

Describe the bug
When using the python that comes with homebrew when installing python code with extension_module() or install_sources() it puts the files in /usr/local/usr/local/lib/python3.11/site-packages/ which is sub optimal.

To Reproduce

meson.build

project('test')
pymod = import('python')
python = pymod.find_installation('python3')
message('python install dir: '+python.get_install_dir())
message('python purelib    : '+python.get_path('purelib'))
message('python platlib    : '+python.get_path('platlib'))

with default options this gets:

Message: python install dir: /usr/local/usr/local/lib/python3.11/site-packages/
Message: python purelib    : /usr/local/lib/python3.11/site-packages
Message: python platlib    : /usr/local/lib/python3.11/site-packages

using --python.install-env=prefix

Message: python install dir: /usr/local/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/usr/local/lib/python3.11/site-packages/
Message: python purelib    : /usr/local/lib/python3.11/site-packages
Message: python platlib    : /usr/local/lib/python3.11/site-packages

Expected behavior
I would python files to be installed to in /usr/local/lib/python3.11/site-packages or somewhere else in sys.path

system parameters

• Is this a cross build or just a plain native build (for the same computer)?
  Native
• what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.)
  MacOS Ventura with Homebrew 4.1.4-36-gf70a49b
• what Python version are you using e.g. 3.8.0
  Python 3.11.4
• what meson --version
  1.2.1
• what ninja --version if it's a Ninja build
  1.11.1

Note
The way the python module determines the install dir is that it calls python_info.py on the target python install.

meson/mesonbuild/scripts/python_info.py

Lines 47 to 48 in 03a2a3a

	empty_vars = {'base': '', 'platbase': '', 'installed_base': ''}
	install_paths = sysconfig.get_paths(vars=empty_vars)

this calls sysconfig.get_paths() with base, platbase and installed_base as empty strings. On most linux installs, the default scheme is posix_prefix and you get answers that look like this:

'purelib': '{base}/lib/python{py_version_short}/site-packages',
'platlib': '{platbase}/{platlibdir}/python{py_version_short}/site-packages',

So that means that with base='' and platbase='' you basically get a relative path to put python files in with a / at the front.

But homebrew's install the default scheme is osx_framework_library which looks like

'purelib': '/usr/local/lib/python{py_version_short}/site-packages',
'platlib': '/usr/local/{platlibdir}/python{py_version_short}/site-packages',

So passing base='' does nothing.

Then the leading '/' is removed at

meson/mesonbuild/modules/python.py

Line 90 in 03a2a3a

rel_path = self.info['install_paths'][key][1:]

and the result is appended to prefix at

meson/mesonbuild/modules/python.py

Lines 129 to 130 in 03a2a3a

	self.platlib_install_path = os.path.join(prefix, python.platlib)
	self.purelib_install_path = os.path.join(prefix, python.purelib)

resulting in the double dose of /user/local

I've been looking into the code and I can think of a few ways to fix this but they all seem like hacks. One way would be to add an if os=='macOS': install_scheme='posix_prefix' to python_info.py. A different idea would be to check the difference between install_paths['platlib'] and paths['plablib'] to see if what you got is an absolute directory and do something. But I think it would be a good thing if the install directory is in sys.path.

[tristan957]

cc @eli-schwartz

[robtaylor]

Any news on this? I could really do with it working for 1.3.0! I took a bit of a look but couldn't figure out a neat approach also :/
maybe rather than if os=='macOS', we could check for HOMEBREW_PREFIX, then less chance of breaking on non homebrew systems.

[robtaylor]

Turns out the homebrew specific logic fitted nicely into the current code, see above PR

[robtaylor]

It seems the fundamental issue here is that 'osx_framework_library' isn't a "prefix" install scheme, either in MacOSX python or in Homebrew python, so should not be being reported by sysconfig.get_preferred_scheme("prefix")

Homebrew discussion started here: https://github.com/orgs/Homebrew/discussions/4858

[robtaylor]

In lieu of any action actually happening at homebrew, or sysconfig becoming sane, could you reconsider this patch as simple, as clean as can be and solves a major issue on macosx that pretty much everyone hits?