feat: don't use vim.g.python3_host_prog as a candidate for python

[williamboman]

This is inconsistent with how other system dependencies are resolved and is not
documented anywhere.

[williamboman]

(technically not a feature but not big enough to warrant a new major version, so this is the only way to get it to show up in the changelog via release-please)

[chrisgrieser]

Hi, I think this might actually be a breaking change.

On macOS, you currently have the system-python on 3.9, and homebrew python on 3.12. Even though homebrew's folders by default are in PATH before system folders, the issue is that the most recent binary is actually python3.12 and not python3. And since it appears mason is looking for python3 in PATH, only the system-python on 3.9 is picked up when installing python packages.

Using vim.g.python3_host_prog, I was able to tell mason to use homebrew's python3.12. However, with this PR, I am not able to do so anymore.

I am not sure how many other uses were – knowingly or not – relying on vim.g.python3_host_prog to have mason use a more recent python version (some packages like autotools outright do not work with python3.9). But given the commonness of homebrew, I think a lot of macOS users will face this problem.

I think an option in the mason config to manually set which python to use might be a solution. Also, due to potentially breacking packages, this PR should be marked as breaking change (to make it show up as such in lazy.nvim etc.)

[williamboman]

Yeah this technically is a breaking change for sure, but I figured it was small/insignificant enough to allow it to go through in a minor bump. The rationale being that it'd simply fallback to the python3/python candidates and python3_host_prog being more of a legacy Vim feature for remote plugins.

I completely forgot about the problems with python versioning that you mention though.. I wonder if we can follow this up by adding more versioned python candidates to attempt (i.e. python3.12, python3.11, python3.10, ...) before attempting python3. I believe it would fix the issue you're describing, no?

I generally lean towards designing things in such a manner that it works out of the box for the vast majority, I'd like to avoid adding new settings as much as it's possible (but still leave room for it when it's needed).

[chrisgrieser]

I wonder if we can follow this up by adding more versioned python candidates to attempt (i.e. python3.12, python3.11, python3.10, ...) before attempting python3. I believe it would fix the issue you're describing, no?

That's a good idea, yeah.

I generally am also in favor of having as few settings as possible, but I feel like the mess that is Python versions probably makes a setting necessary to provide sufficient flexibility

[williamboman]

@chrisgrieser 👋 Would you mind trying out feat/more-python-candidates (#1608) and see if it works ootb for you?

[utkarshgupta137]

This has broken python packages for me on NixOS. I used this to only enable python3 for neovim & not install it system-wide. It sets vim.g.python3_host_prog, which was being used till now. Would appreciate if this support could be restored.

[utkarshgupta137]

FYI, I don't have NodeJS or Ruby installed either, but they show up in require("mason.health").check(), possibly because of "providers": https://neovim.io/doc/user/provider.html#provider-python

[GitMurf]

This has broken python for me as well on Windows using python 3.12 and using pyenv-win to manage my python versions. I also need to use vim.g.python3_host_prog for whatever reason to get the standard neovim healthcheck to pass for python. Using pyenv-win, below is what I needed to set my pythong3 host prog as, but now Mason health check has broken.

vim.g.python3_host_prog = "C:\\Users\\USERNAME\\.pyenv\\pyenv-win\\versions\\3.12.2\\python.exe"

And even though my env path is set fine and my terminal accesses this version of python fine using any of python, python3, or python3.12, without setting the python_3_host_prog I get several different weird behaviors in neovim 🤷‍♂️

[zer09]

This caught me also I'm on openSuse Leap and the neovim is to far behind so I build from source and using the official pynvim from the repo is problematic so also manually installed it and having to python version 3.6 and 3.11. installing pip using 3.6 is giving me warnings, but not on 3.7 and up and the lates version on official repo is 3.11 so I end up using python3.11 for pip.

So far my workaround is using update-alternative to force python3 executable to use 3.11.

[zer09]

This caught me also I'm on openSuse Leap and the neovim is to far behind so I build from source and using the official pynvim from the repo is problematic so also manually installed it and having to python version 3.6 and 3.11. installing pip using 3.6 is giving me warnings, but not on 3.7 and up and the lates version on official repo is 3.11 so I end up using python3.11 for pip.

So far my workaround is using update-alternative to force python3 executable to use 3.11.

I take this back, forcing python3 to 3.11 using update-alternative broke other command like cnf "command-not-found" it need rpm and scout that also needs python3.6

[GitMurf]

Python version compatibility is always the bane of everyone's existence 🤣🤦🏻‍♂️

[williamboman]

@zer09 Does openSUSE not install the "canonical" python3.12, python3.11, etc. binaries?

[koallen]

This has broken things for me as well. RHEL8's system Python is still on 3.6 which will fail to install the latest version of most Python-based LSP (it's not a problem that mason cannot find python). I use pyenv to manage environments and only activate those for specific projects and neovim (by setting python3_host_prog) so they can use a newer Python version. Removing this basically means I have no way of forcing mason to use an alternative Python interpreter.