Python now hard-coded to python 2.7 #25735

Closed
darthmdh opened this Issue Jan 8, 2014 · 21 comments

Projects

None yet

5 participants

@darthmdh
Contributor
darthmdh commented Jan 8, 2014

Following commits like SHA: e341a38 homebrew has been crippled into hardcoding support for only python 2.7 (deprecated 5 years ago). It is now no longer possible to build sip, pyqt etc. against python3 (to name a few affected formulas - the full list is long and makes me sad)

Prior to this commit, "brew install sip" (for example) would build against both python 2 (as sip) and python3 (as sip3) and happily install and work as intended.

@Strayer
Contributor
Strayer commented Jan 11, 2014

This confuses me too... I use PyQt4 with Python 3 in a project myself and currently try compiling PyQt by hand. Apart from the commits I couldn't find any information on why this was done - is Python 3 now kind of unsupported in Homebrew?

@MikeMcQuaid
Member

This is a regrettable situation, I agree and apologise. The reasoning is briefly explained in #24842. To expand somewhat: the Python code we had in the PythonDependency was working for some things (e.g. Python 3 support) and very badly for others (e.g. Mercurial). It was decided that we'd make the changes necessary to support the most active formulae unfortunately knowing that it would affect the less active ones (such as PyQt).

Moving on from here we'll review pull requests to improve these formulae to support Python 3 in PyQt and other places. Longer-term we may consider moving some of these formulae to the new homebrew-python tap where they can be better maintained by those who care and understand the needs of these packages without modifying the Homebrew core.

Apologies again for this.

@darthmdh
Contributor

This issue is not closed until the broken commit is reverted.

Hardcoding the wrong version and making it impossible to do the right thing is not a solution, its a bigger problem.

Python comes with inherent support for multiple instances - both virtual environments of the one interpreter and co-existence of multiple interpreters. Via the standard Unix shebang syntax, you can invoke either 'python' or 'python3' and it will figure it out from the $PATH - and I'm yet to see it get it wrong. But if you have (e.g. for us the likely scenario of a Mavericks system containing system python 2.7 plus homebrew python 2.7) then first on $PATH wins - so set this right (or ask the user to). You can also enforce it by setting $PYTHONHOME, though personally I steer clear of messing with any environment variables other than $PATH. I routinely run with at least three concurrent python instances (system+python2+python3 minimum) not counting venvs, and have no problems (until this nonsense)

Observe what wrappers like venv do via its 'activate' script. See http://docs.python.org/dev/library/venv.html#venv-def

For applications like Mercurial, setuptools (the setup.py script that builds it) will determine its python build environment and act accordingly, setting up the app to use this environment. So you can have 'hg' in the path (e.g. symlinked from /usr/local/bin via brew linkapps) but a completely different python environment activated, and hg will run fine as it'll know to use the one it was built against, which could be different to the currently activated python environment. Under no circumstances do you diddle with $PYTHONPATH - this is probably the root cause of your woes with other python-related recipies; unset it - its completely unnecessary. I note the (cough Ruby-friendly cough) homebrew team advocate its use a lot in other issues and that's the wrong thing to do!

For applications like MacVim which are not python themselves but embed a python interpreter, its the same story - Vim's ./configure script will pick up python or python3 from the $PATH and act accordingly, or you can also pass it a flag to set the python-config-dir for a specific location. I like it from core homebrew not brew-cask as building it from source ceases the problem where Mavericks' libc/c++ move will cause Vim to die should any python be built against different core libraries then either the other python lib or vim itself.

For a library like pyqt, it'll automatically install in the site-packages folder of either the installation or the virtualenv if one is activated. So for multi-version support, build against python2 (either homebrew or system, though personally I avoid messing with the system python), clean, set python3 first in $PATH and build again.

For virtualenv support, just forget it. Homebrew doesn't need to worry about this as Python already has its own build system in pip and setuptools so it would be nonsensically complex to try to help someone build, via homebrew, a library like pyqt inside a virtualenv. The best thing to do is let pyqt install into the standard site-packages folder of the python installation, and from there people can create the venv using --system-site-packages and 'import' can pick it up that way. Anyone who needs a customised pyqt for a specific project and the upstream version for other apps on the system will know what they're doing to make this happen, its waaaaay out of scope for homebrew to support.

@MikeMcQuaid
Member

This issue is not closed until the broken commit is reverted.

With respect, I've explained the situation above. We can't revert the code above because the code it depends on is about to go away because it breaks a bunch of other things.

Homebrew is a community project and no-one's full time job. If you passionately feel something should be changed the best way of doing that is through submitting a pull request for review.

@MikeMcQuaid
Member

I'll reopen this until someone takes a look at it.

@MikeMcQuaid MikeMcQuaid reopened this Jan 12, 2014
@jenshnielsen
Contributor

It would be nice in this information was reflected in the homebrew python wiki page which still mentions the --with-python3 option instead of the option just silently disappearing.

@MikeMcQuaid
Member

@jenshnielsen I'll try and sort out the docs.

@daviewales
Contributor

What's the current status of this issue? If I start creating ...-py3 formulae (e.g. pyqt-py3), will that be useful, or problematic?

@daviewales
Contributor

Also, if we do take the -py3 route, what should we do about symlinking? For instance, I've just finished porting the sip and pyqt formulae to sip-py3 and pyqt-py3 formulae, but the python2.x and python3.x versions clash at the symlink stage.

For instance, pyqt and pyqt-py3 both want to create symlinks in the following locations:

    /usr/local/bin/pyuic4
    /usr/local/bin/pyrcc4
    /usr/local/bin/pylupdate4

Should we just let them clash?
Should we make the python3 formula keg only?
Should we find a way to make different symlinks for the python3 version?

This is the first time I've created Homebrew formulae, so I don't know what the precedents are.

@MikeMcQuaid
Member

We can have -py3 stuff in Homebrew Python but in core we should just have:

depends_on :python => :recommended
depends_on :python3 => :optional

and then this should be handled in the formulae themselves.

@daviewales
Contributor

To clarify, if we have a formula with the following dependencies:

depends_on :python => :recommended
depends_on :python3 => :optional

we should build for Python 2.x, unless the user passes the --with-python3 flag, in which case we should build for both Python 2.x and Python 3.x.
If the user also passes the --without-python flag, then we should build for only Python 3.x.

@MikeMcQuaid
Member

@daviewales Correct. You can use build.with? "python3" to detect this. Python 3 support isn't deprecated, it just had to be scaled back in the previously mentioned formulae in order to perform the long overdue fixing of some Python issues. What was supported in some formulae and won't be any more is building for Python 2 and Python 3 in the same formulae with a single python do block handling both cases.

@daviewales
Contributor

Thanks for the clarification. I'll have another crack at adding Python 3 support to pyqt and sip.

@MikeMcQuaid
Member

Thanks @daviewales.

@daviewales
Contributor

I have sip working, but I have another question.

What should happen if a user specifies --without-python, and doesn't specify --with-python3?

i.e. What should happen if a user tries to build without any python at all?

Should we raise an error, or should we automatically assume --with-python3 for them?
Building without any Python at all is not a valid option for sip.

@MikeMcQuaid
Member

I'm not sure there. Probably throw a failure with odie or similar.

@daviewales
Contributor

@darthmdh What are the other formulae that are missing Python 3 support?

@darthmdh
Contributor
darthmdh commented Feb 8, 2014

pygobject3 only builds for 2.7 even if you specify --with-python3
macvim only finds 2.7 when built dynamically (can't recall if dyn is default or my own hacking as I last touched it after the Mavericks total rebuild).
pyside doesn't even offer, though its supported upstream since 1.0.8 (1.2.1 is brewed)

@MikeMcQuaid
Member

@darthmdh We'll review pull requests to fix these. I can't do them all myself, sorry.

@daviewales
Contributor

I'll have a go at them, but I can't promise anything.

@darthmdh
Contributor
darthmdh commented Feb 8, 2014

No worries, thanks guys. I tried to find more but got API rate limited. @daviewales has fixed the core stuff I raised this issue for, so I'll go ahead and close it. I haven't been submitting pull requests but this is probably a good indicator I should start doing so.

@darthmdh darthmdh closed this Feb 8, 2014
@xu-cheng xu-cheng locked and limited conversation to collaborators Feb 17, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.