Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python runtime is not yet available on OS X #2312

Closed
BanzaiMan opened this issue May 14, 2014 · 51 comments
Closed

Python runtime is not yet available on OS X #2312

BanzaiMan opened this issue May 14, 2014 · 51 comments
Assignees

Comments

@BanzaiMan
Copy link
Member

@BanzaiMan BanzaiMan commented May 14, 2014

No version of Python is available via virtualenv on OS X workers.

https://travis-ci.org/BanzaiMan/travis_production_test/builds/25122306

@BanzaiMan BanzaiMan added mac labels May 14, 2014
@BanzaiMan
Copy link
Member Author

@BanzaiMan BanzaiMan commented May 14, 2014

The version available on Mavericks is 2.7.5.

@joshk
Copy link
Contributor

@joshk joshk commented May 14, 2014

we should install virtualenv like we do with the linux workers, but talk to the great @dstufft about how we might install prebuilt python binaries on demand? :)

@dstufft
Copy link

@dstufft dstufft commented May 14, 2014

I'm not aware of anything like that existing currently. I had some sketches for something like that but I don't have near the time to do that.

@joshk
Copy link
Contributor

@joshk joshk commented May 14, 2014

maybe we can talk more about your ideas, and what you think the time commitment would be to build it, and then we can see how we can make it happen?

@BanzaiMan BanzaiMan added build and removed cookbooks labels May 14, 2014
@dstufft
Copy link

@dstufft dstufft commented May 14, 2014

I'm happy to discuss them, but it's unlikely there'll will be much that'll give me enough time to build it. I'm running on fumes as it is.

@joshk
Copy link
Contributor

@joshk joshk commented May 14, 2014

oh, no no no, i don't expect you to build it, but thinking maybe we can help fund it or something?

@dstufft
Copy link

@dstufft dstufft commented May 14, 2014

Ah. Well I'm happy to advise or whatever :) It's not a very complicated thing I don't think. Fundamentally you just need a way to map an environment and maybe compile options to an identifier that you can use to download and unpack.

@schlamar
Copy link

@schlamar schlamar commented May 15, 2014

crypthography has a recipe to install Python on Travis OSX workers: https://github.com/pyca/cryptography/blob/master/.travis/install.sh#L29

@BanzaiMan
Copy link
Member Author

@BanzaiMan BanzaiMan commented May 15, 2014

@dstufft
Copy link

@dstufft dstufft commented May 15, 2014

It's not actually a C application, it just pretends to be one because Python + OSX on travis-ci means builds error before they execute anything that a project has control over.

@BanzaiMan
Copy link
Member Author

@BanzaiMan BanzaiMan commented May 15, 2014

Sure, but, as far as Travis is concerned, it is a C application. As a result, certain things do not happen for cryptography by default: setting $TRAVIS_PYTHON_VERSION, python key in .travis.yml (if it existed) is not used for constructing the build matrix, etc. If one chooses this route, one is responsible for setting up the sane environment for CI.

@BanzaiMan
Copy link
Member Author

@BanzaiMan BanzaiMan commented May 15, 2014

…which, I guess, is the part about "recipe" that @schlamar mentioned.

@ogrisel
Copy link

@ogrisel ogrisel commented May 16, 2014

The cryptography install script mentioned by @schlamar is using the pyenv (not to be confused with the pyvenv command that comes built-in with Python 3.3+) to build and install Python itself from source and manage shims to switch between Python versions. Using a custom built version of Python will prevent the users to pip install .whl packages with the macosx_10_6_intel.whl suffix.

Instead I would recommend using the official prebuilt .dmg archives from https://www.python.org/downloads/ that contain .mpkg installers. It is possible to script the install of such .dmg + .mpkg installers under OSX with a shell script such as the following:

http://blog.smalleycreative.com/administration/automating-osx-part-one/

Once official Python binaries are installed on OSX, it is possible to install pip with

curl -O https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py
sudo /Library/Frameworks/Python.framework/Versions/3.3/bin/python get-pip.py
sudo /Library/Frameworks/Python.framework/Versions/3.3/bin/pip install virtualenv
mkdir -p ~/virtualenv/
/Library/Frameworks/Python.framework/Versions/3.3/bin/virtualenv ~/virtualenv/python3.3
. ~/virtualenv/python3.3/bin/activate

Note: Python 3.4 (and later) comes with pip and pyvenv (a replacement for virtualenv) installed by default, so there is no need to use the get-pip.py bootstrap and the previous reduces to:

mkdir -p ~/virtualenv/
/Library/Frameworks/Python.framework/Versions/3.4/bin/pyvenv ~/virtualenv/python3.4
. ~/virtualenv/python3.4/bin/activate

I can help you working on such a travis setup for Python under OSX if you wish.

@BanzaiMan
Copy link
Member Author

@BanzaiMan BanzaiMan commented May 16, 2014

If possible, we should stick with a uniform way of managing multiple Python (or any other language) runtimes on all platforms. Supporting different ways sounds appealing, but, in my opinion, support overhead will outweigh the "native" feel benefit in the long run.

@ogrisel
Copy link

@ogrisel ogrisel commented May 16, 2014

@BanzaiMan as far as I know, under Linux, python is installed with a ppa ( https://launchpad.net/~fkrull/+archive/deadsnakes ), hence this method is not possible under OSX. Using the official python.org binary packages for OSX sounds like the most standard way to install Python under OSX and is pretty uniform for all Python versions.

@dstufft
Copy link

@dstufft dstufft commented May 16, 2014

Linux uses pyenv other than for the system site packages variety.

@ogrisel
Copy link

@ogrisel ogrisel commented May 16, 2014

Indeed I just saw the history of the CI env for linux: https://github.com/travis-ci/travis-cookbooks/commits/master/ci_environment/python . The deadsnakes PPA is no longer used since 2 months.

Where is the configuration of the OSX worker image? Is it also managed via the same chef recipes, or is there a specific infrastructure for OSX workers?

@dstufft what is your opinion on using python.org vs pyenv built python under OSX and the impact it has on distutils.util.get_platform?

On Python 3.4 installed from the dmg of python.org I get:

Python 3.4.0 (v3.4.0:04f714765c13, Mar 15 2014, 23:02:41)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.util import get_platform
>>> get_platform()
'macosx-10.6-intel'

On Python 3.4 installed using pyenv, I get:

Python 3.4.0 (default, May 16 2014, 14:36:29)
[GCC 4.7.3] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.util import get_platform
>>> get_platform()
'macosx-10.9-x86_64'

apparently this has an impact on the platform tags when I run the command pip wheel to build a binary distribution and the whl packages accepted when running pip install.

For instance if I run pip install numpy on the latter Python 3.4 (the one from pyenv), it will try to build numpy from the source tarball instead of reusing the official whl package provided by the numpy developers at: https://pypi.python.org/pypi/numpy

If I use the Python 3.4 of python.org, then pip install numpy directly installs the official whl package from pypi without trying to compile numpy from source (which is much faster and user friendlier).

I am not sure whether this is a problem in pip, wheel or a fundamental incompatibility in the 2 builds of the Python interpreter.

I think the default travis-ci should make it easy for python project maintainers to build wheels that they can upload to pypi for their OSX users.

I think that naive Python OSX users are quite likely to use the python installer from Python.org, hence my concerns.

@ogrisel
Copy link

@ogrisel ogrisel commented May 16, 2014

I just found this interesting reference on platform tags, pip wheels and OSX: https://github.com/MacPython/wiki/wiki/Spinning-wheels

@dstufft
Copy link

@dstufft dstufft commented May 16, 2014

My plan was always to have travis on OSX not use the Python.org installer.

We'll probably at some point fix pip so that a 10.9 OSX will load something with the 10.6 SDK as that thing you linked said.

@ogrisel
Copy link

@ogrisel ogrisel commented May 16, 2014

So having read the above reference and @minrk's PR at pypa/pip#1465, it seems that the only way to build fat multi-arch wheels (with the 10_6_intel suffix) is to use a Python built on OSX 10.6 such as those distributed by python.org.

@traumschule traumschule mentioned this issue Nov 23, 2017
gsnedders added a commit to gsnedders/travis-macos-safari-test that referenced this issue Dec 6, 2017
@casperdcl casperdcl mentioned this issue Dec 13, 2017
5 of 12 tasks complete
gf712 pushed a commit to gf712/PyML that referenced this issue Dec 14, 2017
Xuanwo added a commit to qingstor/qsctl that referenced this issue Dec 15, 2017
ref: travis-ci/travis-ci#2312

Signed-off-by: Xuanwo <xuanwo@yunify.com>
Xuanwo added a commit to qingstor/qsctl that referenced this issue Dec 15, 2017
ref: travis-ci/travis-ci#2312

Signed-off-by: Xuanwo <xuanwo@yunify.com>
shirosaki added a commit to plasticboy/vim-markdown that referenced this issue Dec 15, 2017
benmwebb added a commit to ihmwg/python-ihm that referenced this issue Mar 4, 2018
Turns out Travis doesn't support Python on Mac yet,
as per travis-ci/travis-ci#2312.
mishaturnbull added a commit to mishaturnbull/PySpeedTest that referenced this issue Mar 8, 2018
romainrichard added a commit to hacksaurz/nim that referenced this issue Mar 11, 2018
:(
See travis-ci/travis-ci#2312 for details
mikakoi added a commit to Codaone/DEXBot that referenced this issue Mar 12, 2018
Travis currently doesn't support python on mac. See travis-ci/travis-ci#2312
niemasd added a commit to niemasd/FAVITES that referenced this issue Mar 14, 2018
Travis currently doesn't support Python on Mac OS X. See travis-ci/travis-ci#2312
romainrichard added a commit to hacksaurz/nim that referenced this issue Mar 15, 2018
Run nim on Python 3.6, 3.7-dev and pypy3.

Unfortunately Python isn't yet supported on OSX yet on Travis, so we can only ask for Linux
See travis-ci/travis-ci#2312 for details.
lemoney pushed a commit to lemoney/grease that referenced this issue Mar 20, 2018
Signed-off-by: Jimmy <contact@travelerbell.net>
lemoney pushed a commit to lemoney/grease that referenced this issue Mar 20, 2018
Signed-off-by: Jimmy <contact@travelerbell.net>
diegozea added a commit to PhyloSofS-Team/PhyloSofS that referenced this issue Mar 29, 2018
benjspriggs added a commit to PDXCapstoneF/SPECtate that referenced this issue Apr 3, 2018
@travis-ci travis-ci locked and limited conversation to collaborators Apr 8, 2018
@DrTorte
Copy link
Contributor

@DrTorte DrTorte commented Sep 19, 2018

Hi everyone,

At this time we have no plans to implement support for this, so I'll be closing this issue. If it changes, we'll be sure to let everyone know! Thanks for all the feedback.

@DrTorte DrTorte closed this Sep 19, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.