request: virtualenv on OSX unrequire Xcode #168
Comments
|
I don't use OSX myself, but my understanding is that Xcode actually is needed for virtualenv to work at all on OSX; it isn't about installing C extensions. It has to do with using the Apple framework install of Python, and needing to use the "install_name_tool" command (which, IIUC, comes only with Xcode) to tell the virtualenv python executable about its new location. I guess if you compile your own Python and use that with virtualenv instead of the framework install, you might not need install_name_tool -- but if you've compiled your own Python you've clearly got Xcode anyway! If you have more information here, or want to assert that install_name_tool is not in fact required, or is available without Xcode, please start a thread about it on the mailing list (http://groups.google.com/group/python-virtualenv) -- I'd like to get confirmation from some other OSX virtualenv users before making any changes here. |
|
Thanks for the fast response and the information! I couldn't quite remember the mechanism on this one, and my search skills were failing me! |
|
For the record, Xcode for Lion is free (http://itunes.apple.com/us/app/xcode/id448457090?mt=12). |
|
So, talked to Glyph about this at DjangoCon, and he thinks it would be possible to do the same thing that we currently do with install_name_tool using instead the "macholib" library that is available on OSX Pythons. If we could make this change, it would mean Xcode would no longer be required to use virtualenv on OSX, which would be awesome. However, he didn't say it would be easy, so it will probably require an OSX user to devote some significant time to it. Reopening this issue to track this possibility. |
|
Just to clarify; I'm not completely sure about this :). Take a look at what py2app does when it's manipulating its stub binary to stick into an app bundle though, and you might be able to figure something out from there. |
|
Yes, macholib is a possibility but I think the easier and better solution would be to provide a compiled version of Source: http://www.opensource.apple.com/source/cctools/cctools-750/misc/install_name_tool.c |
|
For what it's worth, you download compiled versions from MacOSForge: Leopard: http://src.macosforge.org/Roots/9A581/cctools.root.tar.gz |
|
Jup, but that does not mean we have the right to redistribute these binaries I think. (IANAL) |
|
Carljim, perhaps this will help? from: http://jessenoller.com/2011/07/30/quick-pythondeveloper-tips-for-osx-lion/ https://github.com/kennethreitz/osx-gcc-installer Side note: @kennethreitz, gentleman scholar, has actually done a custom osx-gcc-installer — this contains a system install of GCC and all the goodness you need (such as install_name_tool) for Python hacking. So you might be able to skip the massive XCode install. |
|
@mitsuhiko: Nor am I, but I'd consider it likely that you do, given that |
|
And the S stands for source. Unless I am terribly blind it does not mention binaries at all. Yes, you can build it yourself, but it would be a lot easier for virtualenv if we could just pull in the binaries from macosforge and ship them. |
|
The APSL was approved by both the Open Source Initiative and the Free Software Foundation, so it should give you ample rights to distribute binaries. The links I provided earlier were for Apple's DarwinSource project, which exists in order to allow other people build the base system. I suppose you could ask on darwinbuild-dev if you feel uncertain whether the roots are redistributable, or if it's okay to link to them… |
|
Assuming this is legally possible we could ship the binaries for install_name_tool (last three versions of OS X). A stupid simple compression would be concatenating all three files, remembering the offsets, zlib encoding them and base64ing them. This would come to around 300KB extra size for virtualenv. I think we need all since they support different architectures but I have not verified this. Alternatively (and what I would think makes more sense) would be to provide a script that download install_name_tool and drops it into Regarding licensing I think you can't be too cautious when it comes to apple. |
|
Any additional traction on this? We are gearing up for the next Minneapolis PyStar workshop in 3 weeks, and it would be great to see something happen on this. The Xcode dependency is a major blocker for 'doing things right'. GL |
shouldfix: creating a virtualenv from an OSX Framework build should no longer require XCode to run successfully. implementaion: if ``install_name_tool`` missing, download binaries from macosxforge, and install them to a temp directory rejected: download and compile .c source. TODOS: * review for python2/3 issues * cleanup of tempfiles? Assume the system will do it? * test for robustness * review implications of not having XCode for packages, if any * integrate with pypa#54 pull or its descendents. (trivial) Possible Test Code: #!/bin/sh deactivate # virtualenv on system (Framework) python # assuming I have XCode or ``install_name_tool`` in path. python virtualenv.py t_install_name_tool # creates with no issue t_install_name_tool/bin/python -c "" # remove it from path sudo mv /usr/bin/install_name_tool{,2} # should create with no issue, using tmpfile python virtualenv.py t_tmp_install_name_tool t_tmp_install_name_tool/bin/python -c "" # clean up sudo mv /usr/bin/install_name_tool{2,} command rm -rf t_{,tmp_}install_name_tool
shouldfix: creating a virtualenv from an OSX Framework build should no longer require XCode to run successfully. implementaion: if ``install_name_tool`` missing, download binaries from macosxforge, and install them to a temp directory rejected: download and compile .c source. TODOS: * review for python2/3 issues * cleanup of tempfiles? Assume the system will do it? * test for robustness * review implications of not having XCode for packages, if any * integrate with pypa#54 pull or its descendents. (trivial) Possible Test Code: #!/bin/sh deactivate # virtualenv on system (Framework) python # assuming I have XCode or ``install_name_tool`` in path. python virtualenv.py t_install_name_tool # creates with no issue t_install_name_tool/bin/python -c "" # remove it from path sudo mv /usr/bin/install_name_tool{,2} # should create with no issue, using tmpfile python virtualenv.py t_tmp_install_name_tool t_tmp_install_name_tool/bin/python -c "" # clean up sudo mv /usr/bin/install_name_tool{2,} command rm -rf t_{,tmp_}install_name_tool The original patch was cleaned up, tested against at Python 2.5, 2.6, 2.7, 3.2, and verified to work. Signed-off-by: Yesudeep Mangalapilly <yesudeep@google.com>
|
+1 for this PR; we dont need that FAT xcode here |
|
We're having the issue at Mozilla, and we don't want to force everyone to have to install Xcode, especially when they don't need to deal with compiling code (localizers are such people). |
|
Answering myself: python derives sys.prefix from the library location, which explains why the library is copied. |
|
Pull request #212 / #241 is a really ugly approach that frankly I'd rather avoid. IMO the most promising option discussed here is a pure-Python replacement for |
|
http://kennethreitz.com/xcode-gcc-and-homebrew.html might be the better solution, and the one I would probably recommend, even over my own patches. Lots of other useful libraries (Numpy, Fabric/Paramiko) need compilation, and explaining to users about 'what went wrong' is painful. 10.7 Xcode doesn't even come with GCC and environment (4.3!). As a possible derail, does Rails / Ruby have this issue? (note: most of us on this are Moz people right now, if we want to chat about it in-house :) |
|
I tried to make a macholib replacement for install_name_tool and the resulting binaries only worked on certain OS releases. I can't find my notes at the moment, but I'd guess it worked on 10.6, but failed on 10.7. |
|
Anyone interested in seeing this issue fixed - please test out pull request #289 and post your results there. The diff looks good to me and I'll likely merge it soon, pending any issues discovered in further testing. |
Could there be a solution that installs virtualenv, but doesn't complain about the xcode necessary bits until they are actually needed (i.e., until a c-extension needs to be compiled)? As an alternative, would it make sense to explore a 'minimum xcode install' alternative?
(This has actively bitten me in trying to write/run classes for PyStar.org)
The text was updated successfully, but these errors were encountered: