Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

python framework search path incorrect for homebrew installed python #1838

Closed
springmeyer opened this Issue May 10, 2013 · 15 comments

Comments

Projects
None yet
2 participants
Owner

springmeyer commented May 10, 2013

Thought I fixed this in #1753, but did not fully.

hedin:Versions jj0hns0n$ brew install --HEAD --with-gdal --with-geos mapnik
==> Cloning https://github.com/mapnik/mapnik.git
Updating /Library/Caches/Homebrew/mapnik--git
==> Patching
patching file src/json/feature_collection_parser.cpp
Hunk #1 succeeded at 26 with fuzz 2 (offset 6 lines).
==> python scons/scons.py configure CC="cc" CXX="c++" JOBS=2 PREFIX=/usr/local/Cellar/mapnik/HEAD ICU_INCLUDES=
==> python scons/scons.py install
"c++" -o plugins/input/python.input -F/usr/local/Cellar/python -framework Python -Z -dynamiclib plugins/input/python/python_datasource.os plugins/input/python/python_featureset.os plugins/input/python/python_utils.os -Ldeps/agg -Lsrc -L/usr/local/Cellar/gdal/HEAD/lib -L/usr/local/Cellar/libtiff/4.0.3/lib -L/usr/local/Cellar/proj/4.8.0/lib -L/usr/local/Cellar/jpeg/8d/lib -L/usr/local/Cellar/freetype/2.4.11/lib -L/usr/local/Cellar/icu4c/51.1/lib -L/usr/local/Cellar/boost/1.53.0/lib -L/usr/lib -lmapnik -lboost_python-mt -lboost_system-mt -licuuc
ld: framework not found Python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
scons: *** [plugins/input/python.input] Error 1
scons: building terminated because of errors.

READ THIS: https://github.com/mxcl/homebrew/wiki/troubleshooting
Owner

springmeyer commented May 10, 2013

can be fixed with "FRAMEWORK_SEARCH_PATH=/usr/local/Cellar/python/2.7.4/Frameworks/"

Owner

springmeyer commented May 10, 2013

Looks like the solution is to use HOMEBREW_PREFIX/Frameworks. after mxcl/homebrew#14801. Found this after reading https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python and mxcl/homebrew#11204

Owner

springmeyer commented May 10, 2013

The problem is that this needs to ONLY be set if building against homebrew python (how to detect?):

index 5d88c72..01ba883 100644
--- a/Library/Formula/mapnik.rb
+++ b/Library/Formula/mapnik.rb
@@ -54,6 +54,7 @@ class Mapnik < Formula
              "ICU_INCLUDES=#{icu}/include",
              "ICU_LIBS=#{icu}/lib",
              "PYTHON_PREFIX=#{prefix}",  # Install to Homebrew's site-packages
+             "FRAMEWORK_SEARCH_PATH=#{HOMEBREW_PREFIX}/Frameworks", # For Python
              "JPEG_INCLUDES=#{jpeg}/include",
              "JPEG_LIBS=#{jpeg}/lib",
              "TIFF_INCLUDES=#{libtiff}/include",

@springmeyer springmeyer was assigned May 10, 2013

Owner

springmeyer commented May 20, 2013

pushing off 2.2 milestone, no action on the homebrew side.

@springmeyer springmeyer pushed a commit that referenced this issue May 22, 2013

Dane Springmeyer disable building of python plugin by default - works around #1838 031139f
Owner

springmeyer commented May 22, 2013

also python plugin is crashing on ubuntu raring, just noticed. no idea why:

python_tests.python_plugin_test.test_python_point_init ... ok
python_tests.python_plugin_test.test_python_circle_init ... ok
python_tests.python_plugin_test.test_python_circle_init_with_args ... ok
python_tests.python_plugin_test.test_python_point_rendering ... Segmentation fault (core dumped)
....
cat /var/log/kern.log
May 22 20:16:42 ip-10-197-18-49 kernel: [880070.728855] python[19281]: segfault at 98 ip 0000000000473ba8 sp 00007fff27b07520 error 6 in python2.7[400000+239000]
May 22 20:46:16 ip-10-197-18-49 kernel: [881844.939283] python[19431]: segfault at 98 ip 0000000000473ba8 sp 00007fffe44f2050 error 6 in python2.7[400000+239000]

Hi @springmeyer, I think I have python and mapnik building nicely in mxcl/homebrew#18588. I hope to push soon but I have to adapt some formulae first. Feel free to test my pull request and if it builds mapnik ok.

Please tell me, is this issue here about the mapnik.rb formula that is in homebrew or mapnik's own build script?
If the latter is the case, I think python-config --ldflags or something could also be an option to try.

You asked how to detect if python is from homebrew or system. Do you need this knowledge inside a formula or in general (from extern)?

Let me know if/how I can help.

Owner

springmeyer commented May 27, 2013

hi @samueljohn - thanks for the comments. Your changes to the mapnik.rb in your branch look harmless.

The issue here is very subtle. Overall mapnik core (libmapnik.so/dylib) does not depend on python at all. The python dependencies in mapnik are 1) python 2.x is needed by the scons build system mapnik uses, 2) Mapnik has python bindings that work with either python 2.x/3.x and which do not link to python itself but rather boost python (since python itself can be resolved at runtime based on the interpreter being used), and 3) Mapnik has an optional and experimental python.input datasource plugin that developers can use to stream geodata into Mapnik. This python.input must link to both boost python and the python lib itself because it might be used by a developer that is not controlling/running Mapnik via its python bindings.

So, the issue is that the build script for the python.input that Mapnik uses needs to specify the correct link flags for the current/active python. python-config --ldflags might provide some needed details. But really what I would hope is that homebrew would provide some function to be able to call in the formula that provides the formula develop info on 1) is python installed by homebrew and 2) it is linked or key-only, ie. active.

@springmeyer sounds good, then.
When I pull that changes (#18588), homebrew will provide python.brewed?, python.from_osx?, python.libdir and much more to ease such things. I also fixed the python-config --ldflags to return more robust values, so chances are that it "just works" for brewed python.

I'd love to see 2.x and 3.x support in the mapnik formula then. But I won't do this from the start (only for few formulae). I'll have to write some documentation in the wiki on how to do this.

Owner

springmeyer commented May 28, 2013

sure, happy to help with 2.x and 3.x support in the formula. It should be fairly strait forward. The main issue is that boost_python needs to provide both 2.x and 3.x versions of its lib: https://github.com/mapnik/mapnik/wiki/Python3k

Ok so the challenge is to get boost to build for both? We can tackle this together after I finished basic python 3.x support.

Hi @springmeyer, I finished and pushed the depends_on :python and depends_on :python3 stuff.

A note of warning about adding HOMEBREW_PREFIX/Frameworks, due to a "strange" behavior of Ruby's File.find, Frameworks are not really usable right now in the HOMEBREW_PREFIX because symlinks are not followed. So the Headers symlink Frameworks do have results in an empty directory named Headers. Sucks.

So for the time being, I would suggest to not set the framework path at all. I adapted the python-config --ldflags for brewed python such that it returns linking flags that actually work (by including the full path).

@springmeyer btw. a python core dev told me not to use -framework Python because you don't know if you will get Python 2.x or 3.x (both provide Python.framwork). python-config --ldflags should be preferred, I guess.

Owner

springmeyer commented Jun 11, 2013

@samueljohn - thanks for the update! traveling at the moment but I'll try to take a closer look soon.

Alright. Let me know or ping me if you need assistance!

Owner

springmeyer commented Jun 23, 2013

more details on mxcl/homebrew#19723 (comment) for when I have time to try/test this.

@springmeyer springmeyer closed this Sep 6, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment