New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spack fails to build python on OSX 10.9 #42

Closed
cyrush opened this Issue May 11, 2015 · 24 comments

Comments

Projects
None yet
7 participants
@cyrush
Member

cyrush commented May 11, 2015

I ran into issues building python using spack on OSX 10.9, using either gcc or clang (Xcode 5.1) see below for the gory details, here is one solution to the problem:

If I explicitly set the MACOSX_DEPLOYMENT_TARGET env var to my OSX version (10.9), I avoid the problem. Given this solution, I am not sure if this is something that should go into the spack python package script, or if it could be resolved better in your compiler wrappers. In build_visit, we always set the osx deployment target to avoid issues with various libraries - it may be good practice also to do so in the spack compiler wrappers.

Here is the main symptom that occurs: python’s configure checks fail and report the size of all of the integer types :

checking for uid_t in sys/types.h... yes
checking for uint32_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for uint64_t... yes
checking for int32_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for int64_t... yes
checking for ssize_t... yes
checking size of int... 0
checking size of long... 0
checking size of void *... 0
checking size of short... 0
checking size of float... 0
checking size of double... 0
checking size of fpos_t... 0
checking size of size_t... configure: error: in /private/var/folders/wj/jjhjt_0d3kg1vmk56d5bq_38001kr8/T/spack-stage/spack-stage-r5wJGI/Python-2.7.8': configure: error: cannot compute sizeof (size_t) Seeconfig.log' for more details
==> Error: command './configure --prefix=/Users/harrison37/Work/uberenv/spack/opt/macosx_10.9_x86_64/gcc@4.2.1/python@2.7.8 --without-gcc --enable-shared' returned error code 77

tgamblin added a commit that referenced this issue Nov 28, 2015

Fixed bug #42: problem with satisfies() for virtual dependencies.
- _cross_provider_maps() had suffered some bit rot (map returned was
  ill-formed but still worked for cases with one vdep)

- ProviderIndex.satisfies() was only checking whether the result map
  was non-empty.  It should check whether all common vdeps are *in*
  the result map, as that indicates there is *some* way to satisfy
  *all* of them.  We were checking whether there was some way to
  satisfy *any one* of them, which is wrong.

- Above would cause a problem when there is more than one vdep provider.

- Added test that covers this case.

- Added `constrained()` method to Spec. Analogous to `normalized()`:
  `constrain():constrained() :: normalize():normalized()`

@tgamblin tgamblin added the bug label Dec 20, 2015

@mathstuf

This comment has been minimized.

Show comment
Hide comment
@mathstuf

mathstuf Mar 30, 2016

Collaborator

I'm having no problem with Python2 or Python3 on OS X 10.10 (with Xcode 7.2.1).

Collaborator

mathstuf commented Mar 30, 2016

I'm having no problem with Python2 or Python3 on OS X 10.10 (with Xcode 7.2.1).

@cyrush

This comment has been minimized.

Show comment
Hide comment
@cyrush

cyrush Mar 30, 2016

Member

Looking at the current python spack package -- MACOSX_DEPLOYMENT_TARGET is now set. That was the key to avoiding this issue, so think it is now resolved.

Member

cyrush commented Mar 30, 2016

Looking at the current python spack package -- MACOSX_DEPLOYMENT_TARGET is now set. That was the key to avoiding this issue, so think it is now resolved.

@cyrush cyrush closed this Mar 30, 2016

@tgamblin

This comment has been minimized.

Show comment
Hide comment
@tgamblin

tgamblin Mar 30, 2016

Member

@mathstuf: Should we hard-code the deployment target to 10.6, or take the current OS version (from platform.mac_ver())? I think starting with El Capitan we can't do that and support newer C++ features. So we should probably auto-set it to the latest -- I believe this is what Homebrew does.

Member

tgamblin commented Mar 30, 2016

@mathstuf: Should we hard-code the deployment target to 10.6, or take the current OS version (from platform.mac_ver())? I think starting with El Capitan we can't do that and support newer C++ features. So we should probably auto-set it to the latest -- I believe this is what Homebrew does.

@cyrush

This comment has been minimized.

Show comment
Hide comment
@cyrush

cyrush Mar 30, 2016

Member

@tgamblin - this is the approach I am using in driver code that calls spack:

if "darwin" in platform.system().lower():
    dep_tgt = platform.mac_ver()[0]
    dep_tgt = dep_tgt[:dep_tgt.rfind(".")]
    print "[setting MACOSX_DEPLOYMENT_TARGET to %s]" % dep_tgt
    env["MACOSX_DEPLOYMENT_TARGET"] = dep_tgt
Member

cyrush commented Mar 30, 2016

@tgamblin - this is the approach I am using in driver code that calls spack:

if "darwin" in platform.system().lower():
    dep_tgt = platform.mac_ver()[0]
    dep_tgt = dep_tgt[:dep_tgt.rfind(".")]
    print "[setting MACOSX_DEPLOYMENT_TARGET to %s]" % dep_tgt
    env["MACOSX_DEPLOYMENT_TARGET"] = dep_tgt

@cyrush cyrush reopened this Mar 30, 2016

@tgamblin

This comment has been minimized.

Show comment
Hide comment
@tgamblin

tgamblin Mar 30, 2016

Member

@cyrush: I think Spack should probably set that automatically in the build environment.

@mathstuf @eschnett: do you see any problem with moving MACOSX_DEPLOYMENT_TARGET up a level to the build environment so that package authors do not have to add it?

Member

tgamblin commented Mar 30, 2016

@cyrush: I think Spack should probably set that automatically in the build environment.

@mathstuf @eschnett: do you see any problem with moving MACOSX_DEPLOYMENT_TARGET up a level to the build environment so that package authors do not have to add it?

@mathstuf

This comment has been minimized.

Show comment
Hide comment
@mathstuf

mathstuf Mar 30, 2016

Collaborator

It looks to be well supported, so moving it up seems sensible. What its value should be, however…

Collaborator

mathstuf commented Mar 30, 2016

It looks to be well supported, so moving it up seems sensible. What its value should be, however…

@tgamblin

This comment has been minimized.

Show comment
Hide comment
@tgamblin

tgamblin Mar 30, 2016

Member

My suggestion is what Cyrus recommended: '.'.join(platform.mac_ver()[0].split('.')[:2])

Member

tgamblin commented Mar 30, 2016

My suggestion is what Cyrus recommended: '.'.join(platform.mac_ver()[0].split('.')[:2])

@mathstuf

This comment has been minimized.

Show comment
Hide comment
@mathstuf

mathstuf Mar 30, 2016

Collaborator

Yeah, if OS X support is for development and not deployment, just using the system's version is probably sufficient. Apple doesn't support downgrades, so that is unlikely to be a problem.

Collaborator

mathstuf commented Mar 30, 2016

Yeah, if OS X support is for development and not deployment, just using the system's version is probably sufficient. Apple doesn't support downgrades, so that is unlikely to be a problem.

@tgamblin

This comment has been minimized.

Show comment
Hide comment
@tgamblin

tgamblin Mar 30, 2016

Member

@mathstuf: for the most part I think Spack is being used on peoples' laptops as a dev proxy of what they do on supercomputers. I haven't actually been approached by anyone from the labs who wanted to use a shared NFS volume as a deploy environment, either.

@becker33 should have platform support in pretty soon, at which point we can separate out the mac platforms (and even allow cross-compiling by setting this variable). For now if you want to submit a PR with this hoisted up to the build environment that would be great.

Member

tgamblin commented Mar 30, 2016

@mathstuf: for the most part I think Spack is being used on peoples' laptops as a dev proxy of what they do on supercomputers. I haven't actually been approached by anyone from the labs who wanted to use a shared NFS volume as a deploy environment, either.

@becker33 should have platform support in pretty soon, at which point we can separate out the mac platforms (and even allow cross-compiling by setting this variable). For now if you want to submit a PR with this hoisted up to the build environment that would be great.

@adamjstewart

This comment has been minimized.

Show comment
Hide comment
@adamjstewart

adamjstewart Jun 20, 2016

Member

I am also unable to build Python on macOS. For me, the only error message I'm seeing is:

/System/Library/Frameworks/CoreGraphics.framework/Headers/CGFont.h:53:40: error: initializer element is not constant
Member

adamjstewart commented Jun 20, 2016

I am also unable to build Python on macOS. For me, the only error message I'm seeing is:

/System/Library/Frameworks/CoreGraphics.framework/Headers/CGFont.h:53:40: error: initializer element is not constant
@eschnett

This comment has been minimized.

Show comment
Hide comment
@eschnett

eschnett Jun 21, 2016

Collaborator

@adamjstewart What compiler are you using?

Collaborator

eschnett commented Jun 21, 2016

@adamjstewart What compiler are you using?

@adamjstewart

This comment has been minimized.

Show comment
Hide comment
@adamjstewart

adamjstewart Jun 21, 2016

Member

I was using a Spack-built gcc

Member

adamjstewart commented Jun 21, 2016

I was using a Spack-built gcc

@davydden

This comment has been minimized.

Show comment
Hide comment
@davydden

davydden Jun 21, 2016

Member

on the current develop branch python 2.7.1. builds fine for me on macOS with clang.

Member

davydden commented Jun 21, 2016

on the current develop branch python 2.7.1. builds fine for me on macOS with clang.

@tgamblin

This comment has been minimized.

Show comment
Hide comment
@tgamblin

tgamblin Jun 21, 2016

Member

A lot of things do not build properly with gcc on El Capitan. I have contemplated in the past making clang the preferred compiler on OS X, but I hesitate to do this because it doesn't have good fortran support, and Spack doesn't have the best provenance for mixed compilers. However, I suspect that most people are going to use the system clang + gfortran. What are you guys using, and what's your sense of the common case? I know @eschnett uses mostly gcc. How about others? @cyrush?

Member

tgamblin commented Jun 21, 2016

A lot of things do not build properly with gcc on El Capitan. I have contemplated in the past making clang the preferred compiler on OS X, but I hesitate to do this because it doesn't have good fortran support, and Spack doesn't have the best provenance for mixed compilers. However, I suspect that most people are going to use the system clang + gfortran. What are you guys using, and what's your sense of the common case? I know @eschnett uses mostly gcc. How about others? @cyrush?

@davydden

This comment has been minimized.

Show comment
Hide comment
@davydden

davydden Jun 21, 2016

Member

i use clang+fortran, but i have no problems in the past with building python with gcc as well.

Member

davydden commented Jun 21, 2016

i use clang+fortran, but i have no problems in the past with building python with gcc as well.

@davydden

This comment has been minimized.

Show comment
Hide comment
@davydden

davydden Jul 12, 2016

Member

i confirm the same build error as @adamjstewart on el capitan with gcc 6

In file included from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGContext.h:18:0,
                 from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGBitmapContext.h:9,
                 from /System/Library/Frameworks/CoreGraphics.framework/Headers/CoreGraphics.h:11,
                 from /System/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:35,
                 from /System/Library/Frameworks/Carbon.framework/Headers/Carbon.h:24,
                 from Include/pymactoolbox.h:10,
                 from Python/mactoolboxglue.c:27:
/System/Library/Frameworks/CoreGraphics.framework/Headers/CGFont.h:53:40: error: initializer element is not constant
 static const CGFontIndex kCGGlyphMax = kCGFontIndexMax;
                                        ^~~~~~~~~~~~~~~

p.s.

i have no problems in the past with building python with gcc as well.

here i referred to linux

Member

davydden commented Jul 12, 2016

i confirm the same build error as @adamjstewart on el capitan with gcc 6

In file included from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGContext.h:18:0,
                 from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGBitmapContext.h:9,
                 from /System/Library/Frameworks/CoreGraphics.framework/Headers/CoreGraphics.h:11,
                 from /System/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:35,
                 from /System/Library/Frameworks/Carbon.framework/Headers/Carbon.h:24,
                 from Include/pymactoolbox.h:10,
                 from Python/mactoolboxglue.c:27:
/System/Library/Frameworks/CoreGraphics.framework/Headers/CGFont.h:53:40: error: initializer element is not constant
 static const CGFontIndex kCGGlyphMax = kCGFontIndexMax;
                                        ^~~~~~~~~~~~~~~

p.s.

i have no problems in the past with building python with gcc as well.

here i referred to linux

@davydden

This comment has been minimized.

Show comment
Hide comment
@davydden

davydden Jul 12, 2016

Member

here's the same issue in hashdist hashdist/hashstack#904

Member

davydden commented Jul 12, 2016

here's the same issue in hashdist hashdist/hashstack#904

@cyrush

This comment has been minimized.

Show comment
Hide comment
@cyrush

cyrush Jul 27, 2016

Member

I am now less than 1 day into the El Capitan camp, I can tell there will be lots of fun will be ahead.

Should we open a new issue for El Capitan stuff?

To answer @tgamblin's earlier question -- I use clang + gfortran.

I think for El Capitan I am going to have spack build gfortran for me.

Member

cyrush commented Jul 27, 2016

I am now less than 1 day into the El Capitan camp, I can tell there will be lots of fun will be ahead.

Should we open a new issue for El Capitan stuff?

To answer @tgamblin's earlier question -- I use clang + gfortran.

I think for El Capitan I am going to have spack build gfortran for me.

@eschnett

This comment has been minimized.

Show comment
Hide comment
@eschnett

eschnett Jul 27, 2016

Collaborator

On OS X, I use the system Clang to build my own GCC, then I tell Spack about this GCC. From then on, I build everything with this GCC.

This fails with packages such as Python that don't build with GCC, due to the reasons described above.

Collaborator

eschnett commented Jul 27, 2016

On OS X, I use the system Clang to build my own GCC, then I tell Spack about this GCC. From then on, I build everything with this GCC.

This fails with packages such as Python that don't build with GCC, due to the reasons described above.

@gartung

This comment has been minimized.

Show comment
Hide comment
@gartung

gartung Aug 18, 2016

Member

Hah, I found this with a google search for the size_t error. I am building python 2.7.11 with gcc 6.1.0 on OSX10.11 in a rpmbuild based system (ugh).
Passing MACOSX_DEPLOYMENT_TARGET=10.11
option to ./configure got me past the size_t error.
I avoided the header errors in mactoolbox glue by disabling it with --disable-toolbox-glue. It's not optimal but our software doesn't use mactoolboxglue.

Of course spack built python 2.7.12 with gcc6.1.0 once I modified the package to add --disable-toolbox-glue.

Member

gartung commented Aug 18, 2016

Hah, I found this with a google search for the size_t error. I am building python 2.7.11 with gcc 6.1.0 on OSX10.11 in a rpmbuild based system (ugh).
Passing MACOSX_DEPLOYMENT_TARGET=10.11
option to ./configure got me past the size_t error.
I avoided the header errors in mactoolbox glue by disabling it with --disable-toolbox-glue. It's not optimal but our software doesn't use mactoolboxglue.

Of course spack built python 2.7.12 with gcc6.1.0 once I modified the package to add --disable-toolbox-glue.

@davydden

This comment has been minimized.

Show comment
Hide comment
@davydden

davydden Oct 11, 2016

Member

since #1680 was merged as e6dc448, can we close this issue?

Member

davydden commented Oct 11, 2016

since #1680 was merged as e6dc448, can we close this issue?

@tgamblin

This comment has been minimized.

Show comment
Hide comment
@tgamblin
Member

tgamblin commented Oct 11, 2016

@tgamblin

This comment has been minimized.

Show comment
Hide comment
@tgamblin

tgamblin Oct 11, 2016

Member

whoops sorry -- meant @cyrush?

Member

tgamblin commented Oct 11, 2016

whoops sorry -- meant @cyrush?

@cyrush

This comment has been minimized.

Show comment
Hide comment
@cyrush

cyrush Oct 11, 2016

Member

yes, sorry for the delay

Member

cyrush commented Oct 11, 2016

yes, sorry for the delay

@cyrush cyrush closed this Oct 11, 2016

matz-e added a commit to matz-e/spack that referenced this issue Sep 3, 2018

synapse-tool → synapsetool (spack#42)
Alternative/closes spack#41, fixes spack#40. More in line with the git url of
synapsetool than the name syntool. Also adjust dependencies.

matz-e added a commit to matz-e/spack that referenced this issue Oct 1, 2018

synapse-tool → synapsetool (spack#42)
Alternative/closes spack#41, fixes spack#40. More in line with the git url of
synapsetool than the name syntool. Also adjust dependencies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment