Build Sage on OS X Mountain Lion

[hedtke]

To build Sage on OS X Mountain Lion: update to latest Mountain Lion (10.8.2 as of Sept 19) and Xcode (4.5). Use the new scipy spkg from #13541.

CC: @jhpalmieri @sagetrac-yomcat @mminzlaff

Component: build

Reviewer: Dmitrii Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/13309

[nexttime]

comment:1

Attachment: mercurial-2.2.2.p0.log

MacOS X 10.8 is not yet a supported platform, also because XCode seems to currently be broken on it; see several recent posts on sage-release.

[nexttime]

comment:2

We should probably make this a meta-ticket.

[jdemeyer]

comment:3

Can anybody give remote access to such a system?

[jdemeyer]

comment:4

Given no work has been done here, I don't think we should let it block the sage-5.3 release.

[sagetrac-chsorger]

comment:5

There is a subtle change in Mountain Lion which defaults to that system header files will use Objective C syntax constructs
although we are compiling just pure C. This explains the build error in mercurial above. Setting the compiler flag OS_OBJECT_USE_OBJC=0
reverts this change, so that setting (globally, as I guess there is only pure C in sage)

export CFLAGS="-DOS_OBJECT_USE_OBJC=0"

make

will compile mercurial.

A second build error occurred in gd-2.0.35.p7 where the header file fontconfig/fontconfig.h was not found. If you experience this, adding the include /opt/X11/include helps to compile gd but induces a version incompatibility between the fontconfig and freetype later when starting sage. A workaround for me was to compile fontconfig into the sage local tree using the correct free type libs and then continue with make.

A third error occurred during compilation of scipy, where Mountain Lion does not specify the correct accelerators. Here the following helps.

export CPPFLAGS="-D__ACCELERATE__"

make clean

make

After that sage 5.3 compiled (at least on my 10.8.1).

[jhpalmieri]

comment:6

After upgrading to the latest Mountain Lion (10.8.2) and the latest Xcode (4.5 (4G182)), many of these problems are going away for me. Note that I do not have any version of X11 installed on my machine, which is perhaps good (I don't have any problem with gd, fontconfig, etc.) but perhaps not (there is a small issue with matplotlib).

For me, I need chsorger's fix for scipy, and I have a new spkg ready – see the ticket description for the URL and see the attachments for the difference between the old spkg and the new one. There also seems to be a slightly different, but similar, fix in an unreleased upstream version: see http://projects.scipy.org/scipy/ticket/1710.

Also, when matplotlib is being installed, I get a window saying

To open "gs," you need to install X11.
Would you like to install X11 now?

It turns out that on this machine, /usr/local/bin/gs was a link to /usr/local/bin/gs-X11, and changing the link to point to /usr/local/bin/gs-noX11 fixed the problem.

Other than that, everything seems to work, at least on this machine.

[jhpalmieri]

patch for scipy spkg; for review only

[sagetrac-chsorger]

comment:7

Attachment: trac_13309-scipy.patch.gz

Indeed, upgrading to 10.8.2 and XCode 4.5 worked. No need to specify the compiler flag OS_OBJECT_USE_OBJC any more (apparently Apple silently corrected this with the update). Also jhpalmieri's patch for scipy works very well, so no need to specify by hand the accelerator flag later!

I still run into gd complaining about the missing fontconfig/fontconfig.h file. But on my machine I installed Quartz (Apple no longer installs X11 by default) so that this is obviously related to this. On the other hand, I need X11 for programs like surf/surfex I use. I also installed poppler through brew (which in turn installed fontconfig and free type) as I compile texmaker on my own which depends on poppler. In short, this problem about gd is probably very much related to my own config. Anyhow, I found a workaround for this, so that sage happily compiles now.

[sagetrac-yomcat]

comment:8

I have X11 installed (XQuartz 2.7.3 (xorg-server 1.12.4) to be exact), and both gd and matplotlib build without issues. The build hasn't finished yet, but it's looking good. I haven't done anything to X11. I just installed the one that Apple linked me to when I got that notice jhpalmieri got. So a standard install, including X11/XQuartz, causes no issues.

Oh, Sage 5.4-beta1 and XCode 4.5 on 10.8.2.

Update: Built fine, with no errors. The docs are building now.

[jhpalmieri]

comment:10

I've posted an experimental spkg for version 0.11.0 of scipy. This is experimental because I haven't tried it out very much. It builds on OS X 10.7, OS X 10.8, and on sage.math, but I haven't run any doctests yet.

[jhpalmieri]

patch for 0.11.0 scipy spkg

[jhpalmieri]

comment:11

Attachment: trac_13309-scipy.0.11.0.patch.gz

The new scipy spkg requires a few modifications to doctests, which I'll probably get to later today. Maybe I'll move all of that (i.e., upgrading scipy) to another ticket.

[jhpalmieri]

comment:12

See #13541 for version 0.11.0 of scipy. I wonder if we should just make that a dependency for this ticket, and ignore the scipy spkg here.

[saliola]

comment:14

New iMac, updated to 10.8.2, trying to install Sage.

First issue: there is a new OSX "feature" that puts the machine to sleep during long-running tasks like compiling software, big downloads, Time Machine backups, .... Any ideas on how to get around this? I don't really have the time or patience to wake the machine every 15 minutes. From what I've read, setting the computer to never go to sleep doesn't seem to help either.

[sagetrac-chsorger]

comment:15

Saliola, use the new command "caffeinate" (see man caffeinate).

[saliola]

comment:16

Replying to @sagetrac-chsorger:

Saliola, use the new command "caffeinate" (see man caffeinate).

Thanks! I guess we need to mention this in the installation guide. I'll put this in the "work issues" field for now so that we don't forget, and I'll continue with the review.

[saliola]

Work Issues: add caffeinate to installation guide

[saliola]

comment:17

iMac, 10.8.2. Works great.

Here is what I did:

1. install XCode from the App Store
2. install the XCode command line tools
3. download and extract sage-5.4.rc2.tar
4. switch to $SAGE_ROOT/spkg/standard directory and delete the scipy package (this step is not needed, I think)
5. download the scipy spkg from update scipy to 0.11.0 #13541 into $SAGE_ROOT/spkg/standard
6. caffeinate make

Notes:

5. I used the spkg from update scipy to 0.11.0 #13541 because it has a positive review and will likely be merged into version 5.5.
6. caffeinate prevents the machine from sleeping during the compilation: see here.

Is there anything else that needs to be checked before I set this to positive review?

[jhpalmieri]

comment:18

Two possible ideas, although these could go on new tickets instead: we could always use caffeinate on Mountain Lion, with a patch like this to the root repository:

diff --git a/spkg/install b/spkg/install
--- a/spkg/install
+++ b/spkg/install
@@ -481,10 +481,19 @@ echo "*** ALL ENVIRONMENT VARIABLES BEFO
 env | sort
 echo "***********************************************"
 
+# Use 'caffeinate' to prevent sleep if on OS X Mountain Lion.
+CAFFEINATE=''
+if [ `uname` = "Darwin" ]; then
+    DARWIN_VERSION=`uname -r | cut '-d.' -f1`
+    if [ $DARWIN_VERSION -eq 12 ]; then
+       CAFFEINATE=caffeinate
+    fi
+fi
+
 ###############################################################################
 # NOW do the actual build:
 ###############################################################################
-time $MAKE "$@"
+time $CAFFEINATE $MAKE "$@"
 if [ $? -ne 0 ]; then
     echo >&2 "Error building Sage."
     exit 1

We could also try to avoid potential issues with gs pointing to gs-X11 by creating a new link, for example with this possible change to the matplotlib spkg:

diff --git a/spkg-install b/spkg-install
--- a/spkg-install
+++ b/spkg-install
@@ -12,6 +12,18 @@ if [ "x$SAGE64" = xyes ]; then
    LDFLAGS="-m64 "; export LDFLAGS
 fi
 
+# If on OS X Mountain Lion: see if 'gs' points to
+# /usr/local/bin/gs-X11. If so, link instead to
+# /usr/local/bin/gs-noX11.
+if [ `uname` = "Darwin" ]; then
+    DARWIN_VERSION=`uname -r | cut '-d.' -f1`
+    GS=`command -v gs`
+    GSNOX11=`command -v gs-noX11`
+    if [ $DARWIN_VERSION -eq 12 ] && [ `readlink "$GS"` = "gs-X11" ] && [ -x "$GSNOX11" ]; then
+       ln -s "$GSNOX11" "$SAGE_LOCAL"/bin/gs
+    fi
+fi
+
 # Write a configuration file to src/setup.cfg
 python make-setup-config.py
 

I'm not sure about this second idea, though, because the user might actually have X11 installed, so we would want a more sophisticated check. The linbox configure script also calls gs, so we would need a similar change there.

[dimpase]

comment:19

Replying to @saliola:

iMac, 10.8.2. Works great.
Is there anything else that needs to be checked before I set this to positive review?

I'd also check that the package/patches continues to build on usual systems (say, Debian Linux).
Or you can just give it positive review as it is, assuming that the author was careful to check this.

[jhpalmieri]

comment:20

This ticket no longer contains any patches or packages. It just points out the scipy spkg from #13541 (which already has a positive review). So I'm not sure what we need to do to close this ticket. Get a buildbot running Mountain Lion? Close now as "works for me"? Add information and/or patches re caffeinate?

[dimpase]

comment:21

Replying to @jhpalmieri:

This ticket no longer contains any patches or packages. It just points out the scipy spkg from #13541 (which already has a positive review). So I'm not sure what we need to do to close this ticket. Get a buildbot running Mountain Lion? Close now as "works for me"? Add information and/or patches re caffeinate?

I'm giving it a positive review. I suppose it's better to open a separate ticket on the caffeinate issue. (I'm not doing it, as I don't have OSX 10.8 machine available, sorry.)

[jdemeyer]

Reviewer: Dmitrii Pasechnik

[jdemeyer]

Changed work issues from add caffeinate to installation guide to none

[jdemeyer]

comment:23

Paul-Olivier Dehaye is kindly providing an OS X 10.8 buildslave