Add a gcc package

[jdemeyer]

The aim is to add a GCC package (GNU compiler collection) to Sage with compilers for C, C++ and Fortran. We don't always build it, we would install it by default only on systems where this is needed. This would replace the fortran package.

spkg: http://boxen.math.washington.edu/home/jdemeyer/spkg/gcc-4.6.3.spkg

Apply:

1. attachment: 12369_gcc_root.patch and attachment: 12369_gcc_root_part2.patch to the SAGE_ROOT repository.
2. attachment: 12369_scripts_hgignore.patch to the SCRIPTS repository.
3. attachment: 12369_doc.patch to the Sage library.

See also:

1. When building GCC, build MPIR without the C++ interface #12782: When building GCC, build MPIR without the C++ interface (superseded by Upgrade MPIR to a more recent upstream release #11616).

Depends on #12479
Depends on #12602
Depends on #12608
Depends on #12609
Depends on #12647
Depends on #10492
Depends on #12367
Depends on #12368
Depends on #12405
Depends on #12570
Depends on #12574
Depends on #12423
Depends on #12425
Depends on #12456
Depends on #12363
Depends on #12223
Depends on #12515
Depends on #12519
Depends on #12548
Depends on #12562
Depends on #12629
Depends on #12638
Depends on #12714
Depends on #12647
Depends on #12739
Depends on #12112
Depends on #12631

CC: @vbraun

Component: packages: standard

Author: Jeroen Demeyer

Reviewer: Simon King

Merged: sage-5.0.beta13

Issue created by migration from https://trac.sagemath.org/ticket/12369

[jdemeyer]

Changed dependencies from #12366, #12367, #12368 to #12366, #12367, #12368, #10492

[jdemeyer]

Changed dependencies from #12366, #12367, #12368, #10492 to #11073, #12366, #12367, #12368, #10492, #12405

[kcrisman]

comment:7

# Install gcc if the system-provided one is older than gcc-4.4. 

But that is all Macs. This seems overkill to me; why not only do this on Lion?

Or would this eliminate having Xcode as a prerequisite to building Sage on OS X in general (even for those for which XCode is free, though sometimes annoying to get)? That might be a nice move. I assume this is a prebuilt gcc like our current Fortran package?

Could the message "installing GCC because..." occur several times here?

[vbraun]

comment:8

Updated spkg at http://www.stp.dias.ie/~vbraun/Sage/spkg/gcc-4.6.2.spkg

It turns out to be tricky to convince gcc to use lib64 for glibc but put its own libraries into lib. But I think its safe to just move the generated shared libraries to SAGE_LOCAL/lib. Also, I updated to the newest gcc, install in single-thread, and disabled multilib.

[jdemeyer]

comment:9

First of all: this spkg is all pretty-much proof-of-concept. I haven't even managed to install Sage from source with this spkg.

Replying to @kcrisman:

# Install gcc if the system-provided one is older than gcc-4.4. 

But that is all Macs. This seems overkill to me; why not only do this on Lion?

Maybe you are right, but my idea was to install this gcc spkg if the gcc we ship is more recent than the one the user has. I think many people would benefit from using a more recent compiler.

Or would this eliminate having Xcode as a prerequisite to building Sage on OS X in general (even for those for which XCode is free, though sometimes annoying to get)? That might be a nice move. I assume this is a prebuilt gcc like our current Fortran package?

No, this is a gcc from source. I certainly don't want to include gcc binaries for every platform, that would increase the size way too much.

Could the message "installing GCC because..." occur several times here?

Sure.

[kcrisman]

comment:10

First of all: this spkg is all pretty-much proof-of-concept. I haven't even managed to install Sage from source with this spkg.

Of course; I'm just asking early questions.

But that is all Macs. This seems overkill to me; why not only do this on Lion?

Maybe you are right, but my idea was to install this gcc spkg if the gcc we ship is more recent than the one the user has. I think many people would benefit from using a more recent compiler.

Possibly. I guess what I'm wondering is how one would build this gcc if one didn't have gcc... but this may betray my ignorance. Anyway, the vast majority of Mac users will not have a way to build this gcc if they need gcc until they download/install (from a disk which they've probably thrown out, as William pointed out on sage-devel) XCode anyway, in which case we might as well only worry about Lion anyway.

Or would this eliminate having Xcode as a prerequisite to building Sage on OS X in general (even for those for which XCode is free, though sometimes annoying to get)? That might be a nice move. I assume this is a prebuilt gcc like our current Fortran package?

No, this is a gcc from source. I certainly don't want to include gcc binaries for every platform, that would increase the size way too much.

That's what I figured, but just wanted to clarify.

Could the message "installing GCC because..." occur several times here?

Sure.

Ok. I suppose it doesn't really matter, but could be confusing/alarming.

[jdemeyer]

comment:11

Replying to @kcrisman:

Possibly. I guess what I'm wondering is how one would build this gcc if one didn't have gcc... but this may betray my ignorance.

My hope is that the non-gcc C compiler shipped with XCode 4 might be able to build GCC.

Anyway, the vast majority of Mac users will not have a way to build this gcc if they need gcc until they download/install

That "vast majority" of users will be happy with a binary distribution of Sage (which could actually contain GCC if I manage to make this spkg work).

[kcrisman]

comment:12

Replying to @jdemeyer:

Replying to @kcrisman:

Possibly. I guess what I'm wondering is how one would build this gcc if one didn't have gcc... but this may betray my ignorance.

My hope is that the non-gcc C compiler shipped with XCode 4 might be able to build GCC.

Interesting! Now I see the point of all this.

Anyway, the vast majority of Mac users will not have a way to build this gcc if they need gcc until they download/install

That "vast majority" of users will be happy with a binary distribution of Sage

Good point - until they need to change any code. But this

(which could actually contain GCC if I manage to make this spkg work).

Interesting, as I said. I wonder if this would work:

• Sage compiled with XCode 4, including compiling gcc
• Sage bdisted from this machine
• Sage downloaded in that bdist to a machine WITHOUT Xcode (or MacPorts, or any other compilers)
• gcc included with Sage allows full Sage building fun
  Good luck!

[jdemeyer]

Changed dependencies from #11073, #12366, #12367, #12368, #10492, #12405 to #11073, #12366, #12367, #12368, #10492, #12405, #12416

[jdemeyer]

Changed dependencies from #11073, #12366, #12367, #12368, #10492, #12405, #12416 to #11073, #12366, #12367, #12368, #10492, #12405, #12416, #12422

[jdemeyer]

Changed dependencies from #11073, #12366, #12367, #12368, #10492, #12405, #12416, #12422 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423

[jdemeyer]

Changed dependencies from #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423 to #11073, #10492, #12366, #12367, #12368, #12405, #12416, #12422, #12423, #12425

[ohanar]

comment:17

I haven't looked at the spkg yet (which is probably old anyway), but looking at the root patch that you wrote, I have a few of requests:

1. I agree with Karl. Do not install gcc if gcc is new enough (>=4.0.1), this will cause every mac developer to have to build gcc upon building sage (significantly increasing the build time). I'm not a mac user, but I know that if I had to do this on my system, I would be annoyed.

2. Assume the user knows what they are doing it CC is set (like you do for CXX and SAGE_FORTRAN). While not a very common use case, it would be helpful to not have to go hack the installation file if you were trying to port sage to another compiler (like I am right now).

3. (You might be doing this already) If fortran is the only language that needs to be built (such as on macs), please do not build gcc and g++.

And a couple comments:

1. You assume that CC supports -dumpversion when determining if you should upgrade gcc. I'm fairly certain this is not an option supported by all compilers. Also, the value you would get would be meaningless anyway, since it wouldn't be a gcc version anyway (well, unless it is clang).

2. For OBJC and OBJCXX, I would point to /usr/bin/cc and /usr/bin/cpp instead. If/when apple stops distributing gcc/g++, then these will continue to point to valid compilers.

In response to Karl about a replacement for Xcode (or tools on any platform). This does not provide one, even in binary distributions of sage since we need binutils and make, neither of which this would provide (and neither of which are on OSX without Xcode).

[kcrisman]

comment:18

In response to Karl about a replacement for Xcode (or tools on any platform). This does not provide one, even in binary distributions of sage since we need binutils and make, neither of which this would provide (and neither of which are on OSX without Xcode).

Right, that makes sense. I didn't know exactly what comes from Xcode and what is just pre-included.

[jhpalmieri]

comment:19

I've been wondering for a while if we should change Sage to use autoconf. (When I say "we", I don't actually mean me, since I don't know anything about writing configure scripts, nor do I want to learn.) This could replace spkg-install (especially with the modifications at #10492 that create a Makefile on the fly). Does this make sense as an enhancement? Should I poll sage-devel about it, and then open a ticket if the response is positive?

[ohanar]

comment:20

Replying to @jhpalmieri:

I've been wondering for a while if we should change Sage to use autoconf. (When I say "we", I don't actually mean me, since I don't know anything about writing configure scripts, nor do I want to learn.) This could replace spkg-install (especially with the modifications at #10492 that create a Makefile on the fly). Does this make sense as an enhancement?

I don't know how much additional functionality it would add, but I would argue for it since it is more unix user friendly than our current custom setup (SAGE_ATLAS_LIB is not as user friendly as a fairly standard --with-system-atlas argument for configure). My guess is that it would be a fairly large endeavour to implement a first version of such a change (I don't know autoconf, so I'm just guessing here), although it would hopefully be easier to maintain than spkg-install.

Should I poll sage-devel about it, and then open a ticket if the response is positive?

I don't think it could hurt. Also, that would be a better place to discuss it than on this ticket.

[ohanar]

comment:21

Replying to @ohanar:

2. Assume the user knows what they are doing it CC is set (like you do for CXX and SAGE_FORTRAN). While not a very common use case, it would be helpful to not have to go hack the installation file if you were trying to port sage to another compiler (like I am right now).

Actually, thinking a bit more about it, I think it would be better if we required SAGE_PORT to be set if CC and/or CXX is set.

[jdemeyer]

comment:22

Replying to @ohanar:

I haven't looked at the spkg yet (which is probably old anyway), but looking at the root patch that you wrote, I have a few of requests:

1. I agree with Karl. Do not install gcc if gcc is new enough (>=4.0.1), this will cause every mac developer to have to build gcc upon building sage (significantly increasing the build time). I'm not a mac user, but I know that if I had to do this on my system, I would be annoyed.

Well, we do need to build Fortran on a Mac (I'm assuming that the fortran spkg will go away). And Sage seems to require that the gcc and gfortran versions are the same. It is true that the build time of gcc is significant:

real    21m4.603s
user    15m56.183s
sys     4m26.492s
Successfully installed gcc-4.4.6

(on bsd.math, single-threaded), but it's not that bad. I think the "plus" of having a working Sage compiled with optimizations (as opposed to using XCode 4 with many packages compiled with -O0) outweighs the "minus" of 25 minutes longer build time.

2. Assume the user knows what they are doing it CC is set (like you do for CXX and SAGE_FORTRAN). While not a very common use case, it would be helpful to not have to go hack the installation file if you were trying to port sage to another compiler (like I am right now).

Well, I was thinking anyway of adding an environment variable (or future ./configure option) to either force installation of gcc, or disable installation of gcc. Don't forget that GCC has a few dependencies, and one might set CC to build those dependencies, but still want GCC to be installed. Anyway, I don't care much about this.

3. (You might be doing this already) If fortran is the only language that needs to be built (such as on macs), please do not build gcc and g++.

I thought gcc and gfortran needed to be the same versions?

And a couple comments:

1. You assume that CC supports -dumpversion when determining if you should upgrade gcc. I'm fairly certain this is not an option supported by all compilers. Also, the value you would get would be meaningless anyway, since it wouldn't be a gcc version anyway (well, unless it is clang).

Agreed.

2. For OBJC and OBJCXX, I would point to /usr/bin/cc and /usr/bin/cpp instead. If/when apple stops distributing gcc/g++, then these will continue to point to valid compilers.

Agreed (modulo the fact that you mean c++ instead of the preprocessor cpp).

In response to Karl about a replacement for Xcode (or tools on any platform). This does not provide one, even in binary distributions of sage since we need binutils and make, neither of which this would provide (and neither of which are on OSX without Xcode).

True...

[ohanar]

comment:23

Replying to @jdemeyer:

Well, we do need to build Fortran on a Mac (I'm assuming that the fortran spkg will go away).

Agreed, which is why I would like only fortran to be built if it is the only language needed.

I think the "plus" of having a working Sage compiled with optimizations (as opposed to using XCode 4 with many packages compiled with -O0) outweighs the "minus" of 25 minutes longer build time.

I think this is something that should be brought to the listserv. Once I get a clang port up and running (which shouldn't be too bad on osx, since atlas looks like it will be the only package that is going to be hell to port), this can be resolved by changing the default compiler on osx to clang.

3. (You might be doing this already) If fortran is the only language that needs to be built (such as on macs), please do not build gcc and g++.

I thought gcc and gfortran needed to be the same versions?

I'm not sure about this, I'm using a mixture of clang (which dumps its version as 4.2.1) and gfortran-4.5.3, and occasionally gcc-4.2.4 and g++-4.2.4 to build sage, and currently the entire test suite is passing.

[jdemeyer]

comment:24

Replying to @ohanar:

Agreed, which is why I would like only fortran to be built if it is the only language needed.

Is build time your only concern here? Because if you need to build Fortran, you most likely need to build a stage 1 C compiler anyway.

[jdemeyer]

comment:25

I would like to add that my goal is beyond OS X 10.7 but more generally all systems with bad compilers. That's also why I went for the more stable gcc-4.4.6 as opposed to the more recent (and therefore buggy) gcc-4.6.2.

[jdemeyer]

Merged: sage-5.0.beta13

[nexttime]

comment:239

Replying to @jdemeyer:

Replying to @simon-king-jena:

Or perhaps better exempt the sage* spkgs from re-installation, since my Sage library is patched?

I honestly have no idea what would happen if you re-install the Sage library over a patched version.

That's a noop. Mercurial notices that your current Sage library is based on the one you're trying to reinstall, so won't destroy your patched one. ("Reinstallation" of the Sage library [spkg] almost always happens if you use env SAGE_UPGRADING=yes make after updating/reinstalling some standard spkg.)

[nexttime]

comment:240

Replying to @jdemeyer:

Changed that code to

# If spkg/bin/sage-env doesn't exist, we are surely upgrading (sage-env
# was moved to spkg/bin in sage-5.0).  Manually set SAGE_UPGRADING=yes,
# as old versions of sage-upgrade didn't do this.
# Also avoid RM being set to "rm":
#   https://github.com/sagemath/sage-prod/issues/3537
if [ ! -f "$SAGE_ROOT/spkg/bin/sage-env" ]; then
    SAGE_UPGRADING=yes
    if [ "$RM" = rm ]; then
        unset RM
    fi
fi

This should remove the need to do "unset RM" in various spkg-install files. This change is unrelated to the ticket, but I put it here because I was patching spkg/install anyway and I didn't want to complicate ticket dependencies.

I'd just unset RM (or actually check whether $RM non-existent-file raises an error), since RM could be defined to rm -i, rm -v, /bin/rm or whatever, and not necessarily intentionally set by the user.

[jdemeyer]

comment:241

The new code is

if [ "$RM" = rm ]; then
    export RM="rm -f"
fi

[nexttime]

comment:243

I get GCC build errors on Ubuntu 12.04 (still beta).

Apart from that Ubuntu's [broken] GCC 4.6.3 is unable to build MPFR with -O3 (which I already reported; bug fix is on the way), the GCC from the spkg doesn't find crti.o. After creating a couple of links in /usr/lib/, the third attempt fails in compiling graphite.c, because of a missing ppl_c.h. (Apparently GCC found some parts of CLooG, or mistakenly assumed it was installed; I didn't have it installed, at least not below /usr/.)

More to come, either here or on sage-release.