Experimental support for non-Apple GCCs

[mistydemeo]

This is really preliminary. I didn't change much except a few lines in superenv.

This adds support for building software using a Homebrew-installed, non-Apple GCC. I've tested so far using gcc48 from homebrew/versions, and was able to successfully build gource on Snow Leopard (which can't be built with any Apple-provided compilers on Xcode 3.2.6).

There are a few limitations:

• Only superenv is supported. Not sure if it's worth adding support to stdenv.
• No --use-foo switch is supported, only the HOMEBREW_CC environment variable. For instance, HOMEBREW_CC=gcc-4.8.

To test on < Lion, create a symlink in ENV to enable superenv. (It seems to be working reliably now.)

[samueljohn]

would this (wrongly) capture llvm-gcc-1.2, too?

[mistydemeo]

That's not a supported value for HOMEBREW_CC, which is already sanitized before cc is called.

[samueljohn]

alright.

[samueljohn]

In principle I like this but we should still push clang. I think we had a discussion on the mailing list about this and I remember there was a tendency to do something like suggested here.

[mistydemeo]

Not arguing otherwise! However:

• Some people need newer GCC for whatever reason, and we should support that. We've had numerous requests, and yeah, that mailing list discussion was the inspiration here.
• We already provide formulae for newer GCC, which is obviously of more limited use if we make people then build software outside Homebrew just to make use of the compilers they just built with Homebrew.
• A usable clang just straight up isn't available on Snow Leopard and earlier. I'm not yet ready to throw these users to the wolves where newer C++ stuff is concerned. We may also be able to build newer Apple or non-Apple clang for these platforms, which we could do in addition to supporting newer GCCs.

[samueljohn]

I wasn't arguing against this PR. I am +1 on this.

[cooljeanius]

I am +1 on this.

Me too.

[mistydemeo]

I'll keep this rebased on master frequently. If you want to try it out:

git remote add mistydemeo https://github.com/mistydemeo/homebrew.git
git fetch mistydemeo
git checkout superenv_non_apple_compiler

You'll want to pull periodically for updates.

Any users interested in testing and seeing what does/doesn't work, feedback would be appreciated!

[jacknagel]

Just a note, I'd like to see integration with fails_with before this lands in core.

[jacknagel]

I wrote up some DSL ideas a while back: https://gist.github.com/jacknagel/5149739

[mistydemeo]

Oh, nice! I like that DSL.

Definitely shouldn't merge until the appropriate mechanisms are in place (and it's been tested more).

[MikeMcQuaid]

Looks good to me. Agreed with @jacknagel that fails_with integration wants in first. I also think given that 10.9 is on the horizon, LLVM-GCC is being dropped and we still have a bunch of things that fail with Clang we should just pull GCC keg-only into core and perhaps even never use LLVM-GCC regardless of if it's available.

[samueljohn]

Agree with Mike here but remember there is the bug that formulae removed from taps and put into core break brew update.

[mistydemeo]

we should just pull GCC keg-only into core

Not sure this is a good idea, considering how many versions there are. Directing people to homebrew/versions might be the best bet - especially if stuff that requires GCC goes into a separate tap.

never use LLVM-GCC regardless of if it's available.

Agreed, we shouldn't use llvm-gcc anymore.

[dhess]

The C++ ABI is complex enough that there will probably be occasional compatibility issues for some C++ libraries compiled with one compiler and linked from another, e.g., Boost is compiled with Clang for one project but needed by another that requires GCC. The Clang libc++ page (http://libcxx.llvm.org) states that:

ABI compatibility with gcc's libstdc++ for some low-level features such as exception objects, rtti and memory allocation.

Even different versions of GCC occasionally break compatibility (see http://gcc.gnu.org/gcc-4.7/changes.html, for example).

Whatever the solution that's adopted for supporting GCC(s), it should include a mechanism for compiling and linking against compiler-specific dependencies -- separate taps/kegs/whatever.

[sjackman]

+1

[mistydemeo]

This is just waiting on a fails_with` implementation, which I've been too busy to write.

[mistydemeo]

fails_with implementation in, needs review. A few notes:

• Non-Apple GCCs are returned internally as strings, rather than symbols like Apple compilers.
• I've swapped the compiler priority for llvm-gcc and gcc-4.2, but can drop llvm-gcc from the queue entirely.
• Non-Apple GCCs slot in priority below clang and gcc-4.2, above llvm-gcc and gcc-4.0. Right now they all share the same priority, which isn't really ideal.
• The fails_with code needs better tests. @jacknagel - I was checking the Mocha docs and didn't find anything at a glance, is there a way to return specific values for a stubbed method given a particular argument?

[MikeMcQuaid]

Sounds good to me.

[mistydemeo]

Updated with working tests. (Excuse the one paragraph of fails_with code that sneaked into the first commit.) Should be working well now, review/testing is appreciated.

I'll remove llvm-gcc from the compiler selection altogether once this is merged.

[mistydemeo]

Fixed GCC priority, which is now based on version number, with newest being preferred. e.g. 4.3 gets a priority of 1.3, 4.8 gets a priority of 1.8.

[manphiz]

As much as I'd love to see GCC improvement on Darwin, it is supposed to cause problem as FSF libstdc++ uses a different ABI than the system shipped libstdc++. As the long discussion in macports (http://trac.macports.org/ticket/35770) pointed out, the ultimate solution is to port libstdc++ to use libc++abi on Darwin, which is also used by Clang. However there is an obstacle in license compatibility as libstdc++ is using GPLv3 while libc++abi is using MIT and UIUC dual licenses (which are all BSD-style licenses).

Besides the ABI issue, there is another problem with linking to multiple copies of libstdc++, as in homebrew/versions, multiple versions of GCC ship multiple versions of libstdc++. So if a library is built using gcc47, and then linked to a program built using gcc48, the program will link to two copies of libstdc++ and may end up in problems. Macport solves this problem by only installing one copy of libstdc++, which is either from the latest release or trunk, and let all GCC versions link to that. This generally works unless when ABI incompatible switches are used, such as "--enable-libstdcxx-time=yes". Besides this doesn't fix the first issue above, IMHO I'd suggest one stick to one version of GCC.

Maybe we should also look into providing a homebrew Clang, which supports libc++abi and compatible with system ABI.

[mistydemeo]

This is a tricky situation because there are disadvantages with every option we could go with. A disadvantage to Macports's solution is that platform-specific build errors in the most recent GCC version will wipe out the ability to use any homebrewed GCC on that platform. This isn't a hypothetical and is an unsolved issue over there right now, and is most likely to hurt the platforms that need homebrewed GCC the most - Snow Leopard and older.

An alternate solution would be to record the GCC family used to build a piece of software in its install receipt, and only allow linking dependencies against things built with a compatible stdlib. Might be the simplest solution, and I would imagine it's not too likely that a user will want to mix and match, say, GCC 4.3 and 4.8 in their homebrew installs.

[MikeMcQuaid]

Yeh, the latter seems like a good idea. Another one is just to only allow one version of GCC to be installed at once.

[GuillaumeDIDIER]

From someone who hacked his personal homebrew so that it uses usr/local/bin/clang on Snow Leopard, having homebrew deal with brewed compilers would be quite useful.
Good luck.

[mistydemeo]

Sorry about the delays, I've been too busy with the move to finish up the stdlib tracking. I'll try to get to it sometime.

[MikeMcQuaid]

As an aside: did we allow setting the compiler from the environment like this previously? It seems a bit dangerous...

[samueljohn]

@MikeMcQuaid we have HOMEBREW_USE_LLVM=1 and similar (but I don't like them and would love to deprecate them in favor of HOMEBREW_CC). Also setting HOMEBREW_CC should IMO always print a warning as Misty implemented for gcc here.
I very much prefer the args: --use-clang etc. (though there again we have that misleading --use-llvm but still args seems preferable)

[MikeMcQuaid]

Yeh, I'd rather the args were used too (and aren't sticky). I'm just forseeing blog posts when Mavericks is released telling people to set HOMEBREW_CC globally to GCC or LLVM.

[mistydemeo]

Theoretically this is the "new" way to do things since superenv landed last year, but yeah, I hadn't considered that. Honestly kind of surprised we haven't seen that yet - maybe just because they're poorly documented.

As an alternative, how about not supporting GNU GCC by setting HOMEBREW_CC, and instead using the (non-sticky) option: --use-gnu-gcc=foo I really don't want to overload --use-gcc, which is vague enough as is.

[mistydemeo]

(but I don't like them and would love to deprecate them in favor of HOMEBREW_CC)

They're already deprecated, and using them in superenv will print a warning.

BTW, @samueljohn - fixed the bug you encountered. Accidentally forgot to fix a variable reference when extracting the GNU GCC warning into the shared env extension.

[adamv]

I think we want to move away from --use-[gcc/llvm/clang] altogether, just --cc=<something>.

[jacknagel]

FWIW, I've been setting HOMEBREW_CC=gcc-4.2 on Snow Leopard/Xcode 4.2 (I think we normally choose llvm-gcc, but I was seeing some compilation issues).

[samueljohn]

I think we want to move away from --use-[gcc/llvm/clang] altogether, just --cc=.

Yes --cc="gcc" sounds best, and I am fine with doing that in another PR. Can we already handle command line args with values?

@mistydemeo thanks. Fantastic work here. A lot of people will like this.

[mistydemeo]

@jacknagel I think our next step after this should be to simply remove llvm-gcc from the list of available compilers. There will be enough choices now it shouldn't be necessary, even on platforms where it's the default / an available fallback.

[mistydemeo]

@adamv How about: https://github.com/mistydemeo/homebrew/compare/cc Can push that first, then switch over the GNU GCC to use that and not support HOMEBREW_CC.

When we do that, I'd like to remove the HOMEBREW_USE_LLVM/etc. environment variables, then unify the compiler selection logic.

[mistydemeo]

Updated the branch a bit - added a second commit which unifies the compiler selection logic between superenv and stdenv, and removes the deprecared HOMEBREW_USE_LLVM/etc. variables.

The --cc= option accepts a restricted set of compilers based on their proper executable names, with no aliases: gcc-4.2 not gcc, llvm-gcc not llvm, etc. It's supported by both stdenv and superenv.

[mistydemeo]

If this looks good, I'll push the cc branch, then modify the GNU GCC branch to support --cc= as the only syntax for selecting a non-Apple compiler instead of HOMEBREW_CC=.

[mistydemeo]

Rebased on master with some cleanup to the --cc= code for GNU GCC selection. Seems to work well now!

[samueljohn]

Ok, don't hold this back because of me. Looks great.

[mistydemeo]

Just a heads up that I'm going to merge this this evening.

[mistydemeo]

Merged!

@manphiz Can you take a look at the stub wiki page for C++ stdlibs and make any improvements it needs?

@MikeMcQuaid Can you take a look at the bottle check in 500d40d? Not sure how we want to handle C++ stdlibs in bottles going forward.

[seanjensengrey]

Love the change, couple things:

I am seeing a compilation failure (segfault) in gnu-prolog unless I compile with --env=std

Fatal Error: Segmentation Violation (bad address: 0x10041aada)
compilation failed
make[1]: *** [fd2c.o] Error 1
make: *** [all] Error 1

• Is there a way to just turn off this feature?
• It would be nice if the banner "Warning: Experimental support .." told me the compiler version is picked up
• I cannot --cc=gcc to switch back to my platform gcc, why is that? I can switch to other gcc versions

[mistydemeo]

I am seeing a compilation failure (segfault) in gnu-prolog unless I compile with --env=std

Can you file a separate issue for this, please?

It would be nice if the banner "Warning: Experimental support .." told me the compiler version is picked up

That's a good idea.

I cannot --cc=gcc to switch back to my platform gcc, why is that? I can switch to other gcc versions

--cc= requires a specific compiler version. You can specify --cc=gcc-4.2.

[ohhorob]

I had to use gcc-4.7 to compile protobuf240a on Mavericks.

First step was installing gcc-4.7 (see http://stackoverflow.com/a/19719840/178831)
And then protobuf240a with the cc option:

$ export HOMEBREW_CC=gcc-4.7; export CXX=g++-4.7; ~/homebrew/bin/brew install -v --cc=gcc-4.7 protobuf240a

This was to get the CDH4.5.0 hadoop distro maven target 'idea:idea' to build.. so I had to get the protobuf240a/bin/protoc in the path, and try the maven goal again.

$ mvn idea:idea