Skip to content
This repository has been archived by the owner. It is now read-only.

Experimental support for non-Apple GCCs #20251

Closed

Conversation

@mistydemeo
Copy link
Member

@mistydemeo mistydemeo commented Jun 4, 2013

This is really preliminary. I didn't change much except a few lines in superenv.

This adds support for building software using a Homebrew-installed, non-Apple GCC. I've tested so far using gcc48 from homebrew/versions, and was able to successfully build gource on Snow Leopard (which can't be built with any Apple-provided compilers on Xcode 3.2.6).

There are a few limitations:

  • Only superenv is supported. Not sure if it's worth adding support to stdenv.
  • No --use-foo switch is supported, only the HOMEBREW_CC environment variable. For instance, HOMEBREW_CC=gcc-4.8.

To test on < Lion, create a symlink in ENV to enable superenv. (It seems to be working reliably now.)

'g++-4.2'
when /gcc-4.0/
'g++-4.0'
when /gcc(-\d\.\d)?$/

This comment has been minimized.

@samueljohn

samueljohn Jun 4, 2013
Contributor

would this (wrongly) capture llvm-gcc-1.2, too?

This comment has been minimized.

@mistydemeo

mistydemeo Jun 4, 2013
Author Member

That's not a supported value for HOMEBREW_CC, which is already sanitized before cc is called.

This comment has been minimized.

@samueljohn

samueljohn Jun 4, 2013
Contributor

alright.

@samueljohn
Copy link
Contributor

@samueljohn samueljohn commented Jun 4, 2013

In principle I like this but we should still push clang. I think we had a discussion on the mailing list about this and I remember there was a tendency to do something like suggested here.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 4, 2013

Not arguing otherwise! However:

  • Some people need newer GCC for whatever reason, and we should support that. We've had numerous requests, and yeah, that mailing list discussion was the inspiration here.
  • We already provide formulae for newer GCC, which is obviously of more limited use if we make people then build software outside Homebrew just to make use of the compilers they just built with Homebrew.
  • A usable clang just straight up isn't available on Snow Leopard and earlier. I'm not yet ready to throw these users to the wolves where newer C++ stuff is concerned. We may also be able to build newer Apple or non-Apple clang for these platforms, which we could do in addition to supporting newer GCCs.
@samueljohn
Copy link
Contributor

@samueljohn samueljohn commented Jun 4, 2013

I wasn't arguing against this PR. I am +1 on this.

@cooljeanius
Copy link

@cooljeanius cooljeanius commented Jun 6, 2013

I am +1 on this.

Me too.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 6, 2013

I'll keep this rebased on master frequently. If you want to try it out:

git remote add mistydemeo https://github.com/mistydemeo/homebrew.git
git fetch mistydemeo
git checkout superenv_non_apple_compiler

You'll want to pull periodically for updates.

Any users interested in testing and seeing what does/doesn't work, feedback would be appreciated!

@jacknagel
Copy link
Contributor

@jacknagel jacknagel commented Jun 11, 2013

Just a note, I'd like to see integration with fails_with before this lands in core.

@jacknagel
Copy link
Contributor

@jacknagel jacknagel commented Jun 11, 2013

I wrote up some DSL ideas a while back: https://gist.github.com/jacknagel/5149739

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 11, 2013

Oh, nice! I like that DSL.

Definitely shouldn't merge until the appropriate mechanisms are in place (and it's been tested more).

@MikeMcQuaid
Copy link
Member

@MikeMcQuaid MikeMcQuaid commented Jun 14, 2013

Looks good to me. Agreed with @jacknagel that fails_with integration wants in first. I also think given that 10.9 is on the horizon, LLVM-GCC is being dropped and we still have a bunch of things that fail with Clang we should just pull GCC keg-only into core and perhaps even never use LLVM-GCC regardless of if it's available.

@samueljohn
Copy link
Contributor

@samueljohn samueljohn commented Jun 14, 2013

Agree with Mike here but remember there is the bug that formulae removed from taps and put into core break brew update.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 17, 2013

we should just pull GCC keg-only into core

Not sure this is a good idea, considering how many versions there are. Directing people to homebrew/versions might be the best bet - especially if stuff that requires GCC goes into a separate tap.

never use LLVM-GCC regardless of if it's available.

Agreed, we shouldn't use llvm-gcc anymore.

@dhess
Copy link
Contributor

@dhess dhess commented Jun 17, 2013

The C++ ABI is complex enough that there will probably be occasional compatibility issues for some C++ libraries compiled with one compiler and linked from another, e.g., Boost is compiled with Clang for one project but needed by another that requires GCC. The Clang libc++ page (http://libcxx.llvm.org) states that:

ABI compatibility with gcc's libstdc++ for some low-level features such as exception objects, rtti and memory allocation.

Even different versions of GCC occasionally break compatibility (see http://gcc.gnu.org/gcc-4.7/changes.html, for example).

Whatever the solution that's adopted for supporting GCC(s), it should include a mechanism for compiling and linking against compiler-specific dependencies -- separate taps/kegs/whatever.

@sjackman
Copy link
Member

@sjackman sjackman commented Jun 26, 2013

+1

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 26, 2013

This is just waiting on a fails_with` implementation, which I've been too busy to write.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 29, 2013

fails_with implementation in, needs review. A few notes:

  • Non-Apple GCCs are returned internally as strings, rather than symbols like Apple compilers.
  • I've swapped the compiler priority for llvm-gcc and gcc-4.2, but can drop llvm-gcc from the queue entirely.
  • Non-Apple GCCs slot in priority below clang and gcc-4.2, above llvm-gcc and gcc-4.0. Right now they all share the same priority, which isn't really ideal.
  • The fails_with code needs better tests. @jacknagel - I was checking the Mocha docs and didn't find anything at a glance, is there a way to return specific values for a stubbed method given a particular argument?
@MikeMcQuaid
Copy link
Member

@MikeMcQuaid MikeMcQuaid commented Jun 29, 2013

Sounds good to me.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 29, 2013

Updated with working tests. (Excuse the one paragraph of fails_with code that sneaked into the first commit.) Should be working well now, review/testing is appreciated.

I'll remove llvm-gcc from the compiler selection altogether once this is merged.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jun 30, 2013

Fixed GCC priority, which is now based on version number, with newest being preferred. e.g. 4.3 gets a priority of 1.3, 4.8 gets a priority of 1.8.

@mistydemeo mistydemeo mentioned this pull request Jul 2, 2013
@manphiz
Copy link
Contributor

@manphiz manphiz commented Jul 7, 2013

As much as I'd love to see GCC improvement on Darwin, it is supposed to cause problem as FSF libstdc++ uses a different ABI than the system shipped libstdc++. As the long discussion in macports (http://trac.macports.org/ticket/35770) pointed out, the ultimate solution is to port libstdc++ to use libc++abi on Darwin, which is also used by Clang. However there is an obstacle in license compatibility as libstdc++ is using GPLv3 while libc++abi is using MIT and UIUC dual licenses (which are all BSD-style licenses).

Besides the ABI issue, there is another problem with linking to multiple copies of libstdc++, as in homebrew/versions, multiple versions of GCC ship multiple versions of libstdc++. So if a library is built using gcc47, and then linked to a program built using gcc48, the program will link to two copies of libstdc++ and may end up in problems. Macport solves this problem by only installing one copy of libstdc++, which is either from the latest release or trunk, and let all GCC versions link to that. This generally works unless when ABI incompatible switches are used, such as "--enable-libstdcxx-time=yes". Besides this doesn't fix the first issue above, IMHO I'd suggest one stick to one version of GCC.

Maybe we should also look into providing a homebrew Clang, which supports libc++abi and compatible with system ABI.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jul 9, 2013

This is a tricky situation because there are disadvantages with every option we could go with. A disadvantage to Macports's solution is that platform-specific build errors in the most recent GCC version will wipe out the ability to use any homebrewed GCC on that platform. This isn't a hypothetical and is an unsolved issue over there right now, and is most likely to hurt the platforms that need homebrewed GCC the most - Snow Leopard and older.

An alternate solution would be to record the GCC family used to build a piece of software in its install receipt, and only allow linking dependencies against things built with a compatible stdlib. Might be the simplest solution, and I would imagine it's not too likely that a user will want to mix and match, say, GCC 4.3 and 4.8 in their homebrew installs.

@MikeMcQuaid
Copy link
Member

@MikeMcQuaid MikeMcQuaid commented Jul 9, 2013

Yeh, the latter seems like a good idea. Another one is just to only allow one version of GCC to be installed at once.

@GuillaumeDIDIER
Copy link
Contributor

@GuillaumeDIDIER GuillaumeDIDIER commented Jul 9, 2013

From someone who hacked his personal homebrew so that it uses usr/local/bin/clang on Snow Leopard, having homebrew deal with brewed compilers would be quite useful.
Good luck.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Jul 17, 2013

Sorry about the delays, I've been too busy with the move to finish up the stdlib tracking. I'll try to get to it sometime.

@MikeMcQuaid
Copy link
Member

@MikeMcQuaid MikeMcQuaid commented Aug 23, 2013

As an aside: did we allow setting the compiler from the environment like this previously? It seems a bit dangerous...

@samueljohn
Copy link
Contributor

@samueljohn samueljohn commented Aug 23, 2013

@MikeMcQuaid we have HOMEBREW_USE_LLVM=1 and similar (but I don't like them and would love to deprecate them in favor of HOMEBREW_CC). Also setting HOMEBREW_CC should IMO always print a warning as Misty implemented for gcc here.
I very much prefer the args: --use-clang etc. (though there again we have that misleading --use-llvm but still args seems preferable)

@MikeMcQuaid
Copy link
Member

@MikeMcQuaid MikeMcQuaid commented Aug 23, 2013

Yeh, I'd rather the args were used too (and aren't sticky). I'm just forseeing blog posts when Mavericks is released telling people to set HOMEBREW_CC globally to GCC or LLVM.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 23, 2013

Theoretically this is the "new" way to do things since superenv landed last year, but yeah, I hadn't considered that. Honestly kind of surprised we haven't seen that yet - maybe just because they're poorly documented.

As an alternative, how about not supporting GNU GCC by setting HOMEBREW_CC, and instead using the (non-sticky) option: --use-gnu-gcc=foo I really don't want to overload --use-gcc, which is vague enough as is.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 23, 2013

(but I don't like them and would love to deprecate them in favor of HOMEBREW_CC)

They're already deprecated, and using them in superenv will print a warning.

BTW, @samueljohn - fixed the bug you encountered. Accidentally forgot to fix a variable reference when extracting the GNU GCC warning into the shared env extension.

@adamv
Copy link
Contributor

@adamv adamv commented Aug 23, 2013

I think we want to move away from --use-[gcc/llvm/clang] altogether, just --cc=<something>.

@jacknagel
Copy link
Contributor

@jacknagel jacknagel commented Aug 23, 2013

FWIW, I've been setting HOMEBREW_CC=gcc-4.2 on Snow Leopard/Xcode 4.2 (I think we normally choose llvm-gcc, but I was seeing some compilation issues).

@samueljohn
Copy link
Contributor

@samueljohn samueljohn commented Aug 23, 2013

I think we want to move away from --use-[gcc/llvm/clang] altogether, just --cc=.

Yes --cc="gcc" sounds best, and I am fine with doing that in another PR. Can we already handle command line args with values?

@mistydemeo thanks. Fantastic work here. A lot of people will like this.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 23, 2013

@jacknagel I think our next step after this should be to simply remove llvm-gcc from the list of available compilers. There will be enough choices now it shouldn't be necessary, even on platforms where it's the default / an available fallback.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 24, 2013

@adamv How about: https://github.com/mistydemeo/homebrew/compare/cc Can push that first, then switch over the GNU GCC to use that and not support HOMEBREW_CC.

When we do that, I'd like to remove the HOMEBREW_USE_LLVM/etc. environment variables, then unify the compiler selection logic.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 24, 2013

Updated the branch a bit - added a second commit which unifies the compiler selection logic between superenv and stdenv, and removes the deprecared HOMEBREW_USE_LLVM/etc. variables.

The --cc= option accepts a restricted set of compilers based on their proper executable names, with no aliases: gcc-4.2 not gcc, llvm-gcc not llvm, etc. It's supported by both stdenv and superenv.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 24, 2013

If this looks good, I'll push the cc branch, then modify the GNU GCC branch to support --cc= as the only syntax for selecting a non-Apple compiler instead of HOMEBREW_CC=.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 30, 2013

Rebased on master with some cleanup to the --cc= code for GNU GCC selection. Seems to work well now!

@samueljohn
Copy link
Contributor

@samueljohn samueljohn commented Aug 30, 2013

Ok, don't hold this back because of me. Looks great.

@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Aug 31, 2013

Just a heads up that I'm going to merge this this evening.

mistydemeo added 6 commits Jun 4, 2013
This adds support for non-Apple GCC compilers in the fails_with code.
A fails_with block for a non-Apple compiler looks like:

fails_with :gcc => '4.8.1' do
  cause 'Foo'
end

Non-Apple compilers don't have build numbers, so compiler failures are
based on version strings instead.

Internally non-Apple compilers can be distinguished because they are
passed around as strings instead of symbols.

In addition, this alters the priority list for compilers, with the
following changes:

* Apple GCC 4.2 and LLVM-GCC swap positions, with GCC now taking
  priority. (Maybe LLVM-GCC should just go away.)
* Non-Apple GCC compilers are ranked below GCC 4.2 but above LLVM-GCC
  and Apple GCC 4.0.
The parent process doesn't have access to the selected compiler, which
will be important in the next commit.

Fortunately the child process already has a filtered and massaged
ARGV, so it has enough information to build the tab itself.
There are subtle incompatibilities between Apple's libstdc++ and the
libstdc++ used by the various GNU GCC formulae. In addition, we'll
likely also be supporting libc++ in the future, and that's also
incompatible with the other stdlibs.

Tracking it in the tab lets us make sure that dependencies are all
built against the same stdlib to avoid subtle breakage.
There are now a few possible C++ standard libraries a given build could
be using, with subtle incompatibilities and possibility of breakage
when mixed. This makes sure that the dependency chain was compiled in
a compatible manner.

Fortunately all of the Apple compilers use the same libstdc++, and we
don't yet support building with libc++, so this will primarily only
nag users trying to use GNU gcc who already have software installed
with Apple compilers.

Future TODOs:

* Add general support for building with libc++ (compatibility checking
  already handled here)
* Possibly track formulae which actually build C++ bindings, so that
  users aren't bothered by spurious nagging re: interpreted languages,
  pure-C software, etc.
@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Sep 1, 2013

Merged!

@manphiz Can you take a look at the stub wiki page for C++ stdlibs and make any improvements it needs?

@MikeMcQuaid Can you take a look at the bottle check in 500d40d? Not sure how we want to handle C++ stdlibs in bottles going forward.

@mistydemeo mistydemeo closed this Sep 1, 2013
@mistydemeo mistydemeo deleted the mistydemeo:superenv_non_apple_compiler branch Sep 1, 2013
@seanjensengrey
Copy link

@seanjensengrey seanjensengrey commented Nov 9, 2013

Love the change, couple things:

I am seeing a compilation failure (segfault) in gnu-prolog unless I compile with --env=std

Fatal Error: Segmentation Violation (bad address: 0x10041aada)
compilation failed
make[1]: *** [fd2c.o] Error 1
make: *** [all] Error 1
  • Is there a way to just turn off this feature?
  • It would be nice if the banner "Warning: Experimental support .." told me the compiler version is picked up
  • I cannot --cc=gcc to switch back to my platform gcc, why is that? I can switch to other gcc versions
@mistydemeo
Copy link
Member Author

@mistydemeo mistydemeo commented Nov 9, 2013

I am seeing a compilation failure (segfault) in gnu-prolog unless I compile with --env=std

Can you file a separate issue for this, please?

It would be nice if the banner "Warning: Experimental support .." told me the compiler version is picked up

That's a good idea.

I cannot --cc=gcc to switch back to my platform gcc, why is that? I can switch to other gcc versions

--cc= requires a specific compiler version. You can specify --cc=gcc-4.2.

@ohhorob
Copy link

@ohhorob ohhorob commented Jan 4, 2014

I had to use gcc-4.7 to compile protobuf240a on Mavericks.

First step was installing gcc-4.7 (see http://stackoverflow.com/a/19719840/178831)
And then protobuf240a with the cc option:

$ export HOMEBREW_CC=gcc-4.7; export CXX=g++-4.7; ~/homebrew/bin/brew install -v --cc=gcc-4.7 protobuf240a

This was to get the CDH4.5.0 hadoop distro maven target 'idea:idea' to build.. so I had to get the protobuf240a/bin/protoc in the path, and try the maven goal again.

$ mvn idea:idea
@Homebrew Homebrew locked and limited conversation to collaborators Feb 16, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet