gcc: build gfortran by default.

[MikeMcQuaid]

Don't merge this yet; still work in progress.

This is the starting point to eventually replace the gfortran formula with gcc now it is in core. We'll still allow people to set their own Fortran compilers but default to our gcc formula if it hasn't been set.

[adamv]

(testpath/"in.f90").write(fixture)

[xuhdev]

Any good news?

[MikeMcQuaid]

No, been busy with other stuff. Will get on this eventually.

[MikeMcQuaid]

@adamv @jacknagel @mistydemeo @asparagui This is ready for review. Will probably merge later today if there's no objections.

[adamv]

Looks good.

[jacknagel]

I think the alias will cause problems, for example:

$ mkdir -p Cellar/gcc/4.8.2/bin
$ touch Cellar/gcc/4.8.2/bin/gcc
$ brew rm gfortran
Uninstalling /usr/local/Cellar/gcc/4.8.2...

[MikeMcQuaid]

@mistydemeo expressed similar concerns. Will remove it and we can always add it back in future.

[MikeMcQuaid]

Merged without alias.

[GitLoki]

I've been trying to install gfortran as a dependency for a numpy install and it looks like gcc should now be used instead. Is there a resource anywhere that tells people how to cope with this sort of change? The disappearance of gfortran left me quite puzzled, and it took a while to find the cause...

[kevinushey]

I don't understand. So now if I want gfortran I need all of gcc? Why?

Is there some way that it can be made very obvious that gfortran has moved to gcc?

[xuhdev]

@kevinushey You already have gcc whenever you have gfortran. Gcc is the basis of the compiler collection. Homebrew made it implicit before, now it makes explicit.

[kevinushey]

I see, thanks. I imagine there will be a few other people in my shoes ("huh? brew install gfortran doesn't work anymore?") so I wonder if, instead, gfortran could have some stub formula or otherwise instruct the user to do brew install gcc.

It seems bad to just break brew install gfortran without giving some hint to the user as to where they need to find it now.

[freddycct]

Is it possible to keep gfortran separate from gcc formula in homebrew? I already have gcc installed from the xcode developer commandline tools.

[xuhdev]

@freddycct I don't know much about the formula, but gcc installed from the xcode should be actually clang. You can check it by run:

gcc -v

If you see some thing like Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn), the gcc you are using is only a front end.

[freddycct]

Oh ok, what's the difference between clang gcc and the gcc in homebrew? I know one probably runs on some low-level VM while the other runs natively.

Is the performance better for homebrew gcc ?

[dakcarto]

Ran into an issue with this. On one Mac I have older gfortran formula installed, but not gcc. When trying to reinstall gdal --with-python, compiling numpy resource fails with:

==> python setup.py build --fcompiler=gnu95 install --prefix=/usr/local/osgeo4mac/Cellar/gdal/1.11.0/libexec
  File "/Volumes/Scratch/osgeo4mac-tmp/gdal--numpy-pgRf/numpy-1.8.1/numpy/distutils/fcompiler/gnu.py", line 197, in get_flags_opt
    v = self.get_version()
  File "/Volumes/Scratch/osgeo4mac-tmp/gdal--numpy-pgRf/numpy-1.8.1/numpy/distutils/fcompiler/__init__.py", line 434, in get_version
    raise CompilerNotFound()
numpy.distutils.fcompiler.CompilerNotFound

Adding depends_on :fortran does not fix the issue. gfortran executable is still on PATH, so the requirement passes. Any ideas on what's happening here?

[dakcarto]

So, doing:

brew rm gfortran
brew install gcc

fixed the above issue; but, how can users best be notified to do this? Reinstate gfortran and bump its revision, but actually use the revision to install gcc and unlink/delete Cellar/gfortran?

[mistydemeo]

@NikolausDemmel To be clear, you can always choose what compiler software should be built with. That much software defaults to something named "gcc" doesn't mean you don't have a choice. Homebrew itself already takes care of it internally. Outside Homebrew, you can always export CC=clang CXX=clang++ to make sure that clang gets picked, regardless of what the "gcc" that's first in the PATH is.

[NikolausDemmel]

@xuhdev: Note that I am on 10.9 and I am talking about compiling opencv from source manually (not homebrew). When I do this with gcc installed cmake chooses gcc over clang as a compiler (see #28855 (comment)).

mkdir /tmp/foo
cd /tmp/foo
wget http://downloads.sourceforge.net/project/opencvlibrary/opencv-unix/2.4.9/opencv-2.4.9.zip
unzip opencv-2.4.9.zip
mkdir build
cd build
cmake ../opencv-2.4.9 -DWITH_CUDA=OFF
make
...
[ 22%] Building CXX object modules/highgui/CMakeFiles/opencv_highgui.dir/src/window_cocoa.mm.o
/tmp/opencv/opencv-2.4.9/modules/highgui/src/window_cocoa.mm: In function 'void cvDestroyAllWindows()':
/tmp/opencv/opencv-2.4.9/modules/highgui/src/window_cocoa.mm:200:23: error: expected ';' before 'in'
     for(NSString *key in list) {
                       ^
/tmp/opencv/opencv-2.4.9/modules/highgui/src/window_cocoa.mm:200:23: error: 'in' was not declared in this scope

For other software, e.g. urdfdom I get libstdc++ / libc++ incpatibility problems because dependecies are built with clang:

Linking CXX shared library liburdfdom_model.dylib
Undefined symbols for architecture x86_64:
  "TiXmlElement::SetAttribute(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)", referenced from:
      urdf::exportPose(urdf::Pose&, TiXmlElement*)    in pose.cpp.o
      urdf::exportURDF(urdf::ModelInterface const&) in model.cpp.o
      urdf::exportMaterial(urdf::Material&, TiXmlElement*)    in link.cpp.o
      urdf::exportSphere(urdf::Sphere&, TiXmlElement*)    in link.cpp.o
      urdf::exportBox(urdf::Box&, TiXmlElement*)   in link.cpp.o
      urdf::exportCylinder(urdf::Cylinder&, TiXmlElement*)    in link.cpp.o
      urdf::exportMesh(urdf::Mesh&, TiXmlElement*)    in link.cpp.o
      ...
ld: symbol(s) not found for architecture x86_64

[NikolausDemmel]

@mistydemeo: Yes. I think this is totally reasonable for people who need to use gcc as C/C++ compiler in addition to clang for some projects. I personally can do this, no problem. But I am not speaking for myself, I am advocating for the users who use homebrew as a tool to satisfy dependencies of stuff they install. They might not even use anything installed through homebrew directly. IMHO, if they never intend to do anything not default with their c/c++ compiler, but need gfortran (which Apple does not provide) for example to install popular python packages like numpy, scipy, matplotlib, they should not need to modify their CC/CXX environment variables as a result. As a concrete example, I would like to avoid to extend the already lengthy ROS install instructions with export CC=clang CXX=clang++, especially if the only reason for this is facilitating the installation of gfortran through homebrew. It turns out installing gcc is quite intrusive. Whether there is Apple to blame, or cmake, or thrid party software making wrong assumptions, it does not really matter to me. The point remains that the gcc formula is not just like any other formula installing some software Apple does not provide...

[I said I would not argue this point any longer, but @mistydemeo's comment made me feel like it did not come across that I was not just arguing for my own sake; sorry, I hope I can resist the urge to reply next time.]

[wjwwood]

I understand the position the Homebrew developers are taking, it is not really a dup, so making it keg-only doesn't really make sense. If I install gcc I should be able to use it directly since OS X doesn't really provide it. And it makes total sense that developers can choose which compiler they want explicitly, but for better or worse this change does implicitly change the behavior of CMake and to @NikolausDemmel's point this has caused a tripping point for more novice users which are using Homebrew as a means to an end.

I think our solution will be to create a copy of the gfortran formula in our tap and explicitly note to our users that they should not install gcc from Homebrew, lest they have to deal with the consequences (explicitly selecting clang or gcc) themselves.

This isn't the optimal solution because I don't want to take on the maintenance burden of gfortran nor do I want to have to remind or explain to our users why gcc breaks our from source instructions, but it is what we will do since there doesn't seem to be a solution which meets everyone's needs upstream.

[xuhdev]

I think I know one solution here. The gcc formula should remove the c++ binary under /usr/local/bin, which people see as the default C++ compiler. g++ is enough for people who needs to use gcc for C++ code. As a consequence, cmake should also be able to detect the correct default C++ compiler.

[NikolausDemmel]

@xuhdev: Thanks for the hint. For the record, removing just the c++ does seem to resolve the issue of cmake for projects that project(foo CXX C). Not sure if there are other issues I haven't noticed, but for now we revert to providing our own gfortran anyway and it seems the decision for gcc has fallen on keg-only.

[cmarquardt]

Hi,

I've been hit by this quit badly. Because I needed a re-install of numpy, and the standalone gfortran wasn't available any more, I eventually installed gcc - and hoped that it wouldn't interfere with the Xcode environment. Yes, I know about CC and CXX, but in effect requiring these settings for every upgrade of every homebrew package just to stay consistent with how things worked previously is a nightmare.

Interestingly, @MikeMcQuaid states on https://github.com/Homebrew/homebrew/wiki/Custom-GCC-and-cross-compilers:

Homebrew depends on having an up-to-date version of Xcode because it comes with specific versions of build tools e.g. clang.

Installing a custom version of GCC or autotools into the $PATH has the potential to break lots of compiles so we prefer the Apple or Homebrew provided compilers.

Absolutely true. Why are you doing it then?

Removing the standalone gfortran compiler and requiring to put the entire gcc collection into the main path massively changes how the development environment behaves, and it breaks with homebrew's tradition to rely on the system provided tools.

I'd be very much in favour of having a gcc than I can easily use along with Apple's command line tools There were a number of solutions proposed (different names - similar to glibtool vs. libtool / optionally linking in only parts of the compiler suite / keg-only) that are all fine to work with. But removing a standalone fortran compiler and forcing users to have to use a different compiler as default just because they want to use numpy or scipy (or R or god knows what) isn't a good way forward in my opinion.

Some have argued that gcc isn't the default compiler on a Unix system. That's true formally, but practically, the vast majority of open source software will use a gcc if it finds one. One has to actively do something to enforce the use of another compiler in the presence of gcc. Putting gcc in the PATH means I actively have to do something in order to use the system's default compiler. And that means that for the majority of homebrew packages, the default compiler changes.

Please reconsider this repackaging of gfortran.

[MikeMcQuaid]

Would people rather we installed e.g. gcc-4.8, gfortran-4.8 or just didn't install any of this (including gfortran) into the PATH by default?

[NikolausDemmel]

@MikeMcQuaid: I cannot judge if:

1. Presence of gcc-4.8 would still interfere with the default compiler. Not shawoing the system gcc and c++ binaries sounds like step forward. Not sure if it is sufficient.
2. Presence of gfortran-4.8 would make the various python packages happy.

What I (and it seems some other people) would want is way to make gfortran available with a brew install ... (and no additional env setup) while at the same time not interfering with the systems default c/c++ compiler.

To directly answer your question, out of these two options I probably prefer the latter, but they are both suboptimal.

[MikeMcQuaid]

@NikolausDemmel Can you please investigate those things by editing the formula and/or trying gcc48 from homebrew-versions.

[cmarquardt]

What I (and it seems some other people) would want is way to make gfortran available with a brew install ... (and no additional env setup) while at the same time not interfering with the systems default c/c++ compiler.

Yes. Basically being able to use a fortran compiler without side effects.

Ideally, I'd also like to have a way to use the gcc compiler for compiling software, or even homebrew formulae (I have a use case for that, actually). I understand that one could temporarily add the respective cellar to PATH just for that build. What might be even nicer would be an option in a formula which specifies which compiler to use (maybe that's already possible; sorry, I'm relatively new to Homebrew)

[NikolausDemmel]

@cmarquardt You can use the --cc option to brew install (check the man page) to manually use a specific compiler for a formula. As for specifying a option in the formula to request gcc as a compiler, I think there is no way (@MikeMcQuaid would know), but it shouldn't be hard to add such a requirement if it has use cases.

[cmarquardt]

Thanks for the hint;-)

The use case I mentioned is a library (plain C) which uses OpenMP for parallelization. clang doesn't support it, but with a way to compile the library with gcc if available (or an optional requirement for the package) it might be possible to have that library in a multi-threaded version (if the compiler libraries play well with each other).

Oh but we are digressing from the original topic - sorry.

[NikolausDemmel]

@MikeMcQuaid: At least the specific problem with cmake seems to be caused by presence of /usr/local/c++, so if that were removed or renamed to c++-4.8 that specific problem goes away. However i have no idea about other situations where the c/c++ binaries with -4.8 suffix might affect something.

As for gfortran, it turns out that at least the scipy pip package looks for gfortran specifically and does not pick it up as gfortran-4.8. Same goes for (the current implementation of) the fortran requirement in homebrew.

[MikeMcQuaid]

@cmarquardt You can use fails_with if a formula doesn't work with Clang. grep for it to see usage.

Sounds like the best call may be to bring back prefixing the binaries e.g. gcc-4.8 but just make a symlink specifically for gfortran-4.8 to gfortran. Would that satisfy?

[NikolausDemmel]

@MikeMcQuaid: I think that could be a good practical solution, but I am by no means sure about the non-interference of gcc-4.8 and friends with clang. Short of changing the way homebrew works (along the line of my partial linking suggestion) this might be the best proposal so far IMHO.

[MikeMcQuaid]

@NikolausDemmel I disagree. Can you please try and do some decent research and publish your results here? It feels to me like you're wanting me to spend time fixing your issue without being willing to spend time researching what will or won't fix your problem.

[NikolausDemmel]

@MikeMcQuaid: The issue I personally noticed and reported, namely cmake projects with project(foo C CXX) picking up gcc when the gcc formula is installed while project(foo) picks up clang regardless is (according to my tests, as reported above) dependent on the /usr/local/bin/c++ binary alone. Therefore this particular issue will be addressed with your porposal. The second half of my concern was about gfortran being available to install python packages without modifying the environment, which is also addressed by your proposal (while only installing gfortran-4.8 does not work, as I tested and reported above).

I'm not sure what else you would have me research.

[cmarquardt]

I'll give it a go. I have to rebuild numpy and scipy anyway...

[MikeMcQuaid]

I've re-added the version suffix to everything installed by gcc and added a symlink from gfortran-4.8 to gfortran in #29943. This should be sufficient.

[cmarquardt]

Using your gcc.rb from pull request 29943 (just before you merged it, actually...). Note I'm using a non-standard homebrew prefix /opt/homebrew instead of /usr/local; that never was an issue so far.

brew install -v ./gcc.rb

gives

KASSANDRA (on pull-request-29943) /Users/marq/src/github/homebrew.git/Library/Formula>which gcc
/usr/bin/gcc
KASSANDRA (on pull-request-29943) /Users/marq/src/github/homebrew.git/Library/Formula>which gcc-4.8
/opt/homebrew/bin/gcc-4.8
KASSANDRA (on pull-request-29943) /Users/marq/src/github/homebrew.git/Library/Formula>which g++
/usr/bin/g++
KASSANDRA (on pull-request-29943) /Users/marq/src/github/homebrew.git/Library/Formula>which g++-4.8
/opt/homebrew/bin/g++-4.8
KASSANDRA (on pull-request-29943) /Users/marq/src/github/homebrew.git/Library/Formula>which gfortran
/opt/homebrew/bin/gfortran

Next

brew reinstall -v python # v2.7.7.1

works fine and uses clangas C compiler. But

brew reinstall -v numpy  # v1.8.1

results in errors, saying it can't find a working fortran compiler (I've removed stuff from the output to keep it readable:

==> python setup.py build --fcompiler=gnu95 install --prefix=/opt/homebrew/Cellar/numpy/1.8.1
Running from numpy source directory.
/opt/homebrew/Cellar/python/2.7.7_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'

[....]

  FOUND:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3)]
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']

which means it uses Apple's Acceleratioin framework for Lapack and Blas etc. It also finds the gfortranexecutable:

/opt/homebrew/Cellar/python/2.7.7_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
  warnings.warn(msg)
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
creating build
creating build/src.macosx-10.9-x86_64-2.7
creating build/src.macosx-10.9-x86_64-2.7/numpy
creating build/src.macosx-10.9-x86_64-2.7/numpy/distutils
building library "npymath" sources
customize Gnu95FCompiler
Could not locate executable gfortran
Could not locate executable f95
Found executable /opt/homebrew/bin/gfortran
gnu95: no Fortran 90 compiler found
gnu95: no Fortran 90 compiler found

I think the numpy setup.py tries to link something with the Fortran compiler; the python traceback ends with

[....]
File "numpy/core/setup.py", line 680, in get_mathlib_info
st = config_cmd.try_link('int main(void) { return 0;}')
  File "/opt/homebrew/Cellar/python/2.7.7_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distuti    ls/command/config.py", line 248, in try_link
    self._check_compiler()
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/command/config.py", line 78, in _check_compiler
    self.fcompiler.customize(self.distribution)
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/fcompiler/__init__.py", line 502, in customize
    get_flags('opt', oflags)
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/fcompiler/__init__.py", line 493, in get_flags
    flags.extend(getattr(self.flag_vars, tag))
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/environment.py", line 39, in __getattr__
    return self._get_var(name, conf_desc)
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/environment.py", line 53, in _get_var
    var = self._hook_handler(name, hook)
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/fcompiler/__init__.py", line 700, in _environment_hook
    return hook()
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/fcompiler/gnu.py", line 350, in get_flags_opt
    return GnuFCompiler.get_flags_opt(self)
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/fcompiler/gnu.py", line 197, in get_flags_opt
    v = self.get_version()
  File "/private/tmp/numpy-1wQj/numpy-1.8.1/numpy/distutils/fcompiler/__init__.py", line 434, in get_version
    raise CompilerNotFound()
numpy.distutils.fcompiler.CompilerNotFound

I'll try to find out what's actually tested in setup.py, but did the old gfortran.rb anything to set up libraries?

Thanks!

[cmarquardt]

I'm not sure if this the problem, but IIRC numpy's distutils build objects with the respective language compiler (say, gfortran), and then link them together in shared libraries using the C-compiler.

With test.f90 being

program HelloWorld
   print * "Hello, World!"
end program HelloWorld"

and

gfortran -c test.f90

gcc links this fine:

gcc-4.8 test.o -lgfortran -o test_gcc

but of course clangdoesn't because it doesn't know where to find libgfortran:

clang test.o -o test_clang -lgfortran
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Again, I think the older gfortran formula must have provided a workaround, because numpy installation worked out of the box there.

[cmarquardt]

Sorry - please forget all of the above.

After completely scratching my installation of gcc and python, and updating brew, building python, numpy, scipy and matplotlib build without problems (using clang/clang++ and gfortran).

Thank you @MikeMcQuaid for making that possible again.

[MikeMcQuaid]

@cmarquardt Glad it's working, thanks for your patience and work.

[NikolausDemmel]

Thanks @MikeMcQuaid.

With the latest gcc formula scipy installes through pip and so far no more c/c++ compiler issues popped up for us.