Skip to content

Loading…

better GHC formula with support for 7.6.2 and bindist installs #18779

Closed
wants to merge 2 commits into from

6 participants

@camillol

There have been other pull requests for GHC, but I think this is the one that takes care of all concerns.

  • GHC is out of date: by default this installs the GHC compatible with haskell-platform, but you can pass --current to install the current stable version (these are going to be different most of the time due to the different release schedules).

  • Compiling from source takes forever, and bottles only exist for a given prefix and take forever to make: the GHC bindist can actually be installed into any prefix, so we take advantage of that and we don't need to make bottles again.

  • But someone may want to build from source: that's supported too (--source).

  • But 7.4.2 has a bug and needs a patch: that's why that version builds from source by default (and of course it uses the bottle that is currently available if installing to /usr/local).

Bottom line, this allows people to install 7.6.2 now and makes maintenance easier in the future.

I have tested installing 7.6.2 from binary to a custom prefix, installing 7.4.2 from source to a custom prefix, and installing 7.4.2 from bottle to /usr/local.

@camillol

@samueljohn wanted to be pinged.

@samueljohn

Thanks. I am mostly afk (iPhone only) over the Easter days.

@mikemcquaid
Homebrew member

the GHC bindist can actually be installed into any prefix

Then the new bottles will work in any prefix too. I want us to keep building from source and not use upstream binaries. We're going to start bottling a lot more so any issues with bottles need to be fixed rather than just worked around.

@camillol

No, the bindist is not just a bunch of files that you dump somewhere. AFAICT it contains the compiled object files, but without the final linking step, which is done after you configure it for the final installation prefix. But if you bottle up the final installation, it won't be relocatable.

Also, "I want" is not much of a reason. You're still welcome to make bottles if you want, but there's no reason not to use the upstream binaries when the bottles can't be used or when they haven't been made yet.

@mikemcquaid
Homebrew member

No, the bindist is not just a bunch of files that you dump somewhere. AFAICT it contains the compiled object files, but without the final linking step, which is done after you configure it for the final installation prefix. But if you bottle up the final installation, it won't be relocatable.

If that's the case, fair enough. I'm still not convinced the bottles will not necessarily be relocatable unless there's lots of Qt-style embedding of the prefix as strings inside binaries.

You're still welcome to make bottles if you want, but there's no reason not to use the upstream binaries when the bottles can't be used or when they haven't been made yet.

Well, the same reason we rather not use upstream binaries: we would rather build from source and manage binaries ourselves. We're going to be doing a lot more building of binaries in the next year so I don't want to sidestep issues but instead actually seek to fix them.

@camillol

It's not that there are issues with the bottles, here - and if there were, I'd want them fixed, too. We just don't have a generic mechanism for making all bottles relocatable, since a given formula can depend on the prefix in arbitrary ways - and that's ok. But in this specific case, we can take advantage of GHC's peculiar approach to binary distributions to provide support for arbitrary prefixes.

Also, building GHC from source still requires the GHC binaries, and it takes a really, really, really long time. So we'd be downloading the binaries, using them (which means they have to work!) to build exactly the same thing at a huge expense of time and energy, and then throwing them away. For this formula, I think we should only build from source when we need to apply a patch (as with 7.4.2), or maybe when the user wants to work on GHC itself. But I built GHC from source today, and I never want to do it again unless it's absolutely necessary. It's that slow.

@mikemcquaid
Homebrew member

Considering I've built all the bottles on my own virtual machines: I'm aware of how slow it is. If we want to apply a patch we need to then change the build process again. That's why we currently bottle.

There have, in the last month, been various code changes to make bottles sometimes generically relocatable. If you don't want to build it again that's ok but I want to do so before we use the binaries instead.

@camillol

If we want to apply a patch we need to then change the build process again.

That's FUD. This formula supports both installing from the binaries and building from source. 7.4.2 builds from source by default (because it needs a patch), while 7.6.2 builds from the binaries (because it doesn't). It takes less than one line of code to switch, so it's a non-issue.

Actually, it's an argument in favor of this patch: the existing GHC formula only supported building from source, and I had to spend a lot of time making this patch so that I could install 7.6.2 from the binaries. But if we apply this patch, that won't be a problem any more.

Also, this is not a threat to your work on bottles. You don't have to try to kill it. It's one formula, using a specific approach that only applies to GHC (but makes a lot of sense for GHC), so it doesn't impact the generic mechanisms.

@mikemcquaid
Homebrew member

Whatever. I'll let other maintainers chip in. I'm not merging this until I've tested the bottle support and I'd suggest to other maintainers we don't do so either.

(For future reference when you start tossing around phrases like 'FUD' it kills my motivation to actually have a discussion)

@camillol

We can't have a worthwhile discussion if you won't even look at the patch first.

@mikemcquaid
Homebrew member

I did. Which "less than one line" do you switch to make bottles work with this approach?

@camillol

Bottles already work. If you use this formula and homebrew is in /usr/local, by default you'll get the 7.4.2 bottle. If you pass --current, you'll get 7.6.2, which is installed from the upstream binaries. If you decide to make bottles for it you can simply add them in the appropriate place in the formula (admittedly, a bottle block takes more than one line, but that's not my fault :) ).

The "less than one line" was about switching to building from source so we can apply patches: you just have to change the condition in build_source?. For example, if it turned out that we always need to build from source on Leopard to apply a patch specific to that OS, you'd add or MacOS.version == :leopard (which is less than a whole line).

@mikemcquaid
Homebrew member

The bottle isn't relocatable, I checked. It looks like it could get there with a bit of work as the bulk of GHC has been relocated. Might investigate fixing up the various text files and playing with relocating haddock and Cabal properly.

I think there's some good ideas here but it makes the formula way overcomplicated. I'd be for just sitting on 7.4 until haskell-platform is updated and then updating this to 7.6. I'm open to other maintainers views on this.

@samueljohn

As a side note:
brew install <RAWURL-OF-THIS-PR> -v errors out for me
Perhaps Xcode-only setup. Perhaps non-standard homebrew location.

@samueljohn

update: I have the same issue with the current ghc formula.

@Sharpie

We switched to bootstrapping from source for very good reasons discussed at length in #13519. Specifically, see my comment where I note that the bindist appears to work well for users running the same version of OS X as the developer who rolled it up, but poorly for users of newer or older OS X releases.

Bootstrapping our own bottles is a pain, but ensures that GHC was specifically built for the OS X release that it will be running on.

So, before we ponder switching back to the bindist, we have to answer the question: has the quality of the builds improved enough since 7.4.2 to justify the risk?

@camillol

@samueljohn I've tested using a custom homebrew location, so that shouldn't be the problem.

@Sharpie I've read #13519. The initial issue seemed to be using mismatched versions of GHC and H-P. Then @dysinger mentioned issues with the bindist on Mountain Lion, but he never provided specifics, nor a reproducible testcase. The one specific issue that was mentioned is http://hackage.haskell.org/trac/ghc/ticket/7040 , which is patched in this formula. Of course, having to apply a patch was reason enough to have source builds at that point, and the other supposed issues were never mentioned again.

OTOH, everyone I've talked to on #haskell recommends using the binaries for GHC and/or H-P, and they recommend against using Homebrew. But of course, if you use them directly you lose the convenience of upgrading and uninstalling using Homebrew.

BTW, I think we should follow a policy of explaining non-obvious decisions in comments in the formulas. In this case, though, it turns out that it could not be done simply because no specifics were ever provided. It is entirely possible that the problem was only with 7.4.1, or that it was some other issue, and the version mismatch was just a hunch. Even if the problem was also in 7.4.2, this formula still builds that version from source because it needs the patch.

I think it's wiser to support both source-builds and binary installs, as this patch does. I don't think we should assume that the official binaries are always going to be broken, especially when we have no test case to show that it might be so. And even if it did, it would make sense to contribute the findings upstream and help them fix their builds, so again it wouldn't be a reason to remove the binary path entirely.

@eagleflo

I'm in agreement with @camillol here. If there's a problem with the official binaries, let's help get them fixed upstream. Furthermore, @mikemcquaid's suggestion of sitting on 7.4 until the next version of Haskell Platform is released is not a good way to go for a few reasons:

  • Many Haskell users prefer to use the latest version of GHC and install the packages provided by Haskell Platform from Cabal manually.
  • Haskell Platform has historically had a very erratic release schedule. It could be late by six months again.

I'm of the opinion that the GHC formula should provide the latest official stable release of GHC. Ruby and Python formulas for example seem to be updated almost immediately when a new release is made, while GHC is constantly stuck in an old version due to haskell-platform's dependency on it (7.6 came out in last September).

Could GHC 7.4 be moved to homebrew-versions and have Haskell Platform depend on that until it's updated? Does Homebrew support depending on versioned formulae?

If that's not an option, I'd like to see the latest version provided with a switch at least. We used to use --devel before but --current is probably the better choice, even if slightly apologetic.

@eagleflo

@mistydemeo suggested that a Ghc74 subformula could be created within the Haskell Platform formula and thus separate it from the GHC altogether. This seems like a mighty good idea to me. Also, GHC 7.6.3 has already been released in the meanwhile.

I'll have a go at updating all related formula (ghc, haskell-platform, cabal-install) later tonight.

@camillol

Updated to 7.6.3. Is there any reason left not to merge this?

@mikemcquaid mikemcquaid added a commit that closed this pull request
@mikemcquaid mikemcquaid ghc 7.6.3
Closes #14820.
Closes #18362.
Closes #18779.
8c8ee59
@lovelace lovelace added a commit that referenced this pull request
@mikemcquaid mikemcquaid ghc 7.6.3
Closes #14820.
Closes #18362.
Closes #18779.
287e9e0
@cartazio

I"ll try and read the conversation on this ticket in the next day or so and hopefully make some helpful remarks subsquently

@mikemcquaid
Homebrew member

@cartazio Please don't; this is closed. If you want to discuss these things please take it to the mailing list.
@camillol You appear to be in "piss off the core contributors" mode again.

@camillol

@mikemcquaid what are you saying? You yourself wrote, in this very thread: "I think there's some good ideas here but it makes the formula way overcomplicated. I'd be for just sitting on 7.4 until haskell-platform is updated and then updating this to 7.6. I'm open to other maintainers views on this." This is a great opportunity to get input from the Haskell Mac community. I'm not trying to reopen this ticket, obviously, but we can have a useful discussion on what to do with the GHC formula in the future.

We can choose a different forum if you prefer, but that's no reason for you to get "pissed off".

@mikemcquaid
Homebrew member

This is a closed pull request. We're using our bottles rather than the bindist. The conversation was had.

@mikemcquaid
Homebrew member

I'll accept a pull request that builds the bindist and then we can use a bottle that has cellar :any.

@handyman5 handyman5 pushed a commit to handyman5/homebrew that referenced this pull request
@mikemcquaid mikemcquaid ghc 7.6.3
Closes #14820.
Closes #18362.
Closes #18779.
dea313a
@shelhamer shelhamer added a commit that referenced this pull request
@mikemcquaid mikemcquaid ghc 7.6.3
Closes #14820.
Closes #18362.
Closes #18779.
9b4f559
@eagleflo eagleflo referenced this pull request
Closed

GHC 7.8.2 #28350

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 27, 2013
  1. @camillol
Commits on May 23, 2013
  1. @camillol

    update to GHC 7.6.3

    camillol committed
Showing with 118 additions and 64 deletions.
  1. +118 −64 Library/Formula/ghc.rb
View
182 Library/Formula/ghc.rb
@@ -1,5 +1,21 @@
require 'formula'
+# The GHC bindist can be installed into any prefix.
+# Building from source takes a very, very long time, and requires the bindist anyway.
+# For these reasons, the normal way to install this package should be to download the
+# bindist and install it into our prefix. This way we can achieve fast installation
+# without having to build bottles ourselves, and it works for any prefix.
+# We maintain the option of building from source, for people who want to do so.
+#
+# This formula supports two versions of GHC: the version compatible with the latest
+# Haskell Platform (the default) and the current stable version (use --current). Since
+# H-P is released far less frequently than GHC, these are going to be different versions
+# most of the time.
+#
+# For GHC 7.4.2, we force building from source because there is an important patch that
+# needs to be applied. For this version, it is appropriate to have bottles as well.
+# However, for future versions they will not be needed, saving us a lot of time.
+
class NeedsSnowLeopard < Requirement
satisfy MacOS.version >= :snow_leopard
@@ -11,26 +27,100 @@ def message; <<-EOS.undent
end
end
-class Ghcbinary < Formula
- if Hardware.is_64_bit? and not build.build_32_bit?
- url 'http://www.haskell.org/ghc/dist/7.4.2/ghc-7.4.2-x86_64-apple-darwin.tar.bz2'
- sha1 '7c655701672f4b223980c3a1068a59b9fbd08825'
- else
- url 'http://www.haskell.org/ghc/dist/7.4.2/ghc-7.4.2-i386-apple-darwin.tar.bz2'
- sha1 '60f749893332d7c22bb4905004a67510992d8ef6'
+# Since the bindist package is needed for both binary and source installations, it should
+# be the main formula's download. However, previous versions of this formula had the
+# source package as the main download. Because of name mangling, both end up being called
+# "ghc-7.4.2.tar.bz2", which can cause a checksum mismatch when reinstalling. To make
+# things easier for our users, this disables name mangling for those downloads.
+
+class OriginalNameDownloadStrategy < CurlDownloadStrategy
+ def initialize name, package
+ super '__UNKNOWN__', package
end
- version '7.4.2'
end
class Ghctestsuite < Formula
- url 'https://github.com/ghc/testsuite/tarball/ghc-7.4.2-release'
- sha1 '6b1f161a78a70638aacc931abfdef7dd50c7f923'
+ if build.include? 'current'
+ url 'http://www.haskell.org/ghc/dist/7.6.3/ghc-7.6.3-testsuite.tar.bz2'
+ sha1 'ba8f2555c2678f2c87e75a16631ff959efb95650'
+ else
+ url 'http://www.haskell.org/ghc/dist/7.4.2/ghc-7.4.2-testsuite.tar.bz2'
+ sha1 'b5f38937872f7a10aaf89b11b0b417870d2cff7c'
+ end
+end
+
+class Ghcsource < Formula
+ if build.include? 'current'
+ url 'http://www.haskell.org/ghc/dist/7.6.3/ghc-7.6.3-src.tar.bz2'
+ sha1 '8938e1ef08b37a4caa071fa169e79a3001d065ff'
+ else
+ url 'http://www.haskell.org/ghc/dist/7.4.2/ghc-7.4.2-src.tar.bz2'
+ sha1 '73b3b39dc164069bc80b69f7f2963ca1814ddf3d'
+
+ def patches
+ # Explained: http://hackage.haskell.org/trac/ghc/ticket/7040
+ # Discussed: https://github.com/mxcl/homebrew/issues/13519
+ # Remove: version > 7.4.2
+ 'http://hackage.haskell.org/trac/ghc/raw-attachment/ticket/7040/ghc7040.patch'
+ end
+ end
+
+ def install real_prefix
+ # Fix an assertion when linking ghc with llvm-gcc
+ # https://github.com/mxcl/homebrew/issues/13650
+ ENV['LD'] = 'ld'
+
+ if Hardware.is_64_bit? and not build.build_32_bit?
+ arch = 'x86_64'
+ else
+ ENV.m32 # Need to force this to fix build error on internal libgmp.
+ arch = 'i386'
+ end
+
+ system "./configure", "--prefix=#{real_prefix}",
+ "--build=#{arch}-apple-darwin"
+ system 'make'
+ if build.include? 'tests'
+ Ghctestsuite.new.brew do
+ buildpath.install 'testsuite'
+ cd (buildpath+'testsuite/tests') do
+ system 'make', 'CLEANUP=1', "THREADS=#{ENV.make_jobs}", 'fast'
+ end
+ end
+ end
+ # Temporary j1 to stop an intermittent race condition
+ system 'make', '-j1', 'install'
+ end
end
class Ghc < Formula
homepage 'http://haskell.org/ghc/'
- url 'http://www.haskell.org/ghc/dist/7.4.2/ghc-7.4.2-src.tar.bz2'
- sha1 '73b3b39dc164069bc80b69f7f2963ca1814ddf3d'
+ if build.include? 'current'
+ version '7.6.3'
+ if Hardware.is_64_bit? and not build.build_32_bit?
+ url 'http://www.haskell.org/ghc/dist/7.6.3/ghc-7.6.3-x86_64-apple-darwin.tar.bz2', :using => OriginalNameDownloadStrategy
+ sha1 'fb9f18197852181a9472221e1944081985b75992'
+ else
+ url 'http://www.haskell.org/ghc/dist/7.6.3/ghc-7.6.3-i386-apple-darwin.tar.bz2', :using => OriginalNameDownloadStrategy
+ sha1 '6a312263fef41e06003f0676b879f2d2d5a1f30c'
+ end
+ else
+ version '7.4.2'
+ if Hardware.is_64_bit? and not build.build_32_bit?
+ url 'http://www.haskell.org/ghc/dist/7.4.2/ghc-7.4.2-x86_64-apple-darwin.tar.bz2', :using => OriginalNameDownloadStrategy
+ sha1 '7c655701672f4b223980c3a1068a59b9fbd08825'
+ else
+ url 'http://www.haskell.org/ghc/dist/7.4.2/ghc-7.4.2-i386-apple-darwin.tar.bz2', :using => OriginalNameDownloadStrategy
+ sha1 '60f749893332d7c22bb4905004a67510992d8ef6'
+ end
+
+ bottle do
+ revision 1
+ sha1 '45b4f126123e71613564084851a8470fa4b06e6b' => :mountain_lion
+ sha1 'a93d9aab9e3abfe586f9091f14057c6d90f6fdc0' => :lion
+ sha1 '7d284bd3f3263be11229ac45f340fbf742ebbea6' => :snow_leopard
+ end
+ end
env :std
@@ -38,13 +128,8 @@ class Ghc < Formula
option '32-bit'
option 'tests', 'Verify the build using the testsuite in Fast Mode, 5 min'
-
- bottle do
- revision 1
- sha1 '45b4f126123e71613564084851a8470fa4b06e6b' => :mountain_lion
- sha1 'a93d9aab9e3abfe586f9091f14057c6d90f6fdc0' => :lion
- sha1 '7d284bd3f3263be11229ac45f340fbf742ebbea6' => :snow_leopard
- end
+ option 'source', 'Build from source'
+ option 'current', 'Build the latest stable version'
fails_with :clang do
cause <<-EOS.undent
@@ -53,61 +138,30 @@ class Ghc < Formula
EOS
end
- def patches
- # Explained: http://hackage.haskell.org/trac/ghc/ticket/7040
- # Discussed: https://github.com/mxcl/homebrew/issues/13519
- # Remove: version > 7.4.2
- 'http://hackage.haskell.org/trac/ghc/raw-attachment/ticket/7040/ghc7040.patch'
+ def build_source?
+ # 7.4.2 must always be built from source (unless using bottle)
+ # because we need to apply a patch
+ build.include? 'source' or not build.include? 'current'
end
def install
- # Move the main tarball contents into a subdirectory
- (buildpath+'Ghcsource').install Dir['*']
+ binprefix = build_source? ? buildpath+'binprefix' : prefix
- # Define where the subformula will temporarily install itself
- subprefix = buildpath+'subfo'
+ system "./configure", "--prefix=#{binprefix}"
+ # Temporary j1 to stop an intermittent race condition
+ system 'make', '-j1', 'install'
- Ghcbinary.new.brew do
- system "./configure", "--prefix=#{subprefix}"
- # Temporary j1 to stop an intermittent race condition
- system 'make', '-j1', 'install'
- ENV.prepend 'PATH', subprefix/'bin', ':'
- end
-
- cd 'Ghcsource' do
- # Fix an assertion when linking ghc with llvm-gcc
- # https://github.com/mxcl/homebrew/issues/13650
- ENV['LD'] = 'ld'
-
- if Hardware.is_64_bit? and not build.build_32_bit?
- arch = 'x86_64'
- else
- ENV.m32 # Need to force this to fix build error on internal libgmp.
- arch = 'i386'
- end
-
- system "./configure", "--prefix=#{prefix}",
- "--build=#{arch}-apple-darwin"
- system 'make'
- if build.include? 'tests'
- Ghctestsuite.new.brew do
- (buildpath+'Ghcsource/config').install Dir['config/*']
- (buildpath+'Ghcsource/driver').install Dir['driver/*']
- (buildpath+'Ghcsource/mk').install Dir['mk/*']
- (buildpath+'Ghcsource/tests').install Dir['tests/*']
- (buildpath+'Ghcsource/timeout').install Dir['timeout/*']
- cd (buildpath+'Ghcsource/tests') do
- system 'make', 'CLEANUP=1', "THREADS=#{ENV.make_jobs}", 'fast'
- end
- end
- end
- ENV.j1 # Fixes an intermittent race condition
- system 'make', 'install'
+ ENV.prepend 'PATH', binprefix/'bin', ':'
+ if build_source?
+ Ghcsource.new.brew {|f| f.install prefix}
end
end
def caveats; <<-EOS.undent
This brew is for GHC only; you might also be interested in haskell-platform.
+
+ If `ghc-pkg check` reports broken packages, you may have leftovers from an
+ old installation. Try removing ~/.ghc
EOS
end
end
Something went wrong with that request. Please try again.