Update Haskell-related formulae

[eagleflo]

Hey, I updated some Haskell-related formulae to the newest versions now that Haskell Platform finally got updated:

• GHC 7.6.3
• cabal-install 1.16.0.2
• Haskell Platform 2013.2.0.0

I cleaned up the GHC formula quite a bit and transitioned it back to working with the official bindist packages, based on @camillol's work in #18779.

Tested on OS X 10.8.3. More testing welcome!

[MikeMcQuaid]

I'd rather we built GHC from source and bottled it.

[eagleflo]

Then it's probably better to continue from @camillol's work at #18779.

What does bottling buy us in this case? I'd rather get the official upstream binaries fixed, should any problems arise.

[MikeMcQuaid]

We prefer from-source packages. They allow us more control over the build, optimisation, compiler choice and better supporting individual versions of OSX. I'm not going to be a dictator here though; if other maintainers strongly disagree I'll happily drop this. It's become drastically easier for us to create bottles and will only get easier still.

[eagleflo]

Alright.

From my perspective bottling increases the barrier to contribution. It seems like something only core contributors can do. I have no idea what steps are involved in that process, other than updating the formula to build 7.6.3 from source.

Let's hope someone who is able to create bottles will pick this up then. The other two commits should be usable once GHC is updated.

[MikeMcQuaid]

It does increase barriers to contribution as it is something only core contributors can do. This will improve shortly as we have Mac Minis that semi-automate this process and eventually will completely automate this process.

I can create bottles (and previously created most of them), FYI.

[adamv]

We should pull these for now, close out the accumulated Haskel tickets, and try to build from source after that.

[MikeMcQuaid]

@adamv I'd agree if this didn't delete half of the GHC formula that is currently working to build from source. I'll take a look.

[adamv]

There's the ever unpopular "create homebrew-ghc as a tap and let the community manage it" suggestion.

[MikeMcQuaid]

I don't mind keeping this in core really.

[eagleflo]

I'd love to see someone double-check this on OS X 10.6 before merge, if possible. I just noticed that GHC 7.6.3 bindist is meant for OS X 10.7+ with XCode 4.1+ installed: http://www.haskell.org/ghc/download_ghc_7_6_3#macosx_x86_64

Maybe bottling is the better way to do this after all, if that's the only way to keep support for OS X 10.6.

[camillol]

Thanks for your work, @arkx. But what ended up getting merged is still not satisfactory to me.

I would like to see the discussion in #18779 addressed. Things like:

• The Haskell community recommends using the official binaries;
• The source build is really a source rebuild - it still relies on the official binaries, which means it doesn't work on any system where the official binaries don't work;
• Building from source is incredibly slow, and bottles only work in the default install location, whereas the official binaries are relocatable - this means I'd be stuck building from source forever with the current formula;
• GHC and H-P are only in sync for a very short time each year, so the apparent simplicity of this formula will be short-lived - or we're soon going to be back to shipping an outdated GHC, which is even worse.

And the worst part is that we already had a formula that:

• supported installing the official binaries (in any location);
• supported rebuilding from source when requested by the user, or when it was required for patches;
• supported two version of GHC, one for people who want to the one matching the current H-P, one for people who want the latest GHC;
• supported installing from bottles if available.
  But it was never merged.

I think it's very good that @MikeMcQuaid reminded us of our policy of building from source. But we should not be dogmatic about it: the policy should alwasy be considered, but in this case, there are many good reasons to deviate from it (and it's not even possible to stick to it entirely, since to build the sources we need to install the binaries anyway). Haskell users would like to be able to install the official binaries with homebrew, and to have the latest version of GHC available at all times.

I don't think @MikeMcQuaid is being dogmatic here; he has raised his point, but he said that he'd be ok with other maintainers accepting the counterpoints from the users. The problem is that we don't seem to have any maintainers who really use Haskell or care about this, and so the mild opposition of a single maintainer becomes an unsurmountable roadblock to delivering what the users are asking for - even beyond what that maintainer himself intended. I think this is not how things should work out.

[MikeMcQuaid]

It's not an unsurmountable roadblock; create your own tap or fork, maintain it and get some popular support and we'll consider making it official and/or pulling changes from it.

The discussion went cold because everyone had expressed their views and not enough people agreed with your approach. I spent a bunch of time today getting these formulae working and I'm sorry that you're not satisfied with the result but I'm sure for some people (certainly 10.6 users who would be otherwise losing support for ghc) it is enough.

I'd suggest the best course of action at this point is working out how we can improve bottles and the upstream build process so it supports parallel builds, can use superenv, can produce relocatable binaries and perhaps even self-bootstrap (or use bottles to do so on e.g. 10.5).

Homebrew generally frowns on installing binary packages and, personally, I think this is sensible. The bottling here was made significantly easier by brew test-bot and eventually we're going to be using bottles far more widely. I'd suggest we work out ways to improve this.

[camillol]

Nobody's going to use an alternative tap for a formula that's already in core.

Everyone expressed their views, but some were disproven. For example, @Sharpie mentioned that there were issues with the bindist discussed in #13519; I went through the entire discussion and showed exactly what the issues where and why they were not a problem for my proposed change. But because the discussion was simply dropped, next time we're going to be back to "we have to build from source because the bindist doesn't work", even though that's not true.

10.6 users would not be losing support for GHC, we could easily fallback on a source build with 7.4.2 bootstrapping if needed, since my formula supported source installs too. BTW, did you do the testing that @arkx suggested and verified that 7.6.3 doesn't work on 10.6, or is it an assumption? It would be nice to document that in the formula, if you did.

Homebrew frowns on installing binary-only packages, AFAIK, because it's supposed to be a package manager for open source. If it frowned on using binaries at all, we wouldn't have bottles, which are binary packages.

The upstream build process is fine. It can even produce compiled but not linked binaries that can then be installed in any location, a very nice feature that is usually missing.

I understand that you love your bottles, it's great that you care so much about it, but Homebrew is made for installing software, and the point of the GHC formula is to install GHC. Instead you seem to be approaching this as if its main purpose was to provide another chance to flex the bottle subsystem, with the actual Haskell users as a minor consideration. This is where we really disagree: the solution we have right now is objectively worse at actually installing GHC, but it is better at relying on bottles (except not really, since my proposed formula still used bottles whenever available, specifically to avoid conflict on that point), and at following some general policies (which I'm not sure @mxcl ever actually set in stone).

[MikeMcQuaid]

The problem isn't binary packages but binary packages we can't/don't produce ourselves. The upstream build process takes a long time partly because it doesn't work in parallel and needs -j1 in a bunch of places. That would be a good thing to fix. If you can suggest make a pull request to produce the compiled but not linked binaries that can be installed in any location that sounds like it could make bottles work anywhere so let's do that.

Even if the bindist was literally the best provided upstream binary package in the world then I'd frown on us using it as I think it's better for a mostly source-based distribution to build our own binaries. Perhaps this is because I love my bottles but until other maintainers are going to start disagreeing with me (which I'm totally open to them doing) then this isn't going to change. Part of the reason for the Kickstarter (and the enormous amount of time I've spent getting it ready and since) is to improve our binary package generation and automate the process. This may make some vocal users unhappy but if it enables most users to use Homebrew without Xcode or the CLT and packages install near-instantly after downloading then I think it will be worth some minor teething pains when getting there.

[camillol]

The problem isn't binary packages but binary packages we can't/don't produce ourselves.

Why?

The upstream build process takes a long time partly because it doesn't work in parallel

It would still be painfully slow on my MacBook, even if it were parallelized.

If you can suggest make a pull request to produce the compiled but not linked binaries that can be installed in any location that sounds like it could make bottles work anywhere so let's do that.

It's specific to GHC's build process, it's not something that can be applied to a generic system like bottles. We could call the bindist a bottle, though.

This may make some vocal users unhappy but if it enables most users to use Homebrew without Xcode or the CLT and packages install near-instantly after downloading then I think it will be worth some minor teething pains when getting there.

This is a false dychotomy. I've been telling you since the previous pull request: this is not a threat to your work on bottles in any way. There is literally no conflict. The "minor pain" that you're making Haskell users go through (leading people to recommend against using Homebrew to install Haskell) is completely unnecessary. You could just let us have our bindist support, and it wouldn't impact your bottle work in any way. It's not like this is going to spread to other formulas, either: this is a very specific situation with a very specific build system unique to GHC.

[MikeMcQuaid]

I'm saying if we can make our own version of the bindist we could do that and we'll have a relocatable bottle for free on all the platforms we use. If there's a make bindist-install or something we can call, let's do that.

[camillol]

Apparently it's just a matter of doing ./configure && make && make bindist. Try it. We still need to merge a formula that supports bindist installs, of course.

Also, GHC builds are supposed to support parallelization. People use and recommend make -j2. What race condition did you experience, exactly? Did you report the bug upstream?

Another question that has not been answered: have we actually tested the 7.6.3 bindists on 10.6?

[mistydemeo]

Shin_LAC on IRC points out that we can't directly use a bindist as a bottle because it still requires a make step to install.

[MikeMcQuaid]

@mistydemeo Thanks!

@camillol No, I didn't report upstream or test the bindists on 10.6. Last time I checked you are just as capable as me of doing both. You are also capable of trying to reproduce the race condition. For future reference your communication style seems perfectly optimised to make me want to invest no time in helping you (and I don't think I'm the only one). Well done.

[camillol]

@MikeMcQuaid, I don't have a 10.6 machine. In any case, from the claims you were making about 10.6 compatibility it sounded like you had tested it. Reporting bugs and submitting patches upstream was a Homebrew policy, last I checked, and while I don't believe in inflexible adherence to policies, there has to be a better reason for breaking this one than "someone else can do it". Especially when that someone else had already done all the work to produce a better GHC formula, only to see it ignored, and the detailed technical arguments that came with it unaddressed.

My communication style here is focused on technical points. It would probably be warmer if I were extended the professional courtesy of giving proper answers to my arguments; but in any case, it's not like I hate you or anything.
It sounds like the big obstacle here is that I have rubbed you the wrong way, so how about I buy you a beer and we bury the hatchet?

[MikeMcQuaid]

Reporting bugs and submitting patches upstream is something we try to do but if I tried to do it for every package I fixed (rather than those I have a base knowledge about to work on a patch) I'd simply run out of time. Your formula wasn't ignored, I re-read the thread and the diff before working on what was accepted eventually.

Yes, a beer some time would be good.