Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find system ghc via binary names like "ghc-7.10.3"? #2433

Closed
mgsloan opened this issue Aug 3, 2016 · 29 comments

Comments

@mgsloan
Copy link
Collaborator

commented Aug 3, 2016

In #2221, we're planning to switch away from using system ghc by default. We could consider trusting that the correct version is installed and skipping the version check.

It turns out that for non system ghcs it is also potentially possible to skip the version check. Someone on IRC mentioned that binaries named things like ghc-7.10.3 exist, and that stack should use these. Skipping the check should save a bit of execution time, potentially something like a tenth of a second.

This would mean not gracefully handling cases like #2430 . One option would be to perform such sanity checks after a failing build, if the Cabal errors for these cases are insufficient. This way, the user gets feedback quickly. For failing builds, it will wait a bit longer before exiting, to run the sanity checks.

@Blaisorblade

This comment has been minimized.

Copy link
Collaborator

commented Aug 9, 2016

I think calling ghc-X.X.X, for all GHCs would be huge and would allow having multiple GHCs on the paths; this would make life simpler for all users with trouble with bindists. Right now you can put a GHC path in the stack.yaml, but the result can't be distributed to users typically, since paths are valid locally (except for some scenarios using Docker). Supporting multiple system GHCs, otherwise, would require reading from stack's config a mapping from GHC version to path.

I admit I'm fuzzy on what exactly you have in mind (see below). Interpreting rather liberally, I imagine a few parts to this:

  • call ghc-X.X.X instead of ghc everywhere;
  • same for ghcjs assuming it works (I guess so but I'm not sure);
  • add sanity checks in case of error (maybe just ghc --version, possibly also building and running a minimal program?);
  • also revise sanity checks after GHC setup;
  • remove version check at the start of a build.

On the rest, I'm confused (which is why I didn't answer a week ago). Could you elaborate?

I'm less sure about the changes you propose to the version check, probably because I didn't finish reverse-engineering how it works. In particular, we often don't even call ghc --version but ghc --info. Also, why do we have a version for non system GHCs at all?

In #2221, we're planning to switch away from using system ghc by default. We could consider trusting that the correct version is installed and skipping the version check.

I'm more confused by the link with #2221. Doesn't the version check only make sense for the system GHC? And then that could be dropped by using ghc-X.X.X when looking for a system GHC.

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 9, 2016

same for ghcjs assuming it works (I guess so but I'm not sure);

It works less well for ghcjs right now, because there are a lot of different tarballs all with version 0.2.0. Most tarballs have dates in their name, but many don't have the version number set to something other than 0.2.0.

I'm less sure about the changes you propose to the version check, probably because I didn't finish reverse-engineering how it works. In particular, we often don't even call ghc --version but ghc --info.

Good point, I haven't looked deeply at those details. Could be we can't avoid that. We could cache it?

Also, why do we have a version for non system GHCs at all?

Not sure, I think because it's a mild sanity check, shouldn't be needed though.

I'm more confused by the link with #2221. Doesn't the version check only make sense for the system GHC? And then that could be dropped by using ghc-X.X.X when looking for a system GHC.

If we don't change the default then we'll always have to check the system ghc version.

@Blaisorblade

This comment has been minimized.

Copy link
Collaborator

commented Aug 9, 2016

If we don't change the default then we'll always have to check the system ghc version.

I think that even if you change the default you'll need some check.
Say I reenable system GHC—and I do that by setting system-ghc: true globally.
That doesn't mean you can use my system GHC for all resolvers, so you still need to learn which is the version of the system GHC.

OTOH, I understand all GHC bindists also provide binaries named ghc-X.X.X, not just the one stack installs. I propose to rely on this instead of the version check, by default. Of course this changes the logic from

  • learn GHC's version and match it with what I need
    to
  • look for a system GHC of the version I need.
    I assume this is still doable, but might be more invasive.
@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 9, 2016

Say I reenable system GHC—and I do that by setting system-ghc: true globally.
That doesn't mean you can use my system GHC for all resolvers, so you still need to learn which is the version of the system GHC.

True!

OTOH, I understand all GHC bindists also provide binaries named ghc-X.X.X, not just the one stack installs. I propose to rely on this instead of the version check, by default.

Yeah, maybe we can just go ahead and always trust it for ghc. In theory the logic change shouldn't affect users

@bitemyapp

This comment has been minimized.

Copy link
Contributor

commented Sep 13, 2016

@mgsloan how would this work for the various tools installed with GHC, do they all get suffixed? Some builds are going to fail if not using the associated tool matched to the GHC version.

@Blaisorblade

This comment has been minimized.

Copy link
Collaborator

commented Sep 14, 2016

Thanks for pointing that out! We should surely use the same version for any invocation of ghc-pkg, runghc and haddock-ghc. Do we need to account for other tools? I'm guessing no for alex and happy (whose versions should be controlled otherwise) but I'm no expert there.

$ ls /Applications/ghc-7.8.4.app/Contents/bin/
cabal  ghc  ghc-7.8.4  ghc-pkg  ghc-pkg-7.8.4  ghci  ghci-7.8.4  haddock  haddock-ghc-7.8.4  hp2ps  hpc  hsc2hs  runghc  runghc-7.8.4  runhaskell
$ ls /usr/local/Cellar/ghc/8.0.1/bin/
ghc  ghc-8.0.1  ghc-pkg  ghc-pkg-8.0.1  ghci  ghci-8.0.1  haddock  haddock-ghc-8.0.1  hp2ps  hpc  hsc2hs  runghc  runghc-8.0.1  runhaskell
@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2016

That is indeed a good point! That makes this a bit trickier, but still a good change I hope.

@bitemyapp

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2016

@mgsloan @Blaisorblade not to derail the intent of the thread, but wouldn't inter-operability with tools expecting ordinary names for these utilities be better served by doing the paths like @hvr's GHC distributions? It puts the version in a subdirectory of a consistent path (/opt/...), you can then use the path to switch between them and regular tooling "just works"

This is roughly what rustup does (path blipping) as well.

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2016

@bitemyapp Certainly! However, that does not work with standard global installations of GHC. Are they so tied to happy / alex versions?

Stack implements its own approach for switching sub-directories, which is why we haven't had logic like this.

@bitemyapp

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2016

However, that does not work with standard global installations of GHC.

Not certain what you mean. Users of hvr's PPA usually just pick a version and get on with it.

Are they so tied to happy / alex versions?

I've seen incompatible haddock binaries break builds. I am very impatient with unnecessary build breakage.

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2016

Not certain what you mean. Users of hvr's PPA usually just pick a version and get on with it.

It seems that some users, such as @simonmar do a global install rather than that approach. Their usecase is quite valid, though indeed it may open a can of worms, I'm not certain.

I've seen incompatible haddock binaries break builds. I am very impatient with unnecessary build breakage.

Agreed. You will be entirely safe from this if you have the appropriate GHC installed by stack. I'd recommend not even having a global GHC install.

We are definitely planning to make global GHC a non-default thing in the next release - #2537

@bitemyapp

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2016

It seems that some users, such as @simonmar do a global install rather than that approach. Their usecase is quite valid, though indeed it may open a can of worms, I'm not certain.

You mean putting it somewhere everything can find it? Yeah, I do that with hvr's thing via symlinks. I used to even have a script I would pass the path of the install to, and then it would symlink into /usr/bin for me.

Agreed. You will be entirely safe from this if you have the appropriate GHC installed by stack. I'd recommend not even having a global GHC install.

Having bumped into --*-profiling, yeah.

We are definitely planning to make global GHC a non-default thing in the next release - #2537

Very much looking forward to it 👍 :)

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2016

Great!

Note that you can already adopt this default by just putting system-ghc: false in your config.yaml.

@bitemyapp

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2016

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2016

So, @bitemyapp , are you saying that stack exec -- which ghc is reporting a system ghc despite having system-ghc: false? That's very surprising!

@bitemyapp

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2016

No, nothing's wrong, I'm saying I set that configuration quite awhile ago and I think I was one of the earlier people agitating for making system-ghc: false the default (click one of the referenced issues)

just happy it's getting fixed :)

@simonmar

This comment has been minimized.

Copy link

commented Sep 26, 2016

Can we have an option to tell Stack the path of the GHC binary and have Stack call $GHC --version to find the version? That would basically solve all my problems, and seems easy to implement.

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2016

Yes, it is easy to implement @simonmar , just an oversight due to our typical usecases using official GHC versions. No promises, but we are planning to get it into the next release, likely a couple weeks from now if not sooner.

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Mar 17, 2017

@simonmar I've looked into implementing an option like that. It'd actually be a bit tricky to support - lets say the user provides --with-ghc /home/mgsloan/oss/haskell/ghc/inplace/bin/ghc-stage2. Should it assume that ghc-pkg is a sibling?

Another approach I've considered is a --with-compiler-bin (to match stack path --compiler-bin), to specify a dir to find exes like ghc / ghc-pkg. This is nearly equivalent to extending the PATH, though. The only advantage to having an explicit option is that we could check that we're actually using the GHC from that path. Is it worth it?

Perhaps in the meantime it's sufficient to just extend the PATH? If any renaming needs to be done (e.g. ghc-stage2), symbolic links could be used.

@hvr

This comment has been minimized.

Copy link

commented Mar 17, 2017

@mgsloan Note that cabal has been supporting -w SOMEPATH/ghc-stage2 for ages as the primary way to select a GHC instance and uses heuristics to find the matching ghc-pkg so it can't be too hard ;-)

mgsloan added a commit that referenced this issue Mar 28, 2017

@fommil

This comment has been minimized.

Copy link

commented Aug 22, 2018

I maintain all my ghc installs in separate directories and use a shell script to automatically select the correct ghc based on the presence of a .ghc-version file (this is how java and javac is managed in JVM languages, btw).

It would be good if I could provide a lookup map for each ghc that stack expects to find, to the directory where I have installed it. I could workaround the ghc-X.Y.Z approach with symlinks, if needed, but I'd rather not.

I use ghcup now.

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Aug 22, 2018

@mgsloan — is this still relevant? And if so, what would be needed to resolve the issue?

@mgsloan

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 22, 2018

Yes, I believe it is still relevant. It's too bad that it didn't get resolved previously. I think there's two things:

  1. Allow indicating a path to a GHC - something like cabal's -w option described by @hvr. Would also need to resolve the path to associated tools such as ghc-pkg.

  2. Finding binaries called things like ghc-7.10.3. I don't know the details of this, I never use globally installed GHCs so ¯\_(ツ)_/¯

I have been doing a bit of work on GHC lately, so it's on my todo list to make it easier to use stack with a dev version of GHC. Of course, I encourage anyone else to work on that. Not sure when I'd get around to it.

snoyberg added a commit that referenced this issue Mar 27, 2019

Find GHC on PATH via version suffixes (fixes #2433)
If you enable system-ghc, and have multiple versions of GHC available,
Stack will now find the appropriate one based on the version-suffixed
name.

Note: the code in Stack.Setup is _really_ difficult to follow at this
point. This PR makes the situation a bit worse, to avoid making this
diff larger than it has to be. A later PR to clean up Stack.Setup would
be a great idea.

snoyberg added a commit that referenced this issue Mar 27, 2019

Find GHC on PATH via version suffixes (fixes #2433)
If you enable system-ghc, and have multiple versions of GHC available,
Stack will now find the appropriate one based on the version-suffixed
name.

Note: the code in Stack.Setup is _really_ difficult to follow at this
point. This PR makes the situation a bit worse, to avoid making this
diff larger than it has to be. A later PR to clean up Stack.Setup would
be a great idea.
@snoyberg

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

I've opened up #4668 which addresses this. I've included an integration test and tested it manually on my machine, but it would be great if someone who has the use case described here could test out that PR.

@ygale

This comment has been minimized.

Copy link

commented Mar 27, 2019

In my opinion option 1 of @mgsloan is the more generally useful solution. That is #3931. Option 2 might be helpful for some people who don't mind playing havoc with their PATH. #4668 only addresses option 2.

@snoyberg

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2019

I'm honestly not following the use case for option 1. If you want to manually state which GHC to use on each invocation, you can call Stack with PATH=$GHCDIR:$PATH stack .... The feature in Stack that we're talking about is auto-discovery of the correct GHC location.

Maybe I'm missing something fundamental here.

@Blaisorblade

This comment has been minimized.

Copy link
Collaborator

commented Mar 28, 2019

@snoyberg Well, @ygale objected elsewhere to modifying PATH, but their arguments mystify me.
I imagine maybe beginners would struggle with modifying PATH correctly, but we already have a base setup for beginners, so that doesn't seem to be a problem.

@snoyberg

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2019

Thanks for clarifying @Blaisorblade!

@snoyberg snoyberg closed this in 32eb609 Mar 29, 2019

snoyberg added a commit that referenced this issue Mar 29, 2019

Merge pull request #4668 from commercialhaskell/2433-exe-by-version
Find GHC on PATH via version suffixes (fixes #2433)
@simonmar

This comment has been minimized.

Copy link

commented Apr 1, 2019

Just to note, I think I also asked for option (1) a while ago to support using stack to test locally-built versions of GHC. As far as I'm aware that's not possible right now.

snoyberg added a commit that referenced this issue Apr 4, 2019

A number of integration test improvements
* Use ghc-8.2.2 for init
* Only require GHC 8.2.2 for the #2433 integration test
* More reliable bad-bounds test using local file
* Skip building for no init
* Skip a test on Windows (fails due to bounds issues)

@snoyberg snoyberg referenced this issue Apr 4, 2019

Merged

A number of integration test improvements #4696

0 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.