Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find system ghc via binary names like "ghc-7.10.3"? #2433

Closed
mgsloan opened this issue Aug 3, 2016 · 29 comments
Closed

Find system ghc via binary names like "ghc-7.10.3"? #2433

mgsloan opened this issue Aug 3, 2016 · 29 comments

Comments

@mgsloan
Copy link
Contributor

@mgsloan mgsloan commented Aug 3, 2016

In #2221, we're planning to switch away from using system ghc by default. We could consider trusting that the correct version is installed and skipping the version check.

It turns out that for non system ghcs it is also potentially possible to skip the version check. Someone on IRC mentioned that binaries named things like ghc-7.10.3 exist, and that stack should use these. Skipping the check should save a bit of execution time, potentially something like a tenth of a second.

This would mean not gracefully handling cases like #2430 . One option would be to perform such sanity checks after a failing build, if the Cabal errors for these cases are insufficient. This way, the user gets feedback quickly. For failing builds, it will wait a bit longer before exiting, to run the sanity checks.

@Blaisorblade
Copy link
Collaborator

@Blaisorblade Blaisorblade commented Aug 9, 2016

I think calling ghc-X.X.X, for all GHCs would be huge and would allow having multiple GHCs on the paths; this would make life simpler for all users with trouble with bindists. Right now you can put a GHC path in the stack.yaml, but the result can't be distributed to users typically, since paths are valid locally (except for some scenarios using Docker). Supporting multiple system GHCs, otherwise, would require reading from stack's config a mapping from GHC version to path.

I admit I'm fuzzy on what exactly you have in mind (see below). Interpreting rather liberally, I imagine a few parts to this:

  • call ghc-X.X.X instead of ghc everywhere;
  • same for ghcjs assuming it works (I guess so but I'm not sure);
  • add sanity checks in case of error (maybe just ghc --version, possibly also building and running a minimal program?);
  • also revise sanity checks after GHC setup;
  • remove version check at the start of a build.

On the rest, I'm confused (which is why I didn't answer a week ago). Could you elaborate?

I'm less sure about the changes you propose to the version check, probably because I didn't finish reverse-engineering how it works. In particular, we often don't even call ghc --version but ghc --info. Also, why do we have a version for non system GHCs at all?

In #2221, we're planning to switch away from using system ghc by default. We could consider trusting that the correct version is installed and skipping the version check.

I'm more confused by the link with #2221. Doesn't the version check only make sense for the system GHC? And then that could be dropped by using ghc-X.X.X when looking for a system GHC.

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Aug 9, 2016

same for ghcjs assuming it works (I guess so but I'm not sure);

It works less well for ghcjs right now, because there are a lot of different tarballs all with version 0.2.0. Most tarballs have dates in their name, but many don't have the version number set to something other than 0.2.0.

I'm less sure about the changes you propose to the version check, probably because I didn't finish reverse-engineering how it works. In particular, we often don't even call ghc --version but ghc --info.

Good point, I haven't looked deeply at those details. Could be we can't avoid that. We could cache it?

Also, why do we have a version for non system GHCs at all?

Not sure, I think because it's a mild sanity check, shouldn't be needed though.

I'm more confused by the link with #2221. Doesn't the version check only make sense for the system GHC? And then that could be dropped by using ghc-X.X.X when looking for a system GHC.

If we don't change the default then we'll always have to check the system ghc version.

@Blaisorblade
Copy link
Collaborator

@Blaisorblade Blaisorblade commented Aug 9, 2016

If we don't change the default then we'll always have to check the system ghc version.

I think that even if you change the default you'll need some check.
Say I reenable system GHC—and I do that by setting system-ghc: true globally.
That doesn't mean you can use my system GHC for all resolvers, so you still need to learn which is the version of the system GHC.

OTOH, I understand all GHC bindists also provide binaries named ghc-X.X.X, not just the one stack installs. I propose to rely on this instead of the version check, by default. Of course this changes the logic from

  • learn GHC's version and match it with what I need
    to
  • look for a system GHC of the version I need.
    I assume this is still doable, but might be more invasive.

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Aug 9, 2016

Say I reenable system GHC—and I do that by setting system-ghc: true globally.
That doesn't mean you can use my system GHC for all resolvers, so you still need to learn which is the version of the system GHC.

True!

OTOH, I understand all GHC bindists also provide binaries named ghc-X.X.X, not just the one stack installs. I propose to rely on this instead of the version check, by default.

Yeah, maybe we can just go ahead and always trust it for ghc. In theory the logic change shouldn't affect users

@bitemyapp
Copy link
Contributor

@bitemyapp bitemyapp commented Sep 13, 2016

@mgsloan how would this work for the various tools installed with GHC, do they all get suffixed? Some builds are going to fail if not using the associated tool matched to the GHC version.

@Blaisorblade
Copy link
Collaborator

@Blaisorblade Blaisorblade commented Sep 14, 2016

Thanks for pointing that out! We should surely use the same version for any invocation of ghc-pkg, runghc and haddock-ghc. Do we need to account for other tools? I'm guessing no for alex and happy (whose versions should be controlled otherwise) but I'm no expert there.

$ ls /Applications/ghc-7.8.4.app/Contents/bin/
cabal  ghc  ghc-7.8.4  ghc-pkg  ghc-pkg-7.8.4  ghci  ghci-7.8.4  haddock  haddock-ghc-7.8.4  hp2ps  hpc  hsc2hs  runghc  runghc-7.8.4  runhaskell
$ ls /usr/local/Cellar/ghc/8.0.1/bin/
ghc  ghc-8.0.1  ghc-pkg  ghc-pkg-8.0.1  ghci  ghci-8.0.1  haddock  haddock-ghc-8.0.1  hp2ps  hpc  hsc2hs  runghc  runghc-8.0.1  runhaskell

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Sep 26, 2016

That is indeed a good point! That makes this a bit trickier, but still a good change I hope.

@bitemyapp
Copy link
Contributor

@bitemyapp bitemyapp commented Sep 26, 2016

@mgsloan @Blaisorblade not to derail the intent of the thread, but wouldn't inter-operability with tools expecting ordinary names for these utilities be better served by doing the paths like @hvr's GHC distributions? It puts the version in a subdirectory of a consistent path (/opt/...), you can then use the path to switch between them and regular tooling "just works"

This is roughly what rustup does (path blipping) as well.

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Sep 26, 2016

@bitemyapp Certainly! However, that does not work with standard global installations of GHC. Are they so tied to happy / alex versions?

Stack implements its own approach for switching sub-directories, which is why we haven't had logic like this.

@bitemyapp
Copy link
Contributor

@bitemyapp bitemyapp commented Sep 26, 2016

However, that does not work with standard global installations of GHC.

Not certain what you mean. Users of hvr's PPA usually just pick a version and get on with it.

Are they so tied to happy / alex versions?

I've seen incompatible haddock binaries break builds. I am very impatient with unnecessary build breakage.

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Sep 26, 2016

Not certain what you mean. Users of hvr's PPA usually just pick a version and get on with it.

It seems that some users, such as @simonmar do a global install rather than that approach. Their usecase is quite valid, though indeed it may open a can of worms, I'm not certain.

I've seen incompatible haddock binaries break builds. I am very impatient with unnecessary build breakage.

Agreed. You will be entirely safe from this if you have the appropriate GHC installed by stack. I'd recommend not even having a global GHC install.

We are definitely planning to make global GHC a non-default thing in the next release - #2537

@bitemyapp
Copy link
Contributor

@bitemyapp bitemyapp commented Sep 26, 2016

It seems that some users, such as @simonmar do a global install rather than that approach. Their usecase is quite valid, though indeed it may open a can of worms, I'm not certain.

You mean putting it somewhere everything can find it? Yeah, I do that with hvr's thing via symlinks. I used to even have a script I would pass the path of the install to, and then it would symlink into /usr/bin for me.

Agreed. You will be entirely safe from this if you have the appropriate GHC installed by stack. I'd recommend not even having a global GHC install.

Having bumped into --*-profiling, yeah.

We are definitely planning to make global GHC a non-default thing in the next release - #2537

Very much looking forward to it 👍 :)

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Sep 26, 2016

Great!

Note that you can already adopt this default by just putting system-ghc: false in your config.yaml.

@bitemyapp
Copy link
Contributor

@bitemyapp bitemyapp commented Sep 26, 2016

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Sep 26, 2016

So, @bitemyapp , are you saying that stack exec -- which ghc is reporting a system ghc despite having system-ghc: false? That's very surprising!

@bitemyapp
Copy link
Contributor

@bitemyapp bitemyapp commented Sep 26, 2016

No, nothing's wrong, I'm saying I set that configuration quite awhile ago and I think I was one of the earlier people agitating for making system-ghc: false the default (click one of the referenced issues)

just happy it's getting fixed :)

@simonmar
Copy link

@simonmar simonmar commented Sep 26, 2016

Can we have an option to tell Stack the path of the GHC binary and have Stack call $GHC --version to find the version? That would basically solve all my problems, and seems easy to implement.

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Sep 26, 2016

Yes, it is easy to implement @simonmar , just an oversight due to our typical usecases using official GHC versions. No promises, but we are planning to get it into the next release, likely a couple weeks from now if not sooner.

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Mar 17, 2017

@simonmar I've looked into implementing an option like that. It'd actually be a bit tricky to support - lets say the user provides --with-ghc /home/mgsloan/oss/haskell/ghc/inplace/bin/ghc-stage2. Should it assume that ghc-pkg is a sibling?

Another approach I've considered is a --with-compiler-bin (to match stack path --compiler-bin), to specify a dir to find exes like ghc / ghc-pkg. This is nearly equivalent to extending the PATH, though. The only advantage to having an explicit option is that we could check that we're actually using the GHC from that path. Is it worth it?

Perhaps in the meantime it's sufficient to just extend the PATH? If any renaming needs to be done (e.g. ghc-stage2), symbolic links could be used.

@hvr
Copy link

@hvr hvr commented Mar 17, 2017

@mgsloan Note that cabal has been supporting -w SOMEPATH/ghc-stage2 for ages as the primary way to select a GHC instance and uses heuristics to find the matching ghc-pkg so it can't be too hard ;-)

mgsloan added a commit that referenced this issue Mar 28, 2017
@fommil
Copy link

@fommil fommil commented Aug 22, 2018

I maintain all my ghc installs in separate directories and use a shell script to automatically select the correct ghc based on the presence of a .ghc-version file (this is how java and javac is managed in JVM languages, btw).

It would be good if I could provide a lookup map for each ghc that stack expects to find, to the directory where I have installed it. I could workaround the ghc-X.Y.Z approach with symlinks, if needed, but I'd rather not.

I use ghcup now.

@dbaynard
Copy link
Contributor

@dbaynard dbaynard commented Aug 22, 2018

@mgsloan — is this still relevant? And if so, what would be needed to resolve the issue?

@mgsloan
Copy link
Contributor Author

@mgsloan mgsloan commented Aug 22, 2018

Yes, I believe it is still relevant. It's too bad that it didn't get resolved previously. I think there's two things:

  1. Allow indicating a path to a GHC - something like cabal's -w option described by @hvr. Would also need to resolve the path to associated tools such as ghc-pkg.

  2. Finding binaries called things like ghc-7.10.3. I don't know the details of this, I never use globally installed GHCs so ¯\_(ツ)_/¯

I have been doing a bit of work on GHC lately, so it's on my todo list to make it easier to use stack with a dev version of GHC. Of course, I encourage anyone else to work on that. Not sure when I'd get around to it.

snoyberg added a commit that referenced this issue Mar 27, 2019
If you enable system-ghc, and have multiple versions of GHC available,
Stack will now find the appropriate one based on the version-suffixed
name.

Note: the code in Stack.Setup is _really_ difficult to follow at this
point. This PR makes the situation a bit worse, to avoid making this
diff larger than it has to be. A later PR to clean up Stack.Setup would
be a great idea.
snoyberg added a commit that referenced this issue Mar 27, 2019
If you enable system-ghc, and have multiple versions of GHC available,
Stack will now find the appropriate one based on the version-suffixed
name.

Note: the code in Stack.Setup is _really_ difficult to follow at this
point. This PR makes the situation a bit worse, to avoid making this
diff larger than it has to be. A later PR to clean up Stack.Setup would
be a great idea.
@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Mar 27, 2019

I've opened up #4668 which addresses this. I've included an integration test and tested it manually on my machine, but it would be great if someone who has the use case described here could test out that PR.

@ygale
Copy link

@ygale ygale commented Mar 27, 2019

In my opinion option 1 of @mgsloan is the more generally useful solution. That is #3931. Option 2 might be helpful for some people who don't mind playing havoc with their PATH. #4668 only addresses option 2.

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Mar 28, 2019

I'm honestly not following the use case for option 1. If you want to manually state which GHC to use on each invocation, you can call Stack with PATH=$GHCDIR:$PATH stack .... The feature in Stack that we're talking about is auto-discovery of the correct GHC location.

Maybe I'm missing something fundamental here.

@Blaisorblade
Copy link
Collaborator

@Blaisorblade Blaisorblade commented Mar 28, 2019

@snoyberg Well, @ygale objected elsewhere to modifying PATH, but their arguments mystify me.
I imagine maybe beginners would struggle with modifying PATH correctly, but we already have a base setup for beginners, so that doesn't seem to be a problem.

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Mar 28, 2019

Thanks for clarifying @Blaisorblade!

@snoyberg snoyberg closed this in 32eb609 Mar 29, 2019
snoyberg added a commit that referenced this issue Mar 29, 2019
Find GHC on PATH via version suffixes (fixes #2433)
@simonmar
Copy link

@simonmar simonmar commented Apr 1, 2019

Just to note, I think I also asked for option (1) a while ago to support using stack to test locally-built versions of GHC. As far as I'm aware that's not possible right now.

snoyberg added a commit that referenced this issue Apr 4, 2019
* Use ghc-8.2.2 for init
* Only require GHC 8.2.2 for the #2433 integration test
* More reliable bad-bounds test using local file
* Skip building for no init
* Skip a test on Windows (fails due to bounds issues)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Performance improvements
  
Awaiting triage
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants