Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reinstall of package breaks things #6391

Open
yaxu opened this issue Nov 30, 2019 · 18 comments
Open

Reinstall of package breaks things #6391

yaxu opened this issue Nov 30, 2019 · 18 comments

Comments

@yaxu
Copy link

@yaxu yaxu commented Nov 30, 2019

If I change then install a package I'm developing, ghci can no longer use it, it seems because there are two installations with the same version number. Transcript below. This is with the a fresh install via ghcup under linux.

Also, I don't know why I now have to specify --lib when presumably the module knows it's a library. As a general comment, documentation for v2-install doesn't seem up to date and I can't find clear explanations of the changes and how they can help my project

alex@stanage:~/tmp$ git clone https://github.com/tidalcycles/tidal.git
Cloning into 'tidal'...
remote: Enumerating objects: 125, done.
remote: Counting objects: 100% (125/125), done.
remote: Compressing objects: 100% (76/76), done.
remote: Total 9909 (delta 47), reused 93 (delta 32), pack-reused 9784
Receiving objects: 100% (9909/9909), 6.65 MiB | 1.54 MiB/s, done.
Resolving deltas: 100% (5383/5383), done.
alex@stanage:~/tmp$ cd tidal
alex@stanage:~/tmp/tidal$ cabal install --lib
Wrote tarball sdist to
/home/alex/tmp/tidal/dist-newstyle/sdist/tidal-1.4.6.tar.gz
Wrote tarball sdist to
/home/alex/tmp/tidal/dist-newstyle/sdist/tidal-parse-0.0.1.tar.gz
Resolving dependencies...
Up to date
alex@stanage:~/tmp/tidal$ echo "" >> src/Sound/Tidal/Pattern.hs 
alex@stanage:~/tmp/tidal$ cabal install --lib
Wrote tarball sdist to
/home/alex/tmp/tidal/dist-newstyle/sdist/tidal-1.4.6.tar.gz
Wrote tarball sdist to
/home/alex/tmp/tidal/dist-newstyle/sdist/tidal-parse-0.0.1.tar.gz
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - tidal-1.4.6 (lib) (requires build)
Starting     tidal-1.4.6 (lib)
Building     tidal-1.4.6 (lib)
Installing   tidal-1.4.6 (lib)
Completed    tidal-1.4.6 (lib)
alex@stanage:~/tmp/tidal$ ghci
GHCi, version 8.6.5: http://www.haskell.org/ghc/  :? for help
Loaded package environment from /home/alex/.ghc/x86_64-linux-8.6.5/environments/default
Prelude> import Sound.Tidal.Context 

<no location info>: error:
    Ambiguous module name ‘Sound.Tidal.Context’:
      it was found in multiple packages: tidal-1.4.6 tidal-1.4.6
Prelude> 
Leaving GHCi.
alex@stanage:~/tmp/tidal$ grep tidal ~/.ghc/x86_64-linux-8.6.5/environments/default 
package-id tidal-1.4.6-29102d7626eceae1acf25d4320134825bb64112fb11058d394e0d55078882cde
package-id tidal-1.4.6-53f34a7fc0ce460f98fce9b704f6ab422c96435b533e18602b8728ba5e16553e
@yaxu
Copy link
Author

@yaxu yaxu commented Apr 5, 2020

This bug is still present - I spend time helping people work around this on a daily basis.

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 6, 2020

Hey @yaxu, I should have some time in the next couple of days to investigate this. I'm not familiar with this part of the code (@phadej could you give me some code pointers?). It sounds like install should check the environment for an existing package with the same version (but different hash) and:

a) Fail installation instead of happily installing something which will result in an error later
b) Support overwriting the existing package (with a --force option)

Also, I don't know why I now have to specify --lib when presumably the module knows it's a library.

There's discussion on splitting install --lib into its own command since its different in many ways (see #6481 for discussion).

As a general comment, documentation for v2-install doesn't seem up to date...

Yeah...it's not 🙃 I'm planning to get around to updating the docs soon.

@ertyseidohl
Copy link

@ertyseidohl ertyseidohl commented Apr 28, 2020

Hi - Just want to +1 this bug and ask if there's documentation anywhere about how to get around it.

What's most frustrating to me about this bug is that it relies on global state - I messed up my installations somewhere and reverting my git repo back to my last commit didn't fix things.

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 28, 2020

Hey @ertyseidohl I'm actually actively working on this now, I'm hoping to have a PR out for review this week.

In the meantime, the workaround is to manually edit the GHC env file and remove the entry (or entries) of the libraries you're re-installing before running the install command. For example, say you have previously installed your-library-that-you-want-to-reinstall to the default environment, you will currently have something that looks like this:

$ cat ~/.ghc/x86_64-linux-8.8.3/environments/default 
clear-package-db
global-package-db
package-db /home/mrenaud/.cabal/store/ghc-8.8.3/package.db
package-id ghc-8.8.3
...
<global packages>
...
package-id text-1.2.4.0
package-id your-library-that-you-want-to-reinstall-0.1.2.3-longhashvalue

If you manually edit this file to remove the entry for your-library-that-you-want-to-reinstall before running cabal install --lib then that should work just fine. Let me know if that's not the case.

@yaxu
Copy link
Author

@yaxu yaxu commented Apr 28, 2020

Thanks for looking at this. I've helped many people with broken cabal installs lately, and I'm also seeing this problem with duplicate versions for people who are installing straight from hackage, so presumably it's possible to install the same package version twice even if they have the same hash.
I'm also seeing people who have upgraded the tidal package but ghci is still loading the old one.
The only reliable remedy seems to be removing the environment file, and most reliably the whole of .ghc and .cabal, and installing with v1-install.
Just anecdotes, as I rarely have direct access to the machines.

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 28, 2020

@yaxu do you happen to have a repro case when installing directly from hackage? If the package has the same hash there shouldn't be any modifications made to the env file.

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 28, 2020

Also @yaxu, this is a bit of a meta-comment/question, but I just wanted to make sure that your use case actually requires modifying the global package environment; apologies if you're already aware of this and have determined it doesn't work for you.(this goes back to the v1->v2 behaviour change and workflows is not well documented).

With the v1- commands, in order to use a library in a package/project you're developing you would need to v1-install it to the global package DB and then you would have access to it in your project. With the v2- commands, cabal handles downloading and "installing" libraries into a local environment to your project is handled automatically, as long as you have them listed in the build-depends attribute. So for the ghci above, assuming you are in the tidal project you should just be able to run cabal repl :tidal and it will make all of the modules within the tidal library available to the repl.

@yaxu
Copy link
Author

@yaxu yaxu commented Apr 28, 2020

Thanks @m-renaud, I'd love to understand this more!

The use case I have is supporting a library (tidal) for end-user programmers, in this case musicians who are generally unfamiliar with Haskell and its build systems. The don't want to deal with git repos, or develop any code, they just want to install the tidal library according to the easiest and most reliable method, and use it in the ghci repl (via an editor plugin).

As I understand it there isn't really a global environment with v2- commands, just a 'default' one, is that right? So I assume any problem I'm having with the default package environment would still be a problem if I used another named environment.

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 28, 2020

Gotcha, thanks for the context. Yeah it sounds like using a global environment is probably the best way forward, and the root of the issue is when you make changes to tidal locally and re-install it corrupts the env file. You should soon be able to run:

cabal install --lib --force --package-env=ENV

which would overwrite any existing packages referenced in ENV with the newer versions.

@phadej
Copy link
Collaborator

@phadej phadej commented Apr 28, 2020

I’m sorry to interrupt but I veto any cli changes to cabal install —lib interface until there is a clear plan for the final UI. I have opened various issues and proposed various solutions. The problem is not implementing the changes, but to agree how package environment management should look like.

That said —force is a bad argument name for "don’t consider current environment contents”.

@yaxu
Copy link
Author

@yaxu yaxu commented Apr 28, 2020

Hi @phadej, if the UI isn't finalised, to the extent that bugs like this aren't able to be fixed, why is this already in production? I've found the transition to v2-install a bumpy ride to say the least. I've been spending a hours a week recently helping people with install problems.

@phadej
Copy link
Collaborator

@phadej phadej commented Apr 28, 2020

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 28, 2020

I've got a pretty decent handle on the install --lib code now, I was actually consider doing a pretty substantial prefactor/refactoring before making the change since everything is pretty tangled together at the moment. If desired I can send out a PR for that.

I can start with that and then we can start sketching out what a cabal env API would look like (I have some ideas, where's the best place to put those?).

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 28, 2020

Also, is there a good real-time place to chat about this? #cabal irc? Going back and fourth on comments works for some things but getting on the same page is sometimes easier and more expedient with an "in-person" discussion.

@ertyseidohl
Copy link

@ertyseidohl ertyseidohl commented Apr 29, 2020

If it helps, I've written up my process with all of the errors and fixes at https://organicdonut.com/2020/04/cabal-package-installation-woes/

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 29, 2020

Hey @ertyseidohl I just took a look at your post and it doesn't appear that you need to use cabal install at all. If you are developing an executable that depends on libraries from hackage, you don't need to install the libraries, as long as your .cabal file declares the necessary build-depends all you need to do to build and run your executable is run cabal run :server.

Do you have the full code that I could look at to confirm? Also, what version of cabal are you using?

@ertyseidohl
Copy link

@ertyseidohl ertyseidohl commented Apr 29, 2020

I think part of my problem is just being new to haskell and not understanding what I'm doing with the package management :)

My code (at the time of writing) is here: https://github.com/ertyseidohl/crossword-hs/tree/072b4efa11346dc12b20835d09d4eb7559c13a36

@m-renaud
Copy link
Collaborator

@m-renaud m-renaud commented Apr 29, 2020

Ahh yeah there was a time when you needed to install libraries to the global package namespace to use them but that hasn't been the case for quite some time (although there are random docs scattered across the internet that still say you need to do that). The best resource for getting started is the Getting Started guide in the user docs. If you have any issues following those instructions please file an issue about it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

4 participants