Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`stack clean` fails due to missing ghc-pkg (on latest from master) #4480

Closed
silky opened this issue Jan 3, 2019 · 17 comments

Comments

Projects
None yet
7 participants
@silky
Copy link
Contributor

commented Jan 3, 2019

looks suspiciously related to #4466

to repro

  1. use stack from 5144dd5
  2. go into a project
  3. run stack clean

error:

> stack clean
Executable named ghc-pkg not found on path: [...]

version

11:21 AM noon ∈ Moon-Tile (master*) ♫ stack --version
Version 1.10.0, Git revision 5144dd57bd78553f5051601295a2eb215d1db5a6 x86_64 hpack-0.31.1

@silky silky changed the title `stack clean` fails due to missing ghc-pkg `stack clean` fails due to missing ghc-pkg (on latest from master) Jan 3, 2019

@qrilka qrilka self-assigned this Jan 3, 2019

@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

commented Jan 3, 2019

Confirmed that I can reproduce this

@qrilka

This comment has been minimized.

Copy link
Contributor

commented Jan 4, 2019

It appears to be a bit of a clash between #4412 and #4268
To fix this properly I guess we need some good way to solve #4390 as after #4412 the most of the commands became more dependent on GHC being available

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2019

Fix is probably to change the behaviour of stack clean when GHC is not present. #4466 is probably a coincidence.

We don't have EnvConfig without GHC present, and stack clean currently relies on EnvConfig in order to identify the Cabal library present (there's a documentation PR about these precise details following a conversation with @snoyberg).

The sensible thing would be:

  • if there's only one Cabal directory in dist then use that as the target (as now)
  • if there are multiple Cabal directories, either
    • prompt the user for which to delete, or
    • prompt the user to install ghc, or
    • select a dist directory to clean (unexpected behaviour), or
    • download GHC (undesirable and unexpected behaviour), or
    • delete multiple dist directories (probably undesirable behaviour), or
    • present an error, and introduce flags to control behaviour (seems unnecessarily complicated).

I'd favour the prompts but that leads to issues in scripts.

For now, does running stack setup work around the issue?

@mihaimaruseac

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2019

Prompts with a default and flags for the other would help the scripts.

@qrilka

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2019

@dbaynard I don't understand why do you say that deleting the only one Cabal directory in dist is the way stack clean currently works, see e.g.:

qrilka@qdesktop ~/ws/h/stack-build-test $ ls -l $(dirname `stack path --dist-dir`)
итого 8
drwxr-xr-x 5 qrilka qrilka 4096 янв  7 21:18 Cabal-2.0.1.0
drwxr-xr-x 5 qrilka qrilka 4096 янв  7 21:17 Cabal-2.4.0.1
qrilka@qdesktop ~/ws/h/stack-build-test $ stack --resolver=lts-12.13 clean
qrilka@qdesktop ~/ws/h/stack-build-test $ ls -l $(dirname `stack path --dist-dir`)
итого 8
drwxr-xr-x 5 qrilka qrilka 4096 янв  7 21:18 Cabal-2.0.1.0
drwxr-xr-x 5 qrilka qrilka 4096 янв  7 21:17 Cabal-2.4.0.1

Currently stack clean uses for cleaning only Cabal version shipped with GHC, for that we could use global-hints.yaml and that's what I was asking you about in #4268 (comment) - with that we won't need any user prompts or defaults (thought you could name values in global-hints.yaml as defaults of course). My main concern here is about allowing what is described in #4488 but after reading Michael's words in #stack-collaborators discussion it looks like a proper way to go. I've started a refactoring which will split setupEnv into 2 stages and the 1st one will result in SMActual getting constructed and that should be enough for at least stack clean

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2019

Also see #4385 — I thought that I had already merged it (hence some of my confusion). It changes how stack clean works. And I think it gets the documentation correct.

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Jan 7, 2019

I don't understand why do you say that deleting the only one Cabal directory in dist is the way stack clean currently works

Hold on, I'm confused by your example — lts-12.13 uses Cabal 2.2 so that is exactly the output I'd expect from that command.

EDIT:

stack clean only deletes the dist dir corresponding to the selected resolver. But I've no idea whether anybody uses this functionality — in particular as it isn't documented — and so I think we can change the functionality to do something sensible.

> ls $(stack path --dist-dir)(:h)
Cabal-2.2.0.1  Cabal-2.4.0.1
> stack clean --resolver lts-12
Selected resolver: lts-12.26
> ls $(stack path --dist-dir)(:h)
Cabal-2.4.0.1
> stack build --resolver lts-12
Selected resolver: lts-12.26
…
> ls $(stack path --dist-dir)(:h)
Cabal-2.2.0.1  Cabal-2.4.0.1
> stack clean
> ls $(stack --resolver lts-12 path --dist-dir)(:h)
Selected resolver: lts-12.26
Cabal-2.2.0.1
@qrilka

This comment has been minimized.

Copy link
Contributor

commented Jan 8, 2019

Regarding #4385 - I don't quite see what does it change, I only see it adding stack purge as a synonym for stack clean --full.
Regarding Cabal directories - my concern was about your first formulation of stack clean which appears a bit vague to me: you were saying that it deletes a Cabal directory in dist if there exist one such directory and my argument was more about what you are saying in this last comment - it deletes only directory corresponding to the selected GHC and it doesn't depend on number of Cabal directories in dist.
I'm OK with changing it to something sensible though I don't have good proposals what that could be.

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Jan 8, 2019

my concern was about your first formulation of stack clean which appears a bit vague to me: you were saying that it deletes a Cabal directory in dist if there exist one such directory

Ah, I see my ambiguity, now.

if there's only one Cabal directory in dist then use that as the target (as now)

I was incorrect to say stack clean deletes a Cabal directory if there is only one. It only targets that single directory if the global (compiler) Cabal version matches — otherwise it does nothing.

Yes, I thought that if we can't know which Cabal directory to delete, and there is only one directory, we may wish to assume the user wishes to delete whichever directory is there.

it doesn't depend on number of Cabal directories in dist

Yes — this is the current behaviour, where we use ghc-pkg to identify the correct Cabal directory. Without ghc-pkg, this is not possible in general, but we can get close.


In light of this, it may be better not to assume stack clean by default should delete whatever is present, and we should implement a single solution.

I've started a refactoring which will split setupEnv into 2 stages and the 1st one will result in SMActual getting constructed and that should be enough for at least stack clean

Sounds good 👍

@qrilka qrilka referenced this issue Feb 4, 2019

Merged

Globals (GHC boot packages) pruning #4547

1 of 2 tasks complete
@snoyberg

This comment has been minimized.

Copy link
Contributor

commented Feb 6, 2019

But I've no idea whether anybody uses this functionality — in particular as it isn't documented — and so I think we can change the functionality to do something sensible.

I think this is a great point @dbaynard. I'd be fine with massively simplifying the approach to clean to just getting rid of all of the dist directories, or perhaps even just wiping out the .stack-work directories entirely. I'm not aware of any reason why this should cause breakage. Worst case scenario: some corner case depending on undocumented behavior in Stack ends up having a longer recompilation period.

If someone sees something I don't, please let me know.

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Feb 10, 2019

This is odd — I'm writing an integration test for #4385 (I've rebased on to master) and stack clean has changed. Specifically, it no longer identifies the same GHC variant as stack build.

In my test, when I run stack build it creates x86_64-linux-tinfo6 directories inside the .stack-work/dist and .stack-work/install directory. stack clean ought to delete the corresponding directory inside .stack-work/dist. However, instead it adds a directory: .stack-work/install/x86_64-linux.

The clean command outputs this, at the end. It is using the different install directory.

2019-02-10 22:55:57.995962: [debug] Run process: $HOME/.stack/programs/x86_64-linux/ghc-tinfo6-8.6.3/bin/ghc-pkg init stack/test/integration/tests/3863-purge-command/files/.stack-work/install/x86_64-linux/2038573ca7245a0c03b75f394d0498bd6cd9ca70bb5b1d5833750d0911e12d48/8.6.3/pkgdb/
2019-02-10 22:55:58.026367: [debug] Process finished in 30ms: $HOME/.stack/programs/x86_64-linux/ghc-tinfo6-8.6.3/bin/ghc-pkg init stack/test/integration/tests/3863-purge-command/files/.stack-work/install/x86_64-linux/2038573ca7245a0c03b75f394d0498bd6cd9ca70bb5b1d5833750d0911e12d48/8.6.3/pkgdb/

I'm going to update that PR with the broken integration test.

@dbaynard dbaynard referenced this issue Feb 10, 2019

Merged

Make stack clean be more thorough by default #4385

1 of 2 tasks complete
@vanceism7

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2019

Hey so just to clarify with everyone, should we actually change stack clean to do a full clean by default (i.e just delete the .stack-work dir), or are we preferring to keep the purge synonym and have stack clean do the "shallow" clean like it currently does?

@qrilka

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2019

@dbaynard it looks like a side-effect of SkipDownloadCompiler. At the same time Michael's proposal is to just delete dist without bothering with Cabal/GHC versions. It's a bit of a breaking change but it's not clear what's the point of the current "fine-grained" cleaning and how any code could depend on that. And the question looks to be more about what's @vanceism7 is asking - what will be the default behavior of stack clean - only delete dist or just .stack-work altogether?

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2019

Keep the split functionality — it is useful to be able to delete artefacts (in dist) and keep outputs (in install).

So: delete the build artefacts (dist) with stack clean, I guess for all the cabal subdirectories there. Delete all .stack-work directories with stack purge.

How does that sound?

@snoyberg

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2019

That sounds good to me. To confirm: that would mean that the clean command would never need to know anything about GHC or Cabal version, right?

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2019

Yes. It would need to know the .stack-work/dist directory paths for the project (for when people override .stack-work).

@qrilka

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2019

It looks fine to me too, the main point was about not depending on GHC/Cabal.

@snoyberg snoyberg closed this in 0a9aea1 Mar 7, 2019

snoyberg added a commit that referenced this issue Mar 7, 2019

Merge pull request #4607 from commercialhaskell/4480-fix-stack-clean
stack clean does not need ghc-pkg (fixes #4480)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.