Cabal cradle causes me to constantly need to run cabal configure

[ocharles]

I'm using ghcide 0.2 with Cabal cradles. I now seem to constantly having to do a Cabal configure everytime I do cabal build, because opening new files in causes ghcide to run Cabal configure through hie-bios. The configuration phase takes a substantial amount of time to run (we're talking minutes), so I now seem to spend most of my time watching Cabal configure itself. I imagine this is because hie-bios uses --with-compiler which requires a full recnofiguration. Could this be done in such a way that it doesn't disturb what I do in my working directory?

[fendor]

@ocharles for now, probably not. cabal needs to give us the information necessary to compile a given file. What we need is this pr: haskell/cabal#6241
Once this is merged, we can improve this workflow quite tremendously.

Shouldnt the project only be re-configured once per component?

[lukel97]

I don't think hie-bios currently passes --with-compiler does it? It should be reusing whatever cabal repl would use at the moment. But with that said, I've also been experiencing random re-configuring, and I have a feeling that some files inside dist-newstyle are getting touched thats causing cabal to freak out.

Are there any specific bits of extra information you had in mind for show-build-info @fendor ?

[jneira]

It does:

https://github.com/mpickering/hie-bios/blob/293baaf793f82f639affe734ebe785ccdcafd40b/src/HIE/Bios/Cradle.hs#L439

i didnt know that it could trigger a configure though

[fendor]

@bubba As @jneira wrote, we collect options via cabal repl -w wrapper where our wrapper writes the results into the file $HIE_BIOS_OUTPUT points to. With s-b-i we can use the cabal interface straight away without any hacks. So, implementing proper support is just the general solution to avoid unnecessary rebuilds.

[maksbotan]

This was also discussed here: #177 (comment)

Looking into this is in my long-term list, but hopefully someone from the project itself will fix the problem in a nice way :)

[ocharles]

Shouldnt the project only be re-configured once per component?

Maybe, but this is certainly not happening. In just editing a single file, it seems to repeatedly clobber me running cabal build. I seem to make one change, cabal build and have to reconfigure, make another change and then have to configure again! Occasionally the configure fails, crashing that some /tmp/hie-bios file is missing, so there's also a race condition there.

[maksbotan]

@ocharles that is exactly the problem I reported in #177 (comment).

The problem is that hie-bios calls cabal v2-repl --with-compiler /tmp/some-hie-wrapper to get flags, component dir and stuff. And cabal itself will "remember" this --with-compiler somewhere (cabal.project.local?) and when you do cabal new-build yourself, it will re-configure because --with-compiler "changed", as you did not specify it on the command line.

[maksbotan]

@fendor @jneira I copied dist-newstyle after cabal new-build servant-server:test:spec and then did hie-bios check on the same component. Here's a diff between two dirs:

$ diff -urN dist-with-cabal/ dist-newstyle/
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-0.17/noopt/cache/config and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-0.17/noopt/cache/config differ
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-0.17/noopt/setup-config and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-0.17/noopt/setup-config differ
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/noopt/cache/build and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/noopt/cache/build differ
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/noopt/cache/config and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/noopt/cache/config differ
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/noopt/setup-config and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/noopt/setup-config differ
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/cache/config and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/cache/config differ
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/cache/registration and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/cache/registration differ
Binary files dist-with-cabal/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/setup-config and dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/setup-config differ
Binary files dist-with-cabal/cache/compiler and dist-newstyle/cache/compiler differ
Binary files dist-with-cabal/cache/config and dist-newstyle/cache/config differ
Binary files dist-with-cabal/cache/elaborated-plan and dist-newstyle/cache/elaborated-plan differ
Binary files dist-with-cabal/cache/improved-plan and dist-newstyle/cache/improved-plan differ

Apparantly, cabal stores compiler path in a binary format in a lot of config files, so when a user calls cabal new-build it will detect that new compiler setting (the empty one) does not match the one baked into those files.

What hie-bios could do is to copy dist-newstyle dir and run cabal v2-repl with --dist flag, but that is too overkill IMHO. A linux-specific think would be to mount something like AUFS with copy-on-write mode on top of dist-newstyle, so that hie-bios's cabal run does not overwrite user's files.

[maksbotan]

This trick demonstrates my words (of course it needs bracket):

diff --git a/hie-bios.cabal b/hie-bios.cabal
index f2640f18..0c7c67a1 100644
--- a/hie-bios.cabal
+++ b/hie-bios.cabal
@@ -162,7 +162,8 @@ Library
                         hslogger             >= 1.2 && < 1.4,
                         file-embed           >= 0.0.11 && < 1,
                         conduit              >= 1.3 && < 2,
-                        conduit-extra        >= 1.3 && < 2
+                        conduit-extra        >= 1.3 && < 2,
+                        pathwalk
 
 
 Executable hie-bios
diff --git a/src/HIE/Bios/Cradle.hs b/src/HIE/Bios/Cradle.hs
index c248c605..7f99848e 100644
--- a/src/HIE/Bios/Cradle.hs
+++ b/src/HIE/Bios/Cradle.hs
@@ -42,6 +42,7 @@ import System.IO.Temp
 import System.IO.Error (isPermissionError)
 import Data.List
 import Data.Ord (Down(..))
+import System.Directory.PathWalk
 
 import System.PosixCompat.Files
 import HIE.Bios.Wrappers
@@ -436,16 +437,29 @@ withCabalWrapperTool (ghcPath, ghcArgs) wdir k = do
                         hPutStr h cabalWrapper
                         hClose h
                         setMode loc
-                        k loc)
+                        pathWalk (wdir </> "dist-newstyle") $ \root _ files ->
+                          forM_ files $ \file ->
+                            when (file `elem` fnames) $ do
+                              print ("backing up ", root </> file)
+                              copyFileWithMetadata (root </> file) (root </> file <.> "bak")
+                        r <- k loc
+                        pathWalk (wdir </> "dist-newstyle") $ \root _ files ->
+                          forM_ files $ \file ->
+                            when (file `elem` fnames) $ do
+                              print ("restoring ", root </> file)
+                              copyFileWithMetadata (root </> file <.> "bak") (root </> file)
+                        return r)
 
   where
     setMode wrapper_fp = setFileMode wrapper_fp accessModes
+    fnames = ["config", "compiler-config", "improved-plan", "elaborated-plan", "setup-config", "solver-plan", "build"]

With this patch running cabal new-build after hie-bios check won't reconfigure anything.

[jneira]

That could be acomplished making the ide build use another dist directory.
That is an option that has been presented in several discussion about, although it has its own drawbacks.

[jneira]

That could be acomplished making the ide build use another dist directory.

Now config allows you to set an alternative stack.yaml for stack cradle. If we add support for an alternative cabal project file, we could have a workaround for both build tools, as you can change the build dir using those config files.