Nondeterministic build order makes iterating painful

[robx]

Describe the bug

I seem to run into a sequence like the following frequently:

1. Edit module A
2. Call cabal build, A is rebuilt successfully, then module B is rebuilt because it imports A, and module B fails to build because it needs to be adapted to the change in A.
3. I edit module B to fix that error (or maybe I make a mistake)
4. Call cabal build, module C is rebuilt because it imports A, and only then is module B rebuilt

Sometimes module C will have its own errors, but even just the delay of waiting for C to build before I can see what my change to B did is very disruptive to my edit-compile cycle.

My expectation is that after the following sequence (with nothing else happening inbetween)

1. cabal build fails with an error in module 'B'
2. I edit module 'B'
3. I call cabal build again

that second call to cabal build will pick up exactly where it left off in step 1, that is, try compiling module 'B' again right away.

To Reproduce

A not-quite-minimal example project

diff --git a/app/A.hs b/app/A.hs
new file mode 100644
index 0000000..83a4c9c
--- /dev/null
+++ b/app/A.hs
@@ -0,0 +1,9 @@
+module A where
+
+import Lib
+
+g :: String
+g = f ++ ", ye"
+
+h :: String
+h = g ++ "!"
diff --git a/app/B.hs b/app/B.hs
new file mode 100644
index 0000000..6386bcc
--- /dev/null
+++ b/app/B.hs
@@ -0,0 +1,9 @@
+module B where
+
+import Lib
+
+g :: String
+g = f ++ ", ye"
+
+h :: String
+h = g ++ "!"
diff --git a/app/Lib.hs b/app/Lib.hs
new file mode 100644
index 0000000..90458f6
--- /dev/null
+++ b/app/Lib.hs
@@ -0,0 +1,4 @@
+module Lib where
+
+f :: String
+f = "hello"
diff --git a/app/Main.hs b/app/Main.hs
new file mode 100644
index 0000000..48de6c7
--- /dev/null
+++ b/app/Main.hs
@@ -0,0 +1,7 @@
+module Main where
+
+import qualified A
+import qualified B
+
+main :: IO ()
+main = putStrLn $ A.h ++ B.h
diff --git a/x.cabal b/x.cabal
new file mode 100644
index 0000000..0868c55
--- /dev/null
+++ b/x.cabal
@@ -0,0 +1,18 @@
+cabal-version:      2.4
+name:               x
+version:            0.1.0.0
+
+executable x
+    main-is:          Main.hs
+
+    -- Modules included in this executable, other than Main.
+    other-modules: Lib, A, B
+    build-depends:    base ^>=4.14.3.0
+    hs-source-dirs:   app
+    default-language: Haskell2010

1. cabal v2-build (successful)
2. change type of Lib.f
3. cabal v2-build (fails compiling e.g. A)
4. fix one of the errors in A
5. cabal v2-build (fails compiling B instead of compiling A)

I suspect there's some randomness involved, but in my test right now 5 did the wrong thing quite reliably...

Expected behavior
If step 3 fails with an error in 'A', then any number of cycles of "edit 'A', call 'cabal build'" should keep immediately compiling module 'A' until 'A' is compiled successfully.

System information

• Operating system: macos
• cabal 3.6.2.0, ghc 8.10.7

Additional context

• I imagine this may be something to fix in GHC rather than cabal, feels like it's still an issue to track here?
• It feels like this should be a popular complaint, apologies if I missed a duplicate issue

[Mikolaj]

Yes, that's a terrible emotional roller-coaster. I'm all "woot, the fix to B worked, hurray", I start fixing C and then GHC comes back; "a cute little surprise, B is as broken as it ever was".

[fgaz]

I imagine this may be something to fix in GHC rather than cabal, feels like it's still an issue to track here?

Yes, this is because of ghc. Cabal does not control the order of compilation of modules at all, since it uses ghc --make. There are proposals to have ghc output a dependency graph ( https://gitlab.haskell.org/ghc/ghc/-/wikis/proposals/dependency-tracking-output , ghc-proposals/ghc-proposals#245 and i believe there is another one somewhere), but afaik none was accepted or implemented yet.

Getting a module dependency graph from ghc would also help with parallelism. It'd allow us to implement proper module-level parallelism instead of the current situation where cabal can spawn up to X*Y jobs when using -jX --ghc-options=-jY. I think we have a ticket about that too, but I can't find it

edit: also related: #6658

[AshleyYakeley]

Is there a GHC GitLab issue for this?

[robx]

It seems that there were relevant changes with GHC 9.4, though I can't find anything relevant in the changelog.

But in the concrete example I constructed here (diamond dependencies Main on A, B on Lib), ghc --make now compiles both A and B even if the first of the two is broken.

At this point I'm not sure if the change fixes the core issue, or I need to construct a more involved example. I'll see if I can find the corresponding ghc change.