`cabal repl` causes reconfiguration in subsequent `cabal build`

[mniip]

Describe the bug
Alternating cabal build local-pkg and cabal repl local-pkg commands will cause local-pkg to be endlessly reconfigured.

To Reproduce
Pretty much any cabal project will work, e.g. this empty library:

Example project

cabal.project

packages: .

test.cabal

cabal-version: 3.0
name: test
version: 0
build-type: Simple

library
  build-depends: base
  default-language: Haskell2010
  exposed-modules: Test

Test.hs

module Test where

Run:

$ cabal build test
Configuration is affected by the following files:
- cabal.project
Resolving dependencies...
Build profile: -w ghc-9.8.4 -O1
In order, the following will be built (use -v for more details):
 - test-0 (lib) (first run)
Configuring library for test-0...
Preprocessing library for test-0...
Building library for test-0...
[1 of 1] Compiling Test             ( Test.hs, dist-newstyle/build/x86_64-linux/ghc-9.8.4/test-0/build/Test.o, dist-newstyle/build/x86_64-linux/ghc-9.8.4/test-0/build/Test.dyn_o )

$ cabal repl test < /dev/null
Configuration is affected by the following files:
- cabal.project
Build profile: -w ghc-9.8.4 -O1
In order, the following will be built (use -v for more details):
 - test-0 (interactive) (lib) (configuration changed)
Configuring library for test-0...
Preprocessing library for test-0...
GHCi, version 9.8.4: https://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Test             ( Test.hs, interpreted )
Ok, one module loaded.
ghci> Leaving GHCi.

$ cabal build test
Configuration is affected by the following files:
- cabal.project
Build profile: -w ghc-9.8.4 -O1
In order, the following will be built (use -v for more details):
 - test-0 (lib) (configuration changed)
Configuring library for test-0...
Preprocessing library for test-0...
Building library for test-0...

$ cabal repl test < /dev/null
Configuration is affected by the following files:
- cabal.project
Build profile: -w ghc-9.8.4 -O1
In order, the following will be built (use -v for more details):
 - test-0 (interactive) (lib) (configuration changed)
Configuring library for test-0...
Preprocessing library for test-0...
GHCi, version 9.8.4: https://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling Test             ( Test.hs, interpreted )
Ok, one module loaded.
ghci> Leaving GHCi.

Note that after the initial build, in every command we get "configuration changed", followed by "Configuring library".

Expected behavior
The two commands should not de-configure each other.

System information

• Arch linux
• Cabal-3.16.1.0, GHC 9.8.4

Additional context
In my real workflow this alternation happens because:

• HLS does cabal repl --with-repl .../hie-bios/repl-wrapper OpenedFile.hs to capture GHC arguments,
• I do cabal build all in the terminal.

This means cabal repl has to reconfigure the package containing OpenedFile.hs, and cabal build all has to reconfigure all packages downstream of that file.

Another complication here is that HLS passes --enable-multi-repl and --with-repl, so actually cabal repl has to reconfigure all local packages upstream of OpenedFile.hs, and cabal build has to reconfigure all all local packages, which in my case takes a full minute.

--enable-multi-repl and --with-repl do add constraints to the solver (Cabal >=3.11, Cabal >=3.15 respectively), and initially I thought that this causes reconfiguration. Running cabal build all -v3 2>&1 | grep 'File monitor' we see:

File monitor 'config' unchanged.
File monitor 'improved-plan' changed: monitor value changed
File monitor 'elaborated-plan' changed: monitor value changed
File monitor 'compiler' unchanged.
File monitor 'solver-plan' changed: monitor value changed
File monitor 'source-hashes' unchanged.

But having dumped and analyzed these files, I found that these extra constraints appear in the input part of the cache, which causes the cache to be recomputed, but the output of this recomputation is exactly the same. Indeed, my freeze file has any.Cabal ==3.16.1.0 which is a stronger constraint than both of the above, meaning none of the plans actually change (I've verified this).

Also the reconfiguration happens without --enable-multi-repl nor --with-repl, where all file monitors report "unchanged".

I've done a bisect and e61b658 (#8726) is the first bad commit.

[geekosaur]

For what it's worth, I think I've been seeing this with my xmonad config: sh ./update.sh (cabal update, some git pulls, rebuild everything, echo :q | cabal repl to make subsequent cabal repls faster) followed by mod+q to restart the running xmonad does extra work during the restart.

[ulysses4ever]

Indeed, tried sounds both inconvenient and not straightforward to fix. Can you check that the workaround with setting a custom builddir helps?

Also, /cc @mpickering as the author of the committee in question and an expert in multi-repl in general.

[mniip]

Re: --builddir, normally HLS would use a --builddir for its operations, but then haskell/hie-bios#501 happened which caused this bug to surface.

As expected, setting a --builddir causes the issue to go away.

[ulysses4ever]

Well if the two parties are HLS and you in the terminal, I'd expect you to use the extra option to work around... But thanks for confirming the workaround!

[mniip]

I'd expect you to use the extra option

That would be extremely inconvenient! Considering you can't specify --builddir in a cabal.project.local (and considering HLS will pick up cabal.project.local anyway).

[mpickering]

I think you have diagnosed why the reconfiguration happens, the multi-repl adds some extra constraints and adding extra constraints might change the build plan.

• We could add these extra constraints when running cabal build as well?
• In a few cabal versions, we will only support Cabal library versions which enable multi-repl and this will no longer be an issue (since no extra constraints will be added). It would be good to document what is best to do here though in case a similar need arises in future.

I would be interested to understand your suggestion more about "the outputs don't change", it seems to me the part which is being re-run is the configuration step, whose inputs do change?

[mniip]

The cache file dist-newstyle/cache/elaborated-plan wraps the process of computing the elaborated plan, and as such, the file consists of three parts: the input to this computation, the files used to derive this input, and the output. (c.f. structuredDecodeTriple and checkFileMonitorChanged).

In case of elaborated-plan specifically, the input is of type

( ProjectConfig
, [PackageSpecifier UnresolvedSourcePackage]
, [FilePath] -- $PATH
)

and the output is of type

( ElaboratedInstallPlan
, ElaboratedSharedConfig
, IndexUtils.TotalIndexState
, IndexUtils.ActiveRepos
)

When it's time to produce an elaborated plan, we read this cache file and check whether the new input equals the input that's in the file. If it is so (and the referenced files are unchanged), then we read the output section of the file and use it as the elaborated plan. Otherwise we compute a new elaborated plan and write a new cache file.

So, when --enable-multi-repl is added on the command line, that adds a new constraint that appears in the ProjectConfig, so the input to the plan computation changes, yes. So we recompute the plan, and write a new cache file with a different input section.

But the resulting plan is the exact same, and the written cache file has the exact same output section, which is what I meant by "the output of this recomputation is exactly the same".

Did the plan change? No. Did we write a new plan cache? Yes. These are different things.

So at this point a reasonable question is: why are we reconfiguring local packages based on whether the cache file changed, and not based on whether the plan changed?