hls fails to start in presence of an setup-generated module

[andreabedini]

Hey team,

thanks for your amazing work on HLS! I am running into a problem in presence of a module generated by Setup.hs, see below.

Your environment

Which OS do you use: Ubuntu
Which LSP client (editor/plugin) do you use: Neovim+LanguageClient-neovimhaskell, etc... -->
Describe your project (alternative: link to the project): cabal + hie.yaml

Steps to reproduce

[~/hls-test] tree
.
├── app
│   └── Main.hs
├── CHANGELOG.md
├── hie.yaml
├── hls-test.cabal
└── Setup.hs

1 directory, 5 files

Setup.hs generates a module called Stuff.hs

{-# LANGUAGE Haskell2010 #-}

import Distribution.Simple
    ( defaultMainWithHooks, simpleUserHooks, UserHooks(buildHook) )
import Distribution.Simple.BuildPaths ( autogenPackageModulesDir )
import Distribution.Simple.LocalBuildInfo ( LocalBuildInfo )
import Distribution.Simple.Setup
    ( BuildFlags(buildVerbosity), fromFlagOrDefault )
import Distribution.Simple.Utils ( rewriteFileEx )

import System.Directory ( createDirectoryIfMissing )
import System.FilePath ( (</>) )

main :: IO ()
main = defaultMainWithHooks simpleUserHooks
    { buildHook = \packageDescription localBuildInfo userHooks buildFlags -> do
        generateScriptEnvModule localBuildInfo buildFlags
        buildHook simpleUserHooks packageDescription localBuildInfo userHooks buildFlags
    }

generateScriptEnvModule :: LocalBuildInfo -> BuildFlags -> IO ()
generateScriptEnvModule localBuildInfo buildFlags = do
    let baseDir = autogenPackageModulesDir localBuildInfo
    createDirectoryIfMissing True baseDir
    rewriteFileEx verbosity (baseDir </> "Stuff.hs") $ unlines
      [ "module Stuff where"
      , ""
      , "stuff :: Int"
      , "stuff = 3"
      ]
  where
    verbosity = fromFlagOrDefault minBound (buildVerbosity buildFlags)

the module Stuff is listed in the cabal file:

executable hls-test
    ...
    other-modules:    Stuff
    autogen-modules:  Stuff

hie.yaml was created with gen-hie

cradle:
  cabal:
    - path: "app/Main.hs"
      component: "hls-test:exe:hls-test"

Expected behaviour

Being able to use hls :)

Actual behaviour

hls fails to start.

  Failed to parse result of calling cabal
  Build profile: -w ghc-8.10.7 -O1
  In order, the following will be built (use -v for more details):
  - hls-test-0.1.0.0 (first run)
  Preprocessing executable 'hls-test' for hls-test-0.1.0.0..

  setup: can't find source for Stuff in app,
  /home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/hls-test/autogen,
  /home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/global-autogen

  cabal: repl failed for hls-test-0.1.0.0.

The package builds correctly with cabal. Happy to give into the code if given some pointers.

Include debug information

[~/hls-test] haskell-language-server-wrapper --debug .
Found "/home/andrea/hls-test/hie.yaml" for "/home/andrea/hls-test/a"
Run entered for haskell-language-server-wrapper(haskell-language-server-wrapper) Version 1.6.0.0, Git revision ce41b6459af131c845f942bd39e356f02b6306fa (dirty) x86_64 ghc-8.10.7
Current directory: /home/andrea/hls-test
Operating system: linux
Arguments: ["--debug","."]
Cradle directory: /home/andrea/hls-test
Cradle type: Cabal

Tool versions found on the $PATH
cabal:		3.6.2.0
stack:		2.7.3
ghc:		8.10.7


Consulting the cradle to get project GHC version...
Project GHC version: 8.10.7
haskell-language-server exe candidates: ["haskell-language-server-8.10.7","haskell-language-server"]
Launching haskell-language-server exe at:/home/andrea/.ghcup/bin/haskell-language-server-8.10.7
haskell-language-server version: 1.6.0.0 (GHC: 8.10.7) (PATH: /home/andrea/.ghcup/bin/haskell-language-server-8.10.7~1.6.0.0) (GIT hash: ce41b6459af131c845f942bd39e356f02b6306fa)
2022-02-01 09:43:16.99014224 [ThreadId 4] INFO hls:	Logging heap statistics every 60.00s
 ghcide setup tester in /home/andrea/hls-test.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Step 1/4: Finding files to test in /home/andrea/hls-test
Found 2 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle
  (/home/andrea/hls-test/hie.yaml)

Step 3/4: Initializing the IDE
2022-02-01 09:43:16.993843742 [ThreadId 8] DEBUG hls:	Initializing exports map from hiedb
2022-02-01 09:43:16.994185859 [ThreadId 4] DEBUG hls:	Shake session initialized

Step 2022-02-01 09:43:16.994355221 [ThreadId 8] DEBUG hls:	Done initializing exports map from hiedb (0)
4/4: Type checking the files
2022-02-01 09:43:16.994955876 [ThreadId 32] INFO hls:	Consulting the cradle for "app/Main.hs"
2022-02-01 09:43:16.99518992 [ThreadId 32] DEBUG hls:	Output from setting up the cradle Cradle {cradleRootDir = "/home/andrea/hls-test", cradleOptsProg = CradleAction: Cabal}
2022-02-01 09:43:17.018857693 [ThreadId 38] INFO hie-bios:	Build profile: -w ghc-8.10.7 -O1
2022-02-01 09:43:17.018994521 [ThreadId 38] INFO hie-bios:	In order, the following will be built (use -v for more details):
2022-02-01 09:43:17.019090434 [ThreadId 38] INFO hie-bios:	- hls-test-0.1.0.0 (first run)
2022-02-01 09:43:17.903564203 [ThreadId 38] INFO hie-bios:	Preprocessing executable 'hls-test' for hls-test-0.1.0.0..
2022-02-01 09:43:17.90373787 [ThreadId 36] INFO hie-bios:	setup: can't find source for Stuff in app,
2022-02-01 09:43:17.903838634 [ThreadId 36] INFO hie-bios:	/home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/hls-test/autogen,
2022-02-01 09:43:17.903931985 [ThreadId 36] INFO hie-bios:	/home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/global-autogen
2022-02-01 09:43:17.904025604 [ThreadId 36] INFO hie-bios:	
2022-02-01 09:43:17.908889642 [ThreadId 36] INFO hie-bios:	cabal: repl failed for hls-test-0.1.0.0.
2022-02-01 09:43:17.908966398 [ThreadId 36] INFO hie-bios:	
2022-02-01 09:43:17.920929725 [ThreadId 32] DEBUG hls:	Session loading result: Left [CradleError {cradleErrorDependencies = ["hls-test.cabal","cabal.project","cabal.project.local"], cradleErrorExitCode = ExitFailure 1, cradleErrorStderr = ["Failed to parse result of calling cabal","Build profile: -w ghc-8.10.7 -O1\nIn order, the following will be built (use -v for more details):\n - hls-test-0.1.0.0 (first run)\nPreprocessing executable 'hls-test' for hls-test-0.1.0.0..\n","setup: can't find source for Stuff in app,\n/home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/hls-test/autogen,\n/home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/global-autogen\n\ncabal: repl failed for hls-test-0.1.0.0.\n\n",""]}]
2022-02-01 09:43:17.921336906 [ThreadId 40] INFO hls:	Consulting the cradle for "Setup.hs"
2022-02-01 09:43:17.921571011 [ThreadId 40] DEBUG hls:	Output from setting up the cradle Cradle {cradleRootDir = "/home/andrea/hls-test", cradleOptsProg = CradleAction: Cabal}
2022-02-01 09:43:17.921673174 [ThreadId 40] DEBUG hls:	Session loading result: Left [CradleError {cradleErrorDependencies = [], cradleErrorExitCode = ExitSuccess, cradleErrorStderr = ["Multi Cradle: No prefixes matched","pwd: /home/andrea/hls-test","filepath: /home/andrea/hls-test/Setup.hs","prefixes:","(\"app/Main.hs\",Cabal {component = Just \"hls-test:exe:hls-test\"})"]}]
2022-02-01 09:43:17.922166968 [ThreadId 31] INFO hls:	File:     /home/andrea/hls-test/Setup.hs
Hidden:   no
Range:    1:1-2:1
Source:   cradle
Severity: DsError
Message: 
  Multi Cradle: No prefixes matched
  pwd: /home/andrea/hls-test
  filepath: /home/andrea/hls-test/Setup.hs
  prefixes:
  ("app/Main.hs",Cabal {component = Just "hls-test:exe:hls-test"})
2022-02-01 09:43:17.922890024 [ThreadId 27] INFO hls:	File:     /home/andrea/hls-test/app/Main.hs
Hidden:   no
Range:    1:1-2:1
Source:   cradle
Severity: DsError
Message: 
  Failed to parse result of calling cabal
  Build profile: -w ghc-8.10.7 -O1
  In order, the following will be built (use -v for more details):
  - hls-test-0.1.0.0 (first run)
  Preprocessing executable 'hls-test' for hls-test-0.1.0.0..

  setup: can't find source for Stuff in app,
  /home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/hls-test/autogen,
  /home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/build/x86_64-linux/ghc-8.10.7/hls-test-0.1.0.0/build/global-autogen

  cabal: repl failed for hls-test-0.1.0.0.


  
2022-02-01 09:43:17.923301253 [ThreadId 20] INFO hls:	finish: User TypeCheck (took 0.93s)
2022-02-01 09:43:17.923512067 [ThreadId 61] INFO hls:	finish: GetHie (took 0.00s)
2022-02-01 09:43:17.923747991 [ThreadId 68] INFO hls:	finish: GenerateCore (took 0.00s)
Files that failed:
 * /home/andrea/hls-test/Setup.hs
 * /home/andrea/hls-test/app/Main.hs

Completed (0 files worked, 2 files failed)
haskell-language-server-wrapper: callProcess: /home/andrea/.ghcup/bin/haskell-language-server-8.10.7 "--debug" "." (exit 2): failed

[jneira]

hi, thanks for the bug report, i think gen-hie is not taking in account Stuff.hs. Maybe we could try to add it manually to hie.yaml:

cradle:
  cabal:
    - path: "app/Main.hs"
      component: "hls-test:exe:hls-test"
    - path: "app/Stuff.hs"
      component: "hls-test:exe:hls-test"

and ensure the file has been generated when hls starts (as it seems cabal repl, used by hls underneath, is not triggering its generation)

[andreabedini]

Thanks @jneira.

Adding Stuff.hs to hie.yaml does not make a difference. I double checked and you are correct cabal repl by itself does not generate the module. I am not sure how to guarantee the module gets generated.

[jneira]

Thanks @jneira.

Adding Stuff.hs to hie.yaml does not make a difference. I double checked and you are correct cabal repl by itself does not generate the module. I am not sure how to guarantee the module gets generated.

But if you trigger a build which generates it, does not hls see it?

To make it available in repl i think you could add another hook (replHook? i ever used Setup.hs tbh) and generate the module also in the hook

[andreabedini]

But if you trigger a build which generates it, does not hls see it?

It does not. This is because cabal build works in its own dist-newstyle directory while hls uses a directory under ~/.cache/hie-bios where cabal build never happened.

[andreabedini]

Manually generating the module with cabal build seems to work, hls then can find the module.

$ cabal build --builddir /home/andrea/.cache/hie-bios/dist-hls-test-5ecae760dc1dde62cd04502321ce7a72/

[wz1000]

You should use a postConf hook instead of a buildHook, then HLS will be able to see the autogenerated files.

[andreabedini]

@wz1000 That indeed works. Thank you. If this is going to be a known limitation, we should document it somewhere.

[wz1000]

This is not really a limitation of HLS specifically, just the way custom Setup.hs files work. I am quite sure that if you start with a clean dist-newstyle and then run cabal repl without running cabal build beforehand, the code would fail in the exact same way (indeed that is precisely the failure that occurs - HLS calls cabal repl to get the required flags to build your code).

The buildHook is called by cabal when building your project - however we don't use cabal to build the project, HLS does the building. HLS does use cabal configure to configure your project, which is why postConf works - the code in the hook actually has a chance to run.

So while I'm not opposed to documentation, it really is not an HLS specific issue, it is just a bug in the way the Setup.hs file is implemented. buildHook only runs if you use cabal build to build your code, you can't really expect anything else to call it.

[andreabedini]

I agree with you @wz1000.