support for stack + hpack 0.35

[maerten]

Your environment

Which OS do you use:
MacOS
Which LSP client (editor/plugin) do you use:
Just running haskell-language-server in terminal
Describe your project (alternative: link to the project):
stack.yaml + package.yaml using hpack 0.35 + GHC 9.2.3

Steps to reproduce

using ghcup to install

• ghc 9.2.3
• cabal 3.6.2.0
• stack 2.7.5
• use stack to install hpack 0.35 : stack install hpack-0.35.0 --resolver=nightly-2022-06-30
• install hls from master ghcup compile hls -g master -j 20 --ghc 9.2.3

In a stack project, update it to use hpack 0.35 and generate a new cabal file:

• add this line to stack.yaml:

  with-hpack: /Users/maarten/.local/bin/hpack

• update your package.yaml to use GHC2021 by adding:

• library:
    language: GHC2021 # <-- add this line

• run stack ghci once to make sure a new cabal file is generated (which should now include the line default-language: GHC2021 somewhere in it)

• try running haskell-language-server

Expected behaviour

haskell-language-server should run

Actual behaviour

haskell-language-server fails on every file with this error:

2022-06-30T12:07:37.724770Z | Info | updateFileDiagnostics published different from new diagnostics - file diagnostics: File:
  <myproject/MyFile.hs>
Hidden:   no
Range:    1:1-2:1
Source:   cradle
Severity: DsError
Message:
  ghcide compiled by GHC 9.2 failed to load packages: haskell-language-server: can't find a package
  database at <myproject>/<project>.cabal is up-to-date.
  Please ensure that ghcide is compiled with the same GHC installation as the project.
2022-06-30 14:07:37.848261 [ThreadId 575] INFO hie-bios:	<project>.cabal is up-to-date

Background

The reason for hpack 0.35 is to set the default language of all files in the project to GHC2021, so that a few useful extensions are enabled by default.

[fendor]

Hi, thank you for your bug report! Please share the whole logs!

[maerten]

Full logs for running haskell-language-server src/SomeFile.hs

2022-06-30T12:55:52.562601Z | Info | No log file specified; using stderr.
2022-06-30T12:55:52.562863Z | Info | haskell-language-server version: 1.7.0.0 (GHC: 9.2.3) (PATH: /Users/<username>/.ghcup/bin/haskell-language-server-9.2.3~1.7.0.0)
2022-06-30T12:55:52.563265Z | Info | Directory: /Users/<username>/<myproject>
2022-06-30T12:55:52.563402Z | Info | Logging heap statistics every 60.00s
 ghcide setup tester in /Users/<username>/<myproject>.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Step 1/4: Finding files to test in /Users/<username>/<myproject>
Found 1 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle
  (/Users/<username>/<myproject>/hie.yaml)

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
2022-06-30T12:55:52.571635Z | Info | Cradle path: src/SomeFile.hs
2022-06-30 14:55:52.699308 [ThreadId 28] INFO hie-bios: supermarketdata.cabal is up-to-date
2022-06-30 14:55:52.705889 [ThreadId 29] INFO hie-bios: Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
2022-06-30 14:55:52.74366 [ThreadId 29] INFO hie-bios:  Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
2022-06-30 14:55:53.594341 [ThreadId 29] INFO hie-bios: Configuring GHCi with the following packages: supermarketdata
2022-06-30 14:55:53.797 [ThreadId 34] INFO hie-bios:    supermarketdata.cabal is up-to-date
2022-06-30 14:55:53.803672 [ThreadId 32] INFO hie-bios: Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
2022-06-30 14:55:53.84169 [ThreadId 32] INFO hie-bios:  Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
2022-06-30 14:55:53.947261 [ThreadId 34] INFO hie-bios: supermarketdata.cabal is up-to-date
2022-06-30 14:55:53.954668 [ThreadId 32] INFO hie-bios: Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
2022-06-30 14:55:53.995754 [ThreadId 32] INFO hie-bios: Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
2022-06-30 14:55:54.001775 [ThreadId 34] INFO hie-bios: /Users/<username>/<myproject>/.stack-work/install/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb:/Users/<username>/.stack/snapshots/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb:/Users/<username>/.ghcup/ghc/9.2.3/lib/ghc-9.2.3/lib/package.conf.d
2022-06-30T12:55:54.479174Z | Info | updateFileDiagnostics published different from new diagnostics - file diagnostics: File:     /Users/<username>/<myproject>/src/SomeFile.hs
Hidden:   no
Range:    1:1-2:1
Source:   cradle
Severity: DsError
Message:
  ghcide compiled by GHC 9.2 failed to load packages: haskell-language-server: can't find a package
  database at /Users/<username>/<myproject>/supermarketdata.cabal is up-to-date.
  Please ensure that ghcide is compiled with the same GHC installation as the project.
Files that failed:
 * /Users/<username>/<myproject>/src/SomeFile.hs

Completed (0 files worked, 1 file failed)

[fendor]

Ok, we are still missing some information, please rerun with debug output enabled:
haskell-language-server --debug src/SomeFile.hs

[maerten]

Running with output enabled gives:

main ✗ $ haskell-language-server --debug src/SomeFile.hs                                                                                                 130 ↵
2022-06-30T13:20:36.589232Z | Info | No log file specified; using stderr.
2022-06-30T13:20:36.589518Z | Info | haskell-language-server version: 1.7.0.0 (GHC: 9.2.3) (PATH: /Users/<username>/.ghcup/bin/haskell-language-server-9.2.3~1.7.0.0)
2022-06-30T13:20:36.589893Z | Info | Directory: /Users/<username>/<myproject>
2022-06-30T13:20:36.590033Z | Info | Logging heap statistics every 60.00s
 ghcide setup tester in /Users/<username>/<myproject>.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Step 1/4: Finding files to test in /Users/<username>/<myproject>
Found 1 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle
  (/Users/<username>/<myproject>/hie.yaml)

Step 3/4: Initializing the IDE
2022-06-30T13:20:36.596233Z | Debug | Initializing exports map from hiedb
2022-06-30T13:20:36.596454Z | Debug | Shake session initialized

Step 4/4: Type checking the files
2022-06-30T13:20:36.598326Z | Info | Cradle path: src/SomeFile.hs
2022-06-30T13:20:36.598801Z | Debug | Cradle: Cradle {cradleRootDir = "/Users/<username>/<myproject>", cradleOptsProg = CradleAction: Stack}
2022-06-30T13:20:36.617294Z | Debug | Done initializing exports map from hiedb. Size: 2665
2022-06-30 15:20:36.73961 [ThreadId 29] INFO hie-bios:  myproject.cabal is up-to-date
2022-06-30 15:20:36.755523 [ThreadId 27] INFO hie-bios: Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
2022-06-30 15:20:36.798183 [ThreadId 27] INFO hie-bios: Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
2022-06-30 15:20:37.452121 [ThreadId 27] INFO hie-bios: myproject> initial-build-steps (lib)
2022-06-30 15:20:37.802038 [ThreadId 27] INFO hie-bios: Configuring GHCi with the following packages: myproject
2022-06-30 15:20:38.012031 [ThreadId 34] INFO hie-bios: myproject.cabal is up-to-date
2022-06-30 15:20:38.017785 [ThreadId 32] INFO hie-bios: Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
2022-06-30 15:20:38.05913 [ThreadId 32] INFO hie-bios:  Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
2022-06-30 15:20:38.180892 [ThreadId 34] INFO hie-bios: myproject.cabal is up-to-date
2022-06-30 15:20:38.184161 [ThreadId 32] INFO hie-bios: Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
2022-06-30 15:20:38.228272 [ThreadId 32] INFO hie-bios: Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
2022-06-30 15:20:38.234777 [ThreadId 34] INFO hie-bios: /Users/<username>/<myproject>/.stack-work/install/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb:/Users/<username>/.stack/snapshots/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb:/Users/<username>/.ghcup/ghc/9.2.3/lib/ghc-9.2.3/lib/package.conf.d
2022-06-30T13:20:38.742513Z | Debug | Session loading result: Right (ComponentOptions {componentOptions = ["-i","-odir=/Users/<username>/<myproject>/.stack-work/odir","-hidir=/Users/<username>/<myproject>/.stack-work/odir","-hide-all-packages","-i/Users/<username>/<myproject>/.stack-work/dist/aarch64-osx/Cabal-3.6.3.0/build","-i/Users/<username>/<myproject>/src-database","-i/Users/<username>/<myproject>/src","-i/Users/<username>/<myproject>/src-extraction","-i/Users/<username>/<myproject>/.stack-work/dist/aarch64-osx/Cabal-3.6.3.0/build/autogen","-i/Users/<username>/<myproject>/.stack-work/dist/aarch64-osx/Cabal-3.6.3.0/build/global-autogen","-stubdir=/Users/<username>/<myproject>/.stack-work/dist/aarch64-osx/Cabal-3.6.3.0/build","-I/opt/homebrew/opt/icu4c/include","-L/opt/homebrew/opt/icu4c/lib","-package-id=QuickCheck-2.14.2-DXgIuu4hnyeCHiYYxVmgFq","-package-id=aeson-2.0.3.0-72NF8Ns8VGOKAKHRmA61FI","-package-id=aeson-pretty-0.8.9-Asuqd5iSsXgJ5QMw7NIZQ9","-package-id=base-4.16.2.0","-package-id=beam-core-0.9.2.1-HTO86m74whcBLUMStKYaSZ","-package-id=beam-migrate-0.5.1.2-2wMNL1FLkfLL2J3UQ6KAdK","-package-id=beam-postgres-0.5.2.1-3nIfaMelr9d1CmJUajDszZ","-package-id=bytestring-0.11.3.1","-package-id=conduit-1.3.4.2-26WK0O4ihNPEYAPlmjWS3P","-package-id=conduit-concurrent-map-0.1.3-Gd7sRr3zfgR2f74gfnBJvh","-package-id=containers-0.6.5.1","-package-id=directory-1.3.6.2","-package-id=either-5.0.2-AE1gcSSOCbWEUoYVar0xtP","-package-id=envy-2.1.0.0-HmHiHDuXDhJ3Gf3kmqTHbF","-package-id=exceptions-0.10.4","-package-id=extra-1.7.10-68yffu0l5jPC016XgtaKwN","-package-id=fmt-0.6.3.0-K3L3lfjqusF79Ueop2H8K2","-package-id=formatting-7.1.3-HgoHKFzOJxz8tCL0o3XTpy","-package-id=hspec-2.9.7-EJr56BQceYKArmlgNWMkin","-package-id=hspec-discover-2.9.7-DMyjIaEIktNIdxTjQhRAWL","-package-id=html-entities-1.1.4.6-Dr102yO3R0xF3fgwGZewIU","-package-id=http-client-0.7.11-6sdWOGgWPSi9WSJ9wWZXfk","-package-id=http-client-tls-0.3.6.1-Bs8aS74C9tuKxGMnRFbkYE","-package-id=http-types-0.12.3-2Cj5EG6Bw0n6gTFrZpvPzX","-package-id=hxt-9.3.1.22-9lSUsuqLdph1spvKsEyb7i","-package-id=hxt-xpath-9.1.2.2-3EaKsA2h2C54tYgBIv5c3v","-package-id=lens-5.1.1-8Vh6IAI31B13xqOgOpV5kA","-package-id=lens-aeson-1.2.1-CtxEyYcTyBgCp5TEEcVjiF","-package-id=megaparsec-9.2.1-3mG6PJi4RKfH1aTzhLevhN","-package-id=modern-uri-0.3.4.4-JaRj4t4HyFaKSQahryYiSk","-package-id=mtl-2.2.2","-package-id=optparse-applicative-0.17.0.0-E8W4UtkOw2m450oeyZkxnX","-package-id=postgresql-simple-0.6.4-7xtEBl5Bx5WJup12VmtJxB","-package-id=pretty-simple-4.1.1.0-Blqmne51pIqGDfS9vraM0L","-package-id=pretty-terminal-0.1.0.0-CFT6FJ0GYC7J1lb89SHy2z","-package-id=process-1.6.13.2","-package-id=rate-limit-1.4.2-KJLEi4RTVNqBHgfcVp4MCn","-package-id=raw-strings-qq-1.1-CGSNPrrZyrlChaQnrHZuDD","-package-id=req-3.12.0-G5kQB07zU4NAZTC9gTdWt8","-package-id=scientific-0.3.7.0-EtASQt7qSh0Kt1y2XW2n7Q","-package-id=split-0.2.3.4-J6PjBuGt7em7JpDyPyC9Kv","-package-id=sqlite-simple-0.4.18.0-IGn7AfSny2fIUpjm6uUioW","-package-id=table-layout-0.9.1.0-5I1B8ednjm4ZXGK0vRj08","-package-id=text-1.2.5.0","-package-id=text-format-0.4.0.0-1I9RhfQMLxN8GgIvU4Qov3","-package-id=text-icu-0.7.1.0-FTfOTdV5VQXKgfM8wDFQSP","-package-id=time-1.11.1.1","-package-id=time-units-1.0.0-1FmUezJdMzs3AJprzQd7Ms","-package-id=unliftio-0.2.22.0-LFU1fvpjwP47XXvZbhKXfi","-package-id=unordered-containers-0.2.19.1-FXKbfOyDuRhGRszF2fgF1T","-package-id=vector-0.12.3.1-CglYmqdPc3K6ckiAc0g0Cd","-optP-include","-optP/Users/<username>/<myproject>/.stack-work/ghci/41ea8962/cabal_macros.h","-ghci-script=/private/var/folders/0c/zmpjp7l568xcvnt49pkkn5p00000gn/T/haskell-stack-ghci/d70d78f0/ghci-script","-package-db","myproject.cabal is up-to-date","-package-db","myproject.cabal is up-to-date","-package-db","/Users/<username>/<myproject>/.stack-work/install/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb","-package-db","/Users/<username>/.stack/snapshots/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb","-package-db","/Users/<username>/.ghcup/ghc/9.2.3/lib/ghc-9.2.3/lib/package.conf.d"], componentRoot = "/Users/<username>/<myproject>", componentDependencies = ["myproject.cabal","package.yaml","stack.yaml"]},"/Users/<username>/.ghcup/ghc/9.2.3/lib/ghc-9.2.3/lib")
2022-06-30T13:20:38.792836Z | Info | updateFileDiagnostics published different from new diagnostics - file diagnostics: File:     /Users/<username>/<myproject>/src/SomeFile.hs
Hidden:   no
Range:    1:1-2:1
Source:   cradle
Severity: DsError
Message:
  ghcide compiled by GHC 9.2 failed to load packages: haskell-language-server: can't find a package
  database at /Users/<username>/<myproject>/myproject.cabal is up-to-date.
  Please ensure that ghcide is compiled with the same GHC installation as the project.
2022-06-30T13:20:38.793154Z | Debug | Finished: User TypeCheck Took: 2.20s
2022-06-30T13:20:38.793289Z | Debug | Finished: GetHie Took: 0.00s
2022-06-30T13:20:38.793416Z | Debug | Finished: GenerateCore Took: 0.00s
Files that failed:
 * /Users/<username>/<myproject>/src/SomeFile.hs

Completed (0 files worked, 1 file failed)

[fendor]

From the logs we can see:

"-package-db","myproject.cabal is up-to-date",
"-package-db","myproject.cabal is up-to-date",
"-package-db","/Users/<username>/<myproject>/.stack-work/install/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb",
"-package-db","/Users/<username>/.stack/snapshots/aarch64-osx/e90b006d0ea34f75468edd295d78e10d07eafcf77234b38a7451176eb8b3530e/9.2.3/pkgdb",
"-package-db","/Users/<username>/.ghcup/ghc/9.2.3/lib/ghc-9.2.3/lib/package.conf.d"

which is rather incorrect, especially the first two entries.
Can you check whether stack repl works? Potentially, stack repl <library_component_name>? If it fails, then it is a stack bug, otherwise we rather deal with a bug in hie-bios.

[maerten]

Thanks for looking into this @fendor
Runing stack repl or stack ghci works fine, I have been working with it for a while and have seen no issues.

I'm not entirely sure what you mean by library_component_name but it works fine if i specify one of the executables defined in package.yaml of the project (e.g. stack repl myproject:exe:myprogram).

[fendor]

Can confirm, hie-bios trips up, root cause being:

> stack path --ghc-package-path
stackproj.cabal is up-to-date
Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
stackproj.cabal is up-to-date
Stack has not been tested with GHC versions above 9.0, and using 9.2.3, this may fail
Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
/home/hugin/Documents/haskell/stackproj/.stack-work/install/x86_64-linux-tinfo6/aabc333be4522fc7c449db251b1a90b30d5039a4dd5e5a311c7d06deadf850df/9.2.3/pkgdb:/home/hugin/.stack/snapshots/x86_64-linux-tinfo6/aabc333be4522fc7c449db251b1a90b30d5039a4dd5e5a311c7d06deadf850df/9.2.3/pkgdb:/home/hugin/.stack/programs/x86_64-linux/ghc-tinfo6-9.2.3/lib/ghc-9.2.3/package.conf.d

I don't know why stack is so much chattier with new ghc versions?

EDIT:
Apparently hpack got chattier, with 0.35 it causes stack to dump this additional output, although I have no idea what might have changed.

The question now being, is that a stack bug or hie-bios bug? Seems like something you don't want to have in stack either.