hie-bios can't load tests and executables in subdirs #200

maksbotan · 2020-06-12T22:35:35Z

Hi,

I've noticed this strange behavior on projects with top-level cabal.project and *.cabal files in subdirs, like https://github.com/haskell-servant/servant.

First of all, versions. I use hie and hie-bios used by it is:

hie-bios version 0.5.0 compiled by GHC 8.8.3

Relevant parts of hie.yaml (generated with implicit-hie, but that should not matter):

cradle:
  cabal:
    - path: "servant-server/src"
      component: "lib:servant-server"

    - path: "servant-server/example/greet.hs"
      component: "servant-server:exe:greet"

    - path: "servant-server/test"
      component: "servant-server:test:spec"

Now, what happens if I try to load some file from the test suite:

$ hie-bios check servant-server/test/Servant/ServerSpec.hs 
hie-bios: can't find file: test/Spec.hs

So, for some reason something drops servant-test/ prefix from the path, although it's present in hie.yaml.

From hie logs:

2020-06-13 00:00:47.056703 [ThreadId 15] - New cradle: /Users/maksbotan/temp/servant/servant-server/test/Servant/ServerSpec.hs
2020-06-13 00:00:47.056912 [ThreadId 15] - Found "/Users/maksbotan/temp/servant/hie.yaml" for "/Users/maksbotan/temp/servant/servant-server/test/Servant/ServerSpec.hs"
2020-06-13 00:00:47.057219 [ThreadId 15] - Module "/Users/maksbotan/temp/servant/servant-server/test/Servant/ServerSpec.hs" is loaded by Cradle: Cradle {cradleRootDir = "/Users/mak
sbotan/temp/servant", cradleOptsProg = CradleAction: Cabal}
2020-06-13 00:00:47.057403 [ThreadId 15] - Found cradle: Cradle {cradleRootDir = "/Users/maksbotan/temp/servant", cradleOptsProg = CradleAction: Cabal}
2020-06-13 00:00:47.057557 [ThreadId 5] - <--2--{"jsonrpc":"2.0","params":{"value":{"kind":"end"},"token":464},"method":"$/progress"}
2020-06-13 00:00:47.05763 [ThreadId 5] - <--2--{"jsonrpc":"2.0","params":{"token":465},"method":"window/workDoneProgress/create","id":465}
2020-06-13 00:00:47.057698 [ThreadId 5] - <--2--{"jsonrpc":"2.0","params":{"value":{"kind":"begin","cancellable":false,"title":"Initializing Cabal project","percentage":0},"token":
465},"method":"$/progress"}
2020-06-13 00:00:47.058043 [ThreadId 4] - ---> {"jsonrpc":"2.0","id":465,"result":null}
2020-06-13 00:00:47.058105 [ThreadId 4] - haskell-lsp:Got reply message:"{\"jsonrpc\":\"2.0\",\"id\":465,\"result\":null}"
2020-06-13 00:00:47.058202 [ThreadId 10] - ****** reactor: got message number:1920
2020-06-13 00:00:47.058247 [ThreadId 10] - reactor:got RspFromClient:ResponseMessage {_jsonrpc = "2.0", _id = IdRspInt 465, _result = Just Null, _error = Nothing}
2020-06-13 00:00:47.058282 [ThreadId 10] - ****** reactor: top of loop
2020-06-13 00:00:48.826147 [ThreadId 13886] - Resolving dependencies...
2020-06-13 00:00:49.502951 [ThreadId 13886] - Build profile: -w ghc-8.8.3 -O0
2020-06-13 00:00:49.503037 [ThreadId 13886] - In order, the following will be built (use -v for more details):
2020-06-13 00:00:49.503088 [ThreadId 13886] -  - servant-0.17 (lib) (configuration changed)
2020-06-13 00:00:49.503132 [ThreadId 13886] -  - servant-server-0.17 (lib) (configuration changed)
2020-06-13 00:00:49.503171 [ThreadId 13886] -  - servant-server-0.17 (test:spec) (configuration changed)
2020-06-13 00:00:49.519729 [ThreadId 13886] - Configuring library for servant-0.17..
2020-06-13 00:00:51.352339 [ThreadId 13886] - Preprocessing library for servant-0.17..
2020-06-13 00:00:51.365653 [ThreadId 13886] - Building library for servant-0.17..
2020-06-13 00:00:52.562677 [ThreadId 13886] - Configuring library for servant-server-0.17..
2020-06-13 00:00:54.507245 [ThreadId 13886] - Preprocessing library for servant-server-0.17..
2020-06-13 00:00:54.517199 [ThreadId 13886] - Building library for servant-server-0.17..
2020-06-13 00:00:55.965902 [ThreadId 13886] - Configuring test suite 'spec' for servant-server-0.17..
2020-06-13 00:00:57.200188 [ThreadId 13884] - Warning: The package has an extraneous version range for a dependency on an
2020-06-13 00:00:57.200304 [ThreadId 13884] - internal library: servant-server -any && ==0.17, servant-server -any &&
2020-06-13 00:00:57.200355 [ThreadId 13884] - ==0.17. This version range includes the current package but isn't needed as
2020-06-13 00:00:57.200393 [ThreadId 13884] - the current package's library will always be used.
2020-06-13 00:00:58.096714 [ThreadId 13886] - Preprocessing test suite 'spec' for servant-server-0.17..
2020-06-13 00:00:58.23108 [ThreadId 13881] - Source error on cradle initialisation: can't find file: test/Spec.hs

hie-bios debug:

$ hie-bios debug servant-server/test/Servant/ServerSpec.hs 
Root directory:      /Users/maksbotan/temp/servant
Component directory: /Users/maksbotan/temp/servant/servant-server
GHC options:         -fbuilding-cabal-package -O0 -outputdir /Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/spec-tmp -odir /Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/spec-tmp -hidir /Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/spec-tmp -stubdir /Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/spec-tmp -i -i/Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/spec-tmp -itest -i/Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/autogen -i/Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/global-autogen -I/Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/autogen -I/Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/global-autogen -I/Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/spec-tmp -optP-include -optP/Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/build/spec/autogen/cabal_macros.h -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/maksbotan/.cabal/store/ghc-8.8.3/package.db -package-db /Users/maksbotan/temp/servant/dist-newstyle/packagedb/ghc-8.8.3 -package-db /Users/maksbotan/temp/servant/dist-newstyle/build/x86_64-osx/ghc-8.8.3/servant-server-0.17/t/spec/noopt/package.conf.inplace -package-id QuickCheck-2.13.2-IFlQsVhepoXHrDkLASCySh -package-id aeson-1.4.7.1-CYBfyzHWco0JrS94obsAvi -package-id base-4.13.0.0 -package-id base-compat-0.11.1-4oueEGUxmwHC6tuHqQ6hE2 -package-id base64-bytestring-1.0.0.3-G2dYBQRtOn8Kxdm0ndZRY -package-id bytestring-0.10.10.0 -package-id directory-1.3.6.0 -package-id hspec-2.7.1-aWprQ9Cwpe3oyEKitw2xL -package-id hspec-wai-0.10.1-LnpMCbFbKnxFyLrTCjtYOD -package-id http-types-0.12.3-B9EkuT6UGia2QJglhdFRHH -package-id mtl-2.2.2 -package-id resourcet-1.2.4-6K2EEC3PWvYL6IkanWK5wW -package-id safe-0.3.19-LxDJURZu1921pHCEJqjvRE -package-id servant-0.17-inplace -package-id servant-server-0.17-inplace -package-id should-not-typecheck-2.1.0-Aj1oA7w1Cu9GKPJbdEbgtJ -package-id string-conversions-0.4.0.1-KKjGH1i6WmbIm0l1zjJpaj -package-id temporary-1.3-6rtkIO88VZm9X1XintkbH8 -package-id text-1.2.4.0 -package-id transformers-0.5.6.2 -package-id transformers-compat-0.6.5-4FrvPCFBsgnJ9PLf29P7hZ -package-id wai-3.2.2.1-FkX8kCk55MR1uSuQhWVyx2 -package-id wai-extra-3.0.29.1-F5qpFPbsyBgB3suZR2dFQg -XHaskell2010 test/Spec.hs Servant.ArbitraryMonadServerSpec Servant.Server.ErrorSpec Servant.Server.Internal.ContextSpec Servant.Server.Internal.RoutingApplicationSpec Servant.Server.RouterSpec Servant.Server.StaticFilesSpec Servant.Server.StreamingSpec Servant.Server.UsingContextSpec Servant.Server.UsingContextSpec.TestCombinators Servant.HoistSpec Servant.ServerSpec -Wall -hide-all-packages
System libraries:    /nix/store/qdxkswf4vg9b2ylmg819ff0n8gl8yqip-ghc-shell-for-packages-ghc-8.8.3-env/lib/ghc-8.8.3
Config Location:     /Users/maksbotan/temp/servant/hie.yaml
Cradle:              Cradle {cradleRootDir = "/Users/maksbotan/temp/servant", cradleOptsProg = CradleAction: Cabal}
Dependencies:        cabal.project cabal.project.local

This is annoying, since once I open some of test's files in my editor, hie just starts to consume CPU by restarting in a loop and failing with the same error.

The text was updated successfully, but these errors were encountered:

fendor · 2020-06-13T11:46:08Z

This option: -itest is wrong. It should be an absolute path.

Cabal generates flags for executable and test components passing main-is module as a relative path, therefore cradles with packages in subdirs will fail to load those components. So it has to be made absolute. Fixes haskell#200.

fendor · 2020-06-14T09:51:57Z

I did some more investigation.
Firstly, my first hunch that the option -itest is wrong was incorrect. It is correct, the path is made absolute later in the pipeline which cant be seen here (namely in initSession).
Secondly, $ hie-bios check servant-server/test/Servant/ServerSpec.hs fails due to the same reason cabal repl servant-server/test/Spec.hs fails (a bug in cabal 3.0.0.0 haskell/cabal#6622 that has been fixed in upcoming versions).
Neither of these issues caused the compilation error you observed afaict. Also, I could only reproduce half of the problem. An error was shown when you opened test/Spec.hs but the rest of the test-suite was working fine. The problem of test/Spec.hs seems to be related to hspec-discover (related issue #187).
I was able to load the project with no errors after globally installing hspec-discover.

maksbotan · 2020-06-14T09:58:15Z

You can clearly see this in the command line:

... -XHaskell2010 test/Spec.hs Servant.ArbitraryMonadServerSpec Servant.Server.ErrorSpec ...

I.e., cabal for some reason adds test/Spec.hs as a file, not module name.

BTW, hspec-discover is in $PATH for me (due to nix), so that issue is unrelated.

Have you seen my PR #202? I can confirm that with this patch (rebased on top of 0.5.0) hie works fine with servant.

maksbotan · 2020-06-14T10:00:21Z

And my hie-bios and hie are built with Cabal-3.2.0.0, as far as I can tell. And that version is in $PATH as well.

maksbotan · 2020-06-14T10:02:50Z

Ah, haskell/cabal#6622 has not yet made it to any release, maybe it's the reason I'm getting this error.

So, in other words, we have to wait for Cabal release?

fendor · 2020-06-14T10:06:56Z

You can clearly see this in the command line:

Yes, but the flags are post-processed later on. Unfortunately, they do represent the exact flags that are used by the ghc session anymore.

are built with Cabal-3.2.0.0

I think the fix is going to be shipped with the next minor version cabal-3.2.0.1 release.

So, in other words, we have to wait for Cabal release?

No, it should work. Actually I think I got it wrong again. It should be unrelated to the cabal bug.

I.e., cabal for some reason adds test/Spec.hs as a file, not module name.

That is normal for the main module.

maksbotan · 2020-06-14T10:21:10Z

By the way, this succeeds:

$ cabal new-repl servant-server/test/Servant/ServerSpec.hs

That is normal for the main module.

Indeed, but it's broken for packages in subdirs :) That's why I added preprocessing for this specific case in #202.

fendor · 2020-06-14T10:28:40Z

By the way, this succeeds:

Yeah, the error is thrown by the GHC API.

The fix looks good in general, I am playing around with different ways.

maksbotan · 2020-06-14T10:34:34Z

Okay!

I'm not saying I think my fix is great, I just want to attract your attention to the root of the problem :)

fendor · 2020-06-14T10:44:22Z

I'm not saying I think my fix is great, I just want to attract your attention to the root of the problem :)

I think I have understood it now :) Adding a test and pushing my version of the fix for a review soon.

maksbotan · 2020-06-14T10:56:44Z

Nice to hear!

* Add failing test-case for #200 * Make targets' filepath absolute

fendor · 2020-07-12T21:20:39Z

This should be fixed now! However, someone has to patch it in HIE as well

maksbotan mentioned this issue Jun 13, 2020

Make path to main-is module absolute #202

Closed

fendor added a commit to fendor/hie-bios that referenced this issue Jun 14, 2020

Add failing test-case for haskell#200

d12b262

fendor added a commit to fendor/hie-bios that referenced this issue Jun 14, 2020

Add failing test-case for haskell#200

1300f3c

fendor added a commit to fendor/hie-bios that referenced this issue Jun 14, 2020

Add failing test-case for haskell#200

2ec4661

fendor added a commit to fendor/hie-bios that referenced this issue Jun 14, 2020

Add failing test-case for haskell#200

b92cb45

fendor added a commit to fendor/hie-bios that referenced this issue Jun 15, 2020

Add failing test-case for haskell#200

39ffd59

fendor added a commit to fendor/hie-bios that referenced this issue Jul 1, 2020

Add failing test-case for haskell#200

07171d3

fendor added a commit that referenced this issue Jul 1, 2020

Make main file targets absolute (#205)

2ac1102

* Add failing test-case for #200 * Make targets' filepath absolute

fendor closed this as completed Jul 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hie-bios can't load tests and executables in subdirs #200

hie-bios can't load tests and executables in subdirs #200

maksbotan commented Jun 12, 2020

fendor commented Jun 13, 2020

fendor commented Jun 14, 2020 •

edited

maksbotan commented Jun 14, 2020

maksbotan commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jul 12, 2020

hie-bios can't load tests and executables in subdirs #200

hie-bios can't load tests and executables in subdirs #200

Comments

maksbotan commented Jun 12, 2020

fendor commented Jun 13, 2020

fendor commented Jun 14, 2020 • edited

maksbotan commented Jun 14, 2020

maksbotan commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jun 14, 2020

maksbotan commented Jun 14, 2020

fendor commented Jul 12, 2020

fendor commented Jun 14, 2020 •

edited