Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cabal_macros.h is constantly overwritten #177

Closed
maksbotan opened this issue May 4, 2020 · 25 comments
Closed

cabal_macros.h is constantly overwritten #177

maksbotan opened this issue May 4, 2020 · 25 comments

Comments

@maksbotan
Copy link

I don't know if it's my system broken in a subtle way, or an actual issue, so I report it anyway.

I have GHC 8.8.3 installed from my package manager (I'm on Gentoo Linux):

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.8.3

I also have Cabal-3.2 & cabal-install-3.2 installed from the package manager:

$ cabal --version
cabal-install version 3.2.0.0
compiled using version 3.2.0.0 of the Cabal library 

I use this ghc & cabal to build my project (cabal new-build all).

I also used the same cabal to build haskell-ide-engine and haskell-language-server master branches (I had to change cabal-helper github dep to cabal-helper-1.1.0.0 from Hackage).

Now, when I run hie or haskell-language-server in my project, it rebuilds a lot of modules due to changes to build/autogen/cabal_macros.h.

Here is the diff between cabal_macros.h when I build the project and when HIE/HLS builds it:

--- macros-hand.h	2020-05-04 22:52:17.859642912 +0300
+++ macros-hie.h	2020-05-04 22:53:50.989080885 +0300
@@ -88,17 +88,6 @@
   (major1) == 8 && (major2) == 8 && (minor) <= 3)
 #endif /* MIN_TOOL_VERSION_ghc_pkg */
 
-/* tool haddock-2.23.0 */
-#ifndef TOOL_VERSION_haddock
-#define TOOL_VERSION_haddock "2.23.0"
-#endif /* TOOL_VERSION_haddock */
-#ifndef MIN_TOOL_VERSION_haddock
-#define MIN_TOOL_VERSION_haddock(major1,major2,minor) (\
-  (major1) <  2 || \
-  (major1) == 2 && (major2) <  23 || \
-  (major1) == 2 && (major2) == 23 && (minor) <= 0)
-#endif /* MIN_TOOL_VERSION_haddock */
-
 /* tool hpc-0.67 */
 #ifndef TOOL_VERSION_hpc
 #define TOOL_VERSION_hpc "0.67"

This is very annoying, since each cabal new-build takes a long time due to these rebuilds.

For the record:

$ /usr/lib64/ghc-8.8.3/bin/haddock --version
Haddock version 2.23.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008

(This is not in $PATH).

@maksbotan
Copy link
Author

@mpickering ping? This affects both haskell-ide-engine and haskell-language-server.

@fendor
Copy link
Collaborator

fendor commented May 8, 2020

I will look into it.

@fendor
Copy link
Collaborator

fendor commented May 8, 2020

I can not reproduce with plain hie-bios, so I would not expect that you can reproduce this issue with haskell-ide-engine.

@fendor
Copy link
Collaborator

fendor commented May 8, 2020

I will try to reproduce with cabal 3.2. That is unreleased, right? Just build from snapshot?

@maksbotan
Copy link
Author

Yes. Here's what I changed in haskell-ide-engine's cabal.project:

diff --git a/cabal.project b/cabal.project
index 4b7d2226..0ef8b76e 100644
--- a/cabal.project
+++ b/cabal.project
@@ -1,13 +1,14 @@
 packages:
          ./
          ./hie-plugin-api/
 
          -- ./submodules/HaRe
 
-source-repository-package
-    type: git
-    location: https://github.com/DanielG/cabal-helper.git
-    tag: a18bbb2af92e9b4337e7f930cb80754f2408bcfd
+--source-repository-package
+--    type: git
+--    location: https://github.com/DanielG/cabal-helper.git
+--    tag: a18bbb2af92e9b4337e7f930cb80754f2408bcfd
 
 tests: true
 
@@ -17,8 +18,9 @@ package haskell-ide-engine
 -- Match the flag settings we use in stac builds
 constraints:
         haskell-ide-engine +pedantic,
-        hie-plugin-api     +pedantic
+        hie-plugin-api     +pedantic,
+        Cabal ==3.2.0.0,
+        cabal-helper ==1.1.0.0
 
 write-ghc-environment-files: never
 
-index-state: 2020-05-02T10:11:15Z

cabal-helper-1.1.0.0 seems to already include the commit referenced in the source-repository-package.

I would be happy to provide more information, just tell me what you need.

@mpickering
Copy link
Collaborator

@maksbotan Do you have documention: True in your cabal.project file?

@maksbotan
Copy link
Author

No. I used to, but now it's commented out.

@maksbotan
Copy link
Author

Adding it back did not help, by the way (I tried adding to cabal.project.local).

@fendor
Copy link
Collaborator

fendor commented May 8, 2020

I am unable to reproduce the issue with hie-bios, hie and hls. Whatever I do, cabal_macros.h is only created once and then never modified. Afaict, cabal is reponsible for the creation, maybe you have a modified cabal version? I tried with cabal-3.2.0.0 but could not reproduce.

Can you provide a repository that reproduces the problem? Including rough steps how you reproduce the issue?
Also, do you write a hie.yaml? If yes, what is the content?
If no, then this might be the reason (then the issue might be related to cabal-helper and not hie-bios).
Can you reproduce the issue with ghcide as well?

@maksbotan
Copy link
Author

I fear this depends not on the repo, but on my local setup.
I'll try to look into this more later.

Re hie.yaml - I do have one. With cabal cradle.

Cabal the library that hie is built with comes from hackage, 3.2.0.0 version.
However Cabal the library used by my cabal-install is from my system's package manager, 3.2.0.0 as well. I'll try to check whether this is the problem.

Thanks for the answers so far!

@maksbotan
Copy link
Author

Here is my hie.yaml:

cradle:
    cabal:
        - path: "code/morley"
          component: "lib:morley"
        - path: "code/morley"
          component: "test:morley-test"
        - path: "code/lorentz"
          component: "lib:lorentz"
        - path: "code/morley-ledgers"
          component: "lib:morley-ledgers"
        - path: "code/morley-ledgers"
          component: "exe:morley-ledgers"
        - path: "code/indigo"
          component: "lib:indigo"

The repo is https://gitlab.com/morley-framework/morley, however I don't think it is important.

In plan.json for haskell-ide-engine I see this:

    "install-plan": [
        {
            "type": "pre-existing",
            "id": "Cabal-3.2.0.0-82QQ9INxvFvLuRIYMiY5kY",
            "pkg-name": "Cabal",
            "pkg-version": "3.2.0.0",
...

Meaning that hie is built with the same Cabal I'm building my project with, contrary to what I assumed above. In fact, it comes from here:

/usr/lib64/ghc-8.8.3/package.conf.d/Cabal-3.2.0.0-82QQ9INxvFvLuRIYMiY5kY.conf

re ghcide: I certainly can reproduce this with haskell-language-server. Does it really make sense to check ghcide?

@maksbotan
Copy link
Author

I also did test it with hie-bios itself.

$ /home/maks/work/haskell-ide-engine/dist-newstyle/build/x86_64-linux/ghc-8.8.3/hie-bios-0.4.0/x/hie-bios/build/hie-bios/hie-bios version
hie-bios version 0.4.0 compiled by GHC 8.8.3
$ /home/maks/work/haskell-ide-engine/dist-newstyle/build/x86_64-linux/ghc-8.8.3/hie-bios-0.4.0/x/hie-bios/build/hie-bios/hie-bios debug code/morley/src/
Root directory:      /home/maks/Projects/SRK/morley
GHC options:        XXXXXX
System libraries:    /usr/lib64/ghc-8.8.3
Config Location:     /home/maks/Projects/SRK/morley/hie.yaml
Cradle:              Cradle {cradleRootDir = "/home/maks/Projects/SRK/morley", cradleOptsProg = CradleAction: Cabal}
Dependencies:        cabal.project

And reproduced the same problem. Diff in cabal-macros.h is the same as in my first message.

@maksbotan
Copy link
Author

However, now if I add documentation: True to cabal.project (not .local) hie-bios debug says this:

Cradle failed to load
Exit Code: ExitSuccess
Stderr: Failed to parse result of calling cabal
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O0
In order, the following will be built (use -v for more details):
 - morley-prelude-0.3.0 (lib) (configuration changed)
 - tasty-hunit-compat-0.2 (lib) (configuration changed)
 - tezos-bake-monitor-lib-0.1.0.0 (lib) (configuration changed)
 - morley-1.2.0 (lib) (configuration changed)
Configuring library for tasty-hunit-compat-0.2..
Configuring library for morley-prelude-0.3.0..
Configuring library for tezos-bake-monitor-lib-0.1.0.0..
Preprocessing library for tasty-hunit-compat-0.2..
Preprocessing library for morley-prelude-0.3.0..
Building library for tasty-hunit-compat-0.2..
Building library for morley-prelude-0.3.0..
Preprocessing library for tezos-bake-monitor-lib-0.1.0.0..
Building library for tezos-bake-monitor-lib-0.1.0.0..

cabal: The program 'haddock' version >=2.0 is required but it could not be
found.

cabal: The program 'haddock' version >=2.0 is required but it could not be
found.

cabal: The program 'haddock' version >=2.0 is required but it could not be
found.

Warning: Failed to build documentation for morley-prelude-0.3.0 (which is
required by morley-1.2.0).
Failed to build documentation for tasty-hunit-compat-0.2 (which is required by
morley-1.2.0).
Failed to build documentation for tezos-bake-monitor-lib-0.1.0.0 (which is
required by morley-1.2.0).

I suppose that it is calling cabal v2-repl in some environment where haddock is missing for some reason?

@maksbotan
Copy link
Author

Okay, I believe I understand now.

On Gentoo haddock is at /usr/bin/haddock-ghc-8.8.3. If I copy it as haddock somewhere in $PATH, the issue goes away.

So in a sense, this is the problem of my setup.

However, I still can't understand why cabal called by hie* stuff behaves differently from when I run it myself.

Feel free to close this issue if you don't think fixing this is worth the effort.

@maksbotan
Copy link
Author

Now the only problem I see is that after running hie-bios cabal new-build does this:

Resolving dependencies...
Build profile: -w ghc-8.8.3 -O0
In order, the following will be built (use -v for more details):
 - morley-prelude-0.3.0 (lib) (configuration changed)
 - tasty-hunit-compat-0.2 (lib) (configuration changed)
 - tezos-bake-monitor-lib-0.1.0.0 (lib) (configuration changed)
 - morley-1.2.0 (lib) (configuration changed)
 - morley-1.2.0 (exe:morley) (dependency rebuilt)
Configuring library for tasty-hunit-compat-0.2..
Configuring library for morley-prelude-0.3.0..
Configuring library for tezos-bake-monitor-lib-0.1.0.0..
Preprocessing library for tasty-hunit-compat-0.2..
Building library for tasty-hunit-compat-0.2..
Preprocessing library for morley-prelude-0.3.0..
Building library for morley-prelude-0.3.0..
Preprocessing library for tezos-bake-monitor-lib-0.1.0.0..
Building library for tezos-bake-monitor-lib-0.1.0.0..
Configuring library for morley-1.2.0..
Preprocessing library for morley-1.2.0..
Building library for morley-1.2.0..
Preprocessing executable 'morley' for morley-1.2.0..
Building executable 'morley' for morley-1.2.0..

I.e., configuration changed, but nothing actually gets rebuilt.

@maksbotan
Copy link
Author

Re the last issue: in some rare cases I get this when I compile my project myself:

Building library for lorentz-0.2.0..
/tmp/bios-wrapper20320-2: line 13: : command not found
cabal: Failed to build lorentz-0.2.0 (which is required by indigo-0.1.0.0).

Looks like when hie-bios does cabal v2-repl --with-compiler, this setting gets written down somewhere in dist-newstyle and my cabal new-build is able to pick this up.

Contrary to my haddock problem above, I believe that this issue is very serious.

@fendor
Copy link
Collaborator

fendor commented May 9, 2020

I occassionally see this, too. Could not reproduce reliably and restart fixes it immediately.

@maksbotan
Copy link
Author

Yes, restart helps. However I think this indicates that hie stuff clashes with user's own build system and this not a good sign.

@maksbotan
Copy link
Author

Maybe hie should use separate dist-newstyle dir?

@fendor
Copy link
Collaborator

fendor commented May 9, 2020

It writes this stuff to the filesystem in temp. That is unrelated to dist-newstyle. While I dont really know what causes this, I imagine that it is some kind of race between writing the file to disk/setting the access modes and cabal repl trying to access it.

@maksbotan
Copy link
Author

I'd love to investigate this in my free time, if you could give me some pointers.
I believe hie should get more love :)

@fendor
Copy link
Collaborator

fendor commented May 9, 2020

Sorry, I do not have any pointers :(
You can try to follow the code at https://github.com/mpickering/hie-bios/blob/master/src/HIE/Bios/Cradle.hs#L424
Maybe you see some race and can produce some poc that this race exists?

Just to clarify, hie will not receive love for much longer, but hls will receive way more love :) (https://mpickering.github.io/ide/index.html)

@maksbotan
Copy link
Author

Thanks!
Sorry, by hie I actually meant the whole ide story for haskell. Hie-bios is still going to be used in HLS, right?

@fendor
Copy link
Collaborator

fendor commented May 10, 2020

Sorry, by hie I actually meant the whole ide story for haskell. Hie-bios is still going to be used in HLS, right?

Yes, it is currently the backbone of all ide's :)

@fendor
Copy link
Collaborator

fendor commented Jun 16, 2023

hie-bios uses a different builddir now, so I think this issue is outdated and not relevant anymore.

@fendor fendor closed this as completed Jun 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants