Fix `cabal build` by freezing cabal dependencies #5677

simnalamburt · 2022-02-12T20:20:26Z

To easily package stack to various distributions, one should be able to bootstrap stack without relying on stack binary. So many package maintainers use cabal-install to build stack from source code.

Due to recent update of aeson-2.0.0.0, stack can no longer be built with cabal-install. So I hereby send a PR to freeze cabal dependencies using stackage 17.15 LTS.

Reference:

simnalamburt · 2022-02-16T16:07:15Z

FYI, this PR fixes the major issue with brew install haskell-stack and brew install haskell-stack --HEAD. Currently, the only way for macOS ARM users to use haskell stack is manually compiling it.

- Fixed building from source. Due to recent update of aeson-2.0.0.0, stack can no longer be built with cabal-install. So I patched stack to freeze cabal dependencies using stackage 17.15 LTS. - Add Apple Silicon support. HEAD version of stack has already been patched to support Apple Silicon. However, the next release containing that patch hasn't release yet. So I manually patched stack v2.7.3 to support Apple Silicon. - Add llvm@12 as a dependency only on ARM All ghc versions before 9.2.1 requires LLVM Code Generator as a backend on ARM. Fixes Homebrew#94796 Reference: commercialhaskell/stack#5562 commercialhaskell/stack#5677

- Fixed building from source. Due to recent update of aeson-2.0.0.0, stack can no longer be built with cabal-install. So I patched stack to freeze cabal dependencies using stackage 17.15 LTS. - Add Apple Silicon support. HEAD version of stack has already been patched to support Apple Silicon. However, the next release containing that patch hasn't release yet. So I manually patched stack v2.7.3 to support Apple Silicon. - Add llvm@12 as a dependency only on ARM All ghc versions before 9.2.1 requires LLVM Code Generator as a backend on ARM. Fixes #94796 Reference: commercialhaskell/stack#5562 commercialhaskell/stack#5677

fwcd · 2022-03-18T17:41:53Z

@borsboom Any chance to get this reviewed/merged?

To easily package stack to various distributions, one should be able to bootstrap stack without relying on stack binary. So many package maintainers use cabal-install to build stack from source code. Due to recent update of aeson-2.0.0.0, stack can no longer be built with cabal-install. So I hereby send a PR to freeze cabal dependencies using stackage 17.15 LTS. Reference: https://hackage.haskell.org/package/stack2cabal

philderbeast · 2022-07-11T18:45:55Z

I've tested this locally and can cabal build all --enable-tests with ghc-8.10.7 but with ghc-9.0.2 and ghc-9.2.3 I see failures:

> cabal build all --enable-tests
Resolving dependencies...
Build profile: -w ghc-9.0.2 -O1
...
Failed to build memory-0.15.0.
...
[21 of 29] Compiling Data.ByteArray.ScrubbedBytes

Data/ByteArray/ScrubbedBytes.hs:93:71: error:
    Variable not in scope:
      unsafeCoerce# :: MutableByteArray# RealWorld -> ByteArray#
   |
93 |                 let !scrubber = (getScrubber sz) (byteArrayContents# (unsafeCoerce# mbarr))
   |                                                                       ^^^^^^^^^^^^^

Data/ByteArray/ScrubbedBytes.hs:151:38: error:
    Variable not in scope:
      unsafeCoerce# :: MutableByteArray# RealWorld -> ByteArray#
    |
151 |     a <- f (Ptr (byteArrayContents# (unsafeCoerce# mba)))
    |                                      ^^^^^^^^^^^^^
Error: cabal: Failed to build memory-0.15.0 (which is required by
stack-2.7.6). See the build log above for details.

> cabal build all --enable-tests
Resolving dependencies...
Build profile: -w ghc-9.2.3 -O1
...
Failed to build blaze-builder-0.4.2.1.
...
[ 9 of 11] Compiling Blaze.ByteString.Builder.HTTP

Blaze/ByteString/Builder/HTTP.hs:52:36: error:
    • Couldn't match expected type ‘Word32#’ with actual type ‘Word#’
    • In the first argument of ‘W32#’, namely
        ‘(w `uncheckedShiftRL#` i)’
      In the expression: W32# (w `uncheckedShiftRL#` i)
      In an equation for ‘shiftr_w32’:
          shiftr_w32 (W32# w) (I# i) = W32# (w `uncheckedShiftRL#` i)
   |
52 | shiftr_w32 (W32# w) (I# i) = W32# (w `uncheckedShiftRL#`   i)
   |                                    ^^^^^^^^^^^^^^^^^^^^^^^^^

Failed to build base-compat-0.11.2.
...
[ 63 of 118] Compiling Data.Semigroup.Compat
src/Data/Semigroup/Compat.hs:25:5: error:
    Not in scope: type constructor or class ‘Option’
   |
25 |   , Option(..)
   |     ^^^^^^^^^^

src/Data/Semigroup/Compat.hs:26:5: error: Not in scope: ‘option’
   |
26 |   , option
   |     ^^^^^^

Failed to build basement-0.0.12.
...
[19 of 84] Compiling Basement.Numerical.Conversion

Basement/Numerical/Conversion.hs:84:46: error:
    • Couldn't match expected type ‘Word32#’ with actual type ‘Word#’
    • In the first argument of ‘W32#’, namely
        ‘(uncheckedShiftRL# w64 32#)’
      In the first argument of ‘Word32x2’, namely
        ‘(W32# (uncheckedShiftRL# w64 32#))’
      In the expression:
        Word32x2
          (W32# (uncheckedShiftRL# w64 32#)) (W32# (narrow32Word# w64))
   |
84 | word64ToWord32s (W64# w64) = Word32x2 (W32# (uncheckedShiftRL# w64 32#)) (W32# (narrow32Word# w64))
   |                                              ^^^^^^^^^^^^^^^^^^^^^^^^^

Basement/Numerical/Conversion.hs:84:81: error:
    • Couldn't match expected type ‘Word32#’ with actual type ‘Word#’
    • In the first argument of ‘W32#’, namely ‘(narrow32Word# w64)’
      In the second argument of ‘Word32x2’, namely
        ‘(W32# (narrow32Word# w64))’
      In the expression:
        Word32x2
          (W32# (uncheckedShiftRL# w64 32#)) (W32# (narrow32Word# w64))
   |
84 | word64ToWord32s (W64# w64) = Word32x2 (W32# (uncheckedShiftRL# w64 32#)) (W32# (narrow32Word# w64))
   |                                                                                 ^^^^^^^^^^^^^^^^^

Failed to build cereal-0.5.8.1.
...
[1 of 4] Compiling Data.Serialize.Get

src/Data/Serialize/Get.hs:726:36: error:
    • Couldn't match expected type ‘Word16#’ with actual type ‘Word#’
    • In the first argument of ‘W16#’, namely
        ‘(w `uncheckedShiftL#` i)’
      In the expression: W16# (w `uncheckedShiftL#` i)
      In an equation for ‘shiftl_w16’:
          shiftl_w16 (W16# w) (I# i) = W16# (w `uncheckedShiftL#` i)
    |
726 | shiftl_w16 (W16# w) (I# i) = W16# (w `uncheckedShiftL#`   i)
    |                                    ^^^^^^^^^^^^^^^^^^^^^^^^

src/Data/Serialize/Get.hs:727:36: error:
    • Couldn't match expected type ‘Word32#’ with actual type ‘Word#’
    • In the first argument of ‘W32#’, namely
        ‘(w `uncheckedShiftL#` i)’
      In the expression: W32# (w `uncheckedShiftL#` i)
      In an equation for ‘shiftl_w32’:
          shiftl_w32 (W32# w) (I# i) = W32# (w `uncheckedShiftL#` i)
    |
727 | shiftl_w32 (W32# w) (I# i) = W32# (w `uncheckedShiftL#`   i)
    |                                    ^^^^^^^^^^^^^^^^^^^^^^^^
Error: cabal: Failed to build blaze-builder-0.4.2.1 (which is required by
stack-2.7.6). See the build log above for details.
Failed to build base-compat-0.11.2 (which is required by stack-2.7.6). See the
build log above for details.
Failed to build basement-0.0.12 (which is required by stack-2.7.6). See the
build log above for details.
Failed to build cereal-0.5.8.1 (which is required by stack-2.7.6). See the
build log above for details.

mpilgrem · 2022-07-23T00:14:13Z

Is the original problem - cannot use Cabal (the tool) to build Stack - still a problem? You mention aeson-2.0.0.0 and there have been developments. I'm using a version of Stack (built from master with Stack and --stack-yaml=stack-ghc-902.yaml) and it depends on aeson-2.0.3.0.

simnalamburt · 2022-07-24T09:39:32Z

I'm closing this PR as this patch is now outdated, but we steel do need a way to create a reproducible build without depending on stack itself.

hasufell · 2022-07-24T10:46:19Z

I use stack2cabal (on hackage) to build stack with cabal. It creates freeze file and cabal.project from stack.yaml.

simnalamburt · 2022-07-24T10:47:31Z

Yeah this PR was to embed freeze file into this package not to depend on stack2cabal.

hasufell · 2022-07-24T11:16:00Z

Yeah this PR was to embed freeze file into this package not to depend on stack2cabal.

I agree it's worthwhile, but then there also should be post commit hooks or something that check these don't diverge.

mpilgrem · 2022-07-24T13:27:34Z

I asked myself: What stops Cabal (the tool - I'll refer to it just as 'Cabal') from being able to build Stack? Was there something about Stack's package.yaml file (which generates stack.cabal using Stack's in-built version of Hpack) that was 'malformed', from the perspective of Cabal?

After a little investigation, I think it is simply that Cabal needs to be told about an upper bound on the persistent dependency. I changed that to persistent < 2.14 in package.yaml. Having done that, this was my successful experience (on Windows 11):

I built and installed Stack with Stack (and using GHC 9.0.2): stack --stack-yaml=stack-ghc-902.yaml install. That is straightforward. (As an aside, there is an upstream bug for macOS only in the process package. That is what is currently blocking Stack moving wholesale to GHC 9.0.2 or GHC 9.2.3.)
In another directory, I used Stack to get cabal-install: stack unpack cabal-install. cd cabal-install-3.6.2.0
Following Stack's suggestions, I created a stack.yaml (cabal-install does not provide its own):

resolver: lts-18.28 # cabal-install requires base < 4.15
packages:
- .
extra-deps:
- Cabal-3.6.3.0 # cabal-install requires Cabal >= 3.6.1.0

I built and installed Cabal with Stack: stack install. That is straightforward.
stack exec -- cabal --version yields:

cabal-install version 3.6.2.0
compiled using version 3.6.3.0 of the Cabal library

However, I can't use cabal build in the Stack environment: stack exec -- cabal build yields:

cabal.exe: Use of GHC's environment variable GHC_PACKAGE_PATH is incompatible
with Cabal. Use the flag --package-db to specify a package database (it can be
used multiple times).

Which, as an aside, does not mean anything to me.
7. If I want to use Cabal outside of the Stack environment, I need to tell Cabal where to find the Stack-supplied GHC and to add the Stack-supplied MSYS2 to my PATH (Cabal warns me that MSYS2 is missing, if I forget to do that). So (extracts only of the cabal bulild output:

> stack --stack-yaml=stack-ghc-902.yaml path --compiler-exe
C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.0.2\bin\ghc-9.0.2.exe
> stack exec -- where.exe sh
C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20210604\usr\bin\sh.exe
> $Env:Path = 'C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20210604\usr\bin;' + $Env:Path
> cabal build --with-compiler=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.0.2\bin\ghc-9.0.2.exe
...
Preprocessing executable 'stack' for stack-2.7.6..
Building executable 'stack' for stack-2.7.6..
...
Linking D:\Users\mike\Code\GitHub\stack\dist-newstyle\build\x86_64-windows\ghc-9.0.2\stack-2.7.6\build\stack\stack.exe ...

Success! I'll raise a pull request to put that constraint on persistent.

…ge.yaml` Stack advises that it should be built with Stack, but some users wish to use Cabal (the tool) to build Stack. This is possible if the upper bound on `persistent` is specified.

Fix #5677 Add upper bound for `persistent` in `package.yaml`

simnalamburt mentioned this pull request Feb 12, 2022

Cannot install haskell "stack" HEAD macOS Monterey 12.2 Homebrew/homebrew-core#94796

Closed

2 tasks

simnalamburt mentioned this pull request Feb 12, 2022

haskell-stack: Fix building from source, add Apple Silicon support Homebrew/homebrew-core#95032

Merged

7 tasks

simnalamburt changed the title ~~Freeze cabal dependencies~~ Fix cabal build by freezing cabal dependencies Feb 16, 2022

simnalamburt mentioned this pull request Feb 16, 2022

stack setup installs GHC for aarch64 on M1 Macs #5562

Merged

fwcd mentioned this pull request Feb 25, 2022

Cannot build using GHC 8.10.7 and cabal-install 3.6.2.0 on aarch64 Linux #5684

Closed

hasufell mentioned this pull request Mar 11, 2022

Fail to build with cabal-install #5298

Closed

adamsteen mentioned this pull request Mar 12, 2022

Official OpenBSD support: last steps #3313

Open

5 tasks

simnalamburt force-pushed the cabal-freeze branch from b2a2e56 to fc95a26 Compare March 31, 2022 12:37

simnalamburt force-pushed the cabal-freeze branch from fc95a26 to 2daa257 Compare March 31, 2022 12:38

philderbeast mentioned this pull request Jul 11, 2022

Ignore dist-newstyle folder. #5776

Merged

simnalamburt closed this Jul 24, 2022

mpilgrem added a commit that referenced this pull request Jul 24, 2022

Merge pull request #5791 from mpilgrem/fix5677

5b6fb13

Fix #5677 Add upper bound for `persistent` in `package.yaml`

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix `cabal build` by freezing cabal dependencies #5677

Fix `cabal build` by freezing cabal dependencies #5677

simnalamburt commented Feb 12, 2022 •

edited

simnalamburt commented Feb 16, 2022

fwcd commented Mar 18, 2022

philderbeast commented Jul 11, 2022

mpilgrem commented Jul 23, 2022

simnalamburt commented Jul 24, 2022

hasufell commented Jul 24, 2022

simnalamburt commented Jul 24, 2022

hasufell commented Jul 24, 2022

mpilgrem commented Jul 24, 2022 •

edited

Fix cabal build by freezing cabal dependencies #5677

Fix cabal build by freezing cabal dependencies #5677

Conversation

simnalamburt commented Feb 12, 2022 • edited

Reference:

simnalamburt commented Feb 16, 2022

fwcd commented Mar 18, 2022

philderbeast commented Jul 11, 2022

mpilgrem commented Jul 23, 2022

simnalamburt commented Jul 24, 2022

hasufell commented Jul 24, 2022

simnalamburt commented Jul 24, 2022

hasufell commented Jul 24, 2022

mpilgrem commented Jul 24, 2022 • edited

Fix `cabal build` by freezing cabal dependencies #5677

Fix `cabal build` by freezing cabal dependencies #5677

simnalamburt commented Feb 12, 2022 •

edited

mpilgrem commented Jul 24, 2022 •

edited