Update codebase to GHC to 9.x series

[JordanMartinez]

Summary

Finish what was started in #4326

Motivation

Besides the compiler improvements, this would allow us to use the RecordDotSyntax extension.

Proposal

#4325 was originally blocked by three issues:

• weeder needed to be updated
• protolude needed to be updated
• the GHC API error message (see Update to GHC 9.2.2 #4326's opening thread)
• the change shouldn't cause us to update glibc (see Update to GHC 9.2.2 #4326 (comment))

weeder@2.4.0 and protolude@0.3.1 have both been uploaded to Hackage. So, we're just blocked on the last two issues.

This would be great. I don't think PureScript is actually compilable right now.

The project says it should compile with GHC 8.10.7. However:

• purescript.cabal depends on bower-json==1.0.0.1 (via a "tight" upper/lower bound combination)
• bower-json==1.0.0.1 requires a very old version of aeson. According to Hackage, no successful builds of bower-json have occurred other than with aeson==1.0.2.1.
• All versions of aeson<1.4.4.0 are no longer compilable with recent GHC (including 8.10.7) due to a past breaking language change in the prelude (MonadFail).
• bower-json==1.0.0.1 does not build with aeson==1.4.4.0.
• If you "jailbreak" purescript==0.15.2 and try to build it against bower-json==1.1.0.0 you get build failures with an error nearly identical to this one.

I've always wanted a "strict Haskell", and would love to play around with purescript(-native), but learning a new language is a big time investment, and being able to get the compiler to build predictably, using a process that I understand, is sort of a precondition for me making that investment.

Anyways, the above was the unhappy outcome of the ~10+ hours I spent trying to produce a usable build-from-source of the purescript compiler using cabal2nix. Nixpkgs gave up trying long ago and now simply unpacks the Debian binaries and patchelfs them. Maybe I could wave the magic haskell-stack wand and wind up with a binary that works, but I'd have no idea what stack did in order to produce it... and that isn't a workable starting point for future hacking on the compiler, which I'll want to be able to do.

[rhendric]

@a-m-joseph, for what it's worth, my current dev machine is NixOS, and I have no difficulty compiling the current master branch of PureScript with GHC 8.10.7. I do use stack in my normal workflow, but I just nix-shell'd a cabal-install and that works too. The version of aeson we use is 1.5.6.0, which doesn't seem to cause any issues with bower-json.

The version of aeson we use is 1.5.6.0, which doesn't seem to cause any issues with bower-json.

Hackage seems to disagree: All reported builds failed as of 2022-05-15 for the latest version 1.1.0.0 and last success reported on 2016-12-25 for the previous version 1.0.0.1, using aeson-1.0.0.1 (which no longer compiles using recent GHC). If as you say there were no problems shouldn't we expect to see a successful build in there somewhere?

What version of bower-json are you using? Could you provide the exact version numbers of all the packages in purescript's dependency tree? I think that cabal freeze does this.

@a-m-joseph, for what it's worth, my current dev machine is NixOS

I appreciate your reply, but I think you've misunderstood my concern; this isn't really about Nix. I want to (a) build purescript (including all its dependencies) entirely from source and (b) have a enough understanding of the build process to be able to repeat it without tools that access the network. Writing a Nix expression is an example of a sufficient but not necessary condition for (b) -- assuming it works with --option trusted-substituters "", which is not the default.

I just nix-shell'd a cabal-install and that works too.

Yes of course, and I'm sure there was plenty of network access going on there. And Nix didn't let you put the build results in /nix/store because of that; it doesn't guarantee perfectly reproducible builds, but it's pretty good at preventing obvious "cheating". I've also had a hard time figuring out how to force cabal-install to never download binaries no matter what.

no difficulty compiling the current master branch of PureScript

I'd been using the 0.15.2 release, but just in case I tried e4c98bb instead, and got the following (using bower-json-1.1.0.0):

[111 of 177] Compiling Language.PureScript.CoreFn.ToJSON ( src/Language/PureScript/CoreFn/ToJSON.hs, dist/build/Language/PureScript/CoreFn/ToJSON.o, dist/build/Language/PureScript/CoreFn/ToJSON.dyn_o )
src/Language/PureScript/CoreFn/ToJSON.hs:33:7: error:
    • Couldn't match expected type ‘Key’ with actual type ‘Text’
    • In the first argument of ‘(.=)’, namely ‘T.pack "metaType"’
      In the expression: T.pack "metaType" .= "IsConstructor"
      In the first argument of ‘object’, namely
        ‘[T.pack "metaType" .= "IsConstructor",
          T.pack "constructorType" .= constructorTypeToJSON t,
          T.pack "identifiers" .= identToJSON `map` is]’
   |
33 |     [ T.pack "metaType"         .= "IsConstructor"
   |       ^^^^^^^^^^^^^^^^^
src/Language/PureScript/CoreFn/ToJSON.hs:34:7: error:
    • Couldn't match expected type ‘Key’ with actual type ‘Text’
    • In the first argument of ‘(.=)’, namely
        ‘T.pack "constructorType"’
      In the expression:
        T.pack "constructorType" .= constructorTypeToJSON t
      In the first argument of ‘object’, namely
        ‘[T.pack "metaType" .= "IsConstructor",
          T.pack "constructorType" .= constructorTypeToJSON t,
          T.pack "identifiers" .= identToJSON `map` is]’
   |
34 |     , T.pack "constructorType"  .= constructorTypeToJSON t
   |       ^^^^^^^^^^^^^^^^^^^^^^^^

... and much more of the same.

[rhendric]

If as you say there were no problems shouldn't we expect to see a successful build in there somewhere?

Does Hackage run another build every time a dependency version changes? What I see is consistent with Hackage running a build against the latest version of aeson available at the time and then not rebuilding until bower-json's version bumped. That tells us nothing about which other versions of aeson work with bower-json-1.0.0.1.

Could you provide the exact version numbers of all the packages in purescript's dependency tree?

stack ls dependencies

Cabal 3.2.1.0
Glob 0.10.1
StateVar 1.2.2
adjunctions 4.4
aeson 1.5.6.0
aeson-better-errors 0.9.1.0
aeson-pretty 0.8.9
alex 3.2.6
ansi-terminal 0.11
ansi-wl-pprint 0.6.9
array 0.5.4.0
assoc 1.0.2
async 2.2.4
attoparsec 0.13.2.5
auto-update 0.1.6
base 4.14.3.0
base-compat 0.11.2
base-compat-batteries 0.11.2
base-orphans 0.8.6
basement 0.0.12
bifunctors 5.5.11
binary 0.8.8.0
blaze-builder 0.4.2.2
blaze-html 0.9.1.2
blaze-markup 0.8.2.8
bower-json 1.0.0.1
boxes 0.1.5
bytestring 0.10.12.0
cabal-doctest 1.0.8
call-stack 0.3.0
case-insensitive 1.2.1.0
cborg 0.2.6.0
cheapskate 0.1.1.2
clock 0.8.2
colour 2.3.6
comonad 5.0.8
conduit 1.3.4.2
conduit-extra 1.3.5
constraints 0.13.1
containers 0.6.5.1
contravariant 1.5.5
cryptonite 0.29
css-text 0.1.3.0
data-default 0.7.1.1
data-default-class 0.1.2.0
data-default-instances-containers 0.0.1
data-default-instances-dlist 0.0.1
data-default-instances-old-locale 0.0.1
data-fix 0.3.2
data-ordlist 0.4.7.0
deepseq 1.4.4.0
directory 1.3.6.0
distributive 0.6.2.1
dlist 1.0
easy-file 0.2.2
edit-distance 0.2.2.1
enclosed-exceptions 1.0.3
exceptions 0.10.4
fast-logger 3.0.5
file-embed 0.0.15.0
filepath 1.4.2.1
free 5.1.7
fsnotify 0.3.0.1
ghc-boot-th 8.10.7
ghc-prim 0.6.1
gitrev 1.3.1
half 0.3.1
happy 1.20.0
hashable 1.3.0.0
haskeline 0.8.2
hinotify 0.4.1
http-types 0.12.3
indexed-traversable 0.1.2
indexed-traversable-instances 0.1
integer-gmp 1.0.3.0
integer-logarithms 1.0.3.1
invariant 0.5.5
kan-extensions 5.2.3
language-javascript 0.7.0.0
lens 4.19.2
lifted-async 0.10.2.2
lifted-base 0.2.3.12
memory 0.15.0
monad-control 1.0.3.1
monad-logger 0.3.36
monad-loops 0.4.3
mono-traversable 1.0.15.3
monoidal-containers 0.6.2.0
mtl 2.2.2
mtl-compat 0.2.2
network 3.1.1.1
network-uri 2.6.4.1
newtype 0.2.2.0
old-locale 1.0.0.7
old-time 1.1.0.3
optparse-applicative 0.16.1.0
parallel 3.2.2.0
parsec 3.1.14.0
pattern-arrows 0.0.2
pretty 1.1.3.6
primitive 0.7.3.0
process 1.6.13.1
profunctors 5.6.2
protolude 0.3.0
purescript 0.15.2
random 1.2.0
reflection 2.1.6
regex-base 0.94.0.1
regex-tdfa 1.3.1.1
resourcet 1.2.4.3
rts 1.0.1
safe 0.3.19
scientific 0.3.7.0
semialign 1.1.0.1
semigroupoids 5.3.6
semigroups 0.19.2
serialise 0.2.4.0
shelly 1.9.0
sourcemap 0.1.6.1
split 0.2.3.4
splitmix 0.1.0.4
stm 2.5.0.1
stm-chans 3.0.0.6
streaming-commons 0.2.2.1
strict 0.4.0.1
stringsearch 0.3.6.6
syb 0.7.2.1
tagged 0.8.6.1
tagsoup 0.14.8
template-haskell 2.16.0.0
text 1.2.4.1
th-abstraction 0.4.3.0
th-compat 0.1.3
these 1.1.1.1
time 1.9.3
time-compat 1.9.5
transformers 0.5.6.2
transformers-base 0.4.6
transformers-compat 0.6.6
type-equality 1
typed-process 0.2.7.0
uniplate 1.6.13
unix 2.7.2.2
unix-compat 0.5.3
unix-time 0.4.7
unliftio-core 0.2.0.1
unordered-containers 0.2.14.0
utf8-string 1.0.2
uuid-types 1.0.5
vector 0.12.3.1
vector-algorithms 0.8.0.4
void 0.7.3
witherable 0.4.2
xss-sanitize 0.3.6
zlib 0.6.2.3

Yes of course, and I'm sure there was plenty of network access going on there.

To fetch source tarballs, sure. I didn't audit the network traffic but it sure looked like everything was being built locally. Is that not good enough for your purposes?

I'd been using the 0.15.2 release, but just in case I tried e4c98bb instead, and got the following (using bower-json-1.1.0.0):

Yeah, we don't support bower-json-1.1.0.0. Follow the constraints in purescript.cabal (which cabal2nix famously doesn't do) and you should be okay.

Could you provide the exact version numbers of all the packages in purescript's dependency tree?

stack ls dependencies
...
bower-json 1.0.0.1

Okay:

user@moore:/tmp$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.10.7
user@moore:/tmp$ cabal --version
cabal-install version 3.6.2.0
user@moore:/tmp$ cabal get bower-json==1.0.0.1
Downloading  bower-json-1.0.0.1
Downloaded   bower-json-1.0.0.1
Unpacking to bower-json-1.0.0.1/
user@moore:/tmp$ cd bower-json-1.0.0.1/
user@moore:/tmp/bower-json-1.0.0.1$ cabal build
Resolving dependencies...
Build profile: -w ghc-8.10.7 -O1

... building lots of other packages ...

Configuring library for bower-json-1.0.0.1..
Preprocessing library for bower-json-1.0.0.1..
Building library for bower-json-1.0.0.1..
[1 of 2] Compiling Web.Bower.PackageMeta.Internal ( src/Web/Bower/PackageMeta/Internal.hs, /tmp/bower-json-1.0.0.1/dist-newstyle/build/x86_64-linux/ghc-8.10.7/bower-json-1.0.0.1/build/Web/Bower/PackageMeta/Internal.o, /tmp/bower-json-1.0.0.1/dist-newstyle/build/x86_64-linux/ghc-8.10.7/bower-json-1.0.0.1/build/Web/Bower/PackageMeta/Internal.dyn_o )

src/Web/Bower/PackageMeta/Internal.hs:333:34: error:
    * Couldn't match expected type `Aeson.Key' with actual type `Text'
    * In the first argument of `(.=)', namely `k'
      In the expression: k .= val
      In the expression: [k .= val]
    |
333 | maybePair k = maybe [] (\val -> [k .= val])
    |                                  ^

src/Web/Bower/PackageMeta/Internal.hs:337:24: error:
    * Couldn't match expected type `Aeson.Key' with actual type `Text'
    * In the first argument of `(.=)', namely `k'
      In the expression: k .= xs
      In the expression: [k .= xs]
    |
337 | maybeArrayPair k xs = [k .= xs]
    |                        ^

src/Web/Bower/PackageMeta/Internal.hs:341:31: error:
    * Couldn't match expected type `Aeson.Key' with actual type `Text'
    * In the first argument of `(.=)', namely `k'
      In the expression:
        k .= Aeson.object (map (\ (k', v) -> f k' .= v) xs)
      In the expression:
        [k .= Aeson.object (map (\ (k', v) -> f k' .= v) xs)]
    |
341 | maybeArrayAssocPair f k xs = [k .= A.object (map (\(k', v) -> f k' .= v) xs)]
    |                               ^

src/Web/Bower/PackageMeta/Internal.hs:341:63: error:
    * Couldn't match expected type `Aeson.Key' with actual type `Text'
    * In the first argument of `(.=)', namely f k'
      In the expression: f k' .= v
      In the first argument of `map', namely `(\ (k', v) -> f k' .= v)'
    |
341 | maybeArrayAssocPair f k xs = [k .= A.object (map (\(k', v) -> f k' .= v) xs)]
    |                                                               ^^^^

bower-json==1.0.0.1 does not build with GHC 8.10.7.

[rhendric]

bower-json-1.0.0.1 doesn't build with the latest version of aeson, which is what cabal is going to try if you build it in isolation like that. If you give aeson the same upper bound PureScript uses in bower-json's cabal file, I bet you'll get it to work. (Bet contingent on resolving any similar errors in other dependencies in the same way.)

bower-json-1.0.0.1 doesn't build with the latest version of aeson, which is what cabal is going to try if you build it in isolation like that. If you give aeson the same upper bound PureScript uses in bower-json's cabal file, I bet you'll get it to work.

You win the bet! I hereby award you ten thousand Internets.

(Bet contingent on resolving any similar errors in other dependencies in the same way.)

This was the trick. I needed to force aeson==1.5.6.0 not only in bower-json's dependencies, but throughout all of purescript's transitive dependencies. Specifically cborg, serialize, sourcemap, and monoidal-containers. That was all it took to get a cabal2nix-based build of purescript entirely from source. I'm about to open a PR in nixpkgs to switch it over to building purescript from source, so you won't have any more people pestering you with annoying questions like this one :)

I apologize for hijacking this issue. I have collapsed all of my comments above.

[JordanMartinez]

Building off of Harry's PR to update the dependencies and resolver, I can reproduce the GHC API warning message. However, this only occurs if I run stack test --fast. stack build --fast doesn't emit that warning:

$ stack test --fast
Stack has not been tested with GHC versions above 9.0, and using 9.2.2, this may fail
Stack has not been tested with Cabal versions above 3.4, but version 3.6.3.0 was found, this may fail
purescript-0.15.2: unregistering (dependencies changed)
purescript> configure (lib + exe + test)
Configuring purescript-0.15.2...
Warning:    
    This package indirectly depends on multiple versions of the same package. This is very likely to cause a compile failure.
      package typed-process (typed-process-0.2.10.1-6jQeq0jKfAfFK6D2K5mESq) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package streaming-commons (streaming-commons-0.2.2.4-KOU4k64PnOBKRkqsAnzX4G) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package sourcemap (sourcemap-0.1.7-FTv3GmuhRLCHX3LbATMfPq) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package shelly (shelly-1.10.0-8BZvpSofFEvCGKG29F28cE) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package purescript (purescript-0.15.2) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package purescript (purescript-0.15.2) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package purescript (purescript-0.15.2) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package optparse-applicative (optparse-applicative-0.17.0.0-97wlD8mRycx5wtgSUiQCj) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package haskeline (haskeline-0.8.2-Jo68kxKNNIq8VtDDXYX8ek) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package gitrev (gitrev-1.3.1-A0e5kSeHh2fHAtDy0CBWoZ) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package conduit-extra (conduit-extra-1.3.6-2NJ6RYGqezMIR0V6mEJuBt) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package Cabal (Cabal-3.6.3.0-7bZEXcEY9bD5cTY0vBVGZv) requires process-1.6.13.1-CTZIPxbBPrP7PMGBc5UDkd
      package ghc (ghc-9.2.2) requires process-1.6.13.2

[JordanMartinez]

Looks like hspec-core depends on ghc as of 9.2.3.

[purefunctor]

As of #4351, the compiler is now built using GHC 9.2.3. I think we could keep this issue for tracking RecordDotSyntax changes.

[JordanMartinez]

@purefunctor I think that should be a new issue.

[purefunctor]

Ah, we could close this now then. Resolved by #4351.