Update nightly to 9.6

[TeofilC]

I was wondering what was blocking upgrading nightly to 9.6 and if there's anything that you folks would like help with to speed things up.

[andreasabel]

I think an LTS for 9.4 would be needed first:

• Please create LTS 21 for ghc-9.4 lts-haskell#355

[juhp]

Yep we will need LTS 21 (with ghc-9.4) first, but we can start work on a 9.6 PR for nightly - in fact it is almost^Wreally a prerequisite since we really prefer to minimise the period when both LTS and Nightly are on the same major version as much as possible.

One good thing now is that the docker (nightly) build image is no longer tied to a specific ghc since it just uses ghc via stack, so it might "Just Work" in CI perhaps and if so it could be possible for anyone to contribute (though need to test... may be missing something).

[andreasabel]

Thanks for releasing nightly for GHC 9.6!

Currently tasty is missing from the snapshot (blame says temporily disabled by @juhp), but it will probably be back soon, will it?
Or do you need help from the tasty maintainers?

[TeofilC]

Thanks!

[juhp]

I hope we can bring back tasty soon - there was some "weird" cyclic dependency reported by stack...
Hopefully just a bootstrapping issue

[juhp]

Thanks to everyone who has already worked on ghc-9.6 updating.

I can't help saying the amount of breakage around monad function exports was quite painful
(I assume the new transformers/mtl major versions?), also unix openFD,
and we bumped to optparse-applicative-0.18 which caught some more packages too.
Nevertheless I pleasantly surprised how big the initial snapshot is: almost 2400 packages. :-)

[juhp]

Here is the dependency cycle stack is still complaining about:

       Stack failed to construct a build plan.
       
       While constructing the build plan, Stack encountered the following errors:
       
       In the dependencies for tasty-hunit-0.10.0.3:
           tasty dependency cycle detected: tasty, syb, doctest, prettyprinter-ansi-terminal,
                 optparse-applicative, tasty, STMonadTrans, Agda
       needed since tasty-hunit is a build target.
       
       In the dependencies for syb-0.7.2.3:
           tasty dependency cycle detected: tasty, syb, doctest, prettyprinter-ansi-terminal,
                 optparse-applicative, tasty, STMonadTrans, Agda
       needed since syb is a build target.

Still find it puzzling since all those packages can build (have now built actually) for Nightly in the server.
But somehow restoring all the dependents induces this error again.

Apparently we have run into this before in the past, so we may open a stack issue to investigate further.

cc @mpilgrem

[juhp]

Okay skipping the syb testsuite seemed to do the trick.

See the current list of disabled testsuites and benchmarks due to cycles:

• https://github.com/commercialhaskell/stackage/blob/master/build-constraints.yaml#L8884
• https://github.com/commercialhaskell/stackage/blob/master/build-constraints.yaml#L9994

[mpilgrem]

@juhp, it seems to me the first question is: is that a false report by Stack, or is there, in fact, a dependency cycle? Coincidentally, I have just added a flag to the master branch version of Stack - --plan-in-log - which makes it easier to include the (lengthy) information about build plan construction in its debug output. 'Dependency cycle detected' gets triggered when a package is to be added to the list of packages to be built but it is already on the list. Stack.Build.ConstructPlan.checkCallStackAndAddDep has:

-- | Given a 'PackageName', adds all of the build tasks to build the package.
-- First checks that the package name is not already in the call stack.
checkCallStackAndAddDep ::
     PackageName
  -> M (Either ConstructPlanException AddDepRes)
checkCallStackAndAddDep name = do
  ctx <- ask
  let mpackageInfo = Map.lookup name $ combinedMap ctx
  res <- if name `elem` callStack ctx
    then do
      logDebugPlanS "checkCallStackAndAddDep" $
           "Detected cycle "
        <> fromString (packageNameString name)
        <> ": "
        <> fromString (show $ map packageNameString (callStack ctx))
      pure $ Left $ DependencyCycleDetected $ name : callStack ctx
    else ...

[andreasabel]

Once the dust has settled, would it make sense to alert everyone whose packages dropped out from nightly, like:

• Nightly ghc-9.2 disabled package report (mentioned maintainers are affected) #6486

[ysangkok]

If such an alert is sent, may I propose also pointing out that Hackage now allows for an automatic notification. If more people enabled this feature, it could save us work.

[andreasabel]

@ysangkok : Wow, great feature!
Can I bulk-enable it, or enable it via the command line? (I am not a great fan of clicking sprints...)
Ok, it is not a per-package setting, but a per-user setting. (So no clicking sprint needed.)

[juhp]

Once the dust has settled, would it make sense to alert everyone whose packages dropped out from nightly

Well +1, but I don't think we have a script/tooling for that currently: I am not sure how @bergmark did that?
The most information currently is in the build-constraints.yaml file, together with git history.