July 2021 monthly release (1.3.0) with limited ghc 9.0 support (needs hackage release) #1990

jneira · 2021-06-30T08:42:28Z

The release also would include, among other things:

The lately amazing work done in the wingman plugin by @isovector
Binaries for ghc-9.0.1 (thanks to @anka-213)
- Those binaries dont include all plugins and did not pass the entire test suite but i think we could inlcude it with noting they are in beta state (would love to know your opinion about this)
Artifacts for ghcup for the three os's and a SHA256SUMS with the checksums from all artifacts which could be gpg signed (not sure if we will able to sign them in this one though)
Checklist of upstream packages we need up to date to make the hackage release:
- ghc-api-compat:
  - https://github.com/hsyl20/ghc-api-compat/pull/11
  - https://gitlab.haskell.org/haskell/ghc-api-compat/-/issues/1#note_371055
  - cabal check doesn't account for conditionals cabal#4629
  - (pending pr to remove it from hls)
- remove allow-newer: Clean cabal project #2097

The text was updated successfully, but these errors were encountered:

konn · 2021-06-30T08:51:12Z

I think we could send to lsp clients, when called for ghc 9, some "warning" that noting GHC 9.0.1 support is in the beta stage with partial functionalities and pointing to GHC 9 tracking issue #297.
I think it is likely that some people might not be aware of release note (e.g. automaticaljy updated/installed by lsp client), and sporadically report it as independent issues - so some auxiliary pointer at the editor level could prevent such cases.

jneira · 2021-06-30T10:10:39Z

I think we could send to lsp clients, when called for ghc 9, some "warning" that noting GHC 9.0.1 support is in the beta stage with partial functionalities and pointing to GHC 9 tracking issue #297.

Good idea, a warning popup would be great

anka-213 · 2021-06-30T10:12:24Z

Oh, does it no longer pass the test suite? What broke?

jneira · 2021-06-30T10:17:44Z

Oh, does it no longer pass the test suite? What broke?

Dont worry enabled tests are passing but afaics ghc-9.0 is not tested at all in windows and macOS ~~and the tests of all plugins (execpt hls-haddock-comments-plugin), including the enabled ones for ghc-9.0 are not being executed~~.

After a deeper look, the hls test suite is being executed and some plugins still are included in it (all the enabled ones for ghc-9.0.1)

jneira · 2021-06-30T10:25:27Z

In fact we are not testing macOS at all 😟

anka-213 · 2021-06-30T10:26:12Z

I have run all the tests manually on MacOS on my machine and they worked fine there last I checked. I don't know about windows. But yes, we could enable the tests for ghc9 on windows and macos at the cost of more CI time.

anka-213 · 2021-06-30T10:35:00Z

(I believe both windows and macos are significantly more expensive than ubuntu for github's ci time calculations, but I don't know how that matters for this project)

jneira · 2021-06-30T10:51:17Z

I've activated test for windows and macOs in my repo, lets see how they are going: https://github.com/jneira/haskell-language-server/actions/runs/986014293

jneira · 2021-06-30T12:48:57Z

Not bad at all for Windows (the macOS job was cancelled):

1 out of 559 tests failed (1414.12s)

The failing test:

TH Splice Hover:                                                                                    FAIL (2.22s)
        test\exe\Main.hs:3560:
        failed to find: `Identity` in hover message:
        
        '''haskell
        id :: forall a. a -> a
        '''
        
        *Defined in 'GHC.Base'*
        
        
        
        * * *
        
        '''haskell
        _ :: Q Exp -> Q Exp
        '''

konn · 2021-07-01T04:04:32Z

I think we could send to lsp clients, when called for ghc 9, some "warning" that noting GHC 9.0.1 support is in the beta stage with partial functionalities and pointing to GHC 9 tracking issue #297.

Good idea, a warning popup would be great

OK, I'm tring this in #1992!

jneira · 2021-07-13T20:13:08Z

Would we want to see some pending pr in the next release? In the vscode extension side we have to add the alert when downloading 9.0

jneira · 2021-07-20T09:51:30Z

I am afraid i have not much free time as i would like to lately, if someone want to drive the release the work would be very welcomed 😟

Ailrun · 2021-07-20T21:11:19Z

I can take it over. Probably I have at least that amount of time span...

Ailrun · 2021-07-21T01:21:19Z

BTW; I think it will be great if #1955 can be merged within the timeframe of this release (and I guess it's feasible?). So I will wait that if no one objects.

July541 · 2021-07-21T04:59:11Z

@Ailrun It has the basic function now, and I'll push it today asap, release the current version is OK. But it doesn't support File of Interest, which means the file opened in the editor may not acquire precisive results. Implement it needs at least 3-4 days probably, I have an initial thought and now working for it, would you mind waiting?

Ailrun · 2021-07-21T05:01:50Z

@July541 I think a few days of delay is OK (though it may become August release than July release :) ), and releasing more fully-featured one is worth the delay.

isovector · 2021-07-21T05:50:38Z

I'd like to land #1873 as well

Ailrun · 2021-07-27T21:52:09Z

@isovector, do you want to include #2038 into this release? Otherwise I will prepare the release once #2040 is merged

C.C. @pepeiborra

isovector · 2021-07-27T21:59:56Z

@Ailrun I plan to merge #2038 as soon as CI gets fixed (ideally immediately after #2044), but it's not critical for the release. Feel free to cut the release whenever is convenient for you!

jneira · 2021-07-30T09:52:19Z

@hasufell hi! we should synchronize our release with the ghcup update. Only want to note that, until we include the binary for windows and ghc-8.8.4, as standalone zip and inside the ghcup tar and the sha list, the release is not complete (we will not unmark the prerelease check until that happens)

Ailrun · 2021-07-30T17:15:06Z

@jneira, could you upload GHC 8.8.4 if you have time for that? I think SHA256SUMS should also be fixed with that though

jneira · 2021-07-30T17:34:40Z

the release has got me away from my laptop, will start the build asap
my last trick to make it work was set -O0 for floskell

jneira · 2021-07-31T10:46:03Z

i've uploaded the bin for windows and ghc-8.8.4 and udpated the windows .tar.gz and sha256sum, makint the release definitive, so it is available 🎉
i am afraid that the related vscode extension publish has not been done due to ci problems, but the actual version will download the new version (but with no other warning than the emitted in the hls log)
haskell/vscode-haskell#385 (comment)
//cc @hasufell

jneira · 2021-08-10T11:50:01Z

We will need to backport Build with lsp 1.2.0.1 #2059 to the branch (to be created) 1.3.0-hackage (or restrict the upper bound over lsp/lsp-types). See 1.2.0 hackage release fails to build due to last lsp-types release #2087

Ailrun · 2021-08-10T18:09:06Z

Thank you for the note! I added the branch and created the backport PR

jneira · 2021-08-11T06:22:00Z

We have a terrible list of allow-newer and a source repository package over ghc-api-compat which has to be removed to get the hackage release:

haskell-language-server/cabal.project

Lines 32 to 86 in b8b9620

    
           -- ghc-api-compat-8.6 
        
           source-repository-package 
        
             type: git 
        
             location: https://github.com/hsyl20/ghc-api-compat 
        
             tag: 8fee87eac97a538dbe81ff1ab18cff10f2f9fa15 
        
           write-ghc-environment-files: never 
        
           index-state: 2021-08-08T02:21:16Z 
        
           constraints: 
        
               -- Diagrams doesn't support optparse-applicative >= 0.16 yet 
        
               optparse-applicative < 0.16, 
        
               hyphenation +embed 
        
           allow-newer: 
        
               active:base, 
        
               assoc:base, 
        
               cryptohash-md5:base, 
        
               cryptohash-sha1:base, 
        
               constraints-extras:template-haskell, 
        
               data-tree-print:base, 
        
               deepseq:base, 
        
               dependent-sum:some, 
        
               dependent-sum:constraints, 
        
               diagrams-contrib:base, 
        
               diagrams-contrib:lens, 
        
               diagrams-contrib:random, 
        
               diagrams-core:base, 
        
               diagrams-core:lens, 
        
               diagrams-lib:base, 
        
               diagrams-lib:lens, 
        
               diagrams-postscript:base, 
        
               diagrams-postscript:lens, 
        
               diagrams-svg:base, 
        
               diagrams-svg:lens, 
        
               dual-tree:base, 
        
               -- Does this make any sense? 
        
               entropy:Cabal, 
        
               force-layout:base, 
        
               force-layout:lens, 
        
               floskell:ghc-prim, 
        
               floskell:base, 
        
               hashable:base, 
        
               hslogger:base, 
        
               monoid-extras:base, 
        
               newtype-generics:base, 
        
               parallel:base, 
        
               regex-base:base, 
        
               regex-tdfa:base, 
        
               statestack:base, 
        
               svg-builder:base, 
        
               these:base, 
        
               time-compat:base

Also we should check and fix the optparse-applicative < 0.16 thing

jneira · 2021-08-11T06:26:22Z

And ideally we should make packages not buildables for ghc >= 9.0.1 via base upper bound (as we need lot of dependencies hackage releases)

jneira · 2021-08-13T12:15:40Z

Ok, i am giving a try, removing all things from cabal.project that might block the hackage release:

It finds a build plan and builds succesfully for ghc-8.8.4
But for ghc-8.10.5 obviously it fails

Resolving dependencies...
Error:
    Problem with module re-exports:
      - The module 'GHC.Platform.ARM64'
        is not exported by any suitable package.
        It occurs in neither the 'exposed-modules' of this package,
        nor any of its 'build-depends' dependencies.
    In the stanza 'library'
    In the package 'ghc-api-compat-8.6'

Due to removing

source-repository-package
  type: git
  location: https://github.com/hsyl20/ghc-api-compat
  tag: 8fee87eac97a538dbe81ff1ab18cff10f2f9fa15

So we would need a ghc-api-compat release

jneira · 2021-08-13T12:17:35Z

tbh, given the amount of upstream packages we need up to date in hackage, i dont know if a hackage release will be possible before the next github one

jneira · 2021-08-13T12:31:46Z

I am gonna to start the compilation of packages and correlated prs/issues in the description of the issue 😓

michaelpj · 2021-08-13T13:41:27Z

It may not be a popular position, but if we're serious about releasing to Hackage regularly we need to try really hard to avoid pulling things in via source-repository-package and instead get upstream to do Hackage releases :/

jneira · 2021-08-13T14:59:02Z

same for allow-newer's, for them and source repo packages I would ask to include a comment linking the relevant issue/pr which resolution will allow remove it
will update contribute guidelines with that, just in case someone reads them 😝

pepeiborra · 2021-08-13T18:58:37Z

Ok, i am giving a try, removing all things from cabal.project that might block the hackage release:

It finds a build plan and builds succesfully for ghc-8.8.4

But for ghc-8.10.5 obviously it fails
Resolving dependencies...

Error:

    Problem with module re-exports:

      - The module 'GHC.Platform.ARM64'

        is not exported by any suitable package.

        It occurs in neither the 'exposed-modules' of this package,

        nor any of its 'build-depends' dependencies.

    In the stanza 'library'

    In the package 'ghc-api-compat-8.6'
Due to removing
source-repository-package

  type: git

  location: https://github.com/hsyl20/ghc-api-compat

  tag: 8fee87eac97a538dbe81ff1ab18cff10f2f9fa15
So we would need a ghc-api-compat release

I vaguely remember ghc-api-compat cannot be uploaded to Hackage anymore, at least from HEAD. Maybe we need to backport the 8.10.5 fix and ask for a minor release /cc @hsyl20

Are there any other blockers for 8.10.x?

jneira · 2021-08-13T20:34:34Z

yeah, I commented in the ghc-api-compat issue I linked in the description: https://gitlab.haskell.org/haskell/ghc-api-compat/-/issues/1#note_371055
I have to try removing allow-newer and the constraint and see if they need some upstream releases too.
Some of them are related with benchmarks and my plan is create a special cabal-benchmark.project (in master too) to isolate all the related workarounds (including the custom shake-bench)

jneira · 2021-08-16T20:44:48Z

Ok, finally we got to remove all the blocking allow-newer with no further changes 😅 (so they were obsolete)
The unique blocker then would be the release of ghc-api-compat, which needs fixes in cabal/hackage: haskell/cabal#4629

jneira · 2021-08-18T09:44:27Z

As the needed fixes in cabal/hackage for being able to upload ghc-api-compat will not be done in a short period of time and @fendor has raised good points against its use (maybe we could include here for completeness?) we are gonna take the alternative of drop ghc-api-compat itself.
That has be done for HieDb itself: wz1000/HieDb#36 and could be translated to hls as well.

jneira · 2021-08-25T22:21:15Z

@fendor have you got the chance to start the work to remove ghc-api-compat form hls? thanks!

fendor · 2021-08-26T07:14:30Z

Yes and I made good progress!

 cabal-ghc901.project                                                  |  31 +++-------
 ghcide/ghcide.cabal                                                   |  13 +++-
 ghcide/session-loader/Development/IDE/Session.hs                      |  40 ++++---------
 ghcide/src/Development/IDE/Core/Actions.hs                            |  13 ++--
 ghcide/src/Development/IDE/Core/Compile.hs                            | 108 ++++++++++++----------------------
 ghcide/src/Development/IDE/Core/Preprocessor.hs                       |  83 +++++++++++++-------------
 ghcide/src/Development/IDE/Core/RuleTypes.hs                          |   9 +--
 ghcide/src/Development/IDE/Core/Shake.hs                              |   8 +--
 ghcide/src/Development/IDE/GHC/CPP.hs                                 | 199 +++++++-------------------------------------------------------
 ghcide/src/Development/IDE/GHC/Compat.hs                              | 437 ++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------------------------------------------------------------
 ghcide/src/Development/IDE/GHC/Error.hs                               |  58 +++++++-----------
 ghcide/src/Development/IDE/GHC/ExactPrint.hs                          |  12 ++--
 ghcide/src/Development/IDE/GHC/Orphans.hs                             |  48 +++++++++++----
 ghcide/src/Development/IDE/GHC/Util.hs                                |  95 +++++++++++++++++++-----------
 ghcide/src/Development/IDE/GHC/Warnings.hs                            |  11 ++--
 ghcide/src/Development/IDE/Import/FindImports.hs                      |  49 +++++++---------
 ghcide/src/Development/IDE/Plugin/CodeAction/ExactPrint.hs            |   5 --
 ghcide/src/Development/IDE/Plugin/Completions/Types.hs                |   2 +-
 ghcide/src/Development/IDE/Plugin/Test.hs                             |   1 -
 ghcide/src/Development/IDE/Plugin/TypeLenses.hs                       |   1 -
 ghcide/src/Development/IDE/Spans/AtPoint.hs                           |  48 +++++++--------
 ghcide/src/Development/IDE/Spans/Common.hs                            |  21 +++----
 ghcide/src/Development/IDE/Spans/LocalBindings.hs                     |   9 +--
 ghcide/src/Development/IDE/Types/Exports.hs                           |  19 +++---
 ghcide/src/Development/IDE/Types/HscEnvEq.hs                          |  32 ++++------
 ghcide/src/Development/IDE/Types/Location.hs                          |   8 ++-
 ghcide/src/Development/IDE/Types/Options.hs                           |   4 +-
 haskell-language-server.cabal                                         |   1 -
 hls-plugin-api/hls-plugin-api.cabal                                   |   1 -
 hls-plugin-api/src/Ide/Types.hs                                       |   2 +-
 plugins/hls-call-hierarchy-plugin/hls-call-hierarchy-plugin.cabal     |   1 -
 plugins/hls-class-plugin/hls-class-plugin.cabal                       |   1 -
 plugins/hls-eval-plugin/hls-eval-plugin.cabal                         |   1 -
 plugins/hls-explicit-imports-plugin/hls-explicit-imports-plugin.cabal |   1 -
 plugins/hls-ormolu-plugin/hls-ormolu-plugin.cabal                     |   1 -
 plugins/hls-retrie-plugin/hls-retrie-plugin.cabal                     |   1 -
 36 files changed, 520 insertions(+), 854 deletions(-)

I hope to have a PR for at least lib:ghcide ready today or tomorrow, together with basic GHC 9.2 support (e.g. no plugins).

jneira · 2021-09-08T14:19:04Z

We are exercising the automatic hacka relese here: https://github.com/jneira/haskell-language-server/runs/3545669772

The first error has been with ghcide:

cabal: Could not resolve dependencies:
[__0] trying: ghcide-1.4.1.0 (user goal)
[__1] trying: retrie-1.0.0.0 (dependency of ghcide)
[__2] next goal: ghc (dependency of ghcide)
[__2] rejecting: ghc-8.6.5/installed-8.6.5 (conflict: retrie => ghc>=8.8 &&
<9.2)
[__2] trying: ghc-8.10.2
[__3] next goal: base (dependency of ghcide)
[__3] rejecting: base-4.12.0.0/installed-4.12.0.0 (conflict: ghc => base<0 &&
>=4.14 && <4.15)
[__3] skipping: base-4.15.0.0, base-4.14.3.0, base-4.14.2.0, base-4.14.1.0,
base-4.14.0.0, base-4.13.0.0, base-4.12.0.0, base-4.11.1.0, base-4.11.0.0,
base-4.10.1.0, base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0,
base-4.8.1.0, base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0,
base-4.6.0.1, base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0,
base-4.4.0.0, base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1,
base-4.2.0.0, base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1 (has the
same characteristics that caused the previous version to fail: excluded by
constraint '<0 && >=4.14 && <4.15' from 'ghc')
[__3] fail (backjumping, conflict set: base, ghc, ghcide)
Backjump limit reached (currently 4000, change with --max-backjumps or try to
run with --reorder-goals).

//cc @pepeiborra

pepeiborra · 2021-09-08T15:30:40Z

Retrie dropped support for 8.6.5 a while ago.

jneira · 2021-09-08T16:31:25Z

uhmm and how is working master?

pepeiborra · 2021-09-08T19:06:49Z

ghcide doesn't specify any lower bounds on retrie, so I suppose cabal is just using an older version of retrie which builds with ghc 8.6.

jneira · 2021-09-09T07:48:01Z

https://github.com/haskell/haskell-language-server/actions/runs/1216111548 🤞

jneira · 2021-09-09T10:41:08Z

jneira · 2021-09-09T11:01:14Z

Well it did what we want but the logic to skip existing packages with the same version and no changes (or throw error if there is changes) did not work well: hls-graph has not changed but is has been built

jneira · 2021-09-09T12:22:17Z

So this is done, if the release has any issue it will be in a new one.

Thanks to all helped in this release, specially @Ailrun, who drove the release

jneira added type: enhancement New feature or request old_type: distribution labels Jun 30, 2021

konn mentioned this issue Jul 1, 2021

Warn GHC 9 Compatibility to LSP Client #1992

Merged

jneira pinned this issue Jul 20, 2021

jneira changed the title ~~July monthly release~~ July 2021 monthly release Jul 20, 2021

jneira changed the title ~~July 2021 monthly release~~ July 2021 monthly release with limited ghc 9.0 support Jul 20, 2021

Ailrun mentioned this issue Jul 29, 2021

Prepare 2021 July Release [HLS 1.3.0] #2052

Merged

jneira added this to the 1.3.0 milestone Jul 31, 2021

Ailrun mentioned this issue Aug 11, 2021

Hackage version behind this repo: 1.2.0.0 versus 1.3.0? #2089

Closed

jneira changed the title ~~July 2021 monthly release with limited ghc 9.0 support~~ July 2021 monthly release (1.3.0) with limited ghc 9.0 support (needs hackage release) Aug 12, 2021

jneira mentioned this issue Aug 15, 2021

Clean cabal project #2097

Merged

jneira mentioned this issue Sep 8, 2021

Set bounds for retrie by ghc #2175

Closed

jneira closed this as completed Sep 9, 2021

jneira unpinned this issue Sep 9, 2021

July 2021 monthly release (1.3.0) with limited ghc 9.0 support (needs hackage release) #1990

July 2021 monthly release (1.3.0) with limited ghc 9.0 support (needs hackage release) #1990

Comments

jneira commented Jun 30, 2021 • edited

konn commented Jun 30, 2021

jneira commented Jun 30, 2021

anka-213 commented Jun 30, 2021

jneira commented Jun 30, 2021 • edited

jneira commented Jun 30, 2021

anka-213 commented Jun 30, 2021

anka-213 commented Jun 30, 2021

jneira commented Jun 30, 2021

jneira commented Jun 30, 2021 • edited

konn commented Jul 1, 2021

jneira commented Jul 13, 2021

jneira commented Jul 20, 2021 • edited

Ailrun commented Jul 20, 2021

Ailrun commented Jul 21, 2021

July541 commented Jul 21, 2021

Ailrun commented Jul 21, 2021

isovector commented Jul 21, 2021

Ailrun commented Jul 27, 2021

isovector commented Jul 27, 2021 • edited

jneira commented Jul 30, 2021

Ailrun commented Jul 30, 2021 • edited

jneira commented Jul 30, 2021

jneira commented Jul 31, 2021 • edited

jneira commented Aug 10, 2021 • edited

Ailrun commented Aug 10, 2021

jneira commented Aug 11, 2021 • edited

jneira commented Aug 11, 2021 • edited

jneira commented Aug 13, 2021 • edited

jneira commented Aug 13, 2021

jneira commented Aug 13, 2021 • edited

michaelpj commented Aug 13, 2021

jneira commented Aug 13, 2021

pepeiborra commented Aug 13, 2021

jneira commented Aug 13, 2021 • edited

jneira commented Aug 16, 2021

jneira commented Aug 18, 2021 • edited

jneira commented Aug 25, 2021

fendor commented Aug 26, 2021

jneira commented Sep 8, 2021

pepeiborra commented Sep 8, 2021

jneira commented Sep 8, 2021

pepeiborra commented Sep 8, 2021

jneira commented Sep 9, 2021

jneira commented Sep 9, 2021 • edited

jneira commented Sep 9, 2021 • edited

jneira commented Sep 9, 2021

jneira commented Jun 30, 2021 •

edited

jneira commented Jun 30, 2021 •

edited

jneira commented Jun 30, 2021 •

edited

jneira commented Jul 20, 2021 •

edited

isovector commented Jul 27, 2021 •

edited

Ailrun commented Jul 30, 2021 •

edited

jneira commented Jul 31, 2021 •

edited

jneira commented Aug 10, 2021 •

edited

jneira commented Aug 11, 2021 •

edited

jneira commented Aug 11, 2021 •

edited

jneira commented Aug 13, 2021 •

edited

jneira commented Aug 13, 2021 •

edited

jneira commented Aug 13, 2021 •

edited

jneira commented Aug 18, 2021 •

edited

jneira commented Sep 9, 2021 •

edited

jneira commented Sep 9, 2021 •

edited