Release Planning: unix-2.8.0.0 (GHC 9.6.x) #164

emilypi · 2020-11-16T15:47:11Z

TODO:

Baud Rate

blockSize/blockCount

Doc fixes

Requires an audit

hs-viktor · 2020-11-16T19:34:01Z

Is this a request to review the 4 PRs?

Bodigrim · 2020-11-16T20:07:29Z

Reviews would be nice.
(It's a meta-ticket with everyone in assignees)

hs-viktor · 2020-11-19T19:51:21Z

I've looked some of these over. Not sure what's next. For the Baud rates I really would like to see PatternSynonyms, and support for ad-hoc user-specified code points. For the stat structures, returning Nothing when not supported makes sense to me, and when the associated data types are not defined in base, I'd suggest to just define them to Int, but since we'll always return Nothing, any integral type is as good as any other.

Bodigrim · 2020-11-19T20:01:41Z

PatternSynonyms in #145 require GHC 8.0. Is everyone on board with dropping support for GHC < 8.0? I'm fine with it for unix-2.8 series.

hs-viktor · 2020-11-19T20:03:35Z

Don't know about everyone, but I'm definitely on-board.

hvr · 2020-11-19T20:34:14Z

I'm generally not on board with dropping support for old GHCs unless it is not possible to support those anymore.

As you can see in https://matrix.hackage.haskell.org/package/unix I've made sure that we kept support back till GHC 7.4.2

Bodigrim · 2020-11-19T20:37:04Z

@hvr what's the reason to support GHC < 8.0 in 2020? When will you be ready to abandon it?

hs-viktor · 2020-11-19T20:38:08Z

I'm generally not on board with dropping support for old GHCs unless it is not possible to support those anymore.

As you can see in https://matrix.hackage.haskell.org/package/unix I've made sure that we kept support back till GHC 7.4.2

I'm sympathetic to this on general principles, but find that managing system interfaces portably without PatternSynonyms is just too painful. One ends up locked into rigid algebraic datatypes that map poorly onto a platform-dependent and time-dependent set of constants, and require unnecessary conversions between the underlying numeric values and the supported enums.

So for me GHC system programming really only becomes manageable with GHC 8.0. :-(

[ EDIT: And as I mentioned elsewhere, only GHC 8.8 and 8.10 are still getting bug fixes, thus e.g. the MVar crash I ran into will not be fixed in 7.8, 7.10 or 8.0–8.6. To me that strongly signals that new releases of libraries do not need to target old GHC releases once doing so holds back doing new things "the right way". ]

hs-viktor · 2020-11-19T21:42:02Z

The release date of GHC 7.10.3a (last tagged 7.10.x release) will soon be 5 years in the past:

commit 9b50b5a5da4325e703210b2de2d816debe4c8a64 (tag: ghc-7.10.3a-release)
Author: Ben Gamari <bgamari.foss@gmail.com>
Date:   Tue Dec 8 19:47:57 2015 +0100

And 7.10.1 was released in March of the same year:

commit ca00def1d7093d6b5b2a937ddfc8a01c152038eb (tag: ghc-7.10.1-release)
Author: Austin Seipp <austin@well-typed.com>
Date:   Wed Mar 25 17:33:07 2015 -0500

The 8.0 release has now been out for more than 4 years:

commit 4986837f8168cacf95c24fecc84d7b36c47f3c11 (tag: ghc-8.0.1-release)
Author: Ben Gamari <bgamari.foss@gmail.com>
Date:   Tue May 17 18:05:24 2016 +0200

And the 7.x releases can continue to be used with unix-2.7. Setting the floor for unix-2.8 to GHC 8.0 does not break these older GHC versions. So I think it is safe and reasonable to move on.

Bodigrim · 2020-11-26T22:19:00Z

What is the recommended way to run unix tests, being outside of GHC source tree?

Bodigrim · 2020-11-29T21:37:25Z

Let's try an easier question :)
Has anyone ever run the tests outside of GHC source tree?

vdukhovni · 2020-11-29T22:08:33Z

I just tried something like that now. I say "like that", because I do have the GHC source tree and do my own GHC builds, but I tested the "unix" package from github, not the one in the GHC tree:

$ gmake TOP=$HOME/dev/ghc/testsuite
...
                 
fileStatus.hs:74:10: error:
    • Couldn't match expected type ‘IO System.Posix.Types.Fd’
                  with actual type ‘OpenFileFlags -> IO System.Posix.Types.Fd’
    • Probable cause: ‘openFd’ is applied to too few arguments
      In a stmt of a 'do' block: fd <- openFd f ReadOnly defaultFileFlags
      In the expression:
        do fs <- getFileStatus f
           ls <- getSymbolicLinkStatus f
           fd <- openFd f ReadOnly defaultFileFlags
           fs' <- getFdStatus fd
           ....
      In an equation for ‘getStatus’:
          getStatus f
            = do fs <- getFileStatus f
                 ls <- getSymbolicLinkStatus f
                 fd <- openFd f ReadOnly defaultFileFlags
                 ....

fileStatus.hs:74:28: error:
    • Couldn't match expected type ‘Maybe System.Posix.Types.FileMode’
                  with actual type ‘OpenFileFlags’
    • In the third argument of ‘openFd’, namely ‘defaultFileFlags’
      In a stmt of a 'do' block: fd <- openFd f ReadOnly defaultFileFlags
      In the expression:
        do fs <- getFileStatus f
           ls <- getSymbolicLinkStatus f
           fd <- openFd f ReadOnly defaultFileFlags
           fs' <- getFdStatus fd
           ....
          
*** unexpected failure for fileStatus(normal)

...

fileStatusByteString.hs:73:10: error:
    • Couldn't match expected type ‘IO Fd’
                  with actual type ‘OpenFileFlags -> IO Fd’
    • Probable cause: ‘openFd’ is applied to too few arguments
      In a stmt of a 'do' block: fd <- openFd f ReadOnly defaultFileFlags
      In the expression:
        do fs <- getFileStatus f
           ls <- getSymbolicLinkStatus f
           fd <- openFd f ReadOnly defaultFileFlags
           fs' <- getFdStatus fd
           ....
      In an equation for ‘getStatus’:
          getStatus f
            = do fs <- getFileStatus f
                 ls <- getSymbolicLinkStatus f
                 fd <- openFd f ReadOnly defaultFileFlags
                 ....

fileStatusByteString.hs:73:28: error:
    • Couldn't match expected type ‘Maybe FileMode’
                  with actual type ‘OpenFileFlags’
    • In the third argument of ‘openFd’, namely ‘defaultFileFlags’
      In a stmt of a 'do' block: fd <- openFd f ReadOnly defaultFileFlags
      In the expression:
        do fs <- getFileStatus f
           ls <- getSymbolicLinkStatus f
           fd <- openFd f ReadOnly defaultFileFlags
           fs' <- getFdStatus fd
           ....

*** unexpected failure for fileStatusByteString(normal)
...
Unexpected results from:
TEST="fileStatus fileStatusByteString"
           
SUMMARY for test run started at Sun Nov 29 17:03:02 2020
0:00:29.934497 spent to go through
      29 total tests, which gave rise to
     291 test cases, of which
     261 were skipped
                 
       0 had missing libraries
      28 expected passes
       0 expected failures

       0 caused framework failures
       0 caused framework warnings
       0 unexpected passes
       2 unexpected failures
       0 unexpected stat failures
       0 fragile tests
           
Unexpected failures:
   fileStatus.run            fileStatus [exit code non-0] (normal)
   fileStatusByteString.run  fileStatusByteString [exit code non-0] (normal)
          
gmake: *** [/home/viktor/dev/ghc/testsuite/mk/test.mk:362: test] Error 1

vdukhovni · 2020-11-29T22:19:28Z

Note that the "unix" package in GHC differs from master in at least:

--- System/Posix/IO.hsc 2020-01-24 03:31:15.944438000 -0500
+++ /home/viktor/dev/ghc/libraries/unix/System/Posix/IO.hsc     2019-09-29 21:53:25.804479000 -0400
@@ -76,12 +76,13 @@
 -- for information on how to use the 'FileMode' type.
 openFd :: FilePath
        -> OpenMode
+       -> Maybe FileMode -- ^Just x => creates the file with the given modes, Nothing => the file must exist.
        -> OpenFileFlags
        -> IO Fd
-openFd name how flags =
-   withFilePath name $ \str ->
+openFd name how maybe_mode flags = do
+   withFilePath name $ \str -> do
      throwErrnoPathIfMinus1Retry "openFd" name $
-       open_ str how flags
+       open_ str how maybe_mode flags
 
 -- |Create and open this file in WriteOnly mode.  A special case of
 -- 'openFd'.  See 'System.Posix.Files' for information on how to use
@@ -89,4 +90,4 @@
 
 createFile :: FilePath -> FileMode -> IO Fd
 createFile name mode
-  = openFd name WriteOnly defaultFileFlags{ trunc=True, creat=(Just mode) }
+  = openFd name WriteOnly (Just mode) defaultFileFlags{ trunc=True }

From which I surmise, that my test run used the tests from master, but the tests ran against the "unix" bundled with GHC.

emilypi · 2020-11-30T07:10:55Z

FWIW, according to the Haskell Weekly survey, there seem to be few GHC users for GHC < 8.2. I've recently posed the question of whether long-term support (at least longer than, say, 5 years) is worth the trouble if it holds back the project, or the maintenance burden is too high. Most people, including CLC members, urged dropping GHC versions.

If we are unable to address #145 in a reasonable way due to GHC version blockage (we are), or if there is a clear elegant solution (there is), then we should seriously consider it.

So i agree with @hs-viktor and @Bodigrim here.

Bodigrim · 2020-12-20T12:06:30Z

At the very best a future unix-2.8 will be finished in time for GHC 9.2 (and more probably - for GHC 9.4 only). I believe it is time to pull off the band aid, drop GHC < 8.2 and get both baudRate and blockSize / blockCount sorted. I do not see any other viable approaches: if we cannot provide unconditional API for earlier GHCs, we should not be kicking the can down the road any more. I also do not know any good reasons to support GHCs < 8.2: their user base is shrinking, and remaining consumers most certainly stick to the pre-installed boot libraries.

vdukhovni · 2021-01-18T05:20:42Z

Now that CI is in better shape, it is perhaps time to return the backlog, but first I think we need to settle this issue.
My sense is that setting 8.0 as a floor has rough consensus, and that we can start to use PatternSynonyms and any other relevant 8.0 features where that is the most appropriate way forward on a given PR.

Bodigrim · 2021-01-18T19:49:00Z

8.0 or 8.2? I thought that blockSize / blockCount stuff requires 8.2.

vdukhovni · 2021-01-18T19:54:50Z

8.0 or 8.2? I thought that blockSize / blockCount stuff requires 8.2.

For me personally, the baseline is 8.0, because that makes PatternSynonyms available, which I find indispensable for system programming (modeling various FFI enums and bit fields). I haven't looked at the blockSize issue yet. What 8.2 feature does that require?

Bodigrim · 2021-01-18T19:59:57Z

It requires CBlkCnt and CBlkSize in base AFAIU. I see your comments in #78 that their presense is, strictly speaking, conditional even in GHC 8.2, but I guess there is little sense to target GHCs in which they are unconditionally absent.

vdukhovni · 2021-01-18T20:06:13Z

Yeah, I just found the same comments, I guess I forgot all about it in the meantime. So yes, that PR requires 8.2. If there's no compelling reason to still support 8.0, I am OK with 8.2 as the base, but if #78 builds with 8.0 and just returns Nothing for the conditionally present fields, that may be good enough.

Bodigrim · 2021-02-17T01:20:31Z

FVIW @bgamari asked me whether unix-2.8 is planned to be released in time to be bundled with GHC 9.2. The definition of "in time" remains, however, unclear, because, I believe, GHC 9.2 timeline itself has not been finalized yet. There is no pressure from GHC HQ: they are more than happy to stick to unix-2.7 for the time being.

@vdukhovni what do you think? Given that unix-2.8 is the only chance for breaking changes in the foreseeable future, it makes sense do not rush to an arbitrary deadline.

vdukhovni · 2021-02-17T01:50:57Z

So far we've gotten the project unstuck, with working CI and a more practical minimum GHC version, and have started to clear the PR backlog. But we've not yet had a chance to reassess what architectural debt we want to address that does not yet have a pending PR. If we only get to make one major version bump every few years, then perhaps we should not rush to make a 2.8 release. If on the other hand it'd be OK to have unix-2.8 for GHC 9.2 and unix-2.9 for GHC 9.4 (or whatever) six months to a year later, then we could ship 2.8 comparatively soon, with just the PR backlog cleared, and new work starting after 2.8.

Why can't we do another major bump for the next major GHC release?

Bodigrim · 2021-02-17T19:10:42Z

Why can't we do another major bump for the next major GHC release?

Imagine unix-2.7 shipped with GHC 9.0, unix-2.8 with GHC 9.2 and unix-2.9 bundled with GHC 9.4. If a user wants to maintain compatibility with at least three latest versions of GHC, he/she has to maintain compatibility with all three versions of unix, which could result in a non-negligible amount of CPP. (Life is easier for non-boot packages, because for them it is enough to support the latest major release only)

Bumping upper bounds across the whole ecosystem is also no small business, and it's better do it once instead of twice.

I've found a release schedule for GHC 9.2: https://mail.haskell.org/pipermail/ghc-devs/2021-February/019478.html
It says that a release branch is cut early March.

hasufell · 2022-07-10T11:05:06Z

I want to release unix-2.8.0.0 soon, because of the upcoming filepath release.

This PR will be mandatory for the release: #202

This PR (with a backwards incompatible fix that has an easy workaround by changing imports) can also be included: #219

This one can be included too: #220

Opinions?

status: https://github.com/haskell/unix/milestone/1

Bodigrim · 2022-07-10T20:33:11Z

#220 is not targeted to the master branch, and I prefer #212 / #213 instead anyways.

I'm in favor of #219.

From my perspective #202 awaits for a green CI.

hasufell · 2022-07-10T20:42:19Z

#220 is not targeted to the master branch, and I prefer #212 / #213 instead anyways.

I'm not sure what you mean exactly. We can still commit updated files. network does that too. Old config.guess/config.sub can cause issues.

I'm in favor of #219.

Yeah, me too.

From my perspective #202 awaits for a green CI.

The wasm action is causing trouble.

hasufell · 2022-07-14T15:12:57Z

Ok, I'm going to release filepath independent of unix. I have a feeling making a unix release will take another month at least. There are issues with CI and I can't manually push to master either.

chessai · 2022-07-14T15:23:02Z

Ok, I'm going to release filepath independent of unix. I have a feeling making a unix release will take another month at least. There are issues with CI and I can't manually push to master either.

You can use https://github.com/nektos/act to debug GitHub Actions locally.

In addition to that, could you not just configure the CI yml files to also point to a non-master branch, so that it doesn't get polluted?

I'm not a fan of pushing to master to iterate on CI. However, if you (the maintainers) are okay with it, you can change it in the repo settings, I believe.

hasufell · 2022-07-14T15:29:16Z

I'm not a fan of pushing to master to iterate on CI. However, if you (the maintainers) are okay with it, you can change it in the repo settings, I believe

We're not iterating on CI. But I need PR approvals to merge even simple things that I'm confident about. I don't see how I can manage a timely release in that manner, so I'm going to go ahead with other work.

chessai · 2022-07-14T15:41:13Z

I'm not a fan of pushing to master to iterate on CI. However, if you (the maintainers) are okay with it, you can change it in the repo settings, I believe

We're not iterating on CI. But I need PR approvals to merge even simple things that I'm confident about. I don't see how I can manage a timely release in that manner, so I'm going to go ahead with other work.

I understand the frustration. I have removed the need for PR approvals at this time, this can be re-added later if people decide it's wanted.

Bodigrim · 2022-07-14T18:11:25Z

@hasufell sorry, it's midsummer and people (incuding me) are on holidays, so things take longer than ideal. Let's not merge without approvals please.

hasufell · 2022-07-15T21:33:17Z

@hasufell sorry, it's midsummer and people (incuding me) are on holidays, so things take longer than ideal. Let's not merge without approvals please.

I'm just pushing because these are the last few weeks before my new job, after which I likely won't have time for a while... so bad timing I guess.

hs-viktor · 2022-07-16T18:57:01Z

I'm reviewing the backlog this weekend.

hasufell · 2022-07-17T13:58:25Z

~~All~~ Most PRs merged.

Fix memory leak in handleToFd, fixes issue #4 #222 could go in 2.8.0.0 if someone feels confident about finishing it.
Fix 'semTrywait: invalid argument (Bad file descriptor)' wrt #218 #229 should go in

Otherwise the only thing that's left seems to be to:

adjust changelog (we have one breaking change: Fixes #35: fd{Read,Write} with ByteString payload (Breaking change) #219)
ask @bgamari and @mpickering if there's anything else that could go in
release on hackage
adjust submodule in ghc 9.6 branch

hasufell · 2022-07-29T06:49:55Z

I think we can release. Any last words?

Bodigrim · 2022-07-29T23:10:27Z

Boot libraries must compile without warnings, so (assuming that https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8478 is merged in time for GHC 9.6) we need #238 or equivalent.

hasufell · 2022-07-30T01:04:24Z

Boot libraries must compile without warnings, so (assuming that https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8478 is merged in time for GHC 9.6) we need #238 or equivalent.

That's not released and no one knows if it will be merged and when. That can be a point release.

I don't see why unreleased GHC versions should block a release.

hasufell · 2022-07-30T03:30:12Z

@Bodigrim but feel free to resolve the PR. I'd like to have a release this weekend to unblock certain filepath libraries that can't be uploaded to hackage yet.

Win32 was just released on hackage with AFPP support.

Bodigrim · 2022-07-30T12:35:39Z

Let’s go!

hasufell · 2022-08-03T13:42:00Z

I uploaded the candidate: https://hackage.haskell.org/package/unix-2.8.0.0/candidate

Tag: https://github.com/haskell/unix/releases/tag/v2.8.0.0

Another maintainer please test the candidate (e.g. configure script etc, which are generated on my machine with autoconf-2.69) and then press the publish button on the candidate page.

Bodigrim · 2022-08-03T22:50:59Z

@hasufell please merge #243 and re-upload a candidate.

hasufell · 2022-08-04T04:05:00Z

@hasufell please merge #243 and re-upload a candidate.

Don't we have to bump the version for that? Can candidates be overwritten?

hasufell · 2022-08-04T11:51:02Z

Ok, re-uploaded: https://hackage.haskell.org/package/unix-2.8.0.0/candidate

Bodigrim · 2022-08-04T23:00:58Z

Published: https://hackage.haskell.org/package/unix-2.8.0.0.

emilypi assigned Bodigrim, emilypi and hvr Nov 16, 2020

Bodigrim assigned hs-viktor Nov 16, 2020

Bodigrim mentioned this issue Feb 13, 2021

Drop GHC < 8.2 #175

Merged

hasufell added this to the 2.8.0.0 milestone Jul 10, 2022

hasufell changed the title ~~Release Planning: unix-2.8.0.0~~ Release Planning: unix-2.8.0.0 (GHC 9.6.x) Jul 20, 2022

hasufell added the RELEASE label Jul 20, 2022

Bodigrim closed this as completed Aug 4, 2022

Release Planning: unix-2.8.0.0 (GHC 9.6.x) #164

Release Planning: unix-2.8.0.0 (GHC 9.6.x) #164

Comments

emilypi commented Nov 16, 2020 • edited by hasufell Loading

TODO:

Baud Rate

blockSize/blockCount

Doc fixes

hs-viktor commented Nov 16, 2020

Bodigrim commented Nov 16, 2020

hs-viktor commented Nov 19, 2020

Bodigrim commented Nov 19, 2020

hs-viktor commented Nov 19, 2020

hvr commented Nov 19, 2020

Bodigrim commented Nov 19, 2020

hs-viktor commented Nov 19, 2020 • edited Loading

hs-viktor commented Nov 19, 2020

Bodigrim commented Nov 26, 2020

Bodigrim commented Nov 29, 2020

vdukhovni commented Nov 29, 2020

vdukhovni commented Nov 29, 2020

emilypi commented Nov 30, 2020 • edited Loading

Bodigrim commented Dec 20, 2020 • edited Loading

vdukhovni commented Jan 18, 2021

Bodigrim commented Jan 18, 2021

vdukhovni commented Jan 18, 2021

Bodigrim commented Jan 18, 2021

vdukhovni commented Jan 18, 2021

Bodigrim commented Feb 17, 2021

vdukhovni commented Feb 17, 2021 • edited Loading

Bodigrim commented Feb 17, 2021

hasufell commented Jul 10, 2022 • edited Loading

Bodigrim commented Jul 10, 2022

hasufell commented Jul 10, 2022

hasufell commented Jul 14, 2022

chessai commented Jul 14, 2022

hasufell commented Jul 14, 2022

chessai commented Jul 14, 2022

Bodigrim commented Jul 14, 2022

hasufell commented Jul 15, 2022

hs-viktor commented Jul 16, 2022

hasufell commented Jul 17, 2022 • edited Loading

hasufell commented Jul 29, 2022

Bodigrim commented Jul 29, 2022

hasufell commented Jul 30, 2022

hasufell commented Jul 30, 2022

Bodigrim commented Jul 30, 2022

hasufell commented Aug 3, 2022

Bodigrim commented Aug 3, 2022

hasufell commented Aug 4, 2022

hasufell commented Aug 4, 2022

Bodigrim commented Aug 4, 2022

emilypi commented Nov 16, 2020 •

edited by hasufell

Loading

hs-viktor commented Nov 19, 2020 •

edited

Loading

emilypi commented Nov 30, 2020 •

edited

Loading

Bodigrim commented Dec 20, 2020 •

edited

Loading

vdukhovni commented Feb 17, 2021 •

edited

Loading

hasufell commented Jul 10, 2022 •

edited

Loading

hasufell commented Jul 17, 2022 •

edited

Loading