New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yaml-0.8.31 causes issues on GHC 7.10 and below #165

Closed
ndmitchell opened this Issue Jun 29, 2018 · 36 comments

Comments

Projects
None yet
8 participants
@ndmitchell

ndmitchell commented Jun 29, 2018

I'm seeing this on Travis for HLint: https://travis-ci.org/ndmitchell/hlint/builds/397826785

And the Hackage matrix is showing black squares: https://matrix.hackage.haskell.org/package/yaml

CC @snoyberg. Looking at the code, it has a lower bound on base of GHC 8, so not sure what the problem is...

@jgm

This comment has been minimized.

jgm commented Jun 29, 2018

I'm having the same issue with my pandoc CI builds on ghc 7.10.3 + cabal 2.0.

@jgm

This comment has been minimized.

jgm commented Jun 29, 2018

--- yaml-0.8.30/yaml.cabal	2018-06-29 19:58:56.000000000 +0200
+++ yaml-0.8.31/yaml.cabal	2018-06-29 19:59:56.000000000 +0200
@@ -1,5 +1,5 @@
 name:            yaml
-version:         0.8.30
+version:         0.8.31
 license:         BSD3
 license-file:    LICENSE
 author:          Michael Snoyman <michael@snoyman.com>, Anton Ageev <antage@gmail.com>,Kirill Simonov
@@ -46,7 +46,9 @@
 
 library
     other-extensions: LambdaCase
-    build-depends:   base >= 4 && < 5
+    if impl(ghc < 8.0.2)
+      buildable: False
+    build-depends:   base >= 4.9.1 && < 5
                    , transformers >= 0.1
                    , bytestring >= 0.9.1.4
                    , conduit >= 1.2.8 && < 1.4

Looks like the main thing that changed was raising the minimum bound for base to 4.9.1 and the restriction to ghc >= 8.0.2. But as @ndmitchell says, it's hard to see why cabal's solver didn't just reject the package.

@snoyberg is it impossible to continue to support ghc 7.10.3? We'll have to drop support for it in pandoc solely because of this package. [EDIT: Well, I suppose I can just put an upper bound on the package for older ghcs.]

@ndmitchell

This comment has been minimized.

ndmitchell commented Jun 29, 2018

@jgm as long as Cabal solver is smart enough to pick an older version (which it isn't for no obvious reason), you can continue supporting 7.10.3 even if it results in using an older yaml.

@hvr

This comment has been minimized.

Contributor

hvr commented Jun 29, 2018

@ndmitchell the reason matrix.hho has black squares is that the construct if impl(ghc < 8.0.2) buildable: False on a library component is something the cabal solver doesn't like much and so doesn't backtrack;

The issues with buildable: False have known for some time (and how buildable: False and build-depends were supposed to interact with each other semantically wasn't too obvious initially either), and until recently there was no urgency to act on it; but when http-client started using this construct, a preliminary fix was created (see haskell/cabal#5337) and it's going to be available in the next cabal-2.2 point release; (but the fix is already available in my PPA's cabal-install-2.2 packages);

@jgm btw, if you just need a standards compliant YAML 1.2 parser that avoids FFI and supports GHCs back till GHC 7.4, HsYAML is currently officially the most compliant YAML 1.2 parser available :-)

@ndmitchell

This comment has been minimized.

ndmitchell commented Jun 29, 2018

@hvr perfectly happy with the yaml package, I just want it to install :)

Is the advice that @snoyberg should avoid the impl(ghc) constraint because that breaks older Cabal users? Saying that people on 5 year old compilers will be fine if only they upgraded their Cabal is really just ditching support for them :(.

@phadej

This comment has been minimized.

Member

phadej commented Jun 29, 2018

@ndmitchell Old Cabal (the lib) is fine. People only need to update cabal-install (the tool), which is not a bad idea.

@jgm

This comment has been minimized.

jgm commented Jun 29, 2018

@hvr thanks for the pointer about HsYaml; I'd like to use a pure Haskell library if possible. I'll try it out.

@ndmitchell

This comment has been minimized.

ndmitchell commented Jun 29, 2018

@phadej saying you can use GHC out the box if you upgrade all the things around it to me doesn't work - the people who are on old GHC are also on old Cabal and old cabal-install.

@gbaz

This comment has been minimized.

Contributor

gbaz commented Jun 29, 2018

@ndmitchell true enough, but upgrading from an older ghc is hard -- you may have a codebase which is not compatible library changes, other couplings, etc. On the other hand, updating cabal-install is just replacing a binary, and should otherwise be just fine.

(also: another, certainly nonideal, workaround, would be for the cabal files of their projects, which are likely proprietary, to simply just exclude the offending version of yaml)

snoyberg added a commit to snoyberg/yaml that referenced this issue Jul 1, 2018

@snoyberg

This comment has been minimized.

snoyberg commented Jul 1, 2018

Before I make a release to Hackage, can someone confirm that this release will properly work around the bug in cabal-install? snoyberg/yaml@2735b09

snoyberg added a commit to snoyberg/http-client that referenced this issue Jul 1, 2018

@snoyberg

This comment has been minimized.

snoyberg commented Jul 1, 2018

And here's a workaround for http-client: snoyberg/http-client@1390d6e.

@jgm

This comment has been minimized.

jgm commented Jul 1, 2018

@ndmitchell

This comment has been minimized.

ndmitchell commented Jul 1, 2018

@snoyberg unfortunately my CI is set up thus that it's almost impossible to test before release is made... If you do make it thought I can give quick feedback after that.

@snoyberg

This comment has been minimized.

snoyberg commented Jul 1, 2018

If you add a cabal 2.0 + ghc 7.10.3 combination to your travis CI testing, that should give you the answer.

Afaict it won't. The bug here is in the solver choosing the wrong version of the yaml package, but building a local package won't trigger that code path. I think I'll just move ahead with the release, do some Hackage revisions, and hope for the best.

@snoyberg

This comment has been minimized.

snoyberg commented Jul 1, 2018

I've published a new version of yaml, and added the only revision I could that should fix this: totally breaking yaml-0.8.31 (http://hackage.haskell.org/package/yaml-0.8.31/revisions/). Can you both give it a try and let me know if it works? If so, I'll release http-client as well.

@hvr

This comment has been minimized.

Contributor

hvr commented Jul 1, 2018

Before I make a release to Hackage, can someone confirm that this release will properly work around the bug in cabal-install? snoyberg/yaml@2735b09

Btw, for more or less obvious reasons that won't work due to how constraint solving works

PS: And neither the revision at http://hackage.haskell.org/package/yaml-0.8.31/revisions/ will work

PS2: It's better for you to not do anything to avoid making things worse; I'm looking into addressing this from a more direct and effective angle

PS3: If something's unclear just ask here; there's no need to go complaining on Twitter; the short gist is that backtracking doesn't kick in for the same reason we're having this problem in the first place: once buildable: False is in effect, the build-depends-constraints are ignored, so it's pointless to try to use those to steer the solver away from the affected versions.

@gbaz

This comment has been minimized.

Contributor

gbaz commented Jul 1, 2018

Let me expand a bit, because I had to work it through myself. If there were no version with the buildable flag set to false, but instead of that, an unsatisfiable constraint were used, things would be fine. However, the existence of such a version means that the solver, finding the current version not satisfiable, would backtrack to the bad version, and then encounter that buildable flag, and then go kaput. My immediate thought was "can't we just unprefer the bad version" but it turns out that's not as strong as we'd like, since unprefering is not a hard blacklist, but just makes the solver try the other paths first. So if you have something that pushes the solver past the working paths, then it still follows the failing path instead of recognizing it as a "fully" not working path.

So we will try to work out a fix and sort this out, but the above approach, which does seem reasonable at first, isn't going to suffice.

@ndmitchell

This comment has been minimized.

ndmitchell commented Jul 1, 2018

Would using base versions, skipping any ghc-version test in the Cabal file itself (as this seems to lead to a pot of despair), and then having the Hackage UI display GHC version bounds based on base version bounds be the best approach?

@gbaz

This comment has been minimized.

Contributor

gbaz commented Jul 1, 2018

I don' t know what the best long-term approach is. We do have an older ticket suggesting that certainly hackage displaying the base <-> ghc mapping would be a good idea, regardless: haskell/hackage-server#559

@ndmitchell

This comment has been minimized.

ndmitchell commented Jul 1, 2018

@snoyberg I confirm empirically that it's still bust for me.

hvr added a commit to haskell/hackage-server that referenced this issue Jul 1, 2018

Allow revising component buildability
This makes it possible to resolve issues such as
haskell-infra/hackage-trustees#165 in a simple, direct, and effective
way by using the well-proven Hackage meta-data revision facility.
@hvr

This comment has been minimized.

Contributor

hvr commented Jul 1, 2018

Alright, the situation with yaml and http-client should have been addressed now (EDIT: I knew this was resolved, because I empirically verified this with affected cabal versions; and contrary to the unfoudned accusations this did not break http-client). Yet again, Hackage meta-data revisions saved the day! :-)

As a side-remark, in general I'd advise against using the if impl(ghc < x.y) construct as among other reasons, many people don't realise its subtle non-obvious semantics, it gives worse solver messages, you can't correct mistakes in the impl-condition via Hackage meta-data revisions currently, and it also violates the "test for features, not for versions" mantra, the argument of reinstallable base doesn't hold for lower bounds anyway... and so on. I noticed a blogpost trying to make the case for it but it's a flawed argument and it's really bad advice.

@hvr hvr closed this Jul 1, 2018

@snoyberg

This comment has been minimized.

snoyberg commented Jul 2, 2018

This was an invalid resolution to the situation with http-client: you have broken the most recent version of the package, which people are using, and without a new version to replace it. I do not agree to these kinds of changes to my packages being made, especially without even the courtesy of an explanation being made in this discussion as to what changes you made to my package.

I'll clean this one up myself by uploading a new version of http-client. But the fact that the only version of a package of mine adding features people are using was deprecated via a revision without any explanation that that was your plan is pretty unsettling.

@gbaz

This comment has been minimized.

Contributor

gbaz commented Jul 2, 2018

I think this was just a miscommunication. Since you had uploaded a new yaml and attempted to make the prior one unbuildable, I think it was expected the same process would be taken with http-client. I agree that for clarity's sake, it would have been better to handle yaml, and then leave it to you to handle http-client following the same template.

@snoyberg

This comment has been minimized.

snoyberg commented Jul 2, 2018

I was quite explicit about my intentions and my actions:

I've published a new version of yaml, and added the only revision I could that should fix this: totally breaking yaml-0.8.31 (http://hackage.haskell.org/package/yaml-0.8.31/revisions/). Can you both give it a try and let me know if it works? If so, I'll release http-client as well.

The issue here was that a Hackage Trustee—@hvr—felt it appropriate to tell me to stop maintaining my own packages and performed surgery on them without any explanation of the process being taken. This is precisely why the revision mechanism is so disliked by so many. This is an upsetting combination of unacceptable communication style and the disregard for breakage caused to users by actions. I highly recommend the Hackage Trustee team and haskell.org committee address these concerns, which have been regularly raised and continue to be a major issue.

@gbaz

This comment has been minimized.

Contributor

gbaz commented Jul 2, 2018

Yes you were explicit, which is why this was a miscommunication and misunderstanding. I am glad we had this chance to resolve it to everyone's satisfaction.

@hvr

This comment has been minimized.

Contributor

hvr commented Jul 2, 2018

@snoyberg

This was an invalid resolution to the situation with http-client: you have broken the most recent version of the package

I have not; please get your facts straight before jumping to false conclusions; next time, please ask if there's something you don't understand

@gbaz

This comment has been minimized.

Contributor

gbaz commented Jul 2, 2018

To expand (and here it seems like more misunderstanding and miscommunication): I don't think the revision made broke the then-most-recent version (0.5.13). Looking at http://hackage.haskell.org/package/http-client-0.5.13/http-client.cabal it just seems like it swapped out the not-working buildable thing in the build-depends for the correctly-working base < 0 thing, which should not have broken the package for people.

@ElvishJerricco

This comment has been minimized.

ElvishJerricco commented Jul 2, 2018

@snoyberg Can you elaborate on how the package was broken? @hvr's revision built fine for me with both cabal and stack.

@snoyberg

This comment has been minimized.

snoyberg commented Jul 2, 2018

It looks like I misread the edit, that's my mistake. All of my other statements stand: this was handled in an unacceptable way, and needs to be addressed by improved processes. Gershom's comment "to everyone's satisfaction" is certainly not true, there is no satisfaction on my side from the behavior of the official haskell.org representatives and Hackage/Cabal maintainers present here. Unfortunately the only people to appeal to appear to be the people misbehaving.

@snoyberg

This comment has been minimized.

snoyberg commented Jul 2, 2018

Also, for the record, the reason I got this wrong is that the revision display on Hackage strongly implies that base < 0 was added regardless of flags. I now know when investigating actions taken by Herbert:

  • I cannot hope to have him indicate what his planned actions are, even when discussing one of my own packages
  • I cannot trust the information displayed on the Hackage page, but instead need to look at the raw cabal file

screen shot 2018-07-02 at 10 39 12

@ElvishJerricco

This comment has been minimized.

ElvishJerricco commented Jul 2, 2018

Considering this issue is resolved, I move that the discussion be taken elsewhere.

@snoyberg

This comment has been minimized.

snoyberg commented Jul 2, 2018

I've opened up an issue for the incorrect revisions display: haskell/hackage-server#770.

@hvr

This comment has been minimized.

Contributor

hvr commented Jul 2, 2018

@snoyberg

there is no satisfaction on my side from the behavior of the official haskell.org representatives and Hackage/Cabal maintainers present here. Unfortunately the only people to appeal to appear to be the people misbehaving.

I'm sorry that you feel that way but as history has shown it's difficult to meet your demands and when things aren't done 100% as requested you seem to resort to personal attacks and ad-hominems (like e.g. two years ago just as one of several examples). I stayed up late last night to fix http-client and yaml and all I get in return is more of the same. I for one refuse to interact under these operating conditions. Have a nice day.

@domenkozar

This comment has been minimized.

domenkozar commented Jul 8, 2018

I don't want to heat up this discussion more than needed, but unsupported-ghc-version is not ideal for cabal2nix since it tries to eagerly find out dependencies and it ends up in the inputs.

@snoyberg would using base < 0 instead of non-existing package still satisfy your requirements?

@ElvishJerricco

This comment has been minimized.

ElvishJerricco commented Jul 8, 2018

@domenkozar Yeesh, didn't think of that. Definitely not ideal. As a workaround, can we add unsupported-ghc-version = null; to one of the xyz-configuration.nix files?

@snoyberg

This comment has been minimized.

snoyberg commented Jul 9, 2018

@domenkozar I'm willing to change the cabal file, but I think you're going to have bigger problems than this. The "include a package name as a dependency and the package doesn't exist" approach was cribbed from base:

-- sanity-check to ensure exactly one flag is set
if !((flag(integer-gmp) && !flag(integer-simple)) || (!flag(integer-gmp) && flag(integer-simple)))
    build-depends: invalid-cabal-flag-settings<0

I'd rather move this discussion away from this issue though. If you want to follow up, please do so on the yaml repo instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment