Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renovating GHC's extensions mechanism #628

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

goldfirere
Copy link
Contributor

For a number of years, I've worried that GHC's approach toward extensions has hampered Haskell's adoption. Dealing with extensions can feel bureaucratic and unilluminating. In parallel, the community has recently had much debate over the balance between the dual needs of stability and innovation.

I've pulled my thoughts together in this PR, which I believe could repackage a distinctive part of Haskell in a way that could spur adoption. As the proposed document details, this is not a proper proposal, but instead a position paper describing a direction of travel that I think would be an improvement to GHC. There is a detailed proposal in the middle of it -- and one I think we should consider -- but I'm first seeking discussion around whether the general idea is a good one. Debating the proposal directly at first seems too likely to get caught on its own details.

It's not quite clear to me whether I would want this merged, as such, or just closed (and then a proper proposal, later, would be merged). But a PR is a good place for debate, in the meantime!

Rendered

@Kleidukos
Copy link

Thank you @goldfirere, this is a great position with which I find myself in full agreement.

Only a slight bike-shedding topic from my part that in no case affects my overall support for this path:

Bikeshedding

Shouldn't LambdaCase be part of the Syntax bundle?

appropriately, as detailed in a proposal below. The key points can be summarized
in two sentences:

* **Preparing a package for uploading to Hackage (e.g. ``cabal sdist``) should

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to hammer it home (I think it's implicit but not quite stated), for this to be the case we really need it to be the case that different versions of GHC produce the same warnings when compiling the same code (given the language edition). Otherwise this is quite under-defined - or are we just saying that the user should fix the warnings for the version of GHC that they are currently using?

Even with that, it seems that your Latest edition would give unpredictable warnings. So if the goal is for packages on Hackage to be warning-free then I think we would also need to say that they cannot have the Latest edition?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I don't actually think I want to insist that different versions of GHC produce exactly the same warnings -- that's nice, but probably too expensive to promise. Variability here might be annoying to users. They can always disable the warning, or disable the -Werror behavior. (I don't want to force anyone into anything with this! Just encourage. Not force.) Maybe it would be worth taking a pass through the specific warnings enabled to evaluate them for expected stability.

To clarify my stance here, my goal isn't so much that all packages on Hackage are warning free in a given version of GHC. Instead it's to say that authors of all packages on Hackage have considered the coding problems that are warned about. Authors can suppress the warnings. But they should have to take some action around warn-worthy situations to be able to publish. That's not a very tight specification! But I think it's still worthwhile.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't we need some way of locally suppressing particular warnings for this idea to become practical?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that would be nice. But I still think this aspect of the proposal would be useful even without this mechanism.

manifestoes/extensions.rst Outdated Show resolved Hide resolved
the language editions proposed above with respect to the design here.
I am not an expert in cabal's or haskell-ci's UI, so I will not specify
the details here. However, I would expect all such tools to offer
some way to opt out of the erroring behavior, just to allow e.g. an

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should always be escape hatches!

use to improve their code. Indeed warnings are better than errors for them, because
perhaps their focus at the moment is on a different part of the code than where
the warning arose, or perhaps the user is experimenting and is happy for e.g. an
import to be unused.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it a problem for these to be errors? One reason is that errors stop compilation, so you can't on with working elsewhere while there is an error around. But we can improve that situation! I will shortly be making a HF proposal about that: https://discourse.haskell.org/t/pre-hftp-fault-tolerant-ghc-compiler-pipeline/8445?u=michaelpj

If errors don't stop compilation then the distinction between warnings and errors is much less significant, it mostly just affects the colour of the text (perhaps you need a new category of FATAL errors that do in fact stop compilation).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the work here very much as in sympathy with the fault-tolerance you describe. Already GHC has a bit of fluidity between errors are warnings (though the mechanisms are manifold and confusing, having grown organically). My stance here is essentially that warnings really are errors -- just ones that GHC already knows how to recover from. But I don't want such errors published. :)


* E!: the warning is an error by default and cannot be turned off or made into a warning

* The next several columns describe semantic bundles of options.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on the fence about these. My instinct is to say that I want to say something more like "FancyTypes but without the dubious stuff (for some personal definition of dubious)". But in practice I agree with the statement elsewhere that other languages tend to not give people this level of fine-grained control and it's fine: you just don't use the stuff you don't like.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a Haskeller, I would prefer the fine-grained control. I'll happily declare which features I want and use just those. I like tight specifications.

But I actively want Haskell to be easier to use by people unlike me. "Is that an existential type? or a rank-n type? Both have foralls in weird places..... Oh, forget it, I'll just do without."

The big counterpoint here is that an enterprise might want to choose just certain fancy types. But they still can, if they want to. Or at least as much as our extension system allows: one might reasonably be happy for datatypes to introduce new equalities but not want proper existentials, yet there's no way to say this.

``-Wmissing-kind-signatures``, ``-Wmissing-local-signatures``, ``-Wmissing-pattern-synonym-signatures``,
``-Wmissing-role-annotations``, ``-Wmissing-signatures``

* ``Complete``: There are a number of places where programmers can omit parts of their program

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YES

* ``Stable2024``
* ``Experimental2024``
* ``Latest``
* ``Student2024``

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a lot. One of the costs of making language editions into powerful things instead of collections of defaults is that the overhead tends to grow over time. It seems like it would be easier going forwards if we instead built these out of some component flags. For example, we could have:

--stable-from=2024

Which can be used uniformly to control what stability date you want - then Stable2024 just provides a default value for that.

Similarly, if we provide --error-message-abstraction=basic, then Student2024 can just be --stable-from=2024 --error-message-abstraction=basic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I see the payload of this suggestion. We could break these into components, as you suggest. But how would that reduce complexity / overhead?

@michaelpj
Copy link

While I think the goal of thinking more about user personas is great, I worry that this tries to be a little too opinionated. It is starting to make quite substantive assumptions about what I the user, am like. Just because I'm on Stable now doesn't mean I won't change to Latest at some point. Not giving me the compatibility warnings in that case does me no service.

Now I don't think it's a problem to make strong assumptions, so long as I can always ask for what I want. So I very much do want a way to get the full -Wcompat even if I'm on Stable. The simplest way to do this is to keep doing something similar to what we do today: the language edition controls the default behaviour, but this can all be overridden by the user. I think this principle works across the board: if you want simplified error messages for students, have an error-message-abstraction flag that defaults to "low"... but let me override it to "high" even if I'm using Student.

My experience as a user of software is that at some point you always want an escape hatch from what the software thinks you want... and it's extremely annoying if it's not there.

Various sentences that I disagree with on this basis:

  • "If we think that a warning should be enabled for users going forward, we can turn it on by default, but only in appropriate language editions, such as Experimental or Latest."
  • "For -Wcompat, we add warnings that describe features that will change in the next edition in a given series. So the Stable2024 edition of -Wcompat will warn about features changing in Stable2027."

In a similar vein, I'm quite wary of having features (inc. flags) behave differently across language editions. As the proposal points out, this increases the state space for designers and maintainers to consider, but that applies just as much to users! If a flag does different things in different contexts, then in order to understand what it means I (the user!) have to keep all that in my head, which I absolutely do not want to do. I am already imagining asking people "what language edition are you using?" on IRC and then trying to look up the big matrix in my head to find out what the various flags mean ("ah yes, since you are using Student2024, that flag doesn't enable this feature, even though the stackoverflow post you found says that it does" 😱 ). I think trying to have a control do a single, predictable thing in all contexts is a pretty fundamental UX design principle.

I feel more okay with this if there is a smallish set of "bundle" options that control a bunch of stuff and which can behave differently (like the language extension bundles). Interpreting them will still be a pain, but at least there are relatively few and they're clearly marked as bundles. (So I can say "Hmm, maybe FancyTypes doesn't enable X on your edition, can you try adding LANGUAGE X and see if that does it?")

Various sentences that I disagree with on this basis:

  • "The meaning (or existence) of other flags can depend on language edition"
  • "Even though it might come later in a command line, the edition can affect the meaning of command-line arguments that precede it"
  • "Because of the expressive power of language editions -- with their ability to control all aspects of GHC's behavior -- we now have potentially an even larger configuration space to worry about as implementers"

I think the summary of my comments here and in the doc is: I think the idea of language editions as a way of cutting down the effective state space is good, but I don't agree with the idea of language editions as primitive configuration.

In order to have knobs to override any particular decision a language edition makes, there must be flags to change each decision. This also implies that there must be knobs for every decision, and hence the language edition can't be more than the sum of its knobs.

The knobs should also do the same thing, as much as possible. If we need more knobs, we can add more flags, but hide them inside the language extensions or behind higher level groupings.

I am however, fine with making the knobs bigger, since today we have far too many. I very much like the goal of "users only need to specify a language edition". Perhaps we can just put all the -f flags under a big "Advanced" header.

(I also think the discussion about warnings vs errors is very interesting but I need to think about it more...)

@nomeata
Copy link
Contributor

nomeata commented Jan 4, 2024

Thanks for writing this up Richard. I am broadly in support: Language Editions seem to be a great tool to reduce effective user complexity, and at the same time improving stability (because they indicate when the program was written). They started as merely a set of extensions, but I agree that this does not have to be the case, and they could also affect warning sets, error messages and (possibly) access to some features or semantic changes to extensions.

For the last point I am unsure and @michaelpj has a point, maybe it’s cleaner to have LambdaCase and LambdaCaseV2 in such a case, with the editions triggering one or the other. If more users only interact with language editions, this is less ugly as it looks like, but can help to avoid confusion by giving distinct names to distinct behaviors.

Beside the time axis of language editions, I am less sure about the other (Student, Stable, Experimental, Latest) axis. The usefulness of Stable is rather clear. It seems Experimental20xx is just Stable20xx with more flags allowed (and maybe different warnings). Smells like it could be a -fexperimental flag that unlocks these features. But then that could just be -XExperimental, as proposed here. And once we have that, Student20xx is a nice to have as well (and it could almost be called LTS20xx, because maybe it’s not just students who will opt for the longer support window). So all justifiable, it seems.

Maybe expand on the rationale for Latest (over Experimental + a bunch of extensions)? Is it just so that people can play with new toys without having to think about extensions?

One consequence of Latest seems to be that GHC will have to make a judgment calls for fork-like extensions. If we have ViewPatternInGreen and ViewPatternsInRed, and they are mutually exclusive, and GHC includes one of the two in Latest, that implies some kind of blessing. Not a bad thing.

I see that Stable still allows some extensions to be added. Is the idea that these are precisely those that are optional but stable, and one does not lose the stability guarantees by enabling them?

@Ericson2314
Copy link
Contributor

Ericson2314 commented Jan 4, 2024

Thank you @goldfirere! This is very good. I'm quite happy with the big picture here, and even more happy with the trajectory of the last few discussions/proposals of this sort --- I really don't feel much urge to bring up any quibbles now because based on that trajectory I am confident the details will be sorted out in the more concrete follow-up proposals!


Similar to @nomeata I think editions are crucial for the "stables", but for the "experimental"/"latest" I don't really care so much. E.g. Rust does editions for stable things, but individual feature flags (like us) for experimental things, and that is fine.

Another way of looking at this is that, yes, retroactively, we have stable code that uses language extensions, and so we need editions to version the language extensions. But I think that is mostly an accident of history. If going forward, we have every extension beyond the stable baseline violate the stability guarantee, I think that's actually fine!

Forcing experimental code to be fine-grained has its benefits:

  • I think batching unstable things is a bit artificial, unless it's a "stable edition release candidate"
  • fine-grained experimental opt-in information on experimental code in the wild gives us better insight into what the community is up to

Conversely, removing degrees of freedom of stable code (e.g. no -X... within new Stable20XY going forward) collapses a daunting state space, which makes it easier to abide by our stability guarantee.

Just to give an example of this, I supposed this policy was already in place and old stable edition lacked DeriveFunctor because it predated it.

  • It's important we do not allow code in the old edition to start using DeriveFunctor silently, because then users can accidentally stop supporting old compilers by mistake, when the explicit choice of edition is supposed to prevent that.
  • But conversely in new editions, I see no reason to expose a -warn=derive-functor at all. It's an innocuous feature, who needs to disable it!

The moral of the story to me that things like the -XDeriveFunctor flag only exist because we didn't have edition flags. Once we do, we can just get rid of such flags.


To sum that up, I think:

  1. The first order of business is a single experimental flag, like rust's secret RUSTC_BOOTSTRAP env var or @angerman's Add -experimental flag #617. We need an iron curtain between "stable land" and "unstable land" just so our heads stop spinning.

  2. The second order of business should be this editions for "stable land", i.e. this proposal. Everyone should be able to see that (1) one stable thing is not enough, (2) one stable thing per compiler just gets us back where we started, where language changes and compiler changes have the same release schedule and dev ops chaos ensues. Stable20XY is crucial!

  3. Editions in "unstable land" is low priority for me, and so I soon as well not discuss it at all until "stable land" is sorted out.Indeed, there is no reason the policy for "unstable land" can't be as ill-defined and subject to change as its content! Thus I would just not talk about Experimental20XY or Latest for now.

Student20XY is a nice bonus feature I am personally quite fond of (as we've discussed for years), but can also be dealt with separately, as a novel subset of stable land.

@Ericson2314
Copy link
Contributor

Ericson2314 commented Jan 4, 2024

Even shorter: I'd argue editions is, in fact, little more than returning to the original plan of GHC extensions and frequent Haskell reports!

The denotation of that plan was never wrong. The problem was the connotation that Haskell (N +1) would be mostly a superset of Haskell N: this lead to:

  1. New Haskell language versions being too "high stakes"
  2. New Haskell language versions being much less frequent
  3. GHC extensions accumulate without bound

The beauty of the editions concept is that Edition 2025 is under no obligation to be like Edition 2024. Questions of

Are we sure this is really the right language design

become secondary to

Whether or not this is a good feature, are we confident we can support it concurrently with other editions for the next 6 years

This means:

  1. Edition design/codification is much more free to make mistakes / do things we'll regret (as long as they are not hard to back out of implementation wise)
  2. Edition design/codification can happen frequently, with the lower stakes involved
  3. GHC extensions will no longer accumulate, as people should rarely find themselves needing to stray outside a recent (stable) edition.

@gbaz
Copy link

gbaz commented Jan 4, 2024

I think the editions stuff is useful but I don't think I agree at all with the treatment of warnings and their purpose. I also feel like the discussion on warnings is almost entirely orthogonal to other aspects of this proposal, and would much prefer that it be pulled elsewhere to not have too confused a discussion.

Copy link
Contributor

@adamgundry adamgundry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @goldfirere for putting such a lot of effort into this. The details will need some thought, but the big picture ideas seem compelling to me.

I strongly agree with @michaelpj in #628 (comment) that language editions/bundles should be composed of simple mostly-orthogonal options, both to reduce complexity and to account for edge cases where the defaults are unhelpful. Of course we can de-emphasize those atomic options in our documentation and discourse, but I don't think we should eliminate them.

How do you see the actively underway GHC2024 process evolving in the light of this? I believe the intention was to have GHC2024 ready for GHC 9.10, but I don't think it is realistic to expect this proposal to land on a similar timeframe.

``-``, unless ``Classic`` mode is on.

.. [#]
It is time for the ``Safe`` ecosystem to be dismantled. We should warn
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this, but I was surprised to see this buried in a footnote! For a while I've felt that Safe is not really pulling its weight, but proposing to remove it seemed like it risked sparking too much controversy. I think this proposal provides a good opportunity to argue the case for removal as part of the broader shake-up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've felt that Safe is not really pulling its weight, but proposing to remove it seemed like it risked sparking too much controversy

I understood the outcome of https://discourse.haskell.org/t/deprecating-safe-haskell-or-heavily-investing-in-it/5489 to be that Safe was considered due for removal by a motivated person. N.B. @simonpj's comment

I think it would be plausible to lay it to rest

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a fit of enthusiasm a few years ago, I almost removed Safe. What stayed my hand is that it's goals are worthy, and any replacement of Safe will have to have a similar community buy-in and trust network. So it seemed a shame to disrupt the existing trust network just to have to rebuild it. For whatever reason, I'm now more confident that a replacement isn't coming anytime soon and that it really is time to leave Safe behind.

manifestoes/extensions.rst Outdated Show resolved Hide resolved
Comment on lines +648 to +649
Includes the following: ``DeepSubsumption``, ``IncoherentInstances`` (but without implying
``INCOHERENT`` on every instance), ``NoMonomorphismRestriction``, ``-Wno-warn-orphans``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is too much detail for now, but it isn't obvious what "IncoherentInstances (but without implying INCOHERENT on every instance)" means. Perhaps that writing INCOHERENT pragmas at all is guarded by T3? And the rest of the choices for this group seem strange to me.

manifestoes/extensions.rst Show resolved Hide resolved
@goldfirere
Copy link
Contributor Author

Thanks for engagement here, all.

@michaelpj

There should always be escape hatches!

And there are! The big escape hatch is using Experimental instead of Stable. That is, I want a declaration of Stable to mean "this is stable", not "this is stable, as long as you don't separately defeat the mechanisms that make it stable, except that we won't say which mechanisms these are". Put another way, the whole reason for Experimental is to serve as an escape hatch for Stable. The intent (and maybe my big chart falls short of this intent) is that any Stable program also compiles under Experimental.

too opinionated

I think Haskell has been held back some by its lack of opinions, so yours is not a surprising rejoinder. This point will be hard to convince each other of. My goal is to make one mode of Haskell easier to use than others, subtly (or not so subtly) encouraging its use. I believe this will make Haskell easier to adopt. But I don't have much to back this hunch.

So I very much do want a way to get the full -Wcompat even if I'm on Stable

I don't think we should allow unstable warnings about the future in Stable mode. If the user declares Stable, then when they return to their code in a year or two, it should keep compiling without new noises. If you want otherwise, it sounds like you want Experimental. (Maybe the name Experimental is poor? Really I just mean NotNecessarilyStable. It's not that Experimental features are somehow dangerous.)

I'm quite wary of having features (inc. flags) behave differently across language editions

So am I! This is, I believe, pretty minimal in the details. Maybe we want to ban it outright? But I think that may tie our hands. I do think we should avoid this as much as possible.

In the proposed concrete meanings of everything, we can phrase things so that all the bundles have the same meaning in every edition. (In two cases, Stable would override the bundle's behavior by specifying -XNoRequiredTypeArguments -XNoTransformListComp.) So perhaps this ability is overstated in the text.

I think trying to have a control do a single, predictable thing in all contexts is a pretty fundamental UX design principle.

Strong agree -- but note the word "trying". That is, if the reasons are compelling enough, we might fail to do this. But we should absolutely aim for this.

[I disagree with] "Even though it might come later in a command line, the edition can affect the meaning of command-line arguments that precede it"

I don't see the problem here. This is essentially just an interpretation of "all aspects of GHC work with respect to precisely one language edition" -- including the command line arguments that happen to precede the choice of language edition.

@nomeata

maybe it’s cleaner to have LambdaCase and LambdaCaseV2

Sure. My hope is that users would stop caring about individual extensions, and so doing this is fine.

I am less sure about the other (Student, Stable, Experimental, Latest) axis

You basically rediscovered my motivation in the paragraph that follows this comment. A major goal is Stable, so we have that one. It needs an antonym: Experimental. I'd be content with just those. Latest is nice for those who want to be on the bleeding edge (or, rather, so that those who want to be on the bleeding edge don't need to worry about extensions). Student just seemed like a nice way to honor the educators doing very important work. It's very reasonable to say I've overreached here.

that implies some kind of blessing [of one feature over another that might not be in Latest]

Yes! My opinion these days is that we should have opinions.

Is the idea that [the extensions available in Stable] are precisely those that are optional but stable, and one does not lose the stability guarantees by enabling them?

Yes.

@Ericson2314

Rust does editions for stable things, but individual feature flags (like us) for experimental things, and that is fine.

Sure. We can live without Latest. I still like it, but I can concede the point. Note that having Latest makes it easy for us to have, say, -XLambdaCase and -XLambdaCaseV2 and get people to upgrade.

If going forward, we have every extension beyond the stable baseline violate the stability guarantee, I think that's actually fine!

Over the years, we've accrued a small number of language extensions that will forever remain a bit of configuration for users. I don't see that going away. (Examples: TemplateHaskell, which enables wonders but negatively affects compilation properties; and RebindableSyntax.) So I think some amount of extensions will always remain in Stable mode, and that's ok.

think batching unstable things is a bit artificial, unless it's a "stable edition release candidate"

True -- this is why Experimental does not enable new features beyond Stable. It just allows the user to enable them if they want. (Exception: Experimental evidently doesn't have one warning that Stable does: -Wmissing-poly-kind-signatures. I don't feel like rediscovering why I made that decision, which I'm happy to revisit later.)

The moral of the story to me that things like the -XDeriveFunctor flag only exist because we didn't have edition flags. Once we do, we can just get rid of such flags.

💯 💯 I have presented this in terms of language extensions, because that is the lingua franca of Haskell features. But it needn't be. We can just have features in the language without a named extension -- like your supposed deriving Functor. I agree completely that such a feature needn't be configurable independently.

To sum that up,

Yours is an interesting reduction of the soup I have prepared. I think I agree.

I'd argue editions is, in fact, little more than returning to the original plan of GHC extensions and frequent Haskell reports!

Maybe? But with fewer extensions. I like the conclusions in that post.

@gbaz

I don't think I agree at all with the treatment of warnings and their purpose

I'm not quite sure which aspect of warnings you're referring to here. I feel pretty confident about the relationship between warnings and extensions (that they're really the same thing and ought not to be treated separately). I'm much less confident about the no-warnings-on-deployment story. I'm not retracting my opinions there, but I think there's healthy room for debate about that point.

@adamgundry

language editions/bundles should be composed of simple mostly-orthogonal options

There is a limit to how useful this is. You and I are composed of trillions of cells as determined by billions of bits of DNA. But it's really much easier to understand you as Adam and me as Richard -- unless you're a cell biologist or a doctor trying to debug one of us. My table has 240 lines. That's 240 independently configurable features. Not billions or trillions. But still lots! At some point, it's easier to explain -- especially to users -- large coherent modes rather than many orthogonal features. We implementors may continue to track a growing number of orthogonal features, but I think this should become an internal concern.

How do you see the actively underway GHC2024 process evolving in the light of this?

A great question. I think it should continue unimpeded unless we quickly accept. I'd much rather keep moving forward than to become paralyzed by debate and prevent GHC2024.

There were a few other detail-oriented comments that I'm avoiding responding to. I don't mind seeing the comments, but responses are likely to derail the more important high-level conversation we've all been enjoying here.

@gbaz
Copy link

gbaz commented Jan 6, 2024

I'm not quite sure which aspect of warnings you're referring to here. I feel pretty confident about the relationship between warnings and extensions (that they're really the same thing and ought not to be treated separately). I'm much less confident about the no-warnings-on-deployment story. I'm not retracting my opinions there, but I think there's healthy room for debate about that point.

I disagree with both elements. You're correct that some warnings are coupled to extensions, but not all are.

I just created a simple haskell file with the following.

module Main where

main = print 12

Compiling this with -Wall yields two warnings. First, for a missing type signature, and second for defaulting to Integer.

Neither of these are error-like warnings. I know what the types are and do not need a signature, and I know what a polymorphic use of an integer literal will default to, and my program works just fine. Nonetheless, obeying both warnings will certainly give me higher code quality in production code, make my code easier to read, easier to debug in the future, etc.

I do not see why either warning would want to be coupled to an extension, or why we should want to actively prevent code with these warnings from being uploaded to hackage or whatever else.

I do see the appeal of unifying the mechanisms for controlling warnings and extensions, but I also note the big difference. A language extension controls what code means and so needs to be the same for compiling during development, CI, deployment. However, a warning just controls what the compiler emits and so may (and should) vary during those different phases. I.e. as a developer I may want to see all warnings, in CI I only want to check for some warnings, and in deploy I may want to ignore all warnings.

That said, there are only three sections of this proposal dealing with warnings. The first is general philosophy, which we can debate, the second is making warnings into errors, which you note is not a ghc proposal, and so not actionable, and the third is the already withdrawn idea (but still included in the text) of a full unification of errors and warnings.

So in this sense the warnings stuff is already extraneous to this proposal, in that it includes a number of interesting but contentious ideas, but also nothing actionable as a ghc proposal.

So I would urge removing it for that reason alone.

@simonmar
Copy link

simonmar commented Jan 6, 2024

Thanks for writing this up @goldfirere! It aligns very well with what I and others have been arguing for in other threads, so I'm definitely in favour of the overall direction. In particular

The goal is that a vast majority of our users will be able to specify a language edition, and that's it. No extensions. No warning flags. This simplifies what a user needs to think about when setting up a Haskell project, removing the paralysis of choice that can reign today.

Yes! (although I'm not sure I'm onboard with "no warning flags", more below)

If we ever feel the need to evolve the meaning of an extension, we can do so, by keying the meaning of the extension to the language edition. This allows forward evolution without sacrificing stability. (There may be some implementation duplication to achieve this. There is a cost to stability, but one I think is worth paying.)

Yes! And it's important to state the corollary, that we don't change the meaning of an existing extension for an existing language revision. As others have said, we can avoid changing the meaning of extensions at all and just have explicitly versioned extensions, or more fine-grained extensions - that's also fine and perhaps better. Both alternatives achieve the stability goal that we tie changes to language editions and not GHC releases. (I realise you're not trying to address stability here, but it's closely related)

I'm not sure I'm convinced that we need the extension bundles like FancyTypes, or the extra language editions like Experimental2024. If you're doing something outside the paved path of a particular language revision, then you can use extension flags. Adding another layer here seems to be adding extra complexity that I don't see a clear need for.

I'm also not sure I'm convinced we should be more prescriptive about warnings. The idea of not shipping code with warnings is a good one and I use it myself, but requiring it? I'm not sure at all. Warnings to me are the place where developers can reasonably differ about which stylistic choices they want to make.

@tomjaguarpaw
Copy link
Contributor

tomjaguarpaw commented Jan 7, 2024

Thanks very much for writing up this clear and thorough document @goldfirere! I'm in favour of the direction of travel and I look forward to seeing how the discussion of details pans out.


These are some thoughts and questions I have.

A feature bundle is a collection of Haskell features all require the same assumption of responsibility from users

I really like the idea of describing feature bundles as collections of features that have the same expectations from users. I've never thought of classifying things like that.

A three-year timeline is long enough that we might imagine contributors writing upgrade tools in that timeframe.

Although this is a tangential to the main thrust of the proposal, I feel compelled to point out that almost every time we discuss how to best manage breaking changes someone suggests that tooling can be written to help with migration, yet that tooling has never appeared.

Latest: This language edition is the latest and greatest that GHC has to offer. Compiling with Latest might break between releases.

Is Latest the only edition that can break between releases? That seems to be the implicit meaning. Could we make it explicit?

If language editions come out only every three years

Could we expand on why only every three years? What would be the problem with every year, say?

@goldfirere
Copy link
Contributor Author

@gbaz

My comments relating warnings to extensions starts from this observation: Suppose the birth of Haskell happened without the ability for name shadowing. Writing f x = map (\x -> ...) .... would be an error, because x cannot be shadowed. This would be a perfectly usable programming language. Then someone comes along and says that shadowing would be useful. So, after requisite debate, we introduce -XNameShadowing. Some users turn it on; others don't.

In that alternate universe, we have a Haskell with roughly the same expressiveness as in our current universe, but instead of -Wno-name-shadowing, we have -XNameShadowing. And instead of -Werror=name-shadowing, we have -XNoNameShadowing. (Interestingly, the alternative universe is somewhat lacking: there is no equivalent to -Wname-shadowing.)

The upshot of this little analysis is that whether an idea is captured in an extension or a warning is path-dependent. That is, this dichotomy is a record of our past but says rather little about our present. Essentially, warnings are (optional) removals from the language, while extensions are (optional) additions. In the end, though, to users, the historical context matters not.

To me, this all suggests: whatever we do for extensions, we should do for warnings, and vice versa. That is, if we're considering a new way of wrangling extensions (e.g. the GHC2021 mechanism), that process should encompass warnings on equal footing.

Not all extensions could be rephrased as warnings, as you rightly point out. But many could. On the other hand, I believe all linguistic warnings (excluding e.g. warnings about command-line options) could be turned into extensions, though this would exasperatingly lose their ability to warn. I propose neither such change. Instead, I just propose that we consider extensions and warnings as sibling features, both capable of controlling the range of constructs allowed in a user's program.

The actionable part of this is that the language editions and feature bundles include warnings and extensions. We're not at the point of debating particulars yet, but I do think the high-level point -- that a language edition would be able to control warnings -- is worth debating now.

@goldfirere
Copy link
Contributor Author

@tomjaguarpaw

Is Latest the only edition that can break between releases? That seems to be the implicit meaning. Could we make it explicit?

Could we expand on why only every three years? What would be the problem with every year, say?

This cadence feels like the right balance between our cost, users' stability needs, and lack of stagnation. It's something of an arbitrary choice, but I like it.

No. Experimental can also break. Though I would expect you to need to write e.g. -XExperimental2024 -XLinearTypes to actually get the breakage. Latest is really just there so someone can get all the toys without listing each one. It's not a critical part of this proposal.

@tomjaguarpaw
Copy link
Contributor

It might be worth pointing out that two putative extensions corresponding to @gbaz's example could be -XInferTopLevelTypes and -XAllowAmbiguousLiterals.

@tomjaguarpaw
Copy link
Contributor

Could we expand on why only every three years? What would be the problem with every year, say?

This cadence feels like the right balance between our cost, users' stability needs, and lack of stagnation. It's something of an arbitrary choice, but I like it.

Thanks, I think I'm probably not grasping something. I'm confused about the following.

If we release a new Stable edition every 3 years, users have three years to upgrade before we no longer guarantee support

Suppose I write code targeting Stable2024. When is the first GHC release that will not support Stable2024, i.e. when will my code fail to compile, with a "We don't support that language edition any more" error? Is it in 2027? Or 2030? Or something else?

@Ericson2314
Copy link
Contributor

Ericson2314 commented Jan 8, 2024

@goldfirere

Over the years, we've accrued a small number of language extensions that will forever remain a bit of configuration for users. I don't see that going away. (Examples: TemplateHaskell, which enables wonders but negatively affects compilation properties; and RebindableSyntax.) So I think some amount of extensions will always remain in Stable mode, and that's OK.

I totally agree! However I think, as much as history / old editions permit, we should strive to have separate terminology / separate syntax for such stable configuration options as opposed to actually unstable extensions. Same "territory", just a different "map" so people have the right intuitions.


The upshot of this little analysis is that whether an idea is captured in an extension or a warning is path-dependent. That is, this dichotomy is a record of our past but says rather little about our present. Essentially, warnings are (optional) removals from the language, while extensions are (optional) additions. In the end, though, to users, the historical context matters not.

"Path dependent", "history shouldn't matter", there is so much of this I like! However --- and I imagine, @gbaz might be thinking similar things --- this only works when there is (equivalently):

  1. No fork-like extensions
  2. A "supremum language" for which all other languages can be expressed as it and lints (subtracted functionality)

I think we all agree that there shouldn't be such a supremum between editions. And while perhaps there is such a supremum within stable division, I am not so sure about the unstable world.

To run with the example around name resolution, the essence of me of #608 is that there is a desired fork-like situation here:

  1. Some people want the future student/conservative/non-dependent Haskell to continue having various forms of implicit binding
  2. The future dependent haskell probably should not have implicit binding.

The two approaches coincide as long as there is no shadowing, but once there is, you either need to be fork-like, or through away the Lexical Scoping Principle.

To me, this is pretty clear evidence that the "additive" rather than "subtractive" approach has merrits: again not on "historical evolution grounds" (which I totally agree is bad bogus reasoning, and editions subsumes it) but on "we're still figuring what we want to do, and how many stable "tracks" of the language we need (i.e. stable editions in space not time) going forward" grounds.

edit: Fix Latin :)

@goldfirere
Copy link
Contributor Author

@tomjaguarpaw

Is it in 2027? Or 2030?

I was intending 2030. But this is not spelled out anywhere. That is, a stable edition is live for 3 years, sunsetting for 3 years, and then is due for removal. I could see waiting longer before removal. And I think removal would just be "remove bits of the compiler that are painful to keep around"; maybe many packages would continue to compile even with an officially-no-longer-supported old edition. I'm far from sure about what's right here.

@Ericson2314

Re additive vs subtractive: I generally agree that we might not have a supremum. As long as we do in the stable version. :)

@dixonary
Copy link

dixonary commented Jan 9, 2024

Chiming in as an educator - many thanks to @goldfirere and others for the consideration in the above. Having a Student category of enabled language extensions is a very interesting idea and potentially valuable pedagogically. In an ideal world, such a set would:

  1. Enable exactly the extensions that allow for clean, short, code, and maximise the chance of GHC correctly inferring the meaning of simple code;
  2. Minimise the role of advanced or clever features which are more likely to induce "interesting" type errors.

As a point of reference for what I mean, the dreaded MonomorphismRestriction means that these definitions are not equivalent, even though we would really, really like them to be, when people are first learning Haskell (this specific code tripped up an explanation just today!):

min   x y =             if x < y then x else y
min'  x   =       \y -> if x < y then x else y
min''     = \x -> \y -> if x < y then x else y

And I think that, while it is reasonable for such a thing to be enabled for normal use, having it enabled from day one for new learners does far more harm than good. Hence -XNoMonomorphismRestriction would be a good example of a differentiator between the use cases for Student and Stable [or whatever] featuresets.

All that said, the current state of play is really fine from the teaching perspective, and if such a set would induce a burden of work on the maintainers, and/or on those who create published educational content, then it might be worth focusing attention elsewhere.

@int-index
Copy link
Contributor

int-index commented Jan 9, 2024

A small observation on the branding of the proposed "Student" edition. People typically prefer to learn using "the real thing", so that the acquired skills are applicable in real projects. Perhaps naming this edition "Minimal" would be better, to make clear that it's a restricted subset of real Haskell, not some special student-oriented dialect.

@Ericson2314
Copy link
Contributor

FWIW, the "student" version I think is inspired by the How To Design Programs and its few levels of student languages. As someone that long ago took an intro class using that :), it's something that I think has merit.

Autodidacts (and I've done plenty of that too) certainly like to get their hands on the "real deal", yes, but being able to get better errors by excluding a long tail of bells and whistles (think no "did you mean?" referring to something that is not yet introduce in the curriculum) is quite useful for students in school who might sort of feel everything to do with computers already feels so daunting and complicated.

(Still, as I wrote above, I think we can not figure out the student situation until the main questions around editions can be answered. It could be the answer is "yes this stuff is useful", but only

@gbaz
Copy link

gbaz commented Jan 10, 2024

The actionable part of this is that the language editions and feature bundles include warnings and extensions. We're not at the point of debating particulars yet, but I do think the high-level point -- that a language edition would be able to control warnings -- is worth debating now.

I can see feature bundles controlling warnings just as a convenience -- i.e. we do want to bundle warnings (and already do, via all, etc). However, I do not think that language editions controlling warnings directly makes a ton of sense. This is because different warnings make sense in the same language edition, but in different environments (develop, debug, CI, release). I think it mainly makes sense to control warnings indirectly via extensions, and have language editions simply control those. But also, it makes sense to allow "feature bundles" specifically for warning control for convenience. My view is that no feature bundle should directly control both warnings and extensions, and no language edition should directly control a warnings-only feature bundle.

Finally, I really don't understand why some langauge editions force some warnings to be unable to turned off, while others don't. Seems like it actually undercuts the orthogonality we desire. (in fact, ditto with errors).

@simonpj
Copy link
Contributor

simonpj commented Jan 11, 2024

Some thoughts, focused on the main payload (Section 7)

  • I'm all for language editions as a general principle.

  • "A language edition can control arbitrary behavior of GHC. The meaning (or existence) of other flags can depend on language edition." I think that is stating it too strongly. That could mean that -XExplicitForall could do what it says in GHC2021, but in GHC024 it instead means "enable impredicative polymorphism". That would not be good for users.

    At the extreme, GHC2027 could mean "compile Python", and reject every exisiting Hakell program.

    I can see why you want to allow for the possiblity of changing meaning for existing things, but we should so extremely cautiously. We need to find a way to say that a language edition will not gratuitously change the meaning of existing flags.

  • One Very Good Thing that a language edition could do is to fix which version of Prelude is imported. Currently, import Prelude imports one fixed module. But exactly which module it imports could vary with language edition. That would solve the problem we are experiencing right now, where Prelude now exports foldl' and that is breaking many compilations. If the language edition specified the prelude, that would not happen.

    The neatest was to do this would be for the language edition to specify the version of base; but that would require some version of "reinstallable base".

  • 7.2 says "We introduce several coarse-grained semantic bundles that group together similar features". But you never define what a bundle is. I think you mean something like:

    • A semantic bundle is a synonym for a collection of langauge extensions and warning flags.

    Or maybe you mean something else? Precision is good.

  • 7.2 describes a general framework in which a language edition can do anything. (Too much actually, see above.) 7.3 describes a particaular realisation. But 7.3 lacks a critical piece of preamble, something like this:

    "In this section we describe an initial realision of the general framework in 7.2. In this realisation, a language edition means precisely

    • A set of extensions to be enabled
    • A set of warnings to be enabled as warnings
    • A set of warnings to be enabled as errors
    • A particular version of Prelude
    • ... anything else...?

    Now (but only now) it makes sense to give the table saying exactly which extensions/warnings are on for each language edition.

  • Can a language edition have some but not all of the things in a feature bundle? The two concepts are curiously overlapping, as you can see by the fact that they are both columns of the same table.

@simonpj
Copy link
Contributor

simonpj commented Jan 11, 2024

However, I do not think that language editions controlling warnings directly makes a ton of sense.

I think it does make sense! As #615 describes, it's often a matter of happenstance whether something ends up as a warning or an extension.

For example a language edition PickyHaskell might switch on -Werror=name-shadowing to say that in the PickyHaskell language you can't do name-shadowing.

@adamgundry
Copy link
Contributor

@simonpj

  • One Very Good Thing that a language edition could do is to fix which version of Prelude is imported. Currently, import Prelude imports one fixed module. But exactly which module it imports could vary with language edition. That would solve the problem we are experiencing right now, where Prelude now exports foldl' and that is breaking many compilations. If the language edition specified the prelude, that would not happen.
    The neatest was to do this would be for the language edition to specify the version of base; but that would require some version of "reinstallable base".

This is a very interesting idea. It might even allow us to get closer to editions like Haskell98 actually compiling using the Haskell98 prelude (modulo backwards compatibility concerns, of course). We'd struggle to get all the way, though, because we need to allow code in a single build plan to be compiled using a mixture of language editions (e.g. in different packages). That means we need to have the language editions/preludes agree on the definitions of types/classes, so e.g. going back to a pre-AMP Prelude would probably not be feasible.

@simonpj
Copy link
Contributor

simonpj commented Jan 11, 2024

we need to allow code in a single build plan to be compiled using a mixture of language editions

I hadn't thought of that. It is rather in contradiction with this (my bold):

  • A language edition can control arbitrary behavior of GHC. The meaning (or existence) of other flags can depend on language edition. While we will not implement it this way, we can imagine that GHC becomes a set of programs that happen to share a binary; the choice of which program is chosen by the language edition.

Libraries compiled by different programs might not be compatible. The "arbitrary behaviour" might include a change of calling conventions.

I don't think that is intended. But it would be good to say explicitly that programs compiled with a mix of language editions should still work.

We'd struggle to get all the way, though

Agreed. The same limitations apply to the reinstallable base idea; but I think we could get a long way.

@goldfirere
Copy link
Contributor Author

I've updated the text to incorporate some of what has been written here.

In particular, I feel compelled to call @Ericson2314's attention to the last Open Question that I have added. I suspect he will be pleased with the idea.

@Ericson2314
Copy link
Contributor

Ericson2314 commented Jan 12, 2024

:) Yes I am also torn with, on one hand, wanting to keep matters of language and libraries ("contexts") strictly separate, and, on the other hand, being excited to go hog-wild with some sort of -XComplete.

Comment on lines +349 to +350
* **Preparing a package for uploading to Hackage (e.g. ``cabal sdist``) should
fail if any warnings are produced.**
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree for two very practical reasons:

  1. Cabal can produce sdists very quickly and doesn't require anything else from the environment. Checking for Haskell warnings adds a dependency on GHC and possibly system packages, and makes the process potentially very slow.
  2. It's impossible to enforce from the Hackage side.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reasonable objections. But these are surmountable, if we want to make this change: cabal could retain some information saying whether the last build was warning-free (or even which warnings were produced) and then check the information during sdist. This would not induce a dependency on GHC. And similarly it could upload the evidence if we wanted a Hackage check (although I don't think the Hackage check is necessary).

I'm not arguing here that all of this is a good idea -- just that these implementation matters need not slow us down from the general idea.

@goldfirere
Copy link
Contributor Author

@Ericson2314

language and libraries

Keeping these separate sounds nice... but is it? From the user's perspective, a language and its standard library are really the same thing. Indeed, a great many tools even syntax-highlight standard library functions differently from other ones. (I strongly disagree with this practice, but its prevalence sends a signal about people's understanding of the relationship between a language and its standard library.)

If it helps you feel more comfortable here, we could say that the implicit import of Prelude is definitely a language feature. So we could imagine the language edition just to change which module is implicitly imported. (This isn't great, because it interacts poorly with explicit import Prelude lines. So we may want additionally to treat import Prelude specially.)

@Ericson2314
Copy link
Contributor

Well, since a goal is one can mix modules/packages using different goals, there is firm ceiling on how many much library-level effects editions could have. I am indeed more comfortable making it effect Prelude, but I do think that this does feel like winging it without the theory of private dependencies (or, less anti-modular theory of library coherence) that we'll ultimately need. (See some discussion of this in haskell/cabal#4035 if you anyone is curious.)

Bottom line for me is I think we can still get a lot of mileage of out editions while punting on these problems. Setting base version bounds in Cabal files already works fine for now.

@shlevy
Copy link

shlevy commented Feb 5, 2024

@goldfirere I think if you rebase this on master you might get a readthedocs build (the rendered link is no longer rendered).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet