Extension lifecycle framework proposal (under review)

[david-christiansen]

This proposal suggests a defined lifecycle for language extensions, to more clearly communicate tacit community knowledge about which extensions make sense in which contexts. Additionally, it proposes a set of compiler warning flags that empower users of GHC to choose their own level of tolerance for language extension changes.

This proposal is a product of the Haskell Foundation's Stability Working Group.

Rendered

[simonpj]

The link from the rendered proposal to this thread is broken.

Broadly, looks sensible to me.

[david-christiansen]

I've fixed the broken link - thanks for the heads up!

[Ericson2314]

This looks very good to me. We've been working on some similar stuff over in Nix land for the same reasons.

---

I don't want to derail this thread, but are the issue(s) with RecordPuns as discussed by the stability working group written up anywhere?

[david-christiansen]

RecordPuns is merely a deprecated name for the extension now known as NamedFieldPuns - we didn't discuss anything about the actual technical content of that extension, just that the old name is still around.

[nomeata]

Thanks for writing this up. A few questions:

@aspiwack is in the process of clarifying the purposes of Extensions, e.g. via multiple polls among the committee. It’s not quite the same question, but there is overlap. The two efforts should probably be synchronized to a point.

I agree about the usefulness of -WXDeprecated and -WXLegacy: It seems right that users, probably by default, should be notified when an extension they are already using, or are about to use, is no longer considered a good idea.

I’m less sure about -WXExperimental. Assume I am writing a program, and am about to use the new letmagic feature. I indicate that by using the letmagic syntax, but that doesn’t just work – it’s an extension after all, and GHC tells me to jump through a second hoop, and and {-# LANGUAGE LetMagic #-} to the file. I do that, but now I get a warning that it’s experimental. I probably swear with my internal voice, and add {-# OPTIONS_GHC -Wno-XLetMagic #-} to the file. Did I really gain something?

It seems to me that spelling out {-# LANGUAGE LetMagic #-} should already enough to tell the compiler “yes, I really mean it”. Adding even more hurdles seems … patronizing.

But I see your point: If the opt-in looks the same for experimental and mature extensions, there is nothing forcing the programmer to be aware of the experimental/mature distinction (which, it seems, you think worthwhile). But then maybe the problem isn’t with the experimental extensions (they require an explicit and loud opt-in), but with the mature extensions. If letmagic is mature, shouldn’t it be enough to simply use it? Why the extra hoop there?

One conclusion could be that we don't need -WExperimental and -XMature, and instead mature simply means “enabled by default”, and experimental means “manual explicit loud opt-in needed”. Which is the idea behind the GHC20xx language sets.

Of course, reality isn’t nice enough for that; there are features that are language extensions right now that are mature (in the sense of the current proposal), but for various reasons not suitable to be enabled by default (e.g. not syntax-guarded); a better understanding of this is part of what Arnaud words towards to.

So I don’t have a great answer; I am not keen on having to write {-# LANGUAGE LetMagic #-}{-# OPTIONS_GHC -Wno-XLetMagic #-} though, that seems silly.

Maybe not including -WExperimental in -Wdefault or even -Wall is reasonable; it still gives people the option of enabling it manually to opt-into the double protection against the use of experimental extensions.

[simonpj]

It seems to me that spelling out {-# LANGUAGE LetMagic #-} should already enough to tell the compiler “yes, I really mean it”. Adding even more hurdles seems … patronizing.

I think David's point is that when I build my 800-moudule, 5-person project, I might like an end-to-end check that I'm not using any experimental features, via -WExperimental. So the existence of such a warning sounds sensible; but perhaps it should be off by default

[nomeata]

Indeed, the “large project usecase” is convincing, and there an off-by-default -WXExperimental makes sense. I can see that to take the form of a centrally managed .cabal setting like

  ghc-options: -Werr=XExperimental -Wno-XMagicLet -Wno-XIncoherentInstances

which also declares the project-wide whitelist of experimental extensions that are ok to use, but still get turned-on individually in the appropriate files.

[nomeata]

I am, however, not fully convinced this buys us that more over a simple hlint check that compares enabled extensions against a whitelist. I’d expect that such a large-project-setting likely wants to disable a few mature extensions as well (they might be mature, but not suitable for other reasons). So you already have to curate a list of allowed extensions, and check them somehow.

This proposal has the benefit of providing an initial categorization (mature/experimental) that only has to be refined, rather than written from scratch, and it comes with the linting feature. So I’m not opposed on that grounds, but don’t think I’d miss a lot if I had to implement this using hlint.

[david-christiansen]

There are some further benefits over the hlint solution, I believe:

1. The categorization is maintained in one place, so teams don't need to construct and then maintain their own set. It's much easier to have a minor local deviation from an established standard than it is to create one.
2. Compiler updates that change the status of an extension will trigger warnings without someone needing to remember to modify the configuration of an external linter, and projects that support a range of GHC versions will get the appropriate warnings for each version they run in CI without any extra configuration.
3. Defaults matter most for less-experienced users. How would a new team know that they should install an extra linter and configure it this way? How would they decide between competing linter configurations? It seems valuable to provide them with something that basically reflects the consensus of the experts that comes by default.

[david-christiansen]

I do that, but now I get a warning that it’s experimental. I probably swear with my internal voice, and add {-# OPTIONS_GHC -Wno-XLetMagic #-} to the file. Did I really gain something?

I think that you have indeed not gained anything. However, someone who has no idea what LetMagic is, and who may have written the syntax by mistake (e.g. as a typo, or by copy-pasting code from a blog post), may avoid suddenly depending on something for which the design is in flux without realizing that they're doing so.

It seems to me that spelling out {-# LANGUAGE LetMagic #-} should already enough to tell the compiler “yes, I really mean it”. Adding even more hurdles seems … patronizing.

I think that this is absolutely true if you're a Haskell expert who understands the tradeoffs of LetMagic, including potential near-term breaking changes. It seems less likely to be true for users who are not. You're right that this proposal makes life (slightly) more difficult for experts - I think it does this in exchange for making it easier for non-experts.

[david-christiansen]

So the existence of such a warning sounds sensible; but perhaps it should be off by default.

I think that, after having it -Wcompat for a bit, it should be on by default. The reason for this is that the people who stand to benefit the most from the warning are those who don't yet know to turn it on. But this is definitely a trade-off in convenience between expert users and less experienced users, and there's good arguments to be made for the various points along that continuum.

[telser]

I am, however, not fully convinced this buys us that more over a simple hlint check that compares enabled extensions against a whitelist. I’d expect that such a large-project-setting likely wants to disable a few mature extensions as well (they might be mature, but not suitable for other reasons). So you already have to curate a list of allowed extensions, and check them somehow.

In the general case I think we as a community should reach for quicker moving and less invasive changing tools like hlint and stan (and others!) to achieve various kinds of guarantees and analysis. Some of my reasons for this include:

1. It can move much quicker for much of the community. Adding another tool, or upgrading one is often much less risky than upgrading a compiler. That is in addition to smaller and separate tools can release much quicker than ghc can.

2. It spreads the responsibility. Having more tooling means the responsibility can spread more easily over a larger set of people instead of asking the small group of ghc devs to take on more and more asks and maintenance.

3. Allowing for a greater community subset to input in feedback cycles. Where we have the ability to add something via ghc, or some other tools, watching the implementation of those checks across the community in potentially multiple tools before adding it to ghc gives a much bigger community engagement. Imagine: Three tools implement some kind of check with various overlaps. This gives us a much bigger picture, than if we simply had a proposal here. Note that as the community grows then almost certainly a shrinking percentage of ghc users will be following this repository in any way.

Now that said, context is extremely important. For this case in particular, I think the external tooling approach is subpar.

The following reasons from @david-christiansen are all apart of it:

1. The categorization is maintained in one place, so teams don't need to construct and then maintain their own set. It's much easier to have a minor local deviation from an established standard than it is to create one.
2. Compiler updates that change the status of an extension will trigger warnings without someone needing to remember to modify the configuration of an external linter, and projects that support a range of GHC versions will get the appropriate warnings for each version they run in CI without any extra configuration.
3. Defaults matter most for less-experienced users. How would a new team know that they should install an extra linter and configure it this way? How would they decide between competing linter configurations? It seems valuable to provide them with something that basically reflects the consensus of the experts that comes by default.

In my opinion the last one is really important. Here the categories come from the ghc project itself, which would be the authoritative source for information for the status of extensions as provided by ghc. If someone outside of ghc says something is experimental, or deprecated, that doesn't give nearly the same kind of information as this coming from ghc itself.

Further, I would assert that warning on Experimental by default is one of the more important options. If new to Haskell users get warnings that they are using something that by definition is subject to change, then that sets the expectation of potential pain for this particular feature. That allows the community to get experiments, but also lets the default sit as something that is going to be less prone to unexpected change.

[Ericson2314]

The hlint version also doesn't really help GHC developers. @int-index and I are in the process of:

1. Fixing up a number of things involving @ type application to constructor patterns
2. Making sure there are no breaking changes to e.g. -XScopedTypeVariables

We all intuitively know one of these is very mature and the other isn't, but its rather awkward that this isn't written down. This would make it explicit.

Which is the idea behind the GHC20xx language sets.

So "mature" means it is one of these sets, and "legacy" means it is in addition is no longer in the most recent one?

I suppose that works, but I do like tracking this information per extension. The causality running one way or the other is not entirely also clear to me. A few things:

• I would want individual extensions to be signed off as "ready to take out of experimental" by their maintainers before the extensions are eligible to be included in one of these promoted steps.
• Rather than having to check n many feature sets, it is nice to just check the extension in question for mature vs legacy.
• It is possible that we want something to be deprecated that was in such a set a long time ago.

If we are clever about exactly what we write down, we might not need to "denormalize" anything. And to the extent we do denormalize, we can have tests that ensure nothing drifts out of sync.

[michaelpj]

Nor Deprecated nor Legacy, I would have thought? Or at least, they shouldn't have that status when the language set is created.

[david-christiansen]

Haskell 98 certainly includes NPlusKPatterns, which I think may end up in one of those categories. And over time, it doesn't seem implausible that we'd declare legacy an extension that had been in one of these default sets (after a very, very long notice period, of course)

[michaelpj]

Right, things that are in a set can become deprecated or legacy, but when we create a set we probably shouldn't include anything in those states? e.g. GHC2021 does not contain NPlusKPatterns

[telser]

It certainly seems like a good idea to as a matter of policy say that a new language set does not contain legacy or deprecated extensions. However, if a language set should be added after the fact for some reason, then perhaps it would include a legacy extension. Maybe resolving if a policy should exist for this and what exactly that would be is worthy of a separate discussion?

[david-christiansen]

I'm loathe to impose policy here. I think that the people building such a set can take these extensions into account, and that any policy is either redundant or the kind of thing that would be set aside anyway in the situation.

[david-christiansen]

One more case: the Haskell98 extension set presumably came into existence after some of its component parts were broadly acknowledged as design dead-ends (and thus Legacy). Introducing this set is nonetheless useful, as it helps people improve compatibility with older code. Here, any policy could have been set aside, but making a policy seems to not buy us anything. I certainly agree that we should be surprised if e.g. GHC2035 includes any deprecated extensions when it's introduced in 2035, but how will a written policy actually help?

[cdornan]

I have to say I would rather not link this to the much more ambitious efforts to construct a new framework for understaning extensions. This very modest proposal seeks to in effect codify our existing practice so that they can be better understood more widely and better managed. If we could reach agreement on this proposal (see the early comments) then that could help clarify some things for the more ambitious undertaking.

[aspiwack]

I have to say I would rather not link this to the much more ambitious efforts to construct a new framework for understaning extensions.

I agree. The problem that the effort I'm leading is trying to solve is different, and largely orthogonal to this one (it's essentially about being more principled when answering questions like “should X be an extension”, or “should these two extensions be a single extension”). The discussion ebbs and flows, and the discussions intersect. But the harder discussion oughtn't block the easier one.

[david-christiansen]

I agree with @cdornan and @aspiwack - I think that these efforts are basically orthogonal. Thanks, @nomeata, for pointing us in each others' directions so we could make sure!

[Ericson2314]

#604 Is a small amendment to an existing proposal to change something that I think would be (or ought to be) caught by this: a more mature extension implying a less mature extension.

[simonpj]

Some polishing suggestions

[simonpj]

Suggested change

	* `Stable` extensions are considered to be finished and are not expected to undergo regular changes. These features can be used without worry of unexpected changes, and they are not known to contain serious design or implementation deficiencies. Any breaking change to a stable extension will be announced well in advance of the change being made, with a migration path provided if possible. For an extension to be classified as `Stable` it must be considered `Stable` when used in combination of all possible, non-mutually exclusive, extensions that are already `Stable`. This ensures any extension that is `Stable` will be have the same use without worry when combined with any other extension in the set, while allowing for mutually exclusive extensions to be added. Ideally, no breaking change will be made to a `Stable` extension, with incompatible changes resulting instead in a new, related extension, that is possibly mutually exclusive with the existing one, to enable smooth migration. Some extensions in this category might be `MultiWayIf`, `MonoLocalBinds`, and `ViewPatterns`.
	* `Stable` extensions are considered to be finished and are not expected to undergo regular changes. These features can be used without worry of unexpected changes, and they are not known to contain serious design or implementation deficiencies. More specifically
	* For an extension to be classified as `Stable` it must be considered `Stable` when used in combination of all possible, non-mutually exclusive, extensions that are already `Stable`. This ensures any extension that is `Stable` will be have the same use without worry when combined with any other extension in the set, while allowing for mutually exclusive extensions to be added.
	* Ideally, no breaking change will be made to a `Stable` extension. Instead, it may be better to add a new, related extension, that is possibly mutually exclusive with the existing one, to enable smooth migration.
	* If, however, a breaking change is made to a stable extension, it will be announced well in advance of the change being made, with a migration path provided if possible.
	* Some extensions in the `Stable` category might be `MultiWayIf`, `MonoLocalBinds`, and `ViewPatterns`.

[simonpj]

"This ensures any extension that is Stable will be have the same use without worry " doe not parse.

[simonpj]

Any thoughts on this?

[telser]

I thought the wording changed slightly in the last update but I must have put the existing one back before committing.
Perhaps:

This ensures any extension that is Stable will have the same stability properties when combined with any other extension in the set, while allowing for mutually exclusive extensions to be added.
?

[simonpj]

Suggested change

	* `Deprecated` extensions are considered to be design or implementation dead ends, and should not be used in new code. Deprecating an extension means that GHC intends to remove support for it in a future release (though this decision may be revised in the light of user feedback). When an extension is deprecated, the user's guide and the compiler warning _must_ include a statement about the longevity of the extension, though this need not necessarily commit to a concrete time for removal. Additionally, the user's guide and the warning should direct users to information about migrating away from the deprecated extension. Any extension will be deprecated prior to removal. Some extensions in this category might be `OverlappingInstances`, `Rank2Types`, `RecordPuns`, and `TypeInType`.
	* `Deprecated` extensions are considered to be design or implementation dead ends, and should not be used in new code. Deprecating an extension means that GHC intends to remove support for it in a future release (though this decision may be revised in the light of user feedback).
	* When an extension is deprecated, the user's guide and the compiler warning _must_ include a statement about the longevity of the extension, though this need not necessarily commit to a concrete time for removal. Additionally, the user's guide and the warning should direct users to information about migrating away from the deprecated extension.
	* Any extension will be deprecated prior to removal.
	* Some extensions in the `Deprecated` category might be `OverlappingInstances`, `Rank2Types`, `RecordPuns`, and `TypeInType`.

[simonpj]

Formatting bug. The latter two bullets are over-indented

[telser]

Thank you, the last update should fix the formatting issue.

[simonpj]

Please list all transitions. Otherwise what does "also" mean, precisely?

Or, perhaps better, list the excluded transitions, with some rationale.

[simonpj]

I am keen to list all allowed transitions; or all disallowed transitions. Listing some of each, leaving others unspecified is not a happy state.

[telser]

Updated to list all transitions, including those not explicitly included or excluded in the proposal.

[simonpj]

" Restrictions are places on removal only"

What precisely is the restriction?.

[simonpj]

Any thoughts on this?

[telser]

I updated the set of included transitions to be more complete and added the exclusion list.

[simonpj]

" We explicitly allow for more transitions". More than what? Can you make this precise?

Maybe you have in mind a set of "normal" transitions, ones that are "exceptional" and ones that are "disallowed"? I'm really not sure.

[telser]

Updated For this and the above to note the excluded transitions explicitly.

[simonpj]

I can't see the change here.

[telser]

The explicitly excluded transitions are listed in the block right above this paragraph now.

[simonpj]

"language edition" not "language set"

[telser]

Thank you, updated!

[simonpj]

It still says "language set"

[simonpj]

Why do you pull out two particular transitions for special mention? Why not just say the user guide should document the GHC version in which each transition was made?

[telser]

I've updated the wording to make it more clear the guide should say when each transition was made and that two called out are examples.

[simonpj]

I don't see a change here

[adamgundry]

From a process point of view I think it makes sense to keep the same shepherd by default when a proposal is revised. Of course @simonpj if you'd prefer someone else takes it on, you are welcome to ask for it to be reassigned. But otherwise I think we can discuss it here and you can bring a recommendation to the committee again in due course.

---

Regarding the current state of the proposal itself, I'm generally supportive of accepting this and then bringing forward a new proposal with concrete classifications (as I've been trying to work on in the documents linked from #635). Some big picture comments:

• The proposal talks exclusively about extensions, but GHC has many features that are not extensions as such (e.g. INCOHERENT pragmas). We could surely use the same categorization scheme for them as well, at least in principle (though actually categorizing all existing features is probably infeasible, not least because of the need to pin down what "feature" means).
• I think the Experimental category described here is not quite suitable for Add -experimental flag #617, because the notion of experimental feature envisioned there (with users actively blocked from using the feature) is less widely applicable than the notion envisioned by this proposal (which covers all extensions not yet solid enough to be Stable). Indeed, Add -experimental flag #617 proposed to have its "experimental" set start off empty. I'm content to leave that subdivision until later, though, pending revision of Add -experimental flag #617.
• I'm not entirely sold on the Deprecated/Legacy distinction. I can see it is useful to communicate "this extension is definitely going to go away soon" vs "this extension will probably be supported indefinitely". But there are many extensions currently described as "deprecated" where there is no pressing need to remove them, so we might end up just rewriting the user's guide to say "legacy" instead of "deprecated" everywhere. Yet in both cases the advice for end users is the same: move away from using this extension. So overall I'd prefer just to have a single Deprecated category, with each deprecated extension carrying a statement about its expected longevity in the user's guide.

[simonpj]

The proposal talks exclusively about extensions, but GHC has many features that are not extensions as such

In the interests of getting something done, I'd be inclined not to widen the scope of this proposal.

I'd be quite happy to merge Legacy and Deprecated for now, just as I am happy to have one Experimental category not two (or more). The fewer categories we have, the easier it is to place things in categories.

Does anyone else have a view? Or @telser ?

[telser]

The proposal talks exclusively about extensions, but GHC has many features that are not extensions as such

In the interests of getting something done, I'd be inclined not to widen the scope of this proposal.

I agree with this. Making some progress feels important even if it isn't as broad as many, myself included, would ideally prefer.

* I'm not entirely sold on the `Deprecated`/`Legacy` distinction. I can see it is useful to communicate "this extension is definitely going to go away soon" vs "this extension will probably be supported indefinitely". But there are many extensions currently described as "deprecated" where there is no pressing need to remove them, so we might end up just rewriting the user's guide to say "legacy" instead of "deprecated" everywhere. Yet in both cases the advice for end users is the same: move away from using this extension. So overall I'd prefer just to have a single `Deprecated` category, with each deprecated extension carrying a statement about its expected longevity in the user's guide.

I'd be quite happy to merge Legacy and Deprecated for now, just as I am happy to have one Experimental category not two (or more). The fewer categories we have, the easier it is to place things in categories.

Does anyone else have a view? Or @telser ?

It has become clear from conversations that the state of extensions is critical. While I certainly believe that for GHC it is not pressing to remove some of these extensions, for a subset of the community the frustration with extensions is extreme. In talking with those users as part of forming this proposal, it was a very loud complaint that extensions do not go away, causing ever more discussions. There are two actions that seem lightweight and form an olive branch to that subset of users. First is removing some of these extensions that have been marked as deprecated for years or decades, or are undocumented. Second being very clear on the status of extensions by having two categorizations here. Let some of the extensions be Deprecated and let's get them removed and others as Legacy and commit to them sticking around for a while.

[simonpj]

it was a very loud complaint that extensions do not go away, causing ever more discussions.

Interesting. Can you say what extensions, specifically, caused these loud complaints? And what were the complaints about? Extensions that aren't used (except perhaps by a small minority) should not cause any trouble -- but perhaps they do? Specifics would help, since that makes the case for a legacy/deprecated distinction.

My anxiety is that the moment we say "X will go away", a new set of loud complaints will emerge about its absence :-)

[telser]

it was a very loud complaint that extensions do not go away, causing ever more discussions.

Interesting. Can you say what extensions, specifically, caused these loud complaints? And what were the complaints about? Extensions that aren't used (except perhaps by a small minority) should not cause any trouble -- but perhaps they do? Specifics would help, since that makes the case for a legacy/deprecated distinction.

Extensions that are not widely used do cause problems. They increase the surface area of the language and introduce more points of team debate. In some cases re-litigation of this happens every time a team gets a new member.

My anxiety is that the moment we say "X will go away", a new set of loud complaints will emerge about its absence :-)

Most of the complaints are more about the volume rather than having certain functionality. The most frequent is probably the hardest to solve, and out of scope for this proposal. However that seems to me to make the olive branch for handling others even more important.

Generally speaking the complaints are as follows:

• Extensions that are listed as Deprecated or "does nothing", like Rank2Types and NullaryTypeClasses, among others. These seem like the lowest hanging fruit to handle. Under this proposal these extensions could be marked as Deprecated with a timeline for removal.

• Extensions that seem like a dead-end like Safe, CUSKs, or DeepSubsumption. This grouping is probably harder to solve and depends on the specifics of each. Some, like Safe would appear to have general consensus on removal. Others though like DeepSubsumption clearly need to stay for backwards compatibility. By having a distinction between those extensions that are a dead end but are not immediately going away, teams have more information to establish policy that eases the extension debate.

• Extensions that users don't understand why they still exist when there seems to be a replacement that has been developed. An example that I've been given for this is that it seems ApplicativeDo could been remade with QualifiedDo. When asked why not move towards QualifiedDo, I don't have an answer.

• Finally is that extensions don't simply become part of the language. This ranges from syntactic items like TupleSections to things that touch type checking like MultiparamTypeClasses. Language editions don't really help here as users are still confronted with the extensions individually in a number of respects. They can be debated to turn on/off independently, and they are in the list of extensions individually forcing users to deal with that reality. Again, this is certainly out of scope of this proposal, but it certainly motivates handling the easier cases.

[Tritlo]

• Extensions that seem like a dead-end like Safe, CUSKs, or DeepSubsumption. This grouping is probably harder to solve and depends on the specifics of each. Some, like Safe would appear to have general consensus on removal. Others though like DeepSubsumption clearly need to stay for backwards compatibility. By having a distinction between those extensions that are a dead end but are not immediately going away, teams have more information to establish policy that eases the extension debate.

As an aside: Safe is being used in a mission critical way in at least one company doing differential privacy, where it allows them to ensure the safety of a DSL transitively throughout the toolchain (whereas similar frameworks in e.g. rust do the check, but can only go so deep). So here's an example of a "loud complaint" that would surface were it to be removed.

[telser]

• Extensions that seem like a dead-end like Safe, CUSKs, or DeepSubsumption. This grouping is probably harder to solve and depends on the specifics of each. Some, like Safe would appear to have general consensus on removal. Others though like DeepSubsumption clearly need to stay for backwards compatibility. By having a distinction between those extensions that are a dead end but are not immediately going away, teams have more information to establish policy that eases the extension debate.

As an aside: Safe is being used in a mission critical way in at least one company doing differential privacy, where it allows them to ensure the safety of a DSL transitively throughout the toolchain (whereas similar frameworks in e.g. rust do the check, but can only go so deep). So here's an example of a "loud complaint" that would surface were it to be removed.

@Tritlo That's extremely interesting! Some time ago there was a discourse thread and I don't recall anyone speaking up about concrete usage. Either way though the point of my comment and this entire proposal is to not decide any specifics. Any particular extensions in my comment are only to provide illustrative examples.

So this is interesting but I'd greatly prefer it if we do not get into any particular extensions further on this particular proposal to avoid any particular extension causing us to not have progress on this, extension agnostic proposal.

[simonpj]

@telser I have made various suggestions, some of which you have responded to, but not all.

I understand that you want to stick to the Legacy/Deprecated distinction. Fine. We can vote on that.

Please ping me me when you are ready. I have not yet made a recommendation to the committee, pending your final version.

[adamgundry]

• Extensions that are listed as Deprecated or "does nothing", like Rank2Types and NullaryTypeClasses, among others. These seem like the lowest hanging fruit to handle. Under this proposal these extensions could be marked as Deprecated with a timeline for removal.

What do we gain by actually removing them, though, rather than simply emitting a warning when they are used? I don't think many teams are debating whether to enable Rank2Types or NullaryTypeClasses, nor do they increase the language surface area. The only cost I see to keeping them is that they take up space if one looks at a list of all extensions (e.g. in the user's guide or with ghc --supported-extensions).

[simonpj]

Please ping me me when you are ready. I have not yet made a recommendation to the committee, pending your final version.

Actually to make this clearer I have changed the status to "needs revision". Let me know when you want to resubmit it.

Everyone: it would help the committee to have views from the community about whether you actively want a distinction between Deprecated and Legacy, or whether a single category would suffice. The strongest reason for making a distinction is that we can actually remove Deprecated extensions from GHC; if we'd have to rely on the user manual to give the removal timeline (if any) for a deprecated/legacy extension.

My own instinct is to keep it simple, have a single category; but Trevis's instinct is different. Hence wanting community views.

[michaelpj]

I think there is some meaningful action-guiding difference for a user between Deprecated and Legacy. If something is deprecated I actively want to get rid of it ASAP. If something is legacy, then that suggests that using it is maybe not a great idea but it's not urgent.

An analogy might be between warnings in compat and some of the others:

• A compat warning tells me something is going to actively break if I don't do something about it, a big deal.
• Another warning tells me something might be a bad idea, but it's not urgent. Maybe I want to get rid of the name shadowing in my code, but it's not like GHC is going to make it illegal.

The cost of merging Deprecated and Legacy depends on which one we are left with:

• If all of them will be Deprecated, then people will probably go to extra effort to remove extensions that are not in fact going to go away, which might be wasted work. People who rely on Legacy extensions like DeepSubsumption may be annoyed at having to make an exception to a "no deprecated extensions" policy.
• If all of them will be Legacy, then I wouldn't expect them to ever be removed. If some Legacy extensions might get removed, then this is quite annoying and makes it harder for me to have a policy on when they're okay, since I need to reason on a case-by-case basis and read the manual for each one.

[JackKelly-Bellroy]

I think it's unwise to merge Deprecated and Legacy. Example: I think DeepSubsumption is a good candidate for a Legacy extension: we don't use IHP, but AFAIK its design fundamentally requires -XDeepSubsumption for some of its types to work. Keeping -XDeepSubsumption around as a legacy extension seems like a good idea here.

But for extensions which are actively recognised as mistakes or are completely obsolescent, I think having a Deprecated category and a path to removal is healthy, and possibly good for GHC maintenance too.

[adamgundry]

I've been mulling this over. I can see that there is a useful distinction here to signal to users.

The reason I don't like the Deprecated/Legacy split as currently proposed is that for a long-deprecated but minimal-impact extension like NullaryTypeClasses, it gives us two options:

• Rewrite the user's guide to describe it as "legacy". This gives a misleading signal as there is no real reason for users not to move away from it, if they think about the issue at all. (It has historically been described as "deprecated", and that seems a better description.)
• Commit to removing support for it. But that risks breaking existing code without a particularly good reason.

In this case my preferred course would be to say NullaryTypeClasses is Deprecated, but never in fact remove it from the compiler.

[simonpj]

In this case my preferred course would be to say NullaryTypeClasses is Deprecated, but never in fact remove it from the compiler.

Deprecated means only "may be removed" not "will be removed". So what you suggest is fine.

[adamgundry]

Well, the current text says:

Deprecating an extension means that GHC intends to remove support for it in a future release (though this decision may be revised in the light of user feedback). When an extension is deprecated, the user's guide and the compiler warning must include a statement about the longevity of the extension, though this need not necessarily commit to a concrete time for removal.

So yes there are caveats, but the intention is for it to be removed. If there is consensus that "may be removed" is fine, then I'm happy, but I think we should reflect that in the text, e.g.:

Deprecating an extension means that GHC may remove support for it in a future release (though this decision may be revised in the light of user feedback). When an extension is deprecated, the user's guide and the compiler warning must include a statement about the longevity of the extension, though this need not necessarily commit to a concrete time for removal. Deprecated extensions may remain supported or partially supported for backwards compatibility purposes, even after the bulk of their functionality has been removed.

[simonpj]

As far as I can tell, there is some user sentiment in favour of offering a Deprectated/Legacy distinction, and no argument against except Occam's razor. I'm content to follow user sentiment here.

Status check: the proposal is in "Needs revision" state, awaiting the author (currently @telser) to revise and resubmit. Correct?

[adamgundry]

I believe so. I'd appreciate @telser's comments on my suggestion in #601 (comment).

I've updated my spreadsheet with a rough draft categorisation of extensions based on a variant of this where I distinguish between Internal and Experimental (explained in more detail in the accompanying document). In particular, GHCForeignImportPrim counts as Internal, as well as essentially-unimplemented extensions like JavascriptFFI and OverloadedRecordUpdate. I continue to think that it's useful to have an Internal category where we explicitly disclaim all stability guarantees.

[telser]

I believe so. I'd appreciate @telser's comments on my suggestion in #601 (comment).

I've updated the proposal to reflect the "may be removed" sentiment. Though I fear, at least some of, the users who have spoken at length with me on this will be disappointed in this change. However, if this change is acceptable to @adamgundry, I'm happy to resubmit.

[adamgundry]

Thanks @telser! This is looking good to me.

I've updated the proposal to reflect the "may be removed" sentiment. Though I fear, at least some of, the users who have spoken at length with me on this will be disappointed in this change.

I would love to understand better the source of this disappointment. Perhaps there are ways to ameliorate it (e.g. would dropping Deprecated extensions from the list of extensions in the docs help?).

[adamgundry]

I'm not convinced about excluding these. Something we have been known to do in the past is to introduce a new extension name for existing functionality (of the base language or another extension) that is being deprecated or otherwise restricted. For example:

• DeepSubsumption was arguably Legacy when it was introduced, because its functionality was already present in previous releases but we wanted to move towards a world in which NoDeepSubsumption became the default. (I'm not sure whether we would have considered NoDeepSubsumption to be Experimental or Stable on its introduction.)
• If we want to partially deprecate an extension, I think the right way is to split it into multiple extensions and introduce one or more of them as Deprecated (e.g. I can imagine splitting up IncoherentInstances = IncoherentInstanceDeclarations + IncoherentClassRoles + IncoherentSolving with IncoherentInstanceDeclarations being Deprecated).

So I'd suggest moving these to the "discretionary" category.

[telser]

I'm not convinced about excluding these. Something we have been known to do in the past is to introduce a new extension name for existing functionality (of the base language or another extension) that is being deprecated or otherwise restricted. For example:

• DeepSubsumption was arguably Legacy when it was introduced, because its functionality was already present in previous releases but we wanted to move towards a world in which NoDeepSubsumption became the default. (I'm not sure whether we would have considered NoDeepSubsumption to be Experimental or Stable on its introduction.)

This is a really good point.

• If we want to partially deprecate an extension, I think the right way is to split it into multiple extensions and introduce one or more of them as Deprecated (e.g. I can imagine splitting up IncoherentInstances = IncoherentInstanceDeclarations + IncoherentClassRoles + IncoherentSolving with IncoherentInstanceDeclarations being Deprecated).

I would have expected all of them to be Experimental at first. That way if the way IncoherentInstances was cut needed to change then it is less problematic as the entire set was Experimental. To be more concrete, if it was realized after being "in the wild" that it is better defined as IncoherentInstances = IncoherentDeclarationsAndClassRoles + IncoherentSolving or maybe IncoherentInstances = IncoherentInstanceDeclarations + IncoherentClassRolesAndSolving then the only removals/changes would have been made to Experimental extensions.

That said, I'm fine with moving this to the "discretionary" category.

[adamgundry]

A small point, but what about classifying "negative" extensions? For example FieldSelectors was effectively Stable when introduced (whereas NoFieldSelectors was definitely Experimental).

I guess the simplest thing would be to say that we classify both the enabled and disabled states, but the normal case is for an extension to be Stable when disabled (with NoFieldSelectors being one of a small number of exceptions).

[adamgundry]

One other thought: what should the existing -Wdeprecated-flags warning do once the classification is established? At the moment a handful of extensions are reported by -Wdeprecated-flags, including some that will likely be Deprecated (e.g. OverlappingInstances) but also some that will likely be Legacy (e.g. DatatypeContexts).

We could say that -Wdeprecated-flags warns only for Deprecated and not Legacy (so it will cease warning for DatatypeContexts), or that it will warn for both (so it will start warning for e.g. DeepSubsumption). Neither feels ideal but it would seem a shame if we need to have a separate notion of whether each extension triggers -Wdeprecated-flags.

[telser]

One other thought: what should the existing -Wdeprecated-flags warning do once the classification is established? At the moment a handful of extensions are reported by -Wdeprecated-flags, including some that will likely be Deprecated (e.g. OverlappingInstances) but also some that will likely be Legacy (e.g. DatatypeContexts).

We could say that -Wdeprecated-flags warns only for Deprecated and not Legacy (so it will cease warning for DatatypeContexts), or that it will warn for both (so it will start warning for e.g. DeepSubsumption). Neither feels ideal but it would seem a shame if we need to have a separate notion of whether each extension triggers -Wdeprecated-flags.

I would be in support of a flag for legacy in addition to deprecated, but I also think that is out of the scope of this proposal.