Merge pull request #625 from ghc-proposals/wip/general-rules

README.rst

@@ -46,8 +46,8 @@ Should the GHC maintainers deem a change significant or controversial enough to
 warrant that, they may, at their discretion, involve the committee and ask the
 contributor to write a formal proposal.
 
-Proposals may contain amendments to our principles_, which serve as a guideline
-for future proposals and for the general evolution of GHC.
+Proposals are evaluated against our principles_, which cover both language *design*
+and language *stability*
 
 .. _principles: principles.rst
 
@@ -419,6 +419,8 @@ Review criteria
 ---------------
 Here are some characteristics that a good proposal should have.
 
+* *It should follow our design principles*.  These principles_ cover both the language *design* and its *stability* over time.
+
 * *It should be self-standing*.  Some proposals accumulate a long and interesting discussion
   thread, but in ten years' time all that will be gone (except for the most assiduous readers).
   Before acceptance, therefore, the proposal should be edited to reflect the fruits of

principles.rst

@@ -17,6 +17,12 @@ Note that neither direction is (at all) a guarantee: there will be excpetions
 in both directions. Yet by writing down the principles, we can have an informed
 discussion of the tradeoffs of accepted or rejecting an individual proposal.
 
+These priniciples are divided into two section:
+
+* `Language design principles <#2language-design-principles>`_ (Section 2)
+* `GHC stability principles <#3GHC-stability-principles>`_ (Section 3)
+
+
 How to update these principles
 ------------------------------
 
@@ -42,7 +48,7 @@ given the multitude of principles in play. Including the abbreviation is useful
 the abbreviation is used in other fora (e.g. email, GitHub, etc.) to allow the curious
 to find (e.g. with Ctrl+F) the abbreviation within this page.
 
-Accepted principles
+Language design principles
 -------------------
 
 .. _`#281`: proposals/0281-visible-forall.rst
@@ -303,3 +309,154 @@ down GHC even when those features are not active. Yet this principle is importan
 as we hope not to make the current situation worse.
 
 From `#378`_, slightly generalized.
+
+
+
+GHC stability principles
+--------------------------
+
+The Haskell ecosystem has a built-in tension between stability and innovation.
+
+One the one hand, breaking changes impose heavy costs:
+
+* Users keep having to update their code
+* Library authors keep having to update their code
+* Updating to a later version of a library can in turn force taking on new dependencies.
+
+These difficulties add friction, discourage adoption, and make Haskell an unpredictable investment.
+
+On the other hand, GHC has always sought to be a laboratory for innovation.  If the stability guarantees are too onerous, we risk imposing a different set of costs:
+
+* Volunteers may get discouraged by the hoops they have to jump through to get a change agreed.
+* The language may become stuck in a local optimum, because moving to a better design would require breaking changes.
+* We want Haskell to be a beautiful, elegant language, not one riddled with inconsistencies, grandfathered in simply because fixing the inconsistency would risk some breakage.
+
+We can't navigate this tension with simple all-or-nothing rules.  We have to take each case on its merits, aware of both sets of costs.  This section describes some general rules, or principles, that we use to guide our stability-related discussions for GHC itself.  See also `GHC base library proposal <https://github.com/haskellfoundation/tech-proposals/blob/main/proposals/accepted/051-ghc-base-libraries.rst>`_, which focuses on the stability of the ``base`` library API.
+
+
+Assumptions
+~~~~~~~~~~~~
+
+We assume that we have identified some extensions as "Experimental".  These extensions may well change, and are subject to much weaker stability goals.  `GHC Proposal #635 <https://github.com/ghc-proposals/ghc-proposals/pull/635>`_ is devoted to agreeing this list.
+
+Terminology
+~~~~~~~~~~~~~~~
+
+We define a **stable Haskell package** as follows. A stable Haskell package
+
+* Explicitly specifies a language edition (``Haskell98``, ``GHC2021``), in the source code or the build configuration.
+* Does not use Experimental extensions.
+* Does not use experimental features.  (Examples: use of the Javascript or Wasm back end, builds on non-tier-1 platforms.)
+* Does not rely on explicitly-undefined behaviour. (Example: ``INCOHERENT`` instance selection.)
+* Does not use ``-Werror`` in its default build configuration.
+* Does not use ``-dxxx`` debug flags in its default build configuration.
+
+Stability (GR1)
+~~~~~~~~~~~~~~~~
+
+Our over-arching stability goal is: if it works with GHC(X), it should work with GHC(X+1).  More precisely:
+
+**General rule (GR1)**.  *A stable Haskell package P that works with GHC(X) should continue to work in subsequent releases of GHC, say GHC(X+1),
+provided that, for each of P's direct dependencies D,
+(a) D works with GHC(X+1), possibly after an update to D, and (b) the API of the bits of D that P uses is unchanged by the update to D.*
+
+In this context the term "API" should be taken to include types, semantics, and performance characteristics.
+
+Consider building one of P's dependencies, say D, with GHC(X+1):
+
+* D may work unchanged with GHC(X+1).
+* D may depend on (say) ``ghc-internals``, and require some small change to accomodate the change to ``ghc-internals``, but one that does not require changing D's API; so D will have a minor version bump.
+* D may depend on (say) ``ghc-internals``, those changes may force a change to D's API; so D will have a major version bump.
+
+In all three cases, **provided the bits of D's API that P uses are unchanged**, compiling P with GHC(X+1) should work.
+
+Notes and clarifications:
+
+* The general goal of "works" includes both "compiles successfully" and "runs successfully, as fast as it did before".  That is a very demanding goal, so it has the status of a highly-sought-after aspiration rather than an unbreakable promise.
+
+* *Dependencies*.   If a package P depends directly on a package Q that suffers a major version bump to work with the new GHC, then (of course) P may not work with the new Q. A particular case in point is where P depends directly on packages that are tightly coupled to GHC:
+
+  * GHC-internal packages (e.g. ``ghc-internal``, ``ghc-prim``, ``ghc-bignum``) may change API without notice, and may have a major version bump even with minor releases of GHC.
+  * The ``ghc`` package currently has a huge API that changes with each minor release of GHC. (There is a separate project to define a more stable GHC API, but that is out of scope for this document.)
+  * The ``ghc-experimental`` package will typically expose more functions with each release; and existing functions may change.  It will usually have a major version bump with major releases of GHC, but only a minor version bump with minor releases.
+  * The ``template-haskell`` package contains syntax-tree datatypes which need to be changed as the language evolves.  Again, we expect onl a minor version bump with minor release of GHC.
+  * Reminder: the package versions of all these GHC packages follow the PVP -- see the `GHC base library proposal <https://github.com/haskellfoundation/tech-proposals/blob/main/proposals/accepted/051-ghc-base-libraries.rst#3things-we-all-agree-about>`_.)
+
+* *The base library* is under the careful management of the Core Libraries Committee. Its API grows slowly, usually requiring a major version bump with each major GHC release.  Since almost every package depends directly on ``base``, it would be highly desirable for a new GHC to be released with version(s) of ``base`` that expose earlier ``base`` APIs (requiring a minor bump only) -- the so-called "reinstallable ``base``" goal.  That is, *``base`` should ideally behave like any other package*.  This is a work in progress.
+
+* *Language editions*.
+
+  * A stable package should specify an explicit language edition because subsequent releases of GHC might change the default language edition.  So if the package does not pin a specific language edition, it might then fail when compiled with a later release.
+  * Language editions like ``GHC202x`` should use only stable extensions.
+
+* *Experimental*.  Rule (GR1) applies only to the stable (non-experimental) parts of GHC. The intent is to allow scope for experimentation, while still allowing users to stick to the stable parts of GHC subject to (GR1).
+
+  * We will need to enumerate the "experimental extensions" and "experimental features" mentioned above.
+  * As mentioned above, the ``ghc-experimental`` and GHC-internal libraries are likely to have a major version bump (API change) with every major release, so any package that depends directly on these will not be subect to (GR1).
+
+* *Stability of warnings*. There is no stability guarantee that a later GHC will emit the same warnings as an earlier GHC. In particular, rule (GR1) does *not* guarantee that if a package compiles warning-free with one version of GHC, it will still be warning-free with a later version. In general, warnings should not be regarded as stable.
+
+  A notable case in point is deprecations, where a later GHC may advise authors to (say) import a function from a different module; while an earlier GHC obviously will not.  In general, GHC should, by default, warn about upcoming changes so that users can adapt their code at leisure; see (GR3) below.
+
+* ``-Werror``
+
+  * ``-Werror`` is excluded from the definition of a stable package, because otherwise (GR1) would be broken whenever GHC adds new a warning.
+  * Similarly ``-Werror=wombat`` is excluded from the definition of a stable package, so that a later GHC make ``-Wwombat`` warn in more cases without breaking (GR1).
+  * It's fine for a stable package to use ``-Werror`` in a CI build, to help the author find warnings.  But not in the default configuration, used by others when installing the package.
+
+* *Debug flags*. The behaviour of debug flags ``-dxxx`` (e.g. ``-ddump-simpl``) may vary without warning.  They are for debugging!
+
+* Notice that
+
+  * For new language extensions, (GR1) is trivially satisfied if the change is gated behind an extension flag.
+  * (GR1) is certainly broken if, say, we turn a warning into an error.
+
+
+Exceptions (GR2)
+~~~~~~~~~~~~~~~~
+
+**General rule (GR2)**.   *We may break (GR1), but only with a compelling reason, and with careful consideration of impact.*
+
+Compelling reasons include
+
+* To fix a security loophole.
+* To fix an outright bug in GHC.  It is possible that some code might accidentally rely on that bug; but we can't use that as a reason to grandfather the bug indefinitely.  (Imagine that 2+77 = 80.)  There is a judgement call here, about what a "bug" is.
+* To fix a design flaw in the language. For this, a better path is usually to fix the design flaw with a new language extension, embodied in a new language edition.  Only exceptionally would we fix a design flaw in a way that breaks programs compiled with existing language editions.
+
+This list is not exhaustive, but the emphasis is on "compelling reason", bearing in mind the costs that any change imposes on users.
+
+Choices should be informed by the amount of breakage, e.g. by compiling thousands of packages in Hackage.  Changes that break little or nothing need a much less compelling reason than changes that break a lot.  For example, changing the meaning of the ``LambdaCase`` extensions to include ``\cases`` as well as ``\case`` would break a program that used ``cases`` as a lambda-bound variable.  But we judged that this risk was small compared to the bureaucratic overhead of having two extensions.
+
+Choices can occasionally also be influenced by implementation considerations. If, for example, a major (and desirable) refactoring of the type inference engine changes the behavior of some under-specified edge cases (of which GHC's type system has plenty), it would be unreasonable to attempt to keep the old and the new behaviour.
+
+Changes, especially significant changes, should be introduced gradually: the subject of (GR3).
+
+
+Warning of upcoming changes (GR3)
+~~~~~~~~~~~~~~~~~~~~
+
+**General rule (GR3)**.  *If we break (GR1), for a compelling reason (GR2), we should whenever possible provide a deprecation cycle, as well as appropriate publicity, so that users are alerted (via warnings) of the upcoming change, have time to adapt, and can raise concerns.*
+
+"Provide a deprecation cycle" means that (if possible) when a change in GHC will break existing code,
+then
+
+* GHC(N) should *warn* of the upcoming change, and only GHC(N+1) should make the breaking change
+* The library author can adapt their code in such a way that it works in *both* GHC(N) *and* GHC(N+1)
+
+This gives at least one GHC major release for authors to make changes.  In the case of changes that are more difficult to accommodate, we may consider a second cycle.
+
+Note that (GR3) is not a reason to say that (GR1) is unimportant.  A deprecation cycle defers costs; it does not reduce or eliminate them. However, deprecation cycles are very important.  They give users and library authors time to adapt to changes, without a mad panic.  In particular, they allow users to upgrade GHC (e.g. to get an important bugfix) in one step, and then separately, and on a working code base, to adapt to each change individually, instead of having to do it all in one atomic step.
+
+Notes:
+
+* It may not be possible to satisfy (GR3).  A notable example is that of module exports.  Suppose (say) in GHC(N) an export ``foo`` is added to a module ``M``.  Then, in a module ``X``, if that new import of ``foo`` from ``M`` import clashes with some existing definition or import of ``foo``, compilation will break.  The code can be adapted in a backward-compatible way (e.g. by hiding ``foo`` or importing ``M`` qualified), but immediate action must be taken to make the ``M`` compile with GHC(N).
+* Even changes to Experimental extensions should seek to follow (GR3), but with a significantly lower "bar".
+* There is a `separate conversation going on <https://github.com/haskell/pvp/issues/58>`_ about deprecation warnings and the PVP.
+* (GR3) is not the same as a **three-release policy**.  GHC does not currently have a three-release policy; that is a `separate debate <https://github.com/ghc-proposals/ghc-proposals/issues/629>`_.
+
+
+Mechanisms for checking stability
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Having crisper definitions opens up the possibility of also providing mechanical support, through which a user can declare their intent to use only stable packages.  This is the subject
+of `GHC Proposal #617 <https://github.com/ghc-proposals/ghc-proposals/pull/617>`_.

proposals/0000-template.rst

@@ -56,6 +56,9 @@ may include,
 * how the proposed change interacts with existing language or compiler
   features, in case that is otherwise ambiguous
 
+Think about how your proposed design accords with our `language design principles <../principles.rst#2Language-design-principles>`_,
+and articulate that alignment explicitly wherever possible.
+
 Strive for *precision*. The ideal specification is described as a
 modification of the `Haskell 2010 report
 <https://www.haskell.org/definition/haskell2010.pdf>`_. Where that is
@@ -139,6 +142,9 @@ drawbacks that cannot be resolved.
 
 Backward Compatibility
 ----------------------
+How well does your proposal meet the stability principles described in our
+`GHC stability principles <../principles.rst#3GHC-stability-principles>`_ document?
+
 Will your proposed change cause any existing programs to change behaviour or
 stop working? Assess the expected impact on existing code on the following scale: