GHC's base libraries: Combining stability with innovation #51
Conversation
Update 050-ghc-base-libraries.rst
I've been updated that security notices tool did not actually do the algorithm I thought it did, so my claim that actually we're most the way there to this is invalid.
Emphasize concrete goals of individual teams vs shared values.
|
|
||
| - ``cabal check`` (a per-package check) could warn on packages that use ``ghc-internals``. | ||
|
|
||
| - ...what else? |
There was a problem hiding this comment.
We should use all possible social and technical means to discourage people from depending directly on
ghc-internals
Really all? Including GHC/Cabal simply refusing to allow packages called ghc-internals if the package currently built isn't base or another internal whitelisted package?
But it says “discourage”, not “prevent”, which sounds reasonable.
Maybe more realistic: hackage could reject uploads of packages that depend on ghc-internals?
There was a problem hiding this comment.
But it says “discourage”, not “prevent”, which sounds reasonable.
Yes that is key. If people want to tinker with the internals of some open source software, we should not stop them. But we should make sure they are very aware they are stepping outside the stable safe zone. And likewise we ought to someday make anyone that uses the tinker's software (transitively!) also aware they are exposed to instability.
There was a problem hiding this comment.
(The difference between ghc-prim and ghc-internal also isn't clear. But one answer is
GHC devs can structure (and restructure) their internals however they like, including dividing definitions arbitrarily between
ghc-primandghc-internalsif they so desire, so this question doesn't need to be answered in this document.
Another answer is
ghc-primis (mostly, perhaps should be entirely) auto-generated docs for code that doesn't actually exist in textual form (it's just hard-coded in GHC), whereasghc-internalsis human-written code.
)
|
Dear CLC members, following our earlier private discussion, I understand that we are in agreement with the proposal in principle. This PR is a good opportunity to polish wording, clarify statements and generally take one more final look. However, let's not scope creep, unless absolutely necessary. @tomjaguarpaw @chshersh @hasufell @mixphix @parsonsmatt @angerman |
There was a problem hiding this comment.
Just a few minor suggestions, and one point worth noting: "all possible technical and social means to discourage" means not actively encouraging something ourselves. Say something and stick to it!
I rather like @nomeata's suggestion that Hackage reject packages with explicit ghc-internals dependencies, it seems like the right abstraction layer to do so: if someone really, really wants to depend on it, they can still do so privately.
|
|
||
| This is going to disappear from base. | ||
| You probably don't want to use it at all. | ||
| But if you absolutely must, get it from ``ghc-internals``. |
There was a problem hiding this comment.
We can't say in the same document that we should use "all means necessary" to avoid ghc-internals dependencies, and also recommend people to import a function from there.
| But if you absolutely must, get it from ``ghc-internals``. | |
| If you absolutely must, define it yourself locally, or if this is impossible, let the CLC and GHC team know! |
There was a problem hiding this comment.
I would expect ghc-internals to mainly contain things one cannot define locally? I would expect also some zany integrate-with-ghc-so-tightly-someone-turns-blue projects to always need to use ghc-internals too. I think it's less important to deny that category exists, than emphasize that the vast majority of projects are not in that category.
Put another way, I'd expect those projects to know who they are and not reach out to GHC devs or the CLC, and I'd expect almost everyone who does reach out to be told "actually, you don't need to do things that way". Is that what you were thinking too? Or were you imagining we might sometimes move something to ghc-internals (without a reexport elsewhere) that shouldn't be there, by mistake?
There was a problem hiding this comment.
If such projects are relying on functionality from base that GHC developers and the CLC both agree should be moved to ghc-internals, they ought to reach out and say "hey, we use this for such-and-such", and make a case for it to be put in ghc-prim instead.
The point here is to make, FAIAP, a "hidden" library containing code in the strictest intersection of "compiler bootstrap" and "necessary to allow general-purpose programming": stuff like the guts of IO, datatype representations, and the like. There should be nothing in ghc-internals that is both:
- useful for general programming, and
- not exported from
base,ghc-prim, orghc-experimental.
There was a problem hiding this comment.
I agree with the main "There should be nothing in ghc-internals that is both: useful for general programming, and not exported from..." point, but do not that I don't think anyone intends ghc-prim to be any more user-facing than ghc-internals. Perhaps the proposal should make this more explicit?
There was a problem hiding this comment.
not exported from base, ghc-prim, or ghc-experimental.
This should be "not exported fomr base or ghc-experimental". ghc-prim is every bit "internal" as ghc-internals. Anything in ghc-prim that is useful for general programming should be be exposed by base or ghc-experimenal. Indeed maybe we should make that clearer.
I disagree. ghc-internals is internal API, but properly PVP versioned. Power users writing high performance code or alternative Preludes may have very legitimate use cases of importing it. Rejecting those packages on hackage is wrong. |
That sounds very reasonable, but very different from “discourage by all means possible”, and probably deserves clarification. Which one is it? |
"Discourage" is the correct wording, it does not imply "prohibit". |
|
👍 This seems like a step in the right direction. I hope that this is the first step of decoupling GHC and GHC releases from library upgrades rippling through the whole ecosystem. 🤞 |
There was a problem hiding this comment.
I will leave a few comments:
- The title is: "GHC's base libraries: Combining stability with innovation". Yet, it doesn't define any of those precisely. It lists
base,ghc-primsomewhere in the documents. Thenghc-internals,ghc-bignum. We may want to call out what exactly we understand by "base libraries". - What is understood by "stability"?
- What is understood by "innovation"?
The two goals are in risk of conflict, and we have seen such conflict in the past.
Feels like it unnecessarily pits the CLC and GHC team against each other. As someone who has contributed to GHC, and had whose primary interest is stability, and proper system integration, it feels unnecessarily discouraging. We do want people to contribute to GHC, that care about stability.
I strongly disagree that we want to have a CLC (stability stalwarts) vs. GHC (innovators) view set out here. GHC needs to welcome people who care about stability and backwards compatibility and the CLC also needs to welcome advocates for innovation.
This is why I had proposed a different wording that doesn't focus on exact goals, but on the autonomy of GHC and CLC, which makes sense for its own sake:
We want CLC and GHC to be able to exercise different goals (e.g. stability) and workflows (release management/versioning), but be sure that CLC's decision-making is not compromised, e.g. by changes that indirectly affect 'base'. We also want to be efficient and avoid proposals that are not strictly relevant to proper 'base' API. @simonpj was unhappy with "including its dependencies", which I still don't understand, because it's absolutely true. So let me propose this alternative wording again. Let's not focus on exact goals, because they are rather hard to express when two heterogeneous groups are involved. Let's focus on autonomy and how collaboration is supposed to work. |
|
A quick comment on the question of hiding A caveat: for now, the "big honking warnings" should just go in the Haddocks, to decouple this effort from any hypothetical tooling changes. |
If anyone would like to offer to some concrete text, that would be great. We could have a section at the end on "possible workflows". I'm listening to @Bodigrim who counsels against scope creeep. But a clearly-seprarated discussion/illustration section, that doesn't form part of the proposal but has dicussion about how it might work in practice, would be fine I think. It just needs someone to write it! |
No, it is not.
CLC doesn't necessarily care about it at all. But it's a possible mechanism for GHC to:
Proposals may still be rejected. We have full autonomy. GHC SC is welcome (and encouraged) to source CLC's opinion before accepting GHC proposals that possibly touch base, but it's not a requirement. But if you don't, you can't complain later that your proposal as a whole failed due to CLC rejecting it. |
|
Meta question: Is there a picture anywhere that explains the different GHC "groups" and who/what they care about? Feels like someone could do a great service to the community just by making that picture ... (it would certainly help me!) |
When you say "groups" what do you mean? Do you mean bodies like the core libraries committee, GHC steering committee, etc.? |
Yes; it seems to me a lot of time in this thread is being spent on the various domains/interests/roles not being clear? (Could just be my misunderstanding.) |
Committees are documented here: https://www.haskell.org/community/ Efforts to document more of the hierarchy are discussed here: haskell-infra/www.haskell.org#60 Please comment there. |
No; that's not a complete list; but thanks, I didn't know about that other issue. |
Could you please open a new issue on the haskell.org repo explaining which are missing? I'll make sure they get added. |
Ericson2314
force-pushed
the
ghc-base-libraries
branch
from
466af8b
to
54a55a3
Compare
|
I'm happy to announce that the proposal has been approved unanimously by CLC. |
|
Thank you Andrew!
Simon
…On Tue, 11 Jul 2023 at 22:45, ˌbodʲɪˈɡrʲim ***@***.***> wrote:
I'm happy to announce that the proposal
<c157f56>
has been approved unanimously
<https://groups.google.com/g/haskell-core-libraries/c/10SyhA9Lm98/m/Zimb2ZZUAAAJ>
by CLC.
—
Reply to this email directly, view it on GitHub
<#51 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEOY62ZSZ6RZ4LTLPSKSOLXPXCQ5ANCNFSM6AAAAAAZUUZN5E>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Sorry to nitpick, but I don't see the vote from @mixphix on the mailing list. Did they reply privately? |
|
Oh, my apologies, I did indeed reply privately by mistake. |
|
I think it is time to merge @Ericson2314 or does the HF want to do anything before that @david-christiansen ? |
I agree. Relatedly, once we have done so, I'd like to start a conversation about naming conventions for modules. See this thread. Where would be a good place to have that conversation? As an issue (or a proposal) on this HF github? On the GHC github? Where? |
|
For the sake of formalities, TWG will have a vote before merging, but I'm pretty sure it won't be controversial. As far as that other conversation goes, I have some ideas - I'll get back to you soon. |
|
The TWG agrees that this proposal has been approved by the relevant parties and is useful - we unanimously voted to approve. |
|
Thank you all for this process. I think it led to a good outcome! |
This document describes an plan agreed between the GHC Team and the Core Libraries Committee, saying how we plan to work in partnership to reconcile the goals of innovation and stability for GHC and its ecosystem.
Formally, then, it is not so much a proposal as a record of an outcome. We are, nevertheless, using the Haskell Foundation Technical Working Group proposals repo, so that we can have a permanent public record of how we plan to work, so that others can comment, and so that the document can be polished to add clarity where necessary.
Rendered