ExtraCommas (was: Trailing and leading commas in sub-export lists)

[parsonsmatt]

Rendered.

[andrewthad]

The code block are currently really messed up, making the proposal difficult to read.

[parsonsmatt]

Thanks for pointing that out 😄

[andrewthad]

I'm +1 on this proposal, although it probably should require a language pragma. One thing this would help me with is that I'll often have something like this:

module Foo
  ( -- * Types
    Foo
  , Bar
    -- * Functions
  , mkFoo
  ) where

If I decide that I want to flip the order of Foo and Bar in the export list, I have to fiddle with the commas. The proposal would help ease this (incredibly minor) inconvenience.

[isovector]

Does this affect constraint tuples and lists as well? I'd prefer for the same comma semantics across the board if possible.

[andrewthad]

@isovector I'm not totally sure about this, but constraint tuples might be totally out of the question. Supporting normal tuples, at both the type and the value level, would be bad, since in the presence of the TupleSections extension, it would make partially applied tuples ambiguous. I'm not totally sure about this, but I think that constraint tuples, defined in GHC.Classes, are the same thing as type-level tuples, which would make them extending this extension to them problematic because it would required that it be extended to type-level tuples as well.

[parsonsmatt]

@hvr noted that this has been discussed previously in the Phabricator ticket. I'm going to copy what he posted there:

So, basically we already had more or less agreed upon (at least that was my perception) a plan around 2014 (the discussions on the topic are however spread a bit; you can find bits in 2013 as well as in 2016 again on different mailing lists, and now again in 2017 obviously) on which Alexander B was working on back then.

https://mail.haskell.org/pipermail/ghc-devs/2014-September/006410.html

There were a few minor bikeshedding details, but other than that the idea was to

Introduce a language extension {-# LANGUAGE ExtraCommas #-} which you can probably google for (I'm a strong proponent of that myself for several reasons; that's my personal basic requirement for any syntax extension relative to the Haskell Report), which would relax/extend the grammar to allow comma-separators in more positions; this would include

• record-like occurences (declarations, patterns, constructions, etc)
• module export lists
• module import lists
• lists literals (i.e. [], both type-level & term-level)
• fixity declarations
• comma-separated enumerations in type-signatures (e.g. (Monad m, Monoid m) => ...)
• (maybe) pattern guards
• //...maybe one or two more places that are eluding me right now....//

But not

• language pragmas (due to catch-22)
• Tuples (due to conflict w/ TupleSections)

Personally, I was (or at least I'd be now) a proponent for the smallest extension that allows to support both styles people were asking for (but without allowing mixing them in the same enumeration), i.e. allow

• [a,b,c]
• [,a,b,c]
• [a,b,c,]

But do not allow

• [,a,b,c,]
• [,]
• [,,]
• [a,,b]
• and so on

For completeness, I'd point out that there's also an OptionalCommas extension possible, which would make commas optional where the parsing would remain unambiguous.

In any case, I'm really looking forward to ExtraCommas getting implemented, as I keep toying with implementing it myself every now and then, especially every time I run into this problem with the [] and {} enumerations... import/export lists too, but that bothers me less than the [] and {} situations for some reason.

@isovector It would appear that allowing this for constraints was talked about favorably.

[mgsloan]

Agreed with @isovector , nearly all usages of commas and semicolons should be consistent with this decision. The reasoning about CPP here is good, but extends equally well to other delimited sequences in Haskell. Beyond just CPP, it is extremely convenient for code manipulation. Things can be written such that they can be copied / reordered without corner cases for the starts or ends. This is pretty common practice in the JavaScript codebases I've worked on (I know, perhaps emulating JavaScript is not the best way to argue in favor of this, but this is really convenient in practice).

It's curious to me that, in a way, we already have this for case statements, do-notation (without curlies), value / type applications, pattern constructor arguments, etc, because they are delimited by whitespace, and it is allowed to be trailing and leading. So, I think it would be uniform for all sequence like things to allow trailing and leading delimiters

Tuples can be the rare exception, due to TupleSections. It would be way too confusing to have these two extensions give different meanings to the same syntax. I think this is fine, I don't think would find much use there anyway. If you are CPP-ing your tuples, or planning on your tuples being frequently extended or re-ordered, you probably shouldn't be using tuples.

So, in summary I totally agree with the proposal quoted above by @parsonsmatt . At once, it'd be a shame if trying to do the big change meant that nothing gets merged here. Perhaps we could just try it out in the context of extra commas in exports? Would rather avoid the proliferation of extensions, though..

[mohsen3]

With {-# LANGUAGE TupleSections #-} enabled, (,1,) is the same as \x y -> (x, 1, y). Why [,1,] should have a totally different meaning rather than \x y -> [x, 1, y]? I personally prefer a more consistent syntax rather than a convenient one.

[parsonsmatt]

With {-# LANGUAGE TupleSections #-} enabled, (,1,) is the same as \x y -> (x, 1, y). Why [,1,] should have a totally different meaning rather than \x y -> [x, 1, y]? I personally prefer a more consistent syntax rather than a convenient one.

This is a great point. The proposal, as it stands, is only concerned with (at least) export sublists. Including import sublists (for trailing/leading commas) and import/export lists (for leading commas) seems to have high support, and would make the language both more convenient and more consistent (given that import/export lists already support trailing commas).

Adding leading/trailing comma support to other list forms is not part of this specific proposal.

[hvr]

import/export lists (for leading commas) seems to have high support

in that case, let me state that I won't support this proposal unless it also covers the rest of the ExtraCommas extension (not the least so we don't end up with lots of super granular ExtraCommasExportsImport, ExtraCommasLists, ExtraCommasConstraints, ExtraCommasFixityDecls etc language pragmas) which as far as I'm concerned was already considered greenlighted back in 2014, so the only thing missing from my POV is an ExtraCommas patch we can review/discuss/fine-tune ;-)

Btw, if it's only TupleSections that are in the way, maybe we should consider deprecating TupleSections ... ;-)

[nomeata]

Btw, if it's only TupleSections that are in the way, maybe we should consider deprecating TupleSections ... ;-)

There might not be precedence, but we could discuss whether we allow incompatible language extensions. After all, they are opt-in and for experimentation, so I giving the users an exclusive choice between ExtraCommas (which then apply consistently and pervasively) and TupleSections might not be too bad.

I am, however, concerned about (,) as the non-section prefix-form of the tuple constructor.

[mgsloan]

Hmm, making TupleSections exclusive with ExtraCommas seems fairly reasonable. There is at least some precedent for choice of extension changing the meaning of code. For example, DeriveAnyClass vs GeneralizedNewtypeDeriving gives different meaning to deriving clauses (btw in practice this is really quite annoying and I often need to remove DeriveAnyClass). I'd prefer avoiding it, because now I need to know which one is enabled when looking at a fragment of code. Say I jump into the middle of a module, now I might need to scroll to the top or open the cabal file to figure out what it really means.

I am, however, concerned about (,) as the non-section prefix-form of the tuple constructor.
 1

I think it's fine to allow special interpretations when there are only commas and no elements. It is particularly obvious that these are different if there is a rule that you can either have a leading comma or a trailing comma, but not both. When these are used, it means there is a single comma coming after each element, or a single comma after each element. If there are no elements, then there are no commas.

Maybe I shouldn't paint the bikeshed on this, but how about allowing [,,] to mean (\x y z -> [x, y, z])? Perhaps not with this extension, but would be more consistent.

[gbaz]

TupleSections are super convenient and I'd hate to lose them to get this. I think we should choose to have ExtraCommas apply to everything but tuples and leave tuples extra special.

In my mind this makes perfect sense because an n-tuple is absolutely different than an n+1-tuple or an n-1 tuple, but a list of n elements is effectively the same as a list of n+1 or n-1 elements. (typewise)

[nomeata]

In my mind this makes perfect sense because an n-tuple is absolutely different than an n+1-tuple or an n-1 tuple, but a list of n elements is effectively the same as a list of n+1 or n-1 elements. (typewise)

That is a pretty convincing argument.

[aaronchall]

It would seem that the Haskell room on Stack Overflow supports this proposal as well: https://chat.stackoverflow.com/transcript/message/41311833#41311833

[parsonsmatt]

Ah, I knew there was something lurking in my todo list!

Alright, so based on the conversation here, we have:

• Some support and opposition for the proposal in it's current state
• Strong support for the proposal in an extended state (eg, making the comma rule work in many more places)

I will try to revise this proposal to include the extra cases for this soon.

[nomeata]

I will try to revise this proposal to include the extra cases for this soon.

Marking this proposal as such.

[parsonsmatt]

@hvr I'm curious about your opposition to a leading and trailing comma, eg: [, 1, 2, ]. Would you be willing to expand on that? It seems like a relatively innocuous thing to me, and explicitly forbidding it would require complicating the grammar somewhat.

[parsonsmatt]

@simonpj has one remaining unresolved question:

1. What about repeated commas in the middle of a list?

I think that we should not allow repeated commas in the middle of the list.

[mboes]

@mindreader import and export lists are already implemented, and not gated with any LANGUAGE pragma. The following is legal code today:

module Foo (x,y,) where
import Data.List (intercalate,)

x = intercalate
y = 1

[gridaphobe]

I also find this vote process troubling. Three options, out of many that have been discussed, were mentioned in a comment (and as I said I feel the third is a nonstarter, so arguably two options). Then people started "voting" by GitHub comment. Likely, several people did not want to add to a thread just to cast a vote, or for any number of reasons did not want to participate in this vote at all.

If we're serious about polling the community before making a decision, a better process would have been (a) discussing and agreeing what the options are, and (b) setting up a proper survey, presumably off-site. I'm not necessarily suggesting we do this now, as it would protract this issue even further, but rather I'm expressing reservations about how meaningful this process was, and whether we definitively "have a decision". Indeed, it seems the conversation is ongoing, with alternative ideas still being discussed.

To be clear about the vote, it was only supposed to include committee members, and the decision was based only on the votes of the committee members.

I suggested the committee vote on the matter because discussion had stalled repeatedly, and it seemed like everyone’s positions were known. All that was left to do was make a decision. I proposed three options based on the committee discussion (primarily on the mailing list, though we switched to the new github model in the middle).

That said, I think it’s clear that we handled the situation poorly. As several members of the community pointed out, the vote was taking place during the committee discussion period (during which we ask the community to refrain from commenting), and so the broader community should not have participated in the vote. More importantly, the members who pointed this out felt it would be inappropriate to participate themselves for the same reason. This makes it impossible to draw any meaningful conclusions from the community votes, and we should have just locked the thread for the duration of the vote (as @nomeata suggested later).

I am sorry about the confusion this has caused, and have opened #244 to try to improve the process in the future.

[simonpj]

Above I asked

I know I should have said this earlier, but I'd really misunderstood, and now I undersatnd better. But here it is: why don't we accept Matt's original proposal, that is allow extra commas in import and export lists (only). That's where the shoe is pinching

@mboes and @galenhuntington wanted list and record values too.

No one has said that fixity declarations and pattern guards are important to them.

Does anyone else have an opinion? Matt: would that address your main goals?

I ask because

• when a proposal is controversial, my instinct is to err towards the status quo.
• we already allow embedded and trailing extra-commas in import and export lists, so it seems uncontroversial to extend it to (a) allow leading extra commas and (b) include sub-export lists

There would be an advantage to having a rapid and uncontroversial "yes", compared to allowing this to drag on.

[parsonsmatt]

I don't mean to be hostile or unfriendly in this, but my patience with the process has worn pretty thin. I don't intend on pursuing any aspect of this proposal any further. If someone wants to carry the torch, then please fork the repository and make a new PR. I intend on closing this PR within the week and deleting my branch.

[goldfirere]

Whoa! Why close this? @nomeata writes (in #87 (comment)):

So hallelujah, we have a decision: ExtraCommas is accepted in the variant that does not cover tuples.

@bitemyapp, as the shepherd, would you mind updating the proposal text accordingly and let me know so that I can merge it?

It's true that some debate continued after that point, but I don't see a reason to close the PR -- even if its author wants to move on. (@parsonsmatt, thanks for your patience thus far. You indeed may walk without seeming hostile or unfriendly. This proposal hit the process's squishy points.) Maybe we decide as a committee to just drop the proposal, but it seems premature just to close is, especially after both the committee and the community voted to use this proposal to justify a change in GHC.

It does seem like we need a new champion, as Matt is stepping down. I agree that if we can't find one, that suggests we should just close it. I just don't think we're quite there yet.

(Of course, I can't reopen because the branch is gone.)

[nomeata]

I can help with the git foo to recover the branch if a new champion would arise (essentially git fetch origin refs/pull/87/head).

I would have thought that Chris (as the shepherd) would simply update the proposal to reflect the outcome of the vote and then we can simply merge it.

[jkachmar]

I’ve been following this from afar for the nearly two years that it’s been gestating, so in light of recent events I’d like to share my thoughts on the whole situation.

During the time that y’all have discussed this issue I grew from a hobbyist Haskeller to a commercial user and advocate of the language; however the unprofessional manner in which this process has played out has made me seriously question whether it’s worth investing my time into Haskell in the way that Matt has attempted to here.

---

This was supposed to be an extremely minor syntactic modification that would bring Haskell in line with many other programming languages with respect to trailing commas.

Instead of being handled quickly, it’s been the subject of pointless arguments that are wildly disproportionate to the complexity of the proposed changes for such a drawn out period of time that Matt seems to have quite simply burned out.

---

This isn’t the type of environment that’s inclusive to new contributors, which makes me question the manner in which the language will be capable of growing relative to my needs.

Further, it creates an environment of expected mental fatigue such that I find myself much less likely to consider a proposal of my own in the future.

Please consider the way in which the handling of proposals like this reflects upon not just the committee, but the Haskell overall community in the future.

[goldfirere]

Thanks for your thoughts @jkachmar! It inspired me to re-scan the comment thread here in light of your experience. A few reactions:

• As I've admitted above, this proposal has hit our squishy points. No proposal should languish like this.

• That said, I'm not sure what, concretely, we could have done differently. Despite seeming straightforward, various members of the community had differing opinions on the details. Do we (as a committee) cut off conversation and just decide? But when to do that? And who decides what proposals need to be cut off?

• Maybe the majority of Haskellers just want exactly the original proposal (which got a lot of +1s) and didn't feel the need to make their voices heard again? My sense is not this, but it's very hard to know.

• I'm very pleased with at least one aspect of this conversation: the debate has been, at all times (I believe) professional. I did not see insults flying here, and the debate has remained technical.

Really, the only thing I can see that has gone wrong here is the length of time the proposal has been under debate -- which is highly regrettable.

I'd be very curious to know if you (@jkachmar or anyone else) see this differently.

This suggests a possible way to improve our process: set an expiration date on proposals. (I propose this to be 6 months, given that we are but volunteers.) Deadlines spur action. If a proposal is about to expire (or has recently expired), then it is up to the committee to decide what happens next, and quickly. The time for public debate will have ended. We just decide and move on.

Would something like this seem to improve your perceptions, @jkachmar?

[dwijnand]

(Your mention of expiration dates on language proposals reminds me of yesterday's http://smallcultfollowing.com/babysteps/blog/2019/07/10/aic-unbounded-queues-and-lang-design/ which is exploring how to improve Rust's language design process. Apologies if this is too far off-topic.)

[goldfirere]

Many thanks @dwijnand for bringing in that very relevant post. It's a bit of a different proposal than what I brought up, but it's good to think broadly about potential changes to our process.

One interesting note buried in there: it seems that Rust's lang team rewrites accepted proposals to have a tighter specification. That's an interesting idea. Some of the proposals in our process have languished for want of a tighter specification, and perhaps the proposers don't have the experience / knowledge base to do that well. Our resources are stretched, so I don't know about having the committee rewrite proposals, but it's an interesting alternative to consider.

[int-index]

I have realized that there's an alternative which has not been covered in the discussion of this proposal and would work for tuples, lists, records, export/import lists, and very much any other enumeration. No unicode syntax.

Add an extension, -XLayoutEnumerations, to allow the following:

[
  item1
  item2
  multiline
    item
  item3
]

With this extension, when an opening bracket is immediately followed by a newline, we consider this list (or tuple, or record) to be layout-based rather than comma- or semicolon separated. More examples:

-- a tuple
(
  foo
  bar
)

-- a record
{
  x = 5
  y = 10
}

[nomeata]

There are various layout-based proposals in this thread (you have to click hard to find them, the thread is too long), including markdown-style bullets, or an explicit keyword herald (these). Implicit layout is pretty nice too.

But we should find a different venue than this thread. Maybe open an issue to brainstorm them? Or somewhere else (on the Haskell or GHC wiki) where we can explore the design space? (I agree that separators are a bad idea and layout is great)

[nomeata]

Ugh, I just wrote something on the GHC wiki, but when I submit I get an error 500. Luckily I rescued my text from the Firefox console… Ah, I have to login first. Anyways, lets move the discussion to https://gitlab.haskell.org/ghc/ghc/wikis/All-things-layout, feel free to add more sections with variants, and maybe a comment section.

[nomeata]

As Jakob points out in #317 this ExtraCommas PR is in a weird state: It was accepted (with minor changes needed), but the author lost interest (jusitifably so!) and the shepherd (@bitemyapp) closed the PR.

Maybe we should just call our loses and drop this, as Richard suggests in #87 (comment), removing the Accepted label to reflect the reality that nobody wants to touch this anymore?

[neongreen]

+1 for labels reflecting reality. If this is explicitly marked as abandoned, somebody else might pick it up. If it's in limbo, nobody won't.

[aaronchall]

Coming from Python, I have been a big proponent of trailing commas.

I certainly do see the point that Haskell is a different language, and many languages just aren't going to get trailing commas. I also really like the no-comma proposals (I do like lisp for this as well. Maybe I should pick up clojure...)))))))