Bringing back Variable Arity Methods

[apblack]

I would like to bring back variable arity methods, with certain restrictions. Here is the proposal, as we discussed it during today's video meeting.

Syntax

1. In a method declaration, the final parameter of a method can be written paramName ... or paramName:ParamType ...

2. Variable arity parameter lists shall be permitted only as the final parameter of a method. So,

method ex (p) why (q...)

is permitted, but

method ex (p...) why (q)

is not permitted.

3. A method declaration m(p...) shall be treated as declaring methods m(_), m(_,_), m(_,_,_), m(_,_,_,_), and so on, for all methods m(...) with any number of parameters greater than zero. It does not declare a method m with zero parameters. This means that m(p...) overrides inherited declarations of m with any non-zero parameter list.

4. If m(p...) is inherited, and m(_,_) declared locally, does the declaration of m(_,_) override that of m(p...) with all parameter list lengths, or just that for parameter lists of length 2? Or, is such a local declaration illegal?

Semantics

When a method with a variable arity parameter list is requested, the variable-arity parameter is bound to a Sequence of objects of its annotated type. So, if the method header has the form

method m(n, o, p ...) 

n is bound to the first argument supplied in the request of m(...), o is bound to the second, and p is bound to all of the remaining arguments. p is treated as having type Sequence⟦Unknown⟧.

If the method declaration has the form

method m(n, o, p:Tp...) { 
    ...
} 

p is bound to all of the arguments supplied in the request of m, except for the first two, which are bound to n and o, p is treated as having type Sequence⟦Tp⟧. This may generate a type error if any of the 3rd, 4th, 5th ... arguments in the request does not conform to Tp.

[KimBruce]

Looks good. See embedded comments.

I would like to bring back variable arity methods, with certain restrictions. Here is the proposal, as we discussed it during today's video meeting. Syntax In a method declaration, the final parameter of a method can be written paramName ... or paramName:ParamType ...

I have a slight preference for including a comma before the dots: paramName, … or paramName: ParamType, … I think that looks more normal as we normally separate parameters by commas. (We don’t allow dots as prefix operators or as parts of method names, right?)

Variable arity parameter lists shall be permitted only as the final parameter of a method. So, method ex (p) why (q...) is permitted, but method ex (p...) why (q) is not permitted. A method declaration m(p...) shall be treated as declaring methods m(_), m(_,_), m(_,_,_), m(_,_,_,_), and so on, for all methods m(...) with any number of parameters greater than zero. It does not declare a method m with zero parameters. This means that m(p...) overrides inherited declarations of m with any non-zero parameter list. If m(p...) is inherited, and m(_,_) declared locally, does the declaration of m(_,_) override that of m(p...) with all parameter list lengths, or just that for parameter lists of length 2? Or, is such a local declaration illegal?

I'd make it illegal. I think it is too confusing to have a generic definition as well as specific definitions that override a finite number of pieces. In fact, if there is a method declaration method m(a, b, …) {…} then I would not allow a direct definition of m(a,b), m(a,b,c), etc. I.e., this is a slight strengthen of our rules agains static overloading. (I believe this is implicit in what you wrote for 3.) I believe this makes sense as including method m(a,b) {…} means there are two conflicting definitions with two arguments. The semantics seems fine. Kim

[apblack]

Writing this stuff down, especially the overriding rules, and the following discussion, is tending to make me want to withdraw my recantation. In other words, I'm thinking that I should just put list, set, sequence and dictionary with 1, 2, 3, ... 9 parameters into the collections library, and leave it at that.

The types are particularly troubling, too. In a method with header

method m(n:Tn, p:Tp...) { 
    ...
} 

parameter n has type Tn, but parameter p does not have type Tp — it has type Sequence⟦Tp⟧. (That's actually the reason for putting the ... immediately after the type annotation — you have to think of the ... as modifying the annotation.

Of course, we don't request the method with an argument of type Sequence⟦Tp⟧ — in fact, it's impossible to do that.

Dave Ungar suggested that the only reason that I’m suggesting this is that in the year or more since we took out variable arity, I've forgotten just how horrible it was. I think that he has a point.

[kjx]

there are two sides here. * requester - an interface based either on varargs or overloading is cleaner than one based on "lineups" the distinction between "foo(1,2,3) and "foo [1,2,3]" goes away we can get rid of lineups (and transition type parameters to single square brackets) there is still a question about whether "foo" and "foo (1)" should be seem as the same or different requests * implementer - having varags is easier for implementers than not. I'm sure Dave likes multiple methods, it's what you do in Self (or a doesNotUnderstand handler, and I don't think he went that far) varargs still needs a built-in implementation for sequences but it is much less obvious (unless you think of the ... annotation on arguments as being a very weird indirect form of sequence literal) the obvious move is: - remove lineups, design the library as as variadic interface - implementations could keep varargs as an extension - if & when we cannot stomach a library with 50 differenve versions of "list(a1),list(a2)" etc then we put varargs in on the definition side. looking at e.g. JS and Python and everything else, the temptations with varargs will be - to make internal varargs list type support a full collections interface. - to introduce a "spread" operator so you can unpack a list in a request to a (non-variadic) method J

[KimBruce]

As I understand it you want to put in methods: list (a1) list (a1, a2) list (a1, a2, a3) … list (a1, …, a9) Are you also intending to leave in the list method that takes a lineup? If you also leave that in, then if you are going to leave in a version that takes a lineup (or equivalent) then you need to give it a different name: listFromCollection(lst) It would seem odd to leave out the generic conversion from lineup or similar, though obviously it could be programmed by the student. I think at this point I still lean a bit more toward varargs, as long as we are restrictive about overloading rules. Kim P.S. I am hoping to be available for our meeting tomorrow afternoon, but I may have to take my wife to west LA to get a new passport (lost, somehow) for her flight to Africa on Wednesday. I’ll let you know if I can’t get back in time.

…

On Jul 29, 2017, at 10:28 AM, Andrew Black ***@***.***> wrote: Writing this stuff down, especially the overriding rules, and the following discussion, is tending to make me want to withdraw my recantation. In other words, I'm thinking that I should just put list, set, sequence and dictionary with 1, 2, 3, ... 9 parameters into the collections library, and leave it at that. The types are particularly troubling, too. In a method with header method m(n:Tn, p:Tp...) { ... } parameter n has type Tn, but parameter p does not have type Tp — it has type Sequence⟦Tp⟧. (That's actually the reason for putting the ... immediately after the type annotation — you have to think of the ... as modifying the annotation. Of course, we don't request the method with an argument of type Sequence⟦Tp⟧ — in fact, it's impossible to do that. Dave Ungar suggested that the only reason that O'm suggesting this is that in the year or more since we took out variable arity, I've forgotten just how horrible it was. I think that he has a point. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#130 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABuh-vbjuz80QoRi4BxZfnAxaW5Dg_nHks5sS2urgaJpZM4OjQuG>.

[kjx]

Are you also intending to leave in the list method that takes a lineup?

whatever happens with varargs, I really hope we can do away with lineups.

If you also leave that in, then if you are going to leave in a version that takes a lineup (or equivalent) then you need to give it a different name: listFromCollection(lst)

the remaining design question is: whether "list" should be a zero-arg vararg, and so return an empty list (that's what our earlier vararg design would have done, although it leads to odd syntax in places) or whether "list" should be a separate zero-arg request (the smalltalk distinction between #list: and #list) that is *not* part of the variadic "request family", but which, e.g., can return a "class". That class can then offer list.empty[T] and list.from[T]( _ : Collection[T] ), etc I can see the attraction of the “separate” design - list(x,y,x), list.empty, list.from( c ) But it also seems a big magical to me. list(x,y,x), list, listFrom(c) ?? There's a non-magical but still confusing solution: * either the empty list, or any list, has a "from( c )" request (or even an empty request) that makes a brand new list with little reference to the old one.

…

I think at this point I still lean a bit more toward varargs, as long as we are restrictive about overloading rules.

[apblack]

With regard to this row: Explicit syntax Zero-Varargs Syntax Nonzero-varargs Explicit class syntax ... Make a list from another collection (or stream?) list.from(a,b,c) listFrom( c ) list.from( c) list.from(c) we talked about using `from` in preference to `withAll` in today’s meeting. But there is a reason to prefer the current method name `withAll` over `from`, which I had forgotten. Here it is: The methods to add a single element and a collection of elements to an existing list are `add(elm)` and `addAll(coll)`. The methods to create a collection from a single element and from a collection of elements are `with(elm)` and `withAll(coll)`. I like the symmetry of these names; replacing `withAll` with `from` looses that. So I think that I would be opposed to the change. Andrew

[kjx]

On 2/08/2017, at 17:43PM, Andrew Black ***@***.***> wrote: I like the symmetry of these names; replacing `withAll` with `from` looses that. So I think that I would be opposed to the change.

sure. minor point. could go to addFrom(x) I guess. Or + or ++ :-) J

[apblack]

++ already exists — it does the analogous thing for Lists that it does for Strings and Sequences: makes a new collections that's the concatenation of the receiver and the argument.

[apblack]

As far as I recall, the intended resolution of this issue is as follows. Others should correct me.

1. There should be a method list (set, sequence, dictionary) without arguments that represents the list (etc) –factory object. This is so that the collection in the prelude can be uniform with user-defined collections imported from modules.

2. These factory objects should have the following methods:

• empty — returns the appropriate empty collection
• withAll(existingCollection) — returns a new collection containing the elements of existingCollection
• with(elm1) — returns a new collection containing elem1
• with(elm1, elem2) — returns a new collection containing elem1 and elem2
• with(elm1, elem2, elem3) — returns a new collection containing elem1, elem2, and elem3
• and so on, up to
• with(elm1, elem2, elem3, elem4, elem5, elem6, elem7, elem8, elem9) — returns a new collection containing elem1, elem2, elem3, ... elem9.

3. The square-bracket "array constructor" syntax should be repurposed to create a Sequence, not a Lineup.

4. Collections already support the infix operator >> for use in pipelines. By an appropriate use of genericity, we should allow an alternative to writing list.withAll [1, 2, 3]: instead one could write [1, 2, 3] >> list.

5. Dialects, such as beginningStudent, may continue to use list(_), list(_,_), etc. if they so desire.

Oh, I should have added: we will not be re-instituting variable arity methods at this time. The issues with overriding seem too complex.

[apblack]

On 8th August 2017, in an email, @kjx wrote

James' latest bad idea:

a variadic definition

method foo( x... ) 

not only generates

method foo(x1)
method foo(x1,x2)
method foo(x1,x2,x3)
method foo(x1,x2,x3,x4)

but also

method fooAll(xs)

and the variadic forms are officially implemented by calling fooAll.

So: if you have a collection you can call fooAll to pass it into the variadic method.

In theory it would be possible to override the fooAll method separately from the others,
but if we do varargs, we should probably have a rule that you have to override the whole
variadic "family" at one go, and fooAll would be part of the family.

[KimBruce]

Andrew's comment above corresponds to my understanding of our agreement.

[kjx]

I can't remember. This doesn't answer the key question(s) we talked about. Kim asked:

• what should I write in the book?

list(1,2,3) ? list.with(1,2,3) ? or [1,2,3] >> list

• do we want substantial syntax for a built-in collection? That's a philosophical question, unfortunately, about object-oriented programming.

minor points:

• There should be a method ... that represents the list (etc) –factory object
  In (most of) this design, we can call it the list module. That's good thing.

• list (set, sequence, dictionary) without arguments
  I'm not sold on this big distinction between zero-arg and multi-arg methods (list vs list(x) or list(x,x,x)`. I can see the appeal, and can't really think of anything better...

• The square-bracket "array constructor" syntax should

Which I'd still like to get rid of - go back to Scala type syntax...

• The square-bracket "array constructor" syntax should be repurposed to create a Sequence, not a Lineup.
  We need to rationalise the collection design, so this mostly makes sense. Entia non sunt multiplicanda praeter necessitatem. If we went to varargs, and we had Java8 style definite streams, we could make the vararg "collection" a definite stream.

• Collections already support the infix operator >> for use in pipelines.. one could write [1, 2, 3] >> list
  Could or should? what should Kim pick?

• By an appropriate use of genericity,
  Not just genericity. Type inference also - in fact, I'm pretty sure, it needs subtle and novel work on the interactions between generics, inference, and gradual typing, and almost certainly yet more tradeoffs we won't like.

It's possible we could juggle thing to work in just these cases: [ ... ] or list(...) dynamically work out the some (structural? defined?) lub type for their arguments and then use that type to _ dynamically_ instantiate the actual generic arguments of the underlying collection.

// on some collection object method >>( collectionFactory) { var argType : Type = None // ? do { elem -> argType := argType & elem.magicallyGetMostSpecificType } return collectionFactory.withAll[argType](self) //argType isn't manifest. oops. }

At which point we're probably killed quickly by variance. At least we'd need Ceylon-style separate input (contravariant) and output (covariant) interfaces for collections...

Dialects, such as beginningStudent, may continue to use list(), list(,_), etc. if they so desire.

Oh, I should have added: we will not be re-instituting variable arity methods at this time. The issues with overriding seem too complex.

[KimBruce]

As I understand it from Andrew's write-up, we write [1,2,3,4] for a sequence and list.withAll[1,2,3,4] for the list created from the sequence.

[kjx]

we write [1,2,3,4] for a sequence

Well that's the question. That or seq(1,2,3,4) - which isn't too bad.

[kjx]

this was accepted as per Andrew's design