Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Two Warts: Auto-Tupling and Multi-Parameter Infix Operations #4311

Closed
wants to merge 7 commits into from

Conversation

@odersky
Copy link
Contributor

@odersky odersky commented Apr 13, 2018

Prompted by the Scala contributors thread, this was initially an attempt to drop auto-tupling altogether. But it became quickly annoying. The main problem is that it's unnatural to add another pair of parentheses to infix operations. Compare:

(x, y) == z
z == ((x, y))    // !yuck

Same thing for cons:

((1, x)) :: ((2, y)) :: xs   // double yuck!

So at the very least we have to allow auto-tupling for arguments of infix operators. I went a little bit
further and also allowed auto-tupling if the expected type is already a (some kind of) product or tuple of the right arity. The result is this commit.

The language import noAutoTupling has been removed. The previous auto-tupling is still reported under Scala 2 mode, and there is -rewrite support to add the missing (...) automatically.

@odersky
Copy link
Contributor Author

@odersky odersky commented Apr 13, 2018

Comparing with the status quo: Auto-tupling of method arguments is supported if

  1. There are at least two arguments
  2. The method has a single parameter (and in the case of overloaded methods, all alternatives
    have a single parameter.)

Both of these conditions are missing in scalacs auto-tupling. Arguably, all reported confusing situations are already eliminated by these two conditions. So it is not clear why we would want to go further in restricting auto-tupling. It would complicate the spec and implementation, and I don't see really a gain.

So my current position would be to simply drop the noAutoTupling language import and keep the status quo.

@sjrd
Copy link
Member

@sjrd sjrd commented Apr 13, 2018

Note that the two yucks are because we allow infix method application with multiple parameters ... which is never what I want. Is there any known good use case for that?

If infix application only admitted one parameter, then z == (x, y) would parse the same way as z == ((x, y)) which we need now. Same for the :: example.

@odersky
Copy link
Contributor Author

@odersky odersky commented Apr 13, 2018

Is there any known good use case for that?

 buf += ("hello", "world")

Good or not, it's pretty common. If we started from scratch, I would be happy not to support this feature. Maybe we should work on eliminating this first? I don't yet see a clear way for migration, though.

@sjrd
Copy link
Member

@sjrd sjrd commented Apr 13, 2018

buf += ("hello", "world")

Well, see, I don't even know whether that's supposed to add 2 elements to buf ("hello" and "world") or 1 element (("hello", "world")) ...

An easy migration path, if a bit long:

  1. Deprecation warning when multiple parameters are used in an infix method call (without changing any semantics). This forces people to use dot notation if they want it.
  2. Compilation error in the same situation. Still does not change any existing semantics.
  3. Apply the new interpretation, where the argument is necessarily unique, hence a tuple.

-Xsource:n+1 can be used to get one step ahead in an opt-in way, as usual.

def test(tp: Type): Boolean = tp match {
case tp: TypeRef =>
val sym = tp.symbol
defn.isTupleClass(sym) && defn.arity(sym, str.Tuple) == args.length ||
Copy link
Contributor

@adriaanm adriaanm Apr 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about directly comparing to the class with name s"Tuple${args.length}"?

@adriaanm
Copy link
Contributor

@adriaanm adriaanm commented Apr 13, 2018

Very happy to see this! Looks like the best compromise we can get for now. Will work towards this through deprecation in 2.13 (and make it available under -Xsource:2.14)

@odersky
Copy link
Contributor Author

@odersky odersky commented Apr 13, 2018

I dropped the case where we auto-tuple if the expected type is a tuple. Only two lines in the whole codebase broke. So this is clearly a feature that does not pull its own weight. This means the only
issue is operators, and indeed, we should get rid of the variable number of arguments for an operator instead.

One additional measure to take: Warn if a symbolic method takes multiple parameters (since then there is no longer a way to use the method infix.

It would be good to start with this already for Scala 2.13. This also means that the variadic += operators in the collections should be deprecated or dropped. /cc @julienrf @szeiger @lrytz

@szeiger
Copy link
Contributor

@szeiger szeiger commented Apr 13, 2018

I agree with always tupling the RHS of an operator. It never made much sense to me that a single arg could be passed without parens but if you added parens it was an argument list, not a tuple.

We'd have to deprecate the variadic version of Growable.+= now.

Another instance is LongMap.+=:

  def +=(key: Long, value: V): this.type = { update(key, value); this }

Semantics and call syntax are the same as for the inherited += method (which takes a Tuple2 in a Map) but this overload takes two primitive Long values instead of boxing them and wrapping them in a tuple. No longer being able to call it as an operator would cause a performance regression.

@sjrd
Copy link
Member

@sjrd sjrd commented Apr 13, 2018

For LongMap:

@inline override final def +=(t: (Long, V)): this.type = {
  update(t._1, t._2)
  this
}

and let the optimizer get rid of it. If not scalac's, then the JVM's.

Additional benefit: also works for m += x -> v in addition to m += (x, v).

@szeiger
Copy link
Contributor

@szeiger szeiger commented Apr 13, 2018

Nice. I didn't think of using inlining to let the optimizer remove the boxing overhead.

@odersky odersky changed the title Restrict auto-tupling more Remove Two Warts: Auto-Tupling and Multi-Parameter Infix Operations Apr 15, 2018
@odersky
Copy link
Contributor Author

@odersky odersky commented Apr 15, 2018

This is the second part of the cleanup, which turned out to be quite a bit harder than the first. In particular:

  • An infix operation x op (y, z) now unconditionally interprets
    (y, z) as a tuple, unless in Scala-2 mode.
  • In Scala-2 mode, the pattern will give a migration warning with
    option to rewrite in most cases. Excluded from rewrite are
    operator assignments and right-associative infix operations.
  • Auto-tupling is only enabled in Scala-2 mode, and will give
    a migration warning with option to rewrite in that mode.

This change broke a lot more code than the auto-tupling change. As I feared, the pattern of infix operations that take multiple arguments on their right hand side is quite common. But I also think that in most cases the idiom is confusing and benefits from a rewrite to method syntax.

The other problem is that these patterns now mean something different (i.e. tuple on the rhs), which will likely lead to type errors later on, but it's also conceivable that they change behavior. The best remedy is to compile under -language:Scala2 first. This will issue migration warnings for all
problematic cases and rewrite most of them automatically.

@SethTisue
Copy link
Member

@SethTisue SethTisue commented Apr 15, 2018

buf += ("hello", "world")
Good or not, it's pretty common.

I am skeptical it is common. I have barely ever seen this used. I'd be happy to lose this.

As I feared, the pattern of infix operations that take multiple arguments on their right hand side is quite common

In what codebase? In the compiler codebase? If so, perhaps it's an outlier?

@odersky
Copy link
Contributor Author

@odersky odersky commented Apr 15, 2018

In what codebase? In the compiler codebase? If so, perhaps it's an outlier?

Compiler, collections, testing framework, and tests themselves. The most common usage is something like

 xs and (x, y, z)

because the author decided that looks better than

xs.and(x, y, z)

@ritschwumm
Copy link

@ritschwumm ritschwumm commented Apr 15, 2018

ouch,this will probably need a lot of my code to be changed. especially in longer method chains like a foo b bar c quux (c,d) which would have to become sth like a.foo(b).bar(c).quux(c,d) or (a foo b bar c).quux(c,d) then - both of which are imho more difficult to read.

@odersky odersky force-pushed the drop-autotupling branch from ec7c8ac to 66e4a83 Apr 15, 2018
@odersky
Copy link
Contributor Author

@odersky odersky commented Apr 15, 2018

@ritschwumm I would argue that a.foo(b).bar(c).quux(c,d) is easier to read than the other two. But I am also concerned that we require changes in a lot of places here.

@odersky
Copy link
Contributor Author

@odersky odersky commented Apr 15, 2018

languageserver has build issues now. @allanrenucci, @nicolasstucki could you take a look what needs to change?

@nafg
Copy link

@nafg nafg commented Apr 15, 2018

@ritschwumm the change doesn't have to be so invasive. You can write (a foo b bar c).quux(c,d)

In IntelliJ you can just Alt-Enter on quux and say "Convert from infix expression" and it will put the necessary parentheses in the right place automatically.

@odersky is there any sense in distinguishing between symbolic infix methods and alphanumeric infix methods? For instance a less breaking change would be to only not interpret parentheses as an argument list for symbolic infix methods. Of course it would be more elegant not to distinguish.

Also there would be the equivalent of a deprecation cycle, as was done for procedure syntax. -language:Scala2 counts, and @adriaanm already said there would be on the scalac side as well.

I just wonder if it makes any sense at all to deprecate parentheses as an argument list for symbolic infix methods earlier (or stricter in some way) than for alphanumeric infix methods.

Another point here is whether you can call it "auto-untupling" if the primary meaning of the syntax is a single tuple argument, but the method is defined to take separate parameters instead.

@Duhemm
Copy link
Contributor

@Duhemm Duhemm commented Apr 16, 2018

The CI failure is caused by sbt-buildinfo which generates the configuration for the language server in tests. It generates code which looks like

override val toString: String = "..." format (v1, v2, v3)

Which compiles but will throw an exception at runtime. It would be great to have this kind of issues caught by the compiler though; this is going to be painful to migrate.

See sbt/sbt-buildinfo#124

odersky added 5 commits Apr 16, 2018
This was initially an attempt to drop auto-tupling altogether. But it became quickly annoying. The
main problem is that it's unnatural to add another pair of parentheses to infix operations. Compare:

    (x, y) == z
    z == ((x, y))    // !yuck

Same thing for cons:

    ((1, x)) :: ((2, y)) :: xs   // double yuck!

So at the very least we have to allow auto-tupling for arguments of infix operators. I went a little bit
further and also allowed auto-tupling if the expected type is already a (some kind of) product or tuple of the right
arity. The result is this commit.

The language import `noAutoTupling` has been removed. The previous auto-tupling is still reported
under Scala 2 mode, and there is -rewrite support to add the missing (...) automatically.
idiom

    x op (y, z)

will in the future be interpreted as taking a tuple `(y, z)`. So all
infix operations with more than one parameter have to be rewritten to
method calls. This was for the most part done using an automatic rewrite
(introduced in one of the next commits). Most of the tests were not re-formatted
afterwards so one sees the traces of the rewrite. E.g. the reqrite would yield

    x .op (y, z)

instead of the more idiomatic

    x.op(y. z)
Under Scala-2 mode, issue a migration warning for a symbolic method that is
not unary.

Also: Two more tests updated to new infix operator regime.
 - An infix operation `x op (y, z)` now unconditionally interprets
   `(y, z)` as a tuple, unless in Scala-2 mode.
 - In Scala-2 mode, the pattern will give a migration warning with
   option to rewrite in most cases. Excluded from rewrite are
   operator assignments and right-associative infix operations.
 - Auto-tupling is only enabled in `Scala-2` mode, and will give
   a migration warning with option to rewrite in that mode.
@SethTisue
Copy link
Member

@SethTisue SethTisue commented Mar 1, 2019

meanwhile over in scala/scala, scala/scala#7795

@Ichoran
Copy link

@Ichoran Ichoran commented Mar 12, 2019

Was anyone going to comment on the performance concerns?

@eed3si9n
Copy link
Collaborator

@eed3si9n eed3si9n commented Mar 14, 2019

@Ichoran This looks kind of interesting - #5200

  inline def sum(inline tup: Tuple2[Int, Int]): Int = ~Macros.tup2(tup)

It would be some more work on the library authors, but I think the overall change to limit infix to one argument is good tradeoff since the parameter list and tuple syntax looks the same.

@Ichoran
Copy link

@Ichoran Ichoran commented Mar 18, 2019

@eed3si9n - If the deprecation is going in now, the performance concerns should also be addressed now. How bad are they, how widely used is the high-performance alternative, and will there be a replacement before the deprecated behavior is removed (so you can at least keep the old behavior if you're willing to tolerate the deprecation messages)?

@Ichoran
Copy link

@Ichoran Ichoran commented Mar 18, 2019

I guess it is highly relevant for this PR, but not to the linked one from which it was suggested to discuss the issue here. I think that is actually the wrong decision; this PR shouldn't be cluttered up with decisions about 2.13.

@som-snytt
Copy link
Contributor

@som-snytt som-snytt commented Jun 11, 2020

Scala2 lints all multiarg usages and multiparam definitions (for operators only), since the effort to deprecate (linked above) stalled (probably around migration or coordinating with Dotty):

scala/scala#8951

No attempt to handle op= specially, and += on LongMap is deprecated in favor of wordly method:

scala/scala#9003

The lint has deprecation ergonomics, such as not linting deprecated things:

scala> class C { def ++ (i: Int, j: Int) = println(i+j) }
                     ^
       warning: multiarg infix syntax looks like a tuple and will be deprecated
class C

scala> @deprecated("Uses multiarg infix", since="forever") class C { def ++ (i: Int, j: Int) = println(i+j) }
class C

scala>

On the old forum thread, I mentioned "" + (), which is the subject of a recent fix, which I just tried out 🎉, with the note, "In the PR, I rely on the spec language about multi-arg infix to assert that it doesn’t apply to empty args." So multi-arg infix was also the context for unit-value infix. On that forum post, I also added, "Before someone says it’s fixed in Dotty,...", but I'm afraid I can't edit it, now that it is fixed.

@soronpo
Copy link
Collaborator

@soronpo soronpo commented Jun 24, 2020

In my library I have syntax that looks as follows:

  val foo = DFUInt(32) <> IN init (0, 1, 3, 5, 7)

This limitation completely breaks this syntax and forces it to be:

  val foo = DFUInt(32).<>(IN).init(0, 1, 3, 5, 7)

I don't understand the logic of this limitation within this context. IMO, where the arguments are not allowed to be tuples anyway, there is no point limiting. If there can be an ambiguity, then (and only then) issue an error.

@SethTisue
Copy link
Member

@SethTisue SethTisue commented Jun 24, 2020

Scala 2.13.3, to be released shortly, includes scala/scala#8951 (by @som-snytt), which I guess is why Oron is hitting this now. So I guess we'll find out soon how many other library authors have reservations. (The PR description there now links here.)

@soronpo
Copy link
Collaborator

@soronpo soronpo commented Jun 24, 2020

Scala 2.13.3, to be released shortly, includes scala/scala#8951 (by @som-snytt), which I guess is why Oron is hitting this now. We'll find out soon how many other library authors object. (The PR description there now links here.)

I would have opted for a public discussion first (on contributors), and then deprecation later.

@soronpo
Copy link
Collaborator

@soronpo soronpo commented Jun 24, 2020

In my library I have syntax that looks as follows:

  val foo = DFUInt(32) <> IN init (0, 1, 3, 5, 7)

This limitation completely breaks this syntax and forces it to be:

  val foo = DFUInt(32).<>(IN).init(0, 1, 3, 5, 7)

I don't understand the logic of this limitation within this context. IMO, where the arguments are not allowed to be tuples anyway, there is no point limiting. If there can be an ambiguity, then (and only then) issue an error.

I would say though that there is an alternative course for me which is to accept a single argument which can be a tuple and then using macro-magic to handle the rest. Still, seems unnecessary where the context can be better understood otherwise.

@som-snytt
Copy link
Contributor

@som-snytt som-snytt commented Jun 24, 2020

@soronpo it's "just a lint" on scala 2. -Xlint:-multiarg-infix to turn it off.

@soronpo
Copy link
Collaborator

@soronpo soronpo commented Jun 24, 2020

@soronpo it's "just a lint" on scala 2. -Xlint:-multiarg-infix to turn it off.

Although better, now every user of my library would require this flag, since the warning is at the use site. It would be one of those "put this in and not ask questions" kind of documentation for all its users. That is a no-go for me.

@julienrf
Copy link
Collaborator

@julienrf julienrf commented Jun 24, 2020

(Edit [By @soronpo]: Sorry, I didn't notice I was editing this post. Reversing)

@soronpo Taking a single tuple parameter instead of several parameters is not a satisfactory solution for you?

That being said, to be honest when I read the following:

      val foo = DFUInt(32) <> IN init (0, 1, 3, 5, 7)

I find it very hard to parse (it is confusing what is an operator and what is an operand). I prefer:

      val foo = (DFUInt(32) <> IN).init(0, 1, 3, 5, 7)

(YMMV)

@henricook
Copy link

@henricook henricook commented Jul 14, 2020

I'm really enjoying refactoring lots of scala test syntax as a result of this in Scala 2.13 :-/

e.g. deltas.toSet should contain oneOf(... to deltas.toSet should contain.oneOf(...

@soronpo
Copy link
Collaborator

@soronpo soronpo commented Jul 14, 2020

(Edit [By @soronpo]: Sorry, I didn't notice I was editing this post. Reversing)

@soronpo Taking a single tuple parameter instead of several parameters is not a satisfactory solution for you?

That being said, to be honest when I read the following:

      val foo = DFUInt(32) <> IN init (0, 1, 3, 5, 7)

I find it very hard to parse (it is confusing what is an operator and what is an operand). I prefer:

      val foo = (DFUInt(32) <> IN).init(0, 1, 3, 5, 7)

(YMMV)

I'm sorry, I just noticed I edited your post by mistake. I edited again to restore your post.
Here is what I wrote in reply:

I understand what you mean, but you're currently reading it in parts, out of context, and not as a single statement. There are several legal variations to it:

val foo = DFUInt(32)
or
val foo = DFUInt(32) init 0
or
val foo = DFUInt(32) init (0, 1, 2, 3)
or
val foo = DFUInt(32) <> IN
or 
val foo = DFUInt(32) <> IN init 0
or
val foo = DFUInt(32) <> IN init (0, 1, 2, 3)
or
val foo = DFUInt(32) <> IN init (0, 1, 2, 3) !! SomeTag !! SomeOtherTag
or
<more tagged combinations like above>

Adding bracket/dots/whatever to be explicit about associativity undermines the flow and consistency I expect.
It would be like forcing you to use brackets on protected def. I use Scala as a DSL. So are many other users. And many dedicated APIs are essentially DSLs. I understand the purpose of being more opinionated in Scala 3.0, but I feel we might be blocking too much here.

@godenji
Copy link

@godenji godenji commented Aug 29, 2020

My table definitions look like:

def * = id ~ first ~ last <> (User.apply _, User.unapply _)

Forgetting the current spew of spurious (to me) warnings, removing this feature will result in -- well I'm not sure what as I haven't tried to fix the warnings -- presumably something like like:

def * = ((id ~ first ~ last) <>)(User.apply _, User.unapply _)

which is more confusing than the original syntax. Is allowing this feature really that terrible?

@julienrf
Copy link
Collaborator

@julienrf julienrf commented Aug 29, 2020

@godenji I think you will have to write this:

def * = (id ~ first ~ last).<>(User.apply _, User.unapply _)

@smarter
Copy link
Member

@smarter smarter commented Aug 29, 2020

It seems that people are using this issue to complain about a new warning introduced in Scala 2.13.3. This warning is not present in Dotty itself as far as I can tell, so I don't really know why Scala 2.13's behavior was changed to be stricter than Dotty's, but this isn't the correct bug tracker to discuss this.

@smarter
Copy link
Member

@smarter smarter commented Aug 29, 2020

Ah, it looks like that warning has been moved behind a lint flag in the as-yet-unreleased 2.13.4: scala/scala#9089

@som-snytt
Copy link
Contributor

@som-snytt som-snytt commented Aug 29, 2020

I created a topic on the forum. I did not attempt to summarize opinions here, but I did contribute the coinage "autountupling."

https://contributors.scala-lang.org/t/multiarg-infix-application-considered-warty/4490

@odersky
Copy link
Contributor Author

@odersky odersky commented Nov 16, 2020

To be revived for 3.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet