More precise inference for overloaded methods when arguments types align #7631

adriaanm · 2019-01-11T15:01:55Z

Normally, an argument supplied to an application of an overloaded method
is typed without an expected type. We can constrain type inference more
when we have only one candidate expected type between all alternatives.

So, this adds an exception when all alternatives explicitly specify the
same parameter type for an argument (a missing parameter type, due to e.g.
arity differences, is taken as NoType, thus resorting to no expected type).

Also improve a few corner cases with implicits (views), call-by-name arguments, named arguments.

Note that this also aligns us a bit better with dotty, where constraints are solved lazily,
while being pushed through overload resolution, so that this more precise kind of type inference
falls out "naturally" :-)

Fix scala/bug#11015

@SethTisue let's see how much of the community build I broke :-)

adriaanm · 2019-01-11T16:43:20Z

problematic case, not sure why yet:

  def assertSameElements[A, B](expected: Iterable[A], actual: Iterable[B]): Unit = ???
  def assertSameElements[A, B](expected: Iterable[A], actual: IterableOnce[B]): Unit = ???
  assertSameElements("abcd".split('d'), Array("abc"))

SethTisue · 2019-01-16T06:32:40Z

community build run queued:
https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/1692/ (404 til Jenkins wakes up)

retronym · 2019-01-16T07:08:07Z

I was hoping this case would be fixed by this PR:

$ (jabba use openjdk@1.10.0-2; scala -e 'println(java.util.List.of(""))')
/var/folders/b7/xcc2k0ln6ldcv247ffpy2d1w0000gp/T/scalacmd6660322910276770357.scala:1: error: ambiguous reference to overloaded definition,
both method of in object List of type [E](x$1: E*)java.util.List[E]
and  method of in object List of type [E](x$1: E)java.util.List[E]
match argument types (String) and expected result type Any
println(java.util.List.of(""))
                       ^

But it still fails with this PR:

$ (jabba use openjdk@1.10.0-2; scala-pr 7631 -e 'println(java.util.List.of(""))')
/var/folders/b7/xcc2k0ln6ldcv247ffpy2d1w0000gp/T/scalacmd6997157620920767218.scala:1: error: ambiguous reference to overloaded definition,
both method of in class List of type [E](x$1: E*)java.util.List[E]
and  method of in class List of type [E](x$1: E)java.util.List[E]
match argument types (String) and expected result type Any
println(java.util.List.of(""))
                       ^

adriaanm · 2019-01-16T07:55:04Z

I think that would need a change to specificity rules, off the top of my head.

adriaanm · 2019-01-16T21:17:33Z

Community build fail to investigate:

src/test/scala/spray/json/JsonParserSpec.scala:181:35: type mismatch;
[spray-json] [error]  found   : String("{\"key\":1}{\"key\":2}")
[spray-json] [error]  required: spray.json.ParserInput
[spray-json] [error]       val parser = new JsonParser("""{"key":1}{"key":2}""")
[spray-json] [error]                                   ^
[spray-json] [error] one error found

Normally, an argument supplied to an application of an overloaded method is typed without an expected type. We can constrain type inference more when we have only one candidate expected type between all alternatives. So, this adds an exception when all alternatives explicitly specify the same parameter type for an argument (a missing parameter type, due to e.g. arity differences, is taken as `NoType`, thus resorting to no expected type). Also improve a few corner cases with call-by-name arguments, named arguments.

Avoid having to deal with funky prototypes during implicit search.

adriaanm · 2019-01-21T16:24:34Z

Addressed the failure in spray-json with 68f3c9f

szeiger · 2019-01-23T16:57:02Z

Why would the java.util.List case be different than scala/bug#11015? In both cases you have a parameter of type A vs A*. Actually this prompted me to look at the spec for overload resolution again. Previously I fixating on the inferred type parameter Nothing which should be improved, but the spec doesn't even mention repeated parameters. I don't see anything in there that makes either A or A* more specific. So while type inference is not good enough for x.append(Foo(2)), shouldn't it be ambiguous either way? In which case the actual error would be that x.append(Foo[String](2)) compiles.

adriaanm · 2019-01-24T10:17:19Z

Good point. I'll investigate why that even compiles. I wonder if we can incorporate repeated params into the notion of being as-specific. You could think of being applicable to a repeated param involving an implicit conversion from T* to T, ..., T, which would result in a different score than direct applicability. See isStrictlyMoreSpecific, and this case inisAsSpecific:

case mt @ MethodType(_, _) if bothAreVarargs => checkIsApplicable(mt.paramTypes mapConserve repeatedToSingle)

this seems to explain the behavior, but i haven't verified yet

adriaanm · 2019-01-24T10:41:33Z

The original case for both methods being varargs was added in 1b80725 without much explanation.

adriaanm · 2019-01-24T11:39:26Z

Looks like I found the spot. I'll open a separate PR: #7680

retronym · 2019-01-29T05:09:29Z

src/compiler/scala/tools/nsc/typechecker/Typers.scala

-        val result = inferImplicitView(fromNoAnnot, to, tree, context, reportAmbiguous, saveErrors) match {
-          case fail if fail.isFailure => inferImplicitView(byNameType(fromNoAnnot), to, tree, context, reportAmbiguous, saveErrors)
+        val toNorm = normalizeProtoForView(to)
+        val result = inferImplicitView(fromNoAnnot, toNorm, tree, context, reportAmbiguous, saveErrors) match {


Is there a sensible / fast path through inferImplicitView when pt = WildCardType, i.e. the overloaded signatures don't agree on an effective single formal param type?

Sorry, I don't follow. If the overloaded signatures don't agree, proto.underlying will be WildcardType, so that will be normalizeProtoForView's result, but a wildcard for the expected type of a view makes little sense. Are you saying we should add another check to avoid calling inferImplicitView for such a pt? I suggest that would be a separate issue, since this PR doesn't really add new ways we could get here with a wildcard expected type (the path from adapt/isCoercible would not get to their callsite of inferImplicitView with such a pt.

retronym · 2019-01-29T05:26:28Z

When an OverloadedArgProto is used as the pt to Typer.typed, I've got a few questions (!!! comments inline below).

        val ptPlugins = pluginsPt(pt, this, tree, mode) // !!! Analyzer plugins (continuations? any others we know of?) will be unready for `OverloadedArgProto`
        def retypingOk = (
          context.retyping
            && (tree.tpe ne null)
            && (tree.tpe.isErroneous || !(tree.tpe <:< ptPlugins))
          )
        if (retypingOk) {
          tree.setType(null)
          if (tree.hasSymbolField) tree.symbol = NoSymbol
        }
        val alreadyTyped = tree.tpe ne null
        val shouldPrint = !alreadyTyped && !phase.erasedTypes
        val ptWild = if (mode.inPatternMode)
          ptPlugins // scala/bug#5022 don't widen pt for patterns as types flow from it to the case body.
        else
          dropExistential(ptPlugins) // FIXME: document why this is done. // !!! Do we need to make `dropExistential` deal with `OverloadedArgProto`?

From over in your other PR, we also check repeatedness of pt in https://github.com/scala/scala/pull/7680/files?utf8=%E2%9C%93&diff=split&w=1#diff-4eab1aad4533a31c10565971e90f73eaR1151. That might fall out because of the way things are implemented (OverloadedArgProto.typeSymbol unwraps, I suppose).

adriaanm · 2019-02-04T13:50:24Z

!!! Analyzer plugins

yes, they would have to be updated

dropExistential

I can't think of what this does -- my best guess is that it's for performance / workaround for issues doing subtyping/lub of existentials?

OverloadedArgProto.typeSymbol unwraps, I suppose

indeed

retronym · 2019-02-05T05:12:16Z

I commented out the dropExistential part of typed to jog our collective memory 😄

# Failed test paths (this command will update checkfiles)
partest --update-check \
  /Users/jz/code/scala/test/files/pos/patmat.scala \
  /Users/jz/code/scala/test/files/pos/sammy_exist.scala \
  /Users/jz/code/scala/test/files/pos/sammy_overload.scala \
  /Users/jz/code/scala/test/files/neg/logImplicits.scala \
  /Users/jz/code/scala/test/files/neg/sammy_error_exist_no_crash.scala \
  /Users/jz/code/scala/test/files/neg/t6258.scala \
  /Users/jz/code/scala/test/files/neg/t9273.scala

Extending the test to inject an OverlaodedArgProto wrapping an existential at pt:

trait Consumer[T] {
  def consume(x: T): Unit
}

object Test {
  def foo(x: String): Unit = ???
  def foo(): Unit = ???
  val f: Consumer[_ >: String] = foo

  def ol(a: Consumer[_ >: String], b: String) = ()
  def ol(a: Consumer[_ >: String], b: Int) = ()
  ol(foo, "") // fails with "error: type mismatch; found: Unit, required: Consumer[_ >: String]
}

Playing around a bit more reveals a latent bug (I think)

scala> def foo(a: Function[String, _ <: String]): a.type = a
foo: (a: Function[String, _ <: String])a.type

scala> foo(x => x)
<console>:13: error: missing parameter type
       foo(x => x)
           ^

This code doesn't account for the fact that ptNorm baseType FunctionSymbol can be an ExistentialType:

scala/src/compiler/scala/tools/nsc/typechecker/Typers.scala

Lines 2939 to 2945 in f72fa7f

    
           ptNorm baseType FunctionSymbol match { 
        
             case TypeRef(_, _, args :+ res) => (args, res) // if it's a TypeRef, we know its symbol will be FunctionSymbol 
        
             case _                          => { 
        
               val dummyPt = if (pt == ErrorType) ErrorType else NoType 
        
               (List.fill(numVparams)(dummyPt), WildcardType) // dummyPt is in CBN position 
        
             } 
        
           }

retronym · 2019-02-05T05:21:53Z

If we fix that code (naively, for now, by just calling .underlying before pattern matching), we can compile that:

$ qscala -e 'def foo(a: Function[String, _ <: String]): a.type = a; foo(x => x)'

Then, we can remove the dropExistential again and see the failure:

$ qscala -nc -e 'def foo(a: Function[String, _ <: String]): a.type = a; foo(x => x)'
/var/folders/tz/p8vd07wn7wxck3b9v54grlzw0000gp/T/scalacmd7366029521523825229.scala:1: error: type mismatch;
 found   : x.type (with underlying type String)
 required: _$1
def foo(a: Function[String, _ <: String]): a.type = a; foo(x => x)
                                                               ^

retronym · 2019-02-05T06:33:16Z

Here's where my experiments led: https://github.com/retronym/scala/tree/topic/existential-pt

We really need a canonical way to extract the param/result types from a PF or F type. There is some duplication and divergence between, e.g., partialFunctionArgResTypeFromProto, functionOrPfOrSamArgTypes. Contrast with dropByName, repeatedToSingle etc which share the underlying code to deal with alias and wrapper types.

This isn't strictly related to this PR, so we can address separately. Let's discuss today.

adriaanm · 2019-02-06T08:25:23Z

Ok, as discussed, let's merge this and leave the existential worries for later :-)

retronym · 2019-02-07T00:48:42Z

Followup: #7726

adriaanm added the release-notes worth highlighting in next release notes label Jan 11, 2019

adriaanm requested review from szeiger, smarter, retronym and lrytz January 11, 2019 15:01

scala-jenkins added this to the 2.13.0-RC1 milestone Jan 11, 2019

adriaanm force-pushed the t11015 branch from aff6d1b to 2ce8fb7 Compare January 11, 2019 15:56

adriaanm force-pushed the t11015 branch 2 times, most recently from ff18722 to 9094297 Compare January 14, 2019 13:59

adriaanm added 2 commits January 21, 2019 15:07

Strip OverloadedArgProto from view's expected type

68f3c9f

Avoid having to deal with funky prototypes during implicit search.

adriaanm force-pushed the t11015 branch from 9094297 to 68f3c9f Compare January 21, 2019 16:21

This comment has been minimized.

Sign in to view

retronym reviewed Jan 29, 2019

View reviewed changes

adriaanm self-assigned this Feb 4, 2019

adriaanm merged commit b8101d5 into scala:2.13.x Feb 6, 2019

julien-truffaut mentioned this pull request Apr 8, 2019

Overloaded compose methods optics-dev/Monocle#417

Closed

SethTisue mentioned this pull request Apr 8, 2019

Update scala 2.13.0-M5 to 2.13.0-RC1 lightbend-labs/scala-logging#159

Closed

eed3si9n mentioned this pull request May 3, 2019

OverloadedArgProto's toWild also nukes undets #8025

Merged

SethTisue mentioned this pull request Nov 10, 2022

Overloading issue in 2.12.17 but fine in 2.13.10 scala/bug#12688

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More precise inference for overloaded methods when arguments types align #7631

More precise inference for overloaded methods when arguments types align #7631

adriaanm commented Jan 11, 2019 •

edited

Loading

adriaanm commented Jan 11, 2019

SethTisue commented Jan 16, 2019

retronym commented Jan 16, 2019 •

edited

Loading

adriaanm commented Jan 16, 2019

adriaanm commented Jan 16, 2019

adriaanm commented Jan 21, 2019

This comment has been minimized.

szeiger commented Jan 23, 2019

adriaanm commented Jan 24, 2019 •

edited

Loading

adriaanm commented Jan 24, 2019 •

edited by SethTisue

Loading

adriaanm commented Jan 24, 2019 •

edited

Loading

retronym Jan 29, 2019 •

edited

Loading

adriaanm Feb 4, 2019

retronym commented Jan 29, 2019 •

edited

Loading

adriaanm commented Feb 4, 2019 •

edited

Loading

retronym commented Feb 5, 2019

retronym commented Feb 5, 2019

retronym commented Feb 5, 2019 •

edited

Loading

adriaanm commented Feb 6, 2019

retronym commented Feb 7, 2019

More precise inference for overloaded methods when arguments types align #7631

More precise inference for overloaded methods when arguments types align #7631

Conversation

adriaanm commented Jan 11, 2019 • edited Loading

adriaanm commented Jan 11, 2019

SethTisue commented Jan 16, 2019

retronym commented Jan 16, 2019 • edited Loading

adriaanm commented Jan 16, 2019

adriaanm commented Jan 16, 2019

adriaanm commented Jan 21, 2019

This comment has been minimized.

szeiger commented Jan 23, 2019

adriaanm commented Jan 24, 2019 • edited Loading

adriaanm commented Jan 24, 2019 • edited by SethTisue Loading

adriaanm commented Jan 24, 2019 • edited Loading

retronym Jan 29, 2019 • edited Loading

Choose a reason for hiding this comment

adriaanm Feb 4, 2019

Choose a reason for hiding this comment

retronym commented Jan 29, 2019 • edited Loading

adriaanm commented Feb 4, 2019 • edited Loading

retronym commented Feb 5, 2019

retronym commented Feb 5, 2019

retronym commented Feb 5, 2019 • edited Loading

adriaanm commented Feb 6, 2019

retronym commented Feb 7, 2019

adriaanm commented Jan 11, 2019 •

edited

Loading

retronym commented Jan 16, 2019 •

edited

Loading

adriaanm commented Jan 24, 2019 •

edited

Loading

adriaanm commented Jan 24, 2019 •

edited by SethTisue

Loading

adriaanm commented Jan 24, 2019 •

edited

Loading

retronym Jan 29, 2019 •

edited

Loading

retronym commented Jan 29, 2019 •

edited

Loading

adriaanm commented Feb 4, 2019 •

edited

Loading

retronym commented Feb 5, 2019 •

edited

Loading