Bound callable references

[abreslav]

Discussions about the Bound callable references proposal will be held here.

@udalov is the shepherd for this proposal.

Short description

Bound callable references are expression like foo::bar where foo is itself an expression, not a class. A type of foo::toString would be () -> String for any foo.

See full proposal for more details

[b005t3r]

Just to clarify:

class C {
    fun foo() {}
}

fun test(c: C?) {
    c::foo             // error
    c::class           // error
    null::class        // error
}

But this would still work, right?

class C {
    fun foo() {}
}

fun test(c: C?) {
    c?::foo             // error
    c?::class           // error
    null?::class        // error
}

API for "unbinding" a reference

I'd rather go for API for getting an unbound reference from a bound reference (and vice versa).

if unbound already, throw or return null, or provide both?

Of course there would be a way to check if it's bound/unbound, right?

[udalov]

But this would still work, right?

Currently this is not planned, but it's a valid option.

I'd rather go for API for getting an unbound reference from a bound reference (and vice versa).
Of course there would be a way to check if it's bound/unbound, right?

This is exactly what that open question is about. It's not clear at this point whether such API is needed and what declarations would be available there. I'll elaborate on this in the proposal.

[damianw]

I noticed that re-introducing type inference for an empty LHS isn't listed as a "possible future advancement". Has this been considered? i.e.:

val foo = listOf<String>()
foo.filter(::isEmpty) // String::isEmpty is inferred for arguments of type String.() -> Boolean or (String) -> Boolean

It's not specifically related to bound references, but as I recall, it was removed to reserve the syntax for an implicit this for bound references.

I wouldn't mind having to be explicit to resolve ambiguities with this, but I'd take an implicit type on the LHS rather than this any day. Having to specify SomeComplexType::isFrobnicated is often pretty inconvenient... almost as much as using a lambda when there are many arguments.

[b005t3r]

But this would still work, right?

Currently this is not planned, but it's a valid option.

Hmm, I think this would be really limiting and lead to unnecessary tricks (to get the non-nullable reference) in many normally valid scenarios. Like instead of being able to do this:
event += obj?::handler
one would have to do something like this instead (not sure if this is a totally legit code, but I think you get my point):
obj?.let { event += it::handler }

[MichaelRocks]

It would be really nice if we could create something like delegates in C#. It requires bound references to be comparable. I don’t think it’s possible to make bound references for the same member of a single instance equal by reference in general case. But it should be possible to override equals() in a FunctionX subclass to make bound references comparable.

[b005t3r]

Why can't they be comparable?

Just tested this:

val refA = Demo::func
val refB = Demo::func
println(refA == refB)
println(refA === refB)

and got:

true
false

so everything works as expected.

[MichaelRocks]

If you mean comparable by references then I don't see a way ho you can cache an instance of FunctionX without leaking a receiver.

[b005t3r]

No, I mean structural equality, because that's all what's needed: https://kotlinlang.org/docs/reference/equality.html#structural-equality

[MichaelRocks]

That's what my question is about. I would like the bound reference to have equals() overridden so structural equality can work.

[TheRaspPie]

Maybe add support for partial application? Something like
foo::doSomething(x, y).invoke(z)

Of course this could also be separate from this issue.

[MarioAriasC]

@TheRaspPie Partial application is available from funKTionale https://github.com/MarioAriasC/funKTionale/wiki/Partial-application

[abreslav]

It would be really nice if we could create something like delegates in C#. It requires bound references to be comparable.

@MichaelRocks could you clarify: what is the use case you have in mind?

[b005t3r]

I think the use cause would be like this one, but with a bound reference instead of a lambda:

https://gist.github.com/orangy/11178911

[MichaelRocks]

@abreslav Sure. Let's say we have a class that allows us to add/remove and notify observers.

class Event<T> {
  private val observers = ArrayList<(T) -> Unit>()

  operator fun plusAssign(observer: (T) -> Unit) {
    observers += observer
  }

  operator fun minusAssign(observer: (T) -> Unit) {
    observers -= observer
  }

  operator fun invoke(value: T) {
    observers.forEach { it(value) }
  }
}

And also we have a class which method must be invoked when an event occurs.

class Listener {
  fun onEvent(string: String) {
    // Do something.
  }
}

The natural syntax for adding, invoking, and removing an observer would be something like that.

  val event = Event<String>()
  val listener = Listener()

  event += listener::onEvent
  event("Test")
  event -= listener::onEvent

Unfortunately, removing the observer will not work if equals for a bound reference is not implemented properly. So I suggest overriding the equals() method for bound references so that it compares a receiver by instance.

[abreslav]

@udalov please correct me if I'm wrong, but I think that equals() is supposed to work correctly on bound references. Makes sense to mention it in the document, though

[nsk-mironov]

Regarding the @MichaelRocks comment, I think it's worth to mention that the following use case should be supported as well:

interface Foo {
  fun foo(): CharSequence
}

class Bar : Foo {
  override fun foo(): String = ""
}

val bar: Bar = Bar()
val foo: Foo = bar

require(foo::foo == bar::foo)

Foo::foo and Bar::foo have different types (Foo) -> CharSequence vs (Bar) -> String, but foo::foo and bar::foo are still references to the same method.

[sksamuel]

As @damianw said, there should be support for inferring the instance if it is not specified. Being forced to do foo(someVar::someMethod) is actually worse than foo { someMethod(it) }. I'd give a big thumbs up to just ditching the :: completely if the method is in scope (for some definition of in scope), so I can do foo(someMethod) like in Scala.

[udalov]

@sksamuel

Being forced to do foo(someVar::someMethod) is actually worse than foo { someMethod(it) }

You probably mean foo(SomeType::someMethod) in the first call because omitting the type is what @damianw's suggestion is about and because there's no way to leave out a receiver variable when making a call (the compiler couldn't figure out which variable were you supposed to call the method on).

ditching the :: completely <...>, so I can do foo(someMethod)

This is a valid option, although it'll only allow to pass a reference to a function, not to a property. Still, we're investigating this possibility.

[sksamuel]

Yes, sorry for confusion. I am talking about eta expansion of a method into a function, to avoid this::someMethod

[cypressious]

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

Will it be possible to reference extension methods that are instance methods? Currently, this doesn't work:

class Bar

class Foo {
    fun Bar.doStuff() {}

    fun baz(bar: Bar) {
        bar.apply(Bar::doStuff) //doesn't work
    }
}

What about referencing doStuff from outside the class? You'd need to bind one of the two receivers or maybe even both.

And what about partially bound references. Can I bind only Foo to receive a (Bar) -> Unit (or Bar.() -> Unit?) Or maybe the other way round?

[b005t3r]

@udalov is there a timeframe for this feature (a rough one at least), do you plan to do some other things first, etc. I'd (and hopefully I'm not the only one) just like to know how long it's going to take.

[udalov]

@b005t3r

is there a timeframe for this feature (a rough one at least), do you plan to do some other things first, etc. I'd (and hopefully I'm not the only one) just like to know how long it's going to take.

The prototype is being worked on currently and the larger parts are mostly done. Once everything is implemented according to the proposal, we'll start an early access preview for this feature as a part of Kotlin 1.1. There's no date set for the 1.1 release; it will be out when this and other planned features are fully implemented. The first EAP is coming soon (the main development branch is already 1.1).

[abreslav]

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

@cypressious I'm not sure if we have taken care of it already, but I believe we should. It's an important use case. // cc @udalov

[ilya-g]

Information about the original receiver type is absent in the type of a bound reference

Maybe bound reference should be of some subtype of () -> T, which carries the type of the receiver? Thus it can both be assignable to () -> T and have a type-safe unbind: fun unbind(): R.() -> T

[udalov]

@b005t3r

Sorry for the long delay. Re your comment,

Hmm, I think this would be really limiting and lead to unnecessary tricks (to get the non-nullable reference) in many normally valid scenarios. Like instead of being able to do this:
event += obj?::handler
one would have to do something like this instead (not sure if this is a totally legit code, but I think you get my point):
obj?.let { event += it::handler }

If by a?::b you mean "the reference to b if a is non-null and null otherwise", then I think these two examples have different semantics. In the first one, you're adding a null handler to the event if obj is null, whereas in the second one nothing happens. So, what you've probably meant in the first case was obj?::handler?.let { event += it } and that doesn't look prettier than the second case.

Overall, we've discussed this and decided that unless there are obvious reasons why it should be supported immediately, it can be postponed. Fun fact, to support it in the future without breaking source compatibility we must reserve the a?::b syntax in case when a is an object or a companion object, i.e. report an error in that case. I'll explain this in the proposal.

[b005t3r]

Yes, they do have different effect. I took a shortcut with my example, with the event += a?::b I meant null could be ignored in the operator's implementation.

But I'm not sure how is this going to work with nullable references without the ? - it won't be possible at all?

[udalov]

But I'm not sure how is this going to work with nullable references without the ? - it won't be possible at all?

No, it won't be possible, see the end of the Resolution section:

It is an error if the LHS expression has nullable type and the resolved member is not an extension to nullable type.

So the only possible way would be the one you proposed or, if the operator ignores nulls, event += obj?.let { it::handler }.

[udalov]

@damianw

I noticed that re-introducing type inference for an empty LHS isn't listed as a "possible future advancement". Has this been considered? i.e.:

val foo = listOf<String>()
foo.filter(::isEmpty) // String::isEmpty is inferred for arguments of type String.() -> Boolean or (String) -> Boolean

Thanks. This could be implemented, but with some care not to break source compatibility. I'll explain in the proposal.

[cypressious]

Is there something we can do about type mismatch because (T) -> Unit was expected but (T) -> Something was provided?

This makes it hard to use builder-style methods or something like foo(mutableList::remove) where values are returned that are ignored most of the time anyway.

[udalov]

@cypressious

Is there something we can do about type mismatch because (T) -> Unit was expected but (T) -> Something was provided?

Please vote for KT-11723 and share your use case there. Thanks!

[cypressious]

Has there been some progress on instance extension methods? It seems counterintuitive, that you can do foo::doStuff with

class Foo {
    fun doStuff(bar: Bar) {
    }
}

but can't with

class Foo {
    fun Bar.doStuff() {
    }
}

even though they are virtually the same.

[TheRaspPie]

@udalov Sorry for the delay.

The official term in Kotlin is and has always been "function" and it's used everywhere in the code and in the docs. Colloquially, of course we sometimes call functions "methods" because this is what they're called on JVM. But I'm not sure how is this topic important and relevant to this proposal?

The idea is that
fun test1(): Int = 10 would be just syntatic sugar for
val test2: () -> Int = { 10 }

The result of differentiating between function and anonymous function is that you need to write ::test1 to get the same as test2.
The fact that the JVM only knows named methods like C leaks through this abstraction.
This however, does not apply to the JavaScript backend, where everything is a closure.

To answer your question: This topic is only relevant indirectly to this proposal, because it applies to the overall concept of method references which this issue is a part of.

The argument that you want the same notation as in Java is a valid, although inconsistent argument. Then you'd also need the same syntax for constructors, variable declaration etc.

[cypressious]

References to operators are generally unnecessary because you can always use the desugared methods but the comparison operators equals and compareTo are a bit special because they have special logic (null checks for == and intepreting the Int result to a Boolean for < and friends). Would it be possible to provide a way to reference these operators directly?

Use case:

listOfFoos.all(fooProvider.computeFoo()::==)
listOfMeasurements.any(THRESHOLD::<)

[ilya-g]

listOfFoos.all(fooProvider.computeFoo()::==)
listOfMeasurements.any(THRESHOLD::<)

@cypressious why callable references should be used here? Doesn't plain lambdas fit this case more?
I don't think that special logic operators could be easily represented with callable references, e.g. what member in particular they should reference to?

[cypressious]

@ilya-g

why callable references should be used here? Doesn't plain lambdas fit this case more?

One argument is that the lhs of the bound callable reference is cached but more arguments were discussed in this thread.

I don't think that special logic operators could be easily represented with callable references, e.g. what member in particular they should reference to?

I was thinking about this question and one possibility I see is to provide @InlineOnly methods in the runtime which actually implement the behaviour instead of the magic that the compiler does now. In this case, having the operator reference feature wouldn't even be necessary anymore since you could reference the methods (just like you can do now with plus instead of +)

[udalov]

I've reworded the part about resolution of left-hand side in the proposal and added some corner case examples and their solutions, everyone is welcome to review.

[punksta]

What about reference to constructors?

rx.Single.zip(getName(), getPhotos(), Pair<String, List<String>>::new)

[cypressious]

@punksta ::Pair

[weakish]

@udalov

@ebceu4

why it's not possible to reference functions as first-class citizens,

Do you mean the syntax? If yes, then this is because there are also properties, and references to them cannot use that syntax because it would be indistinguishable from a call.

Why would it be indistinguisheable?
What about just disallow propertiy and the function to have the same name?

Currently Kotlin allows propertiy and function with the same name:

val i: Int = 2
fun i(): Int {
    return 2
}
print(i)
print(i())

Diallowing this (I cannot imgae when this will be useful) so we can just refer a function with its name.

[cypressious]

@weakish That would break backward compatibility and is out of the question.

[weakish]

@cypressious I think few code are written with same name property and function. Besides, current Kotlin cannot resolve it correctly with :: syntax:

class KK() {
  val i = 1
  fun i(): Int { return 1 }
}
fun hkk(kk: KK, f: (KK) -> Int) {
  println(f(kk))
}
fun main(args: Array<String>) {
  hkk(KK(), KK::i)
}

Compiled with kotlinc 1.0.3:

error: overload resolution ambiguity:
public final val i: Int defined in KK
public final fun i(): Int defined in KK
  hkk(KK(), KK::i)
                ^

So:

• Disallowing property and function to have the same name will not break backward compatibility in most cases.
• To maintain backward compatibility with current function reference syntax, just make Foo::bar a (deprecated) alias for Foo.bar.

[dzharkov]

The feature has been released in Kotlin 1.1, thus I'm closing the KEEP. Do not hesitate to open a YouTrack issue for any additional suggestions