Bound callable references #5

Closed
abreslav opened this Issue Apr 20, 2016 · 69 comments

Comments

@abreslav
Contributor

abreslav commented Apr 20, 2016

Discussions about the Bound callable references proposal will be held here.

@udalov is the shepherd for this proposal.

Short description

Bound callable references are expression like foo::bar where foo is itself an expression, not a class. A type of foo::toString would be () -> String for any foo.

See full proposal for more details

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r Apr 20, 2016

Just to clarify:

class C {
    fun foo() {}
}

fun test(c: C?) {
    c::foo             // error
    c::class           // error
    null::class        // error
}

But this would still work, right?

class C {
    fun foo() {}
}

fun test(c: C?) {
    c?::foo             // error
    c?::class           // error
    null?::class        // error
}

API for "unbinding" a reference

I'd rather go for API for getting an unbound reference from a bound reference (and vice versa).

if unbound already, throw or return null, or provide both?

Of course there would be a way to check if it's bound/unbound, right?

b005t3r commented Apr 20, 2016

Just to clarify:

class C {
    fun foo() {}
}

fun test(c: C?) {
    c::foo             // error
    c::class           // error
    null::class        // error
}

But this would still work, right?

class C {
    fun foo() {}
}

fun test(c: C?) {
    c?::foo             // error
    c?::class           // error
    null?::class        // error
}

API for "unbinding" a reference

I'd rather go for API for getting an unbound reference from a bound reference (and vice versa).

if unbound already, throw or return null, or provide both?

Of course there would be a way to check if it's bound/unbound, right?

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Apr 20, 2016

Contributor

But this would still work, right?

Currently this is not planned, but it's a valid option.

I'd rather go for API for getting an unbound reference from a bound reference (and vice versa).
Of course there would be a way to check if it's bound/unbound, right?

This is exactly what that open question is about. It's not clear at this point whether such API is needed and what declarations would be available there. I'll elaborate on this in the proposal.

Contributor

udalov commented Apr 20, 2016

But this would still work, right?

Currently this is not planned, but it's a valid option.

I'd rather go for API for getting an unbound reference from a bound reference (and vice versa).
Of course there would be a way to check if it's bound/unbound, right?

This is exactly what that open question is about. It's not clear at this point whether such API is needed and what declarations would be available there. I'll elaborate on this in the proposal.

@damianw

This comment has been minimized.

Show comment
Hide comment
@damianw

damianw Apr 20, 2016

I noticed that re-introducing type inference for an empty LHS isn't listed as a "possible future advancement". Has this been considered? i.e.:

val foo = listOf<String>()
foo.filter(::isEmpty) // String::isEmpty is inferred for arguments of type String.() -> Boolean or (String) -> Boolean

It's not specifically related to bound references, but as I recall, it was removed to reserve the syntax for an implicit this for bound references.

I wouldn't mind having to be explicit to resolve ambiguities with this, but I'd take an implicit type on the LHS rather than this any day. Having to specify SomeComplexType::isFrobnicated is often pretty inconvenient... almost as much as using a lambda when there are many arguments.

damianw commented Apr 20, 2016

I noticed that re-introducing type inference for an empty LHS isn't listed as a "possible future advancement". Has this been considered? i.e.:

val foo = listOf<String>()
foo.filter(::isEmpty) // String::isEmpty is inferred for arguments of type String.() -> Boolean or (String) -> Boolean

It's not specifically related to bound references, but as I recall, it was removed to reserve the syntax for an implicit this for bound references.

I wouldn't mind having to be explicit to resolve ambiguities with this, but I'd take an implicit type on the LHS rather than this any day. Having to specify SomeComplexType::isFrobnicated is often pretty inconvenient... almost as much as using a lambda when there are many arguments.

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r Apr 20, 2016

But this would still work, right?

Currently this is not planned, but it's a valid option.

Hmm, I think this would be really limiting and lead to unnecessary tricks (to get the non-nullable reference) in many normally valid scenarios. Like instead of being able to do this:
event += obj?::handler
one would have to do something like this instead (not sure if this is a totally legit code, but I think you get my point):
obj?.let { event += it::handler }

b005t3r commented Apr 20, 2016

But this would still work, right?

Currently this is not planned, but it's a valid option.

Hmm, I think this would be really limiting and lead to unnecessary tricks (to get the non-nullable reference) in many normally valid scenarios. Like instead of being able to do this:
event += obj?::handler
one would have to do something like this instead (not sure if this is a totally legit code, but I think you get my point):
obj?.let { event += it::handler }

@MichaelRocks

This comment has been minimized.

Show comment
Hide comment
@MichaelRocks

MichaelRocks Apr 20, 2016

It would be really nice if we could create something like delegates in C#. It requires bound references to be comparable. I don’t think it’s possible to make bound references for the same member of a single instance equal by reference in general case. But it should be possible to override equals() in a FunctionX subclass to make bound references comparable.

It would be really nice if we could create something like delegates in C#. It requires bound references to be comparable. I don’t think it’s possible to make bound references for the same member of a single instance equal by reference in general case. But it should be possible to override equals() in a FunctionX subclass to make bound references comparable.

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r Apr 20, 2016

Why can't they be comparable?

Just tested this:

val refA = Demo::func
val refB = Demo::func
println(refA == refB)
println(refA === refB)

and got:

true
false

so everything works as expected.

b005t3r commented Apr 20, 2016

Why can't they be comparable?

Just tested this:

val refA = Demo::func
val refB = Demo::func
println(refA == refB)
println(refA === refB)

and got:

true
false

so everything works as expected.

@MichaelRocks

This comment has been minimized.

Show comment
Hide comment
@MichaelRocks

MichaelRocks Apr 20, 2016

If you mean comparable by references then I don't see a way ho you can cache an instance of FunctionX without leaking a receiver.

MichaelRocks commented Apr 20, 2016

If you mean comparable by references then I don't see a way ho you can cache an instance of FunctionX without leaking a receiver.

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r Apr 20, 2016

No, I mean structural equality, because that's all what's needed: https://kotlinlang.org/docs/reference/equality.html#structural-equality

b005t3r commented Apr 20, 2016

No, I mean structural equality, because that's all what's needed: https://kotlinlang.org/docs/reference/equality.html#structural-equality

@MichaelRocks

This comment has been minimized.

Show comment
Hide comment
@MichaelRocks

MichaelRocks Apr 20, 2016

That's what my question is about. I would like the bound reference to have equals() overridden so structural equality can work.

MichaelRocks commented Apr 20, 2016

That's what my question is about. I would like the bound reference to have equals() overridden so structural equality can work.

@TheRaspPie

This comment has been minimized.

Show comment
Hide comment
@TheRaspPie

TheRaspPie Apr 21, 2016

Maybe add support for partial application? Something like
foo::doSomething(x, y).invoke(z)

Of course this could also be separate from this issue.

Maybe add support for partial application? Something like
foo::doSomething(x, y).invoke(z)

Of course this could also be separate from this issue.

@MarioAriasC

This comment has been minimized.

Show comment
Hide comment
@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Apr 21, 2016

Contributor

It would be really nice if we could create something like delegates in C#. It requires bound references to be comparable.

@MichaelRocks could you clarify: what is the use case you have in mind?

Contributor

abreslav commented Apr 21, 2016

It would be really nice if we could create something like delegates in C#. It requires bound references to be comparable.

@MichaelRocks could you clarify: what is the use case you have in mind?

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r Apr 21, 2016

I think the use cause would be like this one, but with a bound reference instead of a lambda:

https://gist.github.com/orangy/11178911

b005t3r commented Apr 21, 2016

I think the use cause would be like this one, but with a bound reference instead of a lambda:

https://gist.github.com/orangy/11178911

@MichaelRocks

This comment has been minimized.

Show comment
Hide comment
@MichaelRocks

MichaelRocks Apr 21, 2016

@abreslav Sure. Let's say we have a class that allows us to add/remove and notify observers.

class Event<T> {
  private val observers = ArrayList<(T) -> Unit>()

  operator fun plusAssign(observer: (T) -> Unit) {
    observers += observer
  }

  operator fun minusAssign(observer: (T) -> Unit) {
    observers -= observer
  }

  operator fun invoke(value: T) {
    observers.forEach { it(value) }
  }
}

And also we have a class which method must be invoked when an event occurs.

class Listener {
  fun onEvent(string: String) {
    // Do something.
  }
}

The natural syntax for adding, invoking, and removing an observer would be something like that.

  val event = Event<String>()
  val listener = Listener()

  event += listener::onEvent
  event("Test")
  event -= listener::onEvent

Unfortunately, removing the observer will not work if equals for a bound reference is not implemented properly. So I suggest overriding the equals() method for bound references so that it compares a receiver by instance.

@abreslav Sure. Let's say we have a class that allows us to add/remove and notify observers.

class Event<T> {
  private val observers = ArrayList<(T) -> Unit>()

  operator fun plusAssign(observer: (T) -> Unit) {
    observers += observer
  }

  operator fun minusAssign(observer: (T) -> Unit) {
    observers -= observer
  }

  operator fun invoke(value: T) {
    observers.forEach { it(value) }
  }
}

And also we have a class which method must be invoked when an event occurs.

class Listener {
  fun onEvent(string: String) {
    // Do something.
  }
}

The natural syntax for adding, invoking, and removing an observer would be something like that.

  val event = Event<String>()
  val listener = Listener()

  event += listener::onEvent
  event("Test")
  event -= listener::onEvent

Unfortunately, removing the observer will not work if equals for a bound reference is not implemented properly. So I suggest overriding the equals() method for bound references so that it compares a receiver by instance.

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav Apr 21, 2016

Contributor

@udalov please correct me if I'm wrong, but I think that equals() is supposed to work correctly on bound references. Makes sense to mention it in the document, though

Contributor

abreslav commented Apr 21, 2016

@udalov please correct me if I'm wrong, but I think that equals() is supposed to work correctly on bound references. Makes sense to mention it in the document, though

@nsk-mironov

This comment has been minimized.

Show comment
Hide comment
@nsk-mironov

nsk-mironov Apr 21, 2016

Regarding the @MichaelRocks comment, I think it's worth to mention that the following use case should be supported as well:

interface Foo {
  fun foo(): CharSequence
}

class Bar : Foo {
  override fun foo(): String = ""
}

val bar: Bar = Bar()
val foo: Foo = bar

require(foo::foo == bar::foo)

Foo::foo and Bar::foo have different types (Foo) -> CharSequence vs (Bar) -> String, but foo::foo and bar::foo are still references to the same method.

nsk-mironov commented Apr 21, 2016

Regarding the @MichaelRocks comment, I think it's worth to mention that the following use case should be supported as well:

interface Foo {
  fun foo(): CharSequence
}

class Bar : Foo {
  override fun foo(): String = ""
}

val bar: Bar = Bar()
val foo: Foo = bar

require(foo::foo == bar::foo)

Foo::foo and Bar::foo have different types (Foo) -> CharSequence vs (Bar) -> String, but foo::foo and bar::foo are still references to the same method.

@sksamuel

This comment has been minimized.

Show comment
Hide comment
@sksamuel

sksamuel Apr 29, 2016

As @damianw said, there should be support for inferring the instance if it is not specified. Being forced to do foo(someVar::someMethod) is actually worse than foo { someMethod(it) }. I'd give a big thumbs up to just ditching the :: completely if the method is in scope (for some definition of in scope), so I can do foo(someMethod) like in Scala.

sksamuel commented Apr 29, 2016

As @damianw said, there should be support for inferring the instance if it is not specified. Being forced to do foo(someVar::someMethod) is actually worse than foo { someMethod(it) }. I'd give a big thumbs up to just ditching the :: completely if the method is in scope (for some definition of in scope), so I can do foo(someMethod) like in Scala.

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Apr 29, 2016

Contributor

@sksamuel

Being forced to do foo(someVar::someMethod) is actually worse than foo { someMethod(it) }

You probably mean foo(SomeType::someMethod) in the first call because omitting the type is what @damianw's suggestion is about and because there's no way to leave out a receiver variable when making a call (the compiler couldn't figure out which variable were you supposed to call the method on).

ditching the :: completely <...>, so I can do foo(someMethod)

This is a valid option, although it'll only allow to pass a reference to a function, not to a property. Still, we're investigating this possibility.

Contributor

udalov commented Apr 29, 2016

@sksamuel

Being forced to do foo(someVar::someMethod) is actually worse than foo { someMethod(it) }

You probably mean foo(SomeType::someMethod) in the first call because omitting the type is what @damianw's suggestion is about and because there's no way to leave out a receiver variable when making a call (the compiler couldn't figure out which variable were you supposed to call the method on).

ditching the :: completely <...>, so I can do foo(someMethod)

This is a valid option, although it'll only allow to pass a reference to a function, not to a property. Still, we're investigating this possibility.

@sksamuel

This comment has been minimized.

Show comment
Hide comment
@sksamuel

sksamuel Apr 29, 2016

Yes, sorry for confusion. I am talking about eta expansion of a method into a function, to avoid this::someMethod

Yes, sorry for confusion. I am talking about eta expansion of a method into a function, to avoid this::someMethod

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious May 6, 2016

Contributor

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

Will it be possible to reference extension methods that are instance methods? Currently, this doesn't work:

class Bar

class Foo {
    fun Bar.doStuff() {}

    fun baz(bar: Bar) {
        bar.apply(Bar::doStuff) //doesn't work
    }
}

What about referencing doStuff from outside the class? You'd need to bind one of the two receivers or maybe even both.

And what about partially bound references. Can I bind only Foo to receive a (Bar) -> Unit (or Bar.() -> Unit?) Or maybe the other way round?

Contributor

cypressious commented May 6, 2016

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

Will it be possible to reference extension methods that are instance methods? Currently, this doesn't work:

class Bar

class Foo {
    fun Bar.doStuff() {}

    fun baz(bar: Bar) {
        bar.apply(Bar::doStuff) //doesn't work
    }
}

What about referencing doStuff from outside the class? You'd need to bind one of the two receivers or maybe even both.

And what about partially bound references. Can I bind only Foo to receive a (Bar) -> Unit (or Bar.() -> Unit?) Or maybe the other way round?

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r May 9, 2016

@udalov is there a timeframe for this feature (a rough one at least), do you plan to do some other things first, etc. I'd (and hopefully I'm not the only one) just like to know how long it's going to take.

b005t3r commented May 9, 2016

@udalov is there a timeframe for this feature (a rough one at least), do you plan to do some other things first, etc. I'd (and hopefully I'm not the only one) just like to know how long it's going to take.

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov May 10, 2016

Contributor

@b005t3r

is there a timeframe for this feature (a rough one at least), do you plan to do some other things first, etc. I'd (and hopefully I'm not the only one) just like to know how long it's going to take.

The prototype is being worked on currently and the larger parts are mostly done. Once everything is implemented according to the proposal, we'll start an early access preview for this feature as a part of Kotlin 1.1. There's no date set for the 1.1 release; it will be out when this and other planned features are fully implemented. The first EAP is coming soon (the main development branch is already 1.1).

Contributor

udalov commented May 10, 2016

@b005t3r

is there a timeframe for this feature (a rough one at least), do you plan to do some other things first, etc. I'd (and hopefully I'm not the only one) just like to know how long it's going to take.

The prototype is being worked on currently and the larger parts are mostly done. Once everything is implemented according to the proposal, we'll start an early access preview for this feature as a part of Kotlin 1.1. There's no date set for the 1.1 release; it will be out when this and other planned features are fully implemented. The first EAP is coming soon (the main development branch is already 1.1).

@abreslav

This comment has been minimized.

Show comment
Hide comment
@abreslav

abreslav May 10, 2016

Contributor

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

@cypressious I'm not sure if we have taken care of it already, but I believe we should. It's an important use case. // cc @udalov

Contributor

abreslav commented May 10, 2016

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

@cypressious I'm not sure if we have taken care of it already, but I believe we should. It's an important use case. // cc @udalov

@ilya-g

This comment has been minimized.

Show comment
Hide comment
@ilya-g

ilya-g May 10, 2016

Member

Information about the original receiver type is absent in the type of a bound reference

Maybe bound reference should be of some subtype of () -> T, which carries the type of the receiver? Thus it can both be assignable to () -> T and have a type-safe unbind: fun unbind(): R.() -> T

Member

ilya-g commented May 10, 2016

Information about the original receiver type is absent in the type of a bound reference

Maybe bound reference should be of some subtype of () -> T, which carries the type of the receiver? Thus it can both be assignable to () -> T and have a type-safe unbind: fun unbind(): R.() -> T

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov May 11, 2016

Contributor

@b005t3r

Sorry for the long delay. Re your comment,

Hmm, I think this would be really limiting and lead to unnecessary tricks (to get the non-nullable reference) in many normally valid scenarios. Like instead of being able to do this:
event += obj?::handler
one would have to do something like this instead (not sure if this is a totally legit code, but I think you get my point):
obj?.let { event += it::handler }

If by a?::b you mean "the reference to b if a is non-null and null otherwise", then I think these two examples have different semantics. In the first one, you're adding a null handler to the event if obj is null, whereas in the second one nothing happens. So, what you've probably meant in the first case was obj?::handler?.let { event += it } and that doesn't look prettier than the second case.

Overall, we've discussed this and decided that unless there are obvious reasons why it should be supported immediately, it can be postponed. Fun fact, to support it in the future without breaking source compatibility we must reserve the a?::b syntax in case when a is an object or a companion object, i.e. report an error in that case. I'll explain this in the proposal.

Contributor

udalov commented May 11, 2016

@b005t3r

Sorry for the long delay. Re your comment,

Hmm, I think this would be really limiting and lead to unnecessary tricks (to get the non-nullable reference) in many normally valid scenarios. Like instead of being able to do this:
event += obj?::handler
one would have to do something like this instead (not sure if this is a totally legit code, but I think you get my point):
obj?.let { event += it::handler }

If by a?::b you mean "the reference to b if a is non-null and null otherwise", then I think these two examples have different semantics. In the first one, you're adding a null handler to the event if obj is null, whereas in the second one nothing happens. So, what you've probably meant in the first case was obj?::handler?.let { event += it } and that doesn't look prettier than the second case.

Overall, we've discussed this and decided that unless there are obvious reasons why it should be supported immediately, it can be postponed. Fun fact, to support it in the future without breaking source compatibility we must reserve the a?::b syntax in case when a is an object or a companion object, i.e. report an error in that case. I'll explain this in the proposal.

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r May 11, 2016

Yes, they do have different effect. I took a shortcut with my example, with the event += a?::b I meant null could be ignored in the operator's implementation.

But I'm not sure how is this going to work with nullable references without the ? - it won't be possible at all?

b005t3r commented May 11, 2016

Yes, they do have different effect. I took a shortcut with my example, with the event += a?::b I meant null could be ignored in the operator's implementation.

But I'm not sure how is this going to work with nullable references without the ? - it won't be possible at all?

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov May 11, 2016

Contributor

But I'm not sure how is this going to work with nullable references without the ? - it won't be possible at all?

No, it won't be possible, see the end of the Resolution section:

It is an error if the LHS expression has nullable type and the resolved member is not an extension to nullable type.

So the only possible way would be the one you proposed or, if the operator ignores nulls, event += obj?.let { it::handler }.

Contributor

udalov commented May 11, 2016

But I'm not sure how is this going to work with nullable references without the ? - it won't be possible at all?

No, it won't be possible, see the end of the Resolution section:

It is an error if the LHS expression has nullable type and the resolved member is not an extension to nullable type.

So the only possible way would be the one you proposed or, if the operator ignores nulls, event += obj?.let { it::handler }.

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov May 11, 2016

Contributor

@damianw

I noticed that re-introducing type inference for an empty LHS isn't listed as a "possible future advancement". Has this been considered? i.e.:

val foo = listOf<String>()
foo.filter(::isEmpty) // String::isEmpty is inferred for arguments of type String.() -> Boolean or (String) -> Boolean

Thanks. This could be implemented, but with some care not to break source compatibility. I'll explain in the proposal.

Contributor

udalov commented May 11, 2016

@damianw

I noticed that re-introducing type inference for an empty LHS isn't listed as a "possible future advancement". Has this been considered? i.e.:

val foo = listOf<String>()
foo.filter(::isEmpty) // String::isEmpty is inferred for arguments of type String.() -> Boolean or (String) -> Boolean

Thanks. This could be implemented, but with some care not to break source compatibility. I'll explain in the proposal.

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov May 11, 2016

Contributor

@MichaelRocks @abreslav

please correct me if I'm wrong, but I think that equals() is supposed to work correctly on bound references.

This is an interesting question. Technically this is possible, but doing so (and mentioning this in the spec) seemingly will not allow us to use MethodHandles in the future for method references in the same way as JVM 8+ does without breaking users' code, because there's no equality defined on MethodHandles. Not sure if we should be concerned about this though. On the other hand, equality already works for unbound callable references and breaking it should also be avoided (however, it is not specified anywhere and may be considered undefined behavior).

According to Brian Goetz, equality for MethodHandles is (or was) being considered, but I couldn't find any relevant discussions about that:

One area where it might be practical to tweak the definition of equality is with method references, because this would enable them to be used as listeners and be properly unregistered. This is under consideration.

This probably needs more investigation.

Some relevant SO questions: http://stackoverflow.com/q/28190304/288456, http://stackoverflow.com/q/23983832/288456

Contributor

udalov commented May 11, 2016

@MichaelRocks @abreslav

please correct me if I'm wrong, but I think that equals() is supposed to work correctly on bound references.

This is an interesting question. Technically this is possible, but doing so (and mentioning this in the spec) seemingly will not allow us to use MethodHandles in the future for method references in the same way as JVM 8+ does without breaking users' code, because there's no equality defined on MethodHandles. Not sure if we should be concerned about this though. On the other hand, equality already works for unbound callable references and breaking it should also be avoided (however, it is not specified anywhere and may be considered undefined behavior).

According to Brian Goetz, equality for MethodHandles is (or was) being considered, but I couldn't find any relevant discussions about that:

One area where it might be practical to tweak the definition of equality is with method references, because this would enable them to be used as listeners and be properly unregistered. This is under consideration.

This probably needs more investigation.

Some relevant SO questions: http://stackoverflow.com/q/28190304/288456, http://stackoverflow.com/q/23983832/288456

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r May 11, 2016

One area where it might be practical to tweak the definition of equality is with method references, because this would enable them to be used as listeners and be properly unregistered. This is under consideration.

This probably needs more investigation.

BTW, this is exactly what my example was trying to show (apart from using a nullable reference) and it's what I need this functionality for. To be honest, with no way of comparing two references their usage seems pretty limited, up to a point where they are, well, useless.

I understand that not all references can be compared (e.g. it's impossible for two lambdas), but simply checking if they both were created using the same instance and they reference the same method does not sound impossible, does it?

b005t3r commented May 11, 2016

One area where it might be practical to tweak the definition of equality is with method references, because this would enable them to be used as listeners and be properly unregistered. This is under consideration.

This probably needs more investigation.

BTW, this is exactly what my example was trying to show (apart from using a nullable reference) and it's what I need this functionality for. To be honest, with no way of comparing two references their usage seems pretty limited, up to a point where they are, well, useless.

I understand that not all references can be compared (e.g. it's impossible for two lambdas), but simply checking if they both were created using the same instance and they reference the same method does not sound impossible, does it?

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov May 11, 2016

Contributor

@cypressious

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

Yes.

Will it be possible to reference extension methods that are instance methods?

Seems possible, we'll look into it.

What about referencing doStuff from outside the class? You'd need to bind one of the two receivers or maybe even both.

And what about partially bound references. Can I bind only Foo to receive a (Bar) -> Unit (or Bar.() -> Unit?) Or maybe the other way round?

I think the situation here is similar to the normal call of a member extension from the outside, which you can't make without having at least one of the receivers brought implicitly into the context (by with, or an extension function receiver, or being in a subclass, ...). If we're allowing it, we should make it as close to the normal function calls as possible. So, only implicit receivers which are available in the context could be "implicitly bound" by the member extension reference. I think that would make the most sense.

I'll add relevant parts to the proposal, thanks.

Contributor

udalov commented May 11, 2016

@cypressious

Will inlining be supported for bound callable references? Example: list.map(mapper::map)

Yes.

Will it be possible to reference extension methods that are instance methods?

Seems possible, we'll look into it.

What about referencing doStuff from outside the class? You'd need to bind one of the two receivers or maybe even both.

And what about partially bound references. Can I bind only Foo to receive a (Bar) -> Unit (or Bar.() -> Unit?) Or maybe the other way round?

I think the situation here is similar to the normal call of a member extension from the outside, which you can't make without having at least one of the receivers brought implicitly into the context (by with, or an extension function receiver, or being in a subclass, ...). If we're allowing it, we should make it as close to the normal function calls as possible. So, only implicit receivers which are available in the context could be "implicitly bound" by the member extension reference. I think that would make the most sense.

I'll add relevant parts to the proposal, thanks.

udalov added a commit that referenced this issue May 11, 2016

udalov added a commit that referenced this issue May 11, 2016

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov May 12, 2016

Contributor

@ilya-g

Maybe bound reference should be of some subtype of () -> T, which carries the type of the receiver? Thus it can both be assignable to () -> T and have a type-safe unbind: fun unbind(): R.() -> T

This might work. I was reluctant of thinking in that direction because this function would have a different signature for each function type, so it would only be possible to synthesize all of them in the compiler, not declare them in the code. Even now a function reference has an imaginary (non-existing at runtime) KFunctionN<P1, ..., PN, R> type. Bound function reference can then have a KBoundFunctionN<T, P1, ..., PN, R>, inheriting from KFunctionN and providing the magic unbind function. I think this is rather mysterious, but looks like it's the only option to support unbind.

Contributor

udalov commented May 12, 2016

@ilya-g

Maybe bound reference should be of some subtype of () -> T, which carries the type of the receiver? Thus it can both be assignable to () -> T and have a type-safe unbind: fun unbind(): R.() -> T

This might work. I was reluctant of thinking in that direction because this function would have a different signature for each function type, so it would only be possible to synthesize all of them in the compiler, not declare them in the code. Even now a function reference has an imaginary (non-existing at runtime) KFunctionN<P1, ..., PN, R> type. Bound function reference can then have a KBoundFunctionN<T, P1, ..., PN, R>, inheriting from KFunctionN and providing the magic unbind function. I think this is rather mysterious, but looks like it's the only option to support unbind.

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious May 12, 2016

Contributor

@b005t3r

To be honest, with no way of comparing two references their usage seems pretty limited, up to a point where they are, well, useless.

That's a rather extreme opinion and I do not agree. There are plenty of usecases apart from events and listeners.

Contributor

cypressious commented May 12, 2016

@b005t3r

To be honest, with no way of comparing two references their usage seems pretty limited, up to a point where they are, well, useless.

That's a rather extreme opinion and I do not agree. There are plenty of usecases apart from events and listeners.

@b005t3r

This comment has been minimized.

Show comment
Hide comment
@b005t3r

b005t3r May 12, 2016

That's a rather extreme opinion and I do not agree. There are plenty of usecases apart from events and listeners.

I was not taking about listeners only, but about being able to aggregate references, for which you need to be able to distinguish between them (listeners are a subset of this). Without this I can't really imagine a scenario where you need a bound reference and for some reason you can't use a lambda.

b005t3r commented May 12, 2016

That's a rather extreme opinion and I do not agree. There are plenty of usecases apart from events and listeners.

I was not taking about listeners only, but about being able to aggregate references, for which you need to be able to distinguish between them (listeners are a subset of this). Without this I can't really imagine a scenario where you need a bound reference and for some reason you can't use a lambda.

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious May 12, 2016

Contributor

It's not only about whether you can use a lambda but also about which form is more readable.

Contributor

cypressious commented May 12, 2016

It's not only about whether you can use a lambda but also about which form is more readable.

@bashor bashor added the language label Jun 6, 2016

@Groostav

This comment has been minimized.

Show comment
Hide comment
@Groostav

Groostav Jun 10, 2016

How much of this is available under reflection?

according to the spec:

A KFunction instance for a bound callable reference has no receiver parameter. Its parameters property doesn't have this parameter and the argument should not be passed to call

does that mean I can potentially implement a mocking system with

fun aMethodCall(a : Int, b : Float) : String = //...

on(someObject::aMethodCall) then { doStuff(); }

with

fun <A, B, C> on(callToStub : KFunction2<A, B, C>) : OngoingStubbing<C>{
  if(callToStub !is MethodReference) throw NFG(); 
    //in other words on { a, b -> someObject.aMethodCall(a, b) } wouldn't be legal because the invoked method cannot be reflected-out. 
    //but someObject::aMethodCall would be, asuming 'aMethodCall' as a string or Method object is preserved. 
  val boundReciver : Object = callToStub.getThis();
  val methodCall : KMethod = callToStub.getTargetMethod();
  //...
  val proxy = doDynamicProxyWork(boundReciever, boundReciever::class.java, methodCall.getJavaMethod())

  return OngoingStubbing(proxy);
}

--with an overload for each arity up to whatever's reasonable (I think kotlin.Function stops at 22, and microsofts Action stops at 14, so it'l be one of those two)
?

It sounds like the getThis() wont be a problem, but getTargetMethod() might be.

With Java 8, Method calling writeReplace on a Method reference that was created as a Runnable & Serializable or whatever, will give you back an object with all of the relevant information.

alternatively maybe you guys can just hand over the AST a le C# & Linq.Expressions. 💃

Groostav commented Jun 10, 2016

How much of this is available under reflection?

according to the spec:

A KFunction instance for a bound callable reference has no receiver parameter. Its parameters property doesn't have this parameter and the argument should not be passed to call

does that mean I can potentially implement a mocking system with

fun aMethodCall(a : Int, b : Float) : String = //...

on(someObject::aMethodCall) then { doStuff(); }

with

fun <A, B, C> on(callToStub : KFunction2<A, B, C>) : OngoingStubbing<C>{
  if(callToStub !is MethodReference) throw NFG(); 
    //in other words on { a, b -> someObject.aMethodCall(a, b) } wouldn't be legal because the invoked method cannot be reflected-out. 
    //but someObject::aMethodCall would be, asuming 'aMethodCall' as a string or Method object is preserved. 
  val boundReciver : Object = callToStub.getThis();
  val methodCall : KMethod = callToStub.getTargetMethod();
  //...
  val proxy = doDynamicProxyWork(boundReciever, boundReciever::class.java, methodCall.getJavaMethod())

  return OngoingStubbing(proxy);
}

--with an overload for each arity up to whatever's reasonable (I think kotlin.Function stops at 22, and microsofts Action stops at 14, so it'l be one of those two)
?

It sounds like the getThis() wont be a problem, but getTargetMethod() might be.

With Java 8, Method calling writeReplace on a Method reference that was created as a Runnable & Serializable or whatever, will give you back an object with all of the relevant information.

alternatively maybe you guys can just hand over the AST a le C# & Linq.Expressions. 💃

@ebceu4

This comment has been minimized.

Show comment
Hide comment
@ebceu4

ebceu4 Jun 22, 2016

I'm not sure i've got it, but why it's not possible to reference functions as first-class citizens,
like we do in C#?

val action : () -> Unit = Foo().bar; action();

ebceu4 commented Jun 22, 2016

I'm not sure i've got it, but why it's not possible to reference functions as first-class citizens,
like we do in C#?

val action : () -> Unit = Foo().bar; action();

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Jun 22, 2016

Contributor

@ebceu4

I'm not sure i've got it, but why it's not possible to reference functions as first-class citizens,
like we do in C#?

Do you mean the syntax? If yes, then this is because there are also properties, and references to them cannot use that syntax because it would be indistinguishable from a call.

Contributor

udalov commented Jun 22, 2016

@ebceu4

I'm not sure i've got it, but why it's not possible to reference functions as first-class citizens,
like we do in C#?

Do you mean the syntax? If yes, then this is because there are also properties, and references to them cannot use that syntax because it would be indistinguishable from a call.

@TheRaspPie

This comment has been minimized.

Show comment
Hide comment
@TheRaspPie

TheRaspPie Jun 22, 2016

@udalov Right, but that is specific only to properties. I also don't really get why there is a distinction between functions and methods. Is it because the JVM has no support for this, so you wanted to make the fact explicit that you have to create an anonymous inner class and an instance? But this doesn't apply to the JavaScript runtime.

@udalov Right, but that is specific only to properties. I also don't really get why there is a distinction between functions and methods. Is it because the JVM has no support for this, so you wanted to make the fact explicit that you have to create an anonymous inner class and an instance? But this doesn't apply to the JavaScript runtime.

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Jun 22, 2016

Contributor

@TheRaspPie

Right, but that is specific only to properties.

I was implying that we wanted to have the same syntax for function and property references. We also wanted to have the same syntax as Java's method references. That is how we ended up with the double colon operator.

I also don't really get why there is a distinction between functions and methods. Is it because the JVM has no support for this, so you wanted to make the fact explicit that you have to create an anonymous inner class and an instance?

The official term in Kotlin is and has always been "function" and it's used everywhere in the code and in the docs. Colloquially, of course we sometimes call functions "methods" because this is what they're called on JVM. But I'm not sure how is this topic important and relevant to this proposal?

Contributor

udalov commented Jun 22, 2016

@TheRaspPie

Right, but that is specific only to properties.

I was implying that we wanted to have the same syntax for function and property references. We also wanted to have the same syntax as Java's method references. That is how we ended up with the double colon operator.

I also don't really get why there is a distinction between functions and methods. Is it because the JVM has no support for this, so you wanted to make the fact explicit that you have to create an anonymous inner class and an instance?

The official term in Kotlin is and has always been "function" and it's used everywhere in the code and in the docs. Colloquially, of course we sometimes call functions "methods" because this is what they're called on JVM. But I'm not sure how is this topic important and relevant to this proposal?

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Jul 5, 2016

Contributor

@cypressious

Additionally, I would like to point out a possible clash with the KCallable.name intrinsic (actually, that's not specific to properties, it also works for function references). Maybe it will work out of the box, maybe not. Just something to keep in mind.

Thanks, apparently this didn't work out of the box and I've created an issue.

Contributor

udalov commented Jul 5, 2016

@cypressious

Additionally, I would like to point out a possible clash with the KCallable.name intrinsic (actually, that's not specific to properties, it also works for function references). Maybe it will work out of the box, maybe not. Just something to keep in mind.

Thanks, apparently this didn't work out of the box and I've created an issue.

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Jul 5, 2016

Contributor

What's the state of empty LHS? Has a decision been made or is this intentionally left undecided for later?

Contributor

cypressious commented Jul 5, 2016

What's the state of empty LHS? Has a decision been made or is this intentionally left undecided for later?

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Jul 5, 2016

Contributor

@cypressious

What's the state of empty LHS? Has a decision been made or is this intentionally left undecided for later?

No decision yet. If there is one, it will be reflected in the proposal.

Contributor

udalov commented Jul 5, 2016

@cypressious

What's the state of empty LHS? Has a decision been made or is this intentionally left undecided for later?

No decision yet. If there is one, it will be reflected in the proposal.

K0zka added a commit to K0zka/kotlin that referenced this issue Jul 12, 2016

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Jul 22, 2016

Contributor

Is there something we can do about type mismatch because (T) -> Unit was expected but (T) -> Something was provided?

This makes it hard to use builder-style methods or something like foo(mutableList::remove) where values are returned that are ignored most of the time anyway.

Contributor

cypressious commented Jul 22, 2016

Is there something we can do about type mismatch because (T) -> Unit was expected but (T) -> Something was provided?

This makes it hard to use builder-style methods or something like foo(mutableList::remove) where values are returned that are ignored most of the time anyway.

@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Jul 22, 2016

Contributor

@cypressious

Is there something we can do about type mismatch because (T) -> Unit was expected but (T) -> Something was provided?

Please vote for KT-11723 and share your use case there. Thanks!

Contributor

udalov commented Jul 22, 2016

@cypressious

Is there something we can do about type mismatch because (T) -> Unit was expected but (T) -> Something was provided?

Please vote for KT-11723 and share your use case there. Thanks!

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Jul 25, 2016

Contributor

Has there been some progress on instance extension methods? It seems counterintuitive, that you can do foo::doStuff with

class Foo {
    fun doStuff(bar: Bar) {
    }
}

but can't with

class Foo {
    fun Bar.doStuff() {
    }
}

even though they are virtually the same.

Contributor

cypressious commented Jul 25, 2016

Has there been some progress on instance extension methods? It seems counterintuitive, that you can do foo::doStuff with

class Foo {
    fun doStuff(bar: Bar) {
    }
}

but can't with

class Foo {
    fun Bar.doStuff() {
    }
}

even though they are virtually the same.

@TheRaspPie

This comment has been minimized.

Show comment
Hide comment
@TheRaspPie

TheRaspPie Jul 25, 2016

@udalov Sorry for the delay.

The official term in Kotlin is and has always been "function" and it's used everywhere in the code and in the docs. Colloquially, of course we sometimes call functions "methods" because this is what they're called on JVM. But I'm not sure how is this topic important and relevant to this proposal?

The idea is that
fun test1(): Int = 10 would be just syntatic sugar for
val test2: () -> Int = { 10 }

The result of differentiating between function and anonymous function is that you need to write ::test1 to get the same as test2.
The fact that the JVM only knows named methods like C leaks through this abstraction.
This however, does not apply to the JavaScript backend, where everything is a closure.

To answer your question: This topic is only relevant indirectly to this proposal, because it applies to the overall concept of method references which this issue is a part of.

The argument that you want the same notation as in Java is a valid, although inconsistent argument. Then you'd also need the same syntax for constructors, variable declaration etc.

@udalov Sorry for the delay.

The official term in Kotlin is and has always been "function" and it's used everywhere in the code and in the docs. Colloquially, of course we sometimes call functions "methods" because this is what they're called on JVM. But I'm not sure how is this topic important and relevant to this proposal?

The idea is that
fun test1(): Int = 10 would be just syntatic sugar for
val test2: () -> Int = { 10 }

The result of differentiating between function and anonymous function is that you need to write ::test1 to get the same as test2.
The fact that the JVM only knows named methods like C leaks through this abstraction.
This however, does not apply to the JavaScript backend, where everything is a closure.

To answer your question: This topic is only relevant indirectly to this proposal, because it applies to the overall concept of method references which this issue is a part of.

The argument that you want the same notation as in Java is a valid, although inconsistent argument. Then you'd also need the same syntax for constructors, variable declaration etc.

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Aug 6, 2016

Contributor

References to operators are generally unnecessary because you can always use the desugared methods but the comparison operators equals and compareTo are a bit special because they have special logic (null checks for == and intepreting the Int result to a Boolean for < and friends). Would it be possible to provide a way to reference these operators directly?

Use case:

listOfFoos.all(fooProvider.computeFoo()::==)
listOfMeasurements.any(THRESHOLD::<)
Contributor

cypressious commented Aug 6, 2016

References to operators are generally unnecessary because you can always use the desugared methods but the comparison operators equals and compareTo are a bit special because they have special logic (null checks for == and intepreting the Int result to a Boolean for < and friends). Would it be possible to provide a way to reference these operators directly?

Use case:

listOfFoos.all(fooProvider.computeFoo()::==)
listOfMeasurements.any(THRESHOLD::<)
@ilya-g

This comment has been minimized.

Show comment
Hide comment
@ilya-g

ilya-g Aug 8, 2016

Member
listOfFoos.all(fooProvider.computeFoo()::==)
listOfMeasurements.any(THRESHOLD::<)

@cypressious why callable references should be used here? Doesn't plain lambdas fit this case more?
I don't think that special logic operators could be easily represented with callable references, e.g. what member in particular they should reference to?

Member

ilya-g commented Aug 8, 2016

listOfFoos.all(fooProvider.computeFoo()::==)
listOfMeasurements.any(THRESHOLD::<)

@cypressious why callable references should be used here? Doesn't plain lambdas fit this case more?
I don't think that special logic operators could be easily represented with callable references, e.g. what member in particular they should reference to?

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Aug 8, 2016

Contributor

@ilya-g

why callable references should be used here? Doesn't plain lambdas fit this case more?

One argument is that the lhs of the bound callable reference is cached but more arguments were discussed in this thread.

I don't think that special logic operators could be easily represented with callable references, e.g. what member in particular they should reference to?

I was thinking about this question and one possibility I see is to provide @InlineOnly methods in the runtime which actually implement the behaviour instead of the magic that the compiler does now. In this case, having the operator reference feature wouldn't even be necessary anymore since you could reference the methods (just like you can do now with plus instead of +)

Contributor

cypressious commented Aug 8, 2016

@ilya-g

why callable references should be used here? Doesn't plain lambdas fit this case more?

One argument is that the lhs of the bound callable reference is cached but more arguments were discussed in this thread.

I don't think that special logic operators could be easily represented with callable references, e.g. what member in particular they should reference to?

I was thinking about this question and one possibility I see is to provide @InlineOnly methods in the runtime which actually implement the behaviour instead of the magic that the compiler does now. In this case, having the operator reference feature wouldn't even be necessary anymore since you could reference the methods (just like you can do now with plus instead of +)

udalov added a commit that referenced this issue Aug 10, 2016

Update LHS resolution algorithm with a special case for objects (#5)
Reword the whole algorithm, to (hopefully) improve readability

udalov added a commit that referenced this issue Aug 10, 2016

udalov added a commit that referenced this issue Aug 15, 2016

Merge pull request #43
Reword callable reference and class literal resolution algorithm, explain some corner cases (#5)
@udalov

This comment has been minimized.

Show comment
Hide comment
@udalov

udalov Aug 15, 2016

Contributor

I've reworded the part about resolution of left-hand side in the proposal and added some corner case examples and their solutions, everyone is welcome to review.

Contributor

udalov commented Aug 15, 2016

I've reworded the part about resolution of left-hand side in the proposal and added some corner case examples and their solutions, everyone is welcome to review.

@punksta

This comment has been minimized.

Show comment
Hide comment
@punksta

punksta Sep 3, 2016

What about reference to constructors?

rx.Single.zip(getName(), getPhotos(), Pair<String, List<String>>::new)

punksta commented Sep 3, 2016

What about reference to constructors?

rx.Single.zip(getName(), getPhotos(), Pair<String, List<String>>::new)
@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Sep 3, 2016

Contributor

@punksta ::Pair

Contributor

cypressious commented Sep 3, 2016

@punksta ::Pair

@abreslav abreslav modified the milestone: 1.1 Oct 10, 2016

udalov added a commit that referenced this issue Dec 21, 2016

Merge pull request #62 from Kotlin/udalov
Update type checking rules of bound class literals (#5)
@weakish

This comment has been minimized.

Show comment
Hide comment
@weakish

weakish Dec 23, 2016

@udalov

@ebceu4

why it's not possible to reference functions as first-class citizens,

Do you mean the syntax? If yes, then this is because there are also properties, and references to them cannot use that syntax because it would be indistinguishable from a call.

Why would it be indistinguisheable?
What about just disallow propertiy and the function to have the same name?

Currently Kotlin allows propertiy and function with the same name:

val i: Int = 2
fun i(): Int {
    return 2
}
print(i)
print(i())

Diallowing this (I cannot imgae when this will be useful) so we can just refer a function with its name.

weakish commented Dec 23, 2016

@udalov

@ebceu4

why it's not possible to reference functions as first-class citizens,

Do you mean the syntax? If yes, then this is because there are also properties, and references to them cannot use that syntax because it would be indistinguishable from a call.

Why would it be indistinguisheable?
What about just disallow propertiy and the function to have the same name?

Currently Kotlin allows propertiy and function with the same name:

val i: Int = 2
fun i(): Int {
    return 2
}
print(i)
print(i())

Diallowing this (I cannot imgae when this will be useful) so we can just refer a function with its name.

@cypressious

This comment has been minimized.

Show comment
Hide comment
@cypressious

cypressious Dec 23, 2016

Contributor

@weakish That would break backward compatibility and is out of the question.

Contributor

cypressious commented Dec 23, 2016

@weakish That would break backward compatibility and is out of the question.

@weakish

This comment has been minimized.

Show comment
Hide comment
@weakish

weakish Dec 23, 2016

@cypressious I think few code are written with same name property and function. Besides, current Kotlin cannot resolve it correctly with :: syntax:

class KK() {
  val i = 1
  fun i(): Int { return 1 }
}
fun hkk(kk: KK, f: (KK) -> Int) {
  println(f(kk))
}
fun main(args: Array<String>) {
  hkk(KK(), KK::i)
}

Compiled with kotlinc 1.0.3:

error: overload resolution ambiguity:
public final val i: Int defined in KK
public final fun i(): Int defined in KK
  hkk(KK(), KK::i)
                ^

So:

  • Disallowing property and function to have the same name will not break backward compatibility in most cases.
  • To maintain backward compatibility with current function reference syntax, just make Foo::bar a (deprecated) alias for Foo.bar.

weakish commented Dec 23, 2016

@cypressious I think few code are written with same name property and function. Besides, current Kotlin cannot resolve it correctly with :: syntax:

class KK() {
  val i = 1
  fun i(): Int { return 1 }
}
fun hkk(kk: KK, f: (KK) -> Int) {
  println(f(kk))
}
fun main(args: Array<String>) {
  hkk(KK(), KK::i)
}

Compiled with kotlinc 1.0.3:

error: overload resolution ambiguity:
public final val i: Int defined in KK
public final fun i(): Int defined in KK
  hkk(KK(), KK::i)
                ^

So:

  • Disallowing property and function to have the same name will not break backward compatibility in most cases.
  • To maintain backward compatibility with current function reference syntax, just make Foo::bar a (deprecated) alias for Foo.bar.

@abreslav abreslav moved this from KEEP accepted to Implemented in KEEP Kotlin 1.1 Jan 27, 2017

@dzharkov

This comment has been minimized.

Show comment
Hide comment
@dzharkov

dzharkov Feb 14, 2018

Collaborator

The feature has been released in Kotlin 1.1, thus I'm closing the KEEP. Do not hesitate to open a YouTrack issue for any additional suggestions

Collaborator

dzharkov commented Feb 14, 2018

The feature has been released in Kotlin 1.1, thus I'm closing the KEEP. Do not hesitate to open a YouTrack issue for any additional suggestions

@dzharkov dzharkov closed this Feb 14, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment