-
Notifications
You must be signed in to change notification settings - Fork 362
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compile-time Extension Interfaces #87
Conversation
The proposed syntax for companion constrains looks a bit confusing. Take a look at this example:
It is not perfectly clear from this declaration what is the receiver type of I was thinking a bit on how to fit companion-extension syntax in a nicer way and came up with the following alternative syntactic approach. Let's eschew the idea that
Here, in the context of
In this approach Unfortunately, this proposal also opens the whole can of worms on companion-vs-static debate. One of the issues being the fact, that not all classes in Kotlin (and even in Kotlin stdlib) have companions and there is currently no way to declare a companion outside of the class. Moreover, it is hardly possible to fix or to work around due to dynamic nature of JVM. One solution to this conundrum is to stop piggybacking onto companion objects at all. Let us, instead, declare functions on the typeclass itself, like this:
In this approach
The interesting thought direction here is that this kind of extension mechanism can work as a replacement for companion objects altogether. We might just deprecate companions, because any declaration of the form:
can be replaced, for the same effect, with
or, which is not possible with companion objects, if the access to private members of
It is cleaner and more modular, since you are not limited to one companion object anymore, but can define as many as you want. However, with this alternative-to-companion approach to typeclasses we have to work out on how the typeclass instances are going to be named and what is the syntactic approach to explicitly specify this name (this is, at least, needed for JVM interop). We can also try to somehow reuse |
@elizarov makes perfect sense. I'll update the proposal to the new proposed syntax |
proposals/type-classes.md
Outdated
} | ||
``` | ||
|
||
Some of this examples where originally proposed by Roman Elizarov and the Kategory contributors where these features where originally discussed https://kotlinlang.slack.com/archives/C1JMF6UDV/p1506897887000023 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small note seeing this link, you may want to copy/paste the snippets if you believe it provides important context because the Kotlin slack is free and only has the most recent 10k messages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I realise all this stuff comes from Haskell and people who love category theory, but I strongly suggest that to gain better acceptance you drop the conventional FP names for things and use more appropriate names that are less jargony and more Java-like: Monoid -> Combinable etc |
@mikehearn those are just examples and not part of the proposal. Anyone is free to name their own typeclasses whatever they want and this proposal does not address what names users will choose nor it proposes new types for the stdlib. |
How type class implementations ( Elaborating on the example with
and, for example,
and
Now, in package |
@dnpetrov an import matching the contained instances would bring them into scope: import org.fizzbuzz.* In the case where you have two instances that are conflicting the compiler should bail because they are ambiguous and the user needs to disambiguate them. import org.fizzbuzz.*
import org.acme.*
val x: Int = 1.combine(2) // // Ambiguous instances for Monoid: Int. Please redefine`extension Monoid: Int` in the current scope or use tagged instances if they provide different behavior. A natural way for disambiguation may be redefining the instance in the current scope which would take precedence over the imported one.
Another approach would be to place further constrains in the instances like is done today with the
|
Let's assume the example by @dnpetrov
It is pretty straightforward with star imports:
To disambiguate extension in the case of start import from the other package, I suggest to use the simple name of the class that extension is being defined for, like this:
This will bring in scope all This approach has a nice and quite well-intended side-effect that if a class and its extension are both defined in the same package, then a qualified import of a class automatically brings into scope all the extensions defined for this class in the same package. We can even go as far as to forbid redefining class extensions in other packages if they are defined in the original package. This will improve predictability and is quite consistent with a class member vs extension resolution in Kotlin (class member, being defined by the class author, always takes precedence). However, if there is no extension in the class's original package, then different clients can define and/or import different extension instances for this class with the same typeclass. Also note, that a generic method with typeclass upper-bounds should have an implict instance of typeclass in scope behind the scenes (actually passed to it by invoker) and this instance should always take precedence. |
The interesting side-effect of this proposal is that we can almost deprecate
Now a function that was doing something like:
can be replaced with:
This analogy might help to understand how typeclasses shall behave in various scenarios. In particular, generic classes with typeclass upper bounds shall be supported if and only if we roll out support of generic classes with reified type parameters. |
@elizarov that will simplify most of our code a ton since we are currently relying on |
Updated the proposal with a section on overcoming some of the limitations of |
What if I have multiple type class implementations for the same type in the same package?
I'd expect extensions to have names or something. Implicit imports look somewhat fragile. You don't know what else you bring into the scope. Maybe even just follow familiar class inheritance syntax:
Then |
With parametric type classes, type class dependent functions might need some different syntax, like:
|
@dnpetrov As for the syntax makes sense. That syntax is similar to what I originally proposed as well since it is also necessary to implement the fact that certain type classes can't target a single type but target the combinations of many. For example a natural transformation to go from one type to another: typeclass FunctionK<F<_>, G<_>> {
fun <A> invoke(fa: F<A>): G<A>
}
extension Option2List : FunctionK<Option, List> {
fun <A> invoke(fa: Option<A>): List<A> =
fa.fold({ emptyList() }, { listOf(it) })
} In the case above the instance does not target neither @dnpetrov In your example above an alternative way to express fun <T> combineWith(others: Iterable<T>, instance MT : Monoid<T>): T = ... That would allow call sites to replace the instance by explicitly providing one if needed so the method could be invoked in a similar way as:
That also is an alternative way to handle ambiguity when collisions exists beside using concrete imports. |
The latest syntactic proposal by @dnpetrov for typeclass declaration is indeed more general and is easier to adapt to to the can of typeclasses binding multiple types. However, I don't like the need to name typeclasses and more so the need to declare and name typeclass instances in function signature. It is just a boilerplate. IMHO, it defeats the whole purpose of having type-classes as a language feature, since you can already do all that. At this point I would recommend to (re)read Cedric's blog post on ad-hoc polymorphism. In our example, we can already write just:
then write
then declare functions with an explicit type class instance:
It is even shorter than the latest syntactic proposal for typeclasses, since I don't have to add IMHO, the only valuable benefit of having type classes directly supported in the language is making them easier to grasp for beginners by capitalizing on the mental model of interfaces on steroids (aka Swift protocols), and eschewing as much boilerplate as we can in the process. Having to name your type class instance is just a boilerplate, having to pass your type-class instances explicitly is just a boilerplate. Neither should be required. |
A nitpick but in the examples, I think you should be more specific about the name of the |
@elizarov Kategory uses interfaces for typeclasses now and does it like that. The issue is that:
Requires explicitly passing the instance manually which defeats the purpose of type classes. Having a keyword to require the compiler to verify that call sites such as I agree though that the variant based on extensions is much easier to grasp for beginners. We can perhaps encode all complex cases where typeclasses require multiple types if typeclasses allowed for type args:
And in a polymorphic context:
|
Having played more with @dnpetrov syntactic proposal I start to like certain parts of it. First of all, the generic-like declaration for interface indeed looks more concise than my original proposal with the Moreover, instead of a having to reserve a new
This syntax also scales to multiple types (and, maybe HKTs, but I will not cover them here). For example, you can have:
However, as soon as you allow applying this syntax to multiple types, the story with "standalone" (static? but not really) functions like In order to overcome this difficulty, we can use some fresh syntactic construct to bind the type to its typeclass. Look at how the dot (
In this context, Note, that we can allow to add additional types in Similarly, declaration of typeclass instances can ready follow the model of Kotlin's
Since we have indicated that Alas, this notation makes it less obvious on how to specify typeclass constraints for functions. The interface-inspired syntax of
|
This approach has some issues with overloading. E.g., what if I have several different type classes (extension interfaces, whatever) for I'd expect something like:
|
Now, when I look at my example above, it starts to remind me "extension providers" from Xtend (https://www.eclipse.org/xtend/documentation/202_xtend_classes_members.html#extension-methods). @raulraja @elizarov |
One possible motivation could be ad-hoc polymorphism as described by Cedric. For example in klaxon to create a new Json-Object you have the following method: This basically allows everything to be put inside. With typeclasses we can define a JSONValue:
Then all you need inside your JSON-Library is a method like that:
|
@dnpetrov here are a few: Type classes cover many use cases, here are a few interesting ones syntax aside: Compile-Time Dependency Injection
In the example above we used Type restrictions for safer operations
Type safe composable encoder / decoders and reflectionless instrospectionExpanding on what @d3xter mentions in #87 (comment)
The most important use case of them all is that users and library authors can write code with typeclasses and default instances providing complete solutions to their users. At the same time users have the power to override library exposed typeclasses behavior at any time by providing other instances in scope. Defined in user code (not in the json library), this implementation capitalizes the keys.
The library author did not have to add specific support for Pascal Case because overriding a tiny bit of functionaly is easy enough. Entire system and libraries can be composed with typeclass evidences deferring dependencies to users but providing reasonable defaults that are verified at compile time therefore making programs more secure and making easier not to recurre to runtime reflection, costly initialization graph of dependencies etc. I'll be happy to provide more use cases if needed but I believe the controlled and safe power type classes would bring to Kotlin is a great step forward in terms of compile time verification, type safety and performance for many common patterns like the ones demonstrated above. There is also the use case of supporting typed FP with all the abstractions we used in examples previously but I will not bring those examples here because Type Classes is much more than FP, it's all about polymorphism and evidence based compile time verification opening the door to many other use cases most of them in trivial tasks users face in daily programming. |
@dnpetrov I would suggest that "type extension" like
|
@elizarov is |
@raulraja That was a type. Fixed, than you. You can use either The problem with the former, as was discussed before, is that it is not clear how to disambiguate the functions like With the later declaration, given |
@edrd-f This doesn't work in general. Example: interface Monoid<A> {
fun z(): A
fun A.add(other: A): A
}
class PairIsMonoid<A,B>(
val left: Monoid<A>, val right: Monoid<B>
): Monoid<Pair<A, B>>, Monoid<A> by left, Monoid<B> by right {
/*implementation omitted */
} The compiler fails because I cannot implement I therefore think that the assertion that the only viable alternative being explicit |
@ulrikrasmussen I am sure your real-world json serialization problem would be a much appreciated example for many who are not used to monoid terminology. |
@fvasco indeed, but it's possible to have interfaces that solely define extension functions: interface ValidatorExtensions<T> {
fun T.isValid(): Boolean
} These extensions are only visible within the class implementing it, so although the class can be considered a But in case one doesn't want to expose this interface publicly, the approach would be to simply call the validator methods: interface Validator<A> {
fun isValid(record: A): Boolean
}
class ValidatedRepository<A>(private val validator: Validator<A>) : Repository<A> {
val storedUsers: MutableMap<Int, A> = mutableMapOf() // e.g. users stored in a DB
override fun loadAll(): Map<Int, A> {
return storedUsers.filter { validator.isValid(it.value) }
}
override fun loadById(id: Int): A? {
return storedUsers[id]?.let { if (validator.isValid(it)) it else null }
}
override fun A.save() {
storedUsers[generateKey(this)] = this
}
} What's the difference in this case? |
@edrf-f
The extension functions will be visible when used with scoping functions, among other things: val validatedRepository: ValidatedRepository<Foo>
val Foo: Foo
validatedRepository.run {
foo.isValid() // visible here
} |
@fatjoem Sure. Simplified, I just want to have a interface ToJson<A> {
fun A.toJson(): JsonElement
} I want to be able to serialize lists if the elements are serializable. I can write the instance as a class which takes the element class ListToJson<A>(
val elementToJson: ToJson<A>
): ToJson<List<A>>, ToJson<A> by elementToJson {
override fun List<A>.toJson() = /* implement encoder using the A.toJson() instance that is implicitly in scope */
} This fails to compile, however, because I am not allowed to define a class that implements the same generic interface with different type arguments (it tries to implement |
Apart from the verbosity (it is always possible to translate type class code to explicit record passing), you also lose the ability to enforce coherency in the compiler, which can make it harder to reason about your code. That is, the implementations that are passed to you are chosen by the caller, and you cannot rely on the fact that there is at most one instance for a given type. |
@ulrikrasmussen – quoting @elizarov:
|
I think you can workaround this compilation issue using |
I don't understand this line of reasoning. Because object interfaces are coherent, then typeclasses shouldn't have that property? I also don't understand how ad-hoc polymorphism is supposed to be synonymous with incoherence. Typeclasses by the way are well suited for implementing compositional behaviors, something that is not well supported by regular interfaces. Returning to the class List<A : ToJson>(): ToJson { ... } But that prevents me from ever using a list with something that isn't serializable as JSON, which is clearly not what we want. Typeclasses allow me to express this more naturally. |
No, that won't work, for two reasons.
interface ToJson<A> {
@JvmName("foo")
fun A.toJson(): String
} saying that the annotation is not applicable to this declaration. |
I agree with @ulrikrasmussen - the title of this KEEP is even "Extension Interfaces". My understanding was the point was to introduce a new "type" of interface that can be implemented for pre-existing classes. I can add code and say "an integer is a monoid" and then use it everywhere I'm using monoids. The only difference is that it's added after the integer type is defined, by someone who has no access to the original code. |
I actually found an alternative to the "plain Kotlin" pattern proposed by @edrd-f which does not have the limitations I pointed out above. You need to define your extensions in an object inside the body of a function which invokes interface ToJson<A> {
fun A.toJson(): String
}
fun <A> ListToJson(
innerToJson: ToJson<A>
): ToJson<List<A>> =
with(innerToJson) {
object: ToJson<List<A>> {
override fun List<A>.toJson(): String =
"[${joinToString(",") { elem -> elem.toJson() }}]"
}
} Here is another example with two extension constraints: interface Monoid<A> {
fun z(): A
fun A.add(other: A): A
}
fun <A, B> PairIsMonoid(
left: Monoid<A>,
right: Monoid<B>
): Monoid<Pair<A, B>> =
with(left) { with(right) {
object: Monoid<Pair<A, B>> {
// I have to call these explicitly on `left` and `right` because Kotlin cannot dispatch on return type.
override fun z(): Pair<A, B> = Pair(left.z(), right.z())
override fun Pair<A, B>.add(other: Pair<A, B>): Pair<A, B> {
val (x1, y1) = this
val (x2, y2) = other
// The `.add` extensions on `A` and `B` can be used here.
return Pair(x1.add(x2), y1.add(y2))
}
}
} } There may be problems I have overlooked. |
I don't think that's the only difference. Regular interfaces have constraints that every instance implementing it must satisfy. With extension interfaces, you can have constraints for a type, which means you can have constraints on "static" things. This is the most important part from my pov. |
I'm not sure I follow, but I want to - can you elaborate a bit more? I think I'm kinda stuck somewhere in type vs class land or something along those lines and that's where my confusion comes from. |
No problem, the terminology is indeed confusing because many paradigms get mixed here. When you have an interface, you implement it in your class and than you have to satisfy the constraints of that interface. That means when using the interface as a reference type for any implementing instance, you can use those interface's features. Easy.
Look at how each class implements a companion object, because the instanceSizeInBytes is static for a class and only changes when the class's layout changes. Imagine you want to have a function like the following
With the proposed extension interfaces, you can enforce constraints for a type, that means you don't need an instance of that type to do something. The function would look like
Depending on how extension interfaces are implemented, Some might find my example odd, but there are really a lot of other use cases, for example factory/constructor constraints, serializer stuff etc. |
I have now done the same thing with a classical monoid example. In my opinion, using the companion values proposal (from keep 106) leads to more idiomatic kotlin code, than what is proposed in this keep. The classical monoid example would look like this when using companion values: interface IMonoid<A> {
fun zero(): A
fun A.append(other: A): A
}
class StringMonoid : IMonoid<String> {
override fun zero() = ""
override fun String.append(other: String) = this + other
}
fun <A> IMonoid<A>.doMonoidStuff(a: A) {
...
}
fun main() {
// the companion val can go into different scopes.
// in this example we are scoping it locally in this function:
companion val stringMonoid = StringMonoid()
val someString = "hi world"
doMonoidStuff(someString)
} In this example, everything is ordinary Kotlin, except for the companion val, which is the proposed feature of keep 106. It basically is just syntactic sugar for: fun main() = with(StringMonoid()) {
val someString = "hi world"
doMonoidStuff(someString)
} |
Most examples here require the interface to have a type parameter (the class we are extending), which is fine if we have control of the interface, but sometimes we actually don't, eg. Android's I believe the extension class should define both what interface it is implementing and what class it is implementing it for. So building on fatjoem's suggestion: class StringMonoid : IMonoid for String {
override fun zero() = ""
override fun append(other: String) = this + other
}
fun doMonoidStuff(m: IMonoid) {
...
}
fun main() {
private companion val stringMonoid = StringMonoid()
val someString = "hi world"
doMonoidStuff(someString)
}
|
Why make things complicated with Just use the existing syntax provided by Kotlin, and if need be introduce the Correct me if I am wrong here but Kotlin seems to be mostly used by mobile engineers on Android mostly (due to Googles support), and if this is use-case, again I would call out that keeping parity between Consider the following example in Swift:: protocol Event {}
protocol Eventable: class {
func send(_ event: Event)
}
// I am able to extend Apple’s UIKit class here
// this gives me amazing flexibility
extension UIViewController: Eventable {
func send(_ event: Event) {
// Implementation...
}
}
struct AnEvent: Event {}
// Can reference only Eventable now
let eventables: [Eventable] = //...
eventables.forEach {
$0.send(AnEvent())
} Now it would be amazing if I do the following in Kotlin: interface Event {}
interface Eventable {
fun send(event: Event)
}
// Extend Androids APIs giving me
// amazing flexibility
extension Fragment: Eventable {
fun send(event: Event) {
// Implementation..
}
}
object AnEvent: Event {}
val eventables: List<Eventable> = //...
eventables.forEach {
it.send(event = AnEvent())
} Now Furthermore I can now hold an array of What is preventing the above implementation? :-)
|
@ZkHaider The JVM and ART VM prevent this from working with I've been thinking about |
Having re-read all this thread after some years, I still think the proposal from @elizarov on Oct 2, 2017 is the best one. Looking at the Swift world, there are lots of use for |
This issue has had some awesome discussions, rewrite, and a lot of other evolution. Thanks a lot to @raulraja personally and to all the other contributors who are too numerous to name them here. One thing became clear: there is an enormous community interest in adding some kind of extension interfaces to Kotlin. 🚀 The main use-case that gets people excited about this feature is almost available today in Kotlin via extension functions, so we have decided that we will double-down, first, on this particular Kotlin language design aspect and will extend it to support multiple receivers. Not only this will address the long-standing irregularities in the original Kotlin design but will enable many of the use-cases that were mentioned in this discussion without having to introduce entirely new concepts into the language. This work on multiple receivers is tracked at issue 👉 https://youtrack.jetbrains.com/issue/KT-10468 The abovementioned issue does not have a concrete design proposal yet, but lists some key use-cases and basic requirements that were already identified. Please, vote for it, comment on your specific use-cases for multiple receivers and continue discussions there 👍 I understand that multiple receivers will not cover the full scope of features that were envisioned by the original "extension interface proposal". At the current stage of the design process, there are no provisions for any kind of automated creation/import or another kind of acquisition of "extension interface" instances beyond them being explicitly brought into the scope. However, just by itself, it is a big change to the language, so it is imperative to take small and careful steps. The very support for multiple receivers is the necessary first step that will enable other improvements in the future. It is not enough to introduce a new feature into the langauge. We also need to learn how to use it effectively to improve the clarity of code that we write in Kotlin. There are many open questions down this road, and that is why we are starting just with a prototype. Having said that, I'm closing this issue to shift our focus onto the shorter-term goal of supporting multiple receivers in Kotlin first. |
Thanks so much, @elizarov. We look forward to multiple receivers and appreciate the work the Kotlin team is putting in the compiler to accommodate these community use cases 🙏 |
Proposal
The goal of this proposal is to enable compile-time dependency resolution through extension syntax. Overall, we want to enable "contract" interfaces to be declared as program constraints in function or class constructor arguments and enable the compiler to automatically resolve and inject those instances that must be provided with evidence in one of a given set of scopes. The resolution scopes to look for those evidences are predefined and ordered, more details inside the proposal.
In the case of not having evidence of a required interface (program constraints), the compiler would fail and provide the proper error messages both by IntellIJ IDEA inspections and compile-time errors.
This would bring first-class named extension families to Kotlin. Extension families allow us to guarantee that a given data type (class, interface, etc.) satisfies behaviors (a group of functions or properties) that are decoupled from the type's inheritance hierarchy.
Unlike the traditional subtype style composition where users are forced to extend and implement classes and interfaces, extension families favor horizontal composition based on compile-time resolution between types and their extensions.
Current code status
Thanks to @truizlop and @JorgeCastilloPrz,we already have a working POC where we've coded a big part of the KEEP-87. There are still some additional behaviors we'd need to code that have been clarified on the KEEP though, and we'd love to get JetBrains help on iterating those. Instructions about how to test it have been provided within the "How to try?" section of the keep, where we have also provided a code sample.
Rewording
We have revamped the initial proposal given Type Classes are not really our aim here, but compile-time dependency resolution for Kotlin. The nature of the proposal has not changed a lot, just suffered a major refactor so it's more in the line of what a generic language feature like this one would be, and also to fit the compiler implementation feedback we've got from some JetBrains devs these later months.
We believe the compiler could support compile-time dependency resolution out of the box, as other modern laguages already do. We believe some infrastructure patterns would fit better as built-in language features, so users can get their canonical solution for that already provided and focus their work on the actual domain requirements and business logics of their programs. We've seen similar things in other languages that provide built in support for concurrency and other stuff.
How to contribute
Want to help to bring compile time extension resolution to Kotlin?. A fork is being provisioned where a reference implementation based on this proposal will take place at https://github.com/arrow-kt/kotlin