Multiple traits

[timholy]

Thanks for doing this! One specific discussion point:

My counterpoint to this is that we have exactly the same issue with Holy traits, except worse. There we can't even express this kind of relationship with the basic pattern...

Not true. The "depot" function should just compute both traits:

myInterfaceFunc(x) = _myInterfaceFunc(isMyTrait(x), isAnotherTrait(x), x)

and the underscore versions take all 3 args. I think it's basically the same as Meet (with all the same strengths and weaknesses), albeit with far uglier syntax.

[Seelengrab]

Thank you for your comment!

You're right, it's technically just as expressive - with the downside of having to modify the dispatch into a much more complicated, potentially N-ary dispatch that also needs to be modified every time you want to add a new trait (IF the original maintainer is around to do that!). I wouldn't call that pattern "basic" anymore though, since you now also have to modify _myInterfaceFunc to be aware of every new trait :) I did mention as much in the paragraph following the code example:

Now, without modifying myInterfaceFunc, we can't define isMyTrait(::Foo) = IsAnotherTrait() to also support that kind of trait, because that would require giving up on MyTrait. We could introduce a second layer of indirection, to perhaps create a Meet-like of the supported traits, but that then exposes the true problem of the ambiguity between which implementation of _myInterfaceFunc we'd like to use, if both exist. The only way out is yet again defining _myInterfaceFunc(::Foo), breaking the ambiguity.

The "Meet-like" and subsequent breaking of the ambiguity in _myInterfaceFunc is exactly the kind of modification you're suggesting - keep in mind, the example assumed that we wanted to make an existing dispatch aware of our new trait. By simply overloading it with our new trait, we can do just that - something we can't do with Holy Traits without modifying the original source code, to introduce the additional dispatch (we don't always have access to the source code, we'd potentially have to pull in a new dependency into our interface package for the trait function definition... - or we cause an invalidation in the interface, which is surely not desirable. That's bound to lead to lots of recompilation!).

[timholy]

Correct. I think (not sure) some of the traits packages (probably WhereTraits) support extending with new traits: you can do this automatically with Base.delete_method and then automatic method-rewriting, where the new methods take one more trait typed as ::Any.

BUT there are huge downsides to this:

• if used widely, it makes precompilation useless. I'd much rather have precompilation and not have traits.
• you get ambiguities if two packages extend the list of traits in non-intersecting ways.

For those reasons, I personally don't see ever supporting an extensible (in the sense of adding new independent) traits system in Julia; I think there would have to be some social consensus about what traits are needed, and the "depot function" that computes the trait-values would live in some small package (like ArrayInterface). Then we can add new traits and semver it as a breaking change.

[Seelengrab]

Right - extending existing struct types with new traits from the outside is questionable at best, and I wrote as much in the doc page. I'm definitely not talking about deleting methods or anything like automatic method rewriting - I'm not sure how WhereTraits.jl and its mechanism for having predicates to limit dispatch relates to what I discuss in the philosophy page. Could you elaborate? I don't personally see how that gets in the way of treating abstract types as trait-like dispatches directly.

To be clear - I really want to get rid of the "depot" function entirely. From my POV, it's just an additional dispatch layer that's been introduced to work around the limitation of not being able to subtype more than one abstract type. That's why I keep saying that abstract types are exactly the traits we're looking for, were we able to explicitly subtype more than one of them.

[timholy]

I think the non-dispatch solution still amounts to the same thing: you have to invalidate everything that was compiled in a world where it didn't know about the new trait.

[Seelengrab]

Right, but that's something shared across every form of trait implementation, no? Even Holy Traits, since all that's happening there is moving the dispatch one layer in to the "depot function", as you call it - the invalidations still happen just the same, if I'm not mistaken, by lieu of invalidating the isMyTrait functions. I'm not sure that's fundamentally different from... just having the implementation that's required of that trait anyway.

[timholy]

Right, that exactly my point. We have to choose between "extensible traits" and other good things like having precompilation that works. I'm strongly in the camp of keeping precompilation working, even if it means we can never have extensible traits.

Maybe I'm misunderstanding your point, but I interpreted

By simply overloading it with our new trait, we can do just that

to indicate that you thought a Meet-like approach could avoid the negatives of HolyTraits. My point is they both result in invalidation and so are effectively the same thing. But maybe I misunderstood your point.

[Seelengrab]

I guess I'm confused by what exactly you mean with "extensible traits". Do you mean having some way to declare an object implementing a trait? Because that is what I mean with subtyping an abstract type - you have to implement the methods required of an abstract type either way already.

My point with Meet is that it's much easier to combine trait requirements (i.e. "I require at least these methods to work on this object") with it in a function signature than it is with Holy Traits, at least syntactically, and on top of which this "makes sense" semantically.

[timholy]

Using the syntax of WhereTraits but inspired by the example of Traitor: PkgA declares

foo(A::AbstractArray) where {Size(A)<:Big} = 1

but then PkgB declares

PkgA.foo(A::AbstractArray) where {Odor(A)<:Smelly} = 2

[Seelengrab]

I don't understand. That is not at all what I'm talking about, or what RequiredInterfaces.jl would do, even if implemented in Base. With RequiredInterfaces.jl, you'd have abstract type SmallArr <: AbstractArray end (i.e., a type encoding the difference in behavior) and just dispatch on that as usual, just as we do today all the time. Precompilation keeps working just the same.

At least, that's what I take the entire concept of a type to mean, both theoretically and practically in Julia. Are you suggesting we should create some different dispatch mechanism bespoke to this..? That seems a bit redundant to me, like creating a second way of expressing types in Julia.

[timholy]

That is not at all what I'm talking about

OK. I don't think I can answer your question because I've lost sight of what you mean by

with the downside of having to modify the dispatch into a much more complicated, potentially N-ary dispatch that also needs to be modified every time you want to add a new trait

That's the objection I was answering. My point is that dynamic trait/multiple inheritance may require old code to be thrown away regardless of the implementation. At the end of the day, if method calls would dispatch differently in the new world you've created, you're going to have to recompile code (because the dispatches are hard-wired).

[Seelengrab]

I'm talking about your comment that you can just add the new trait to the depot function here:

myInterfaceFunc(x) = _myInterfaceFunc(isMyTrait(x), isAnotherTrait(x), x)

And while that indeed works, if you now want to add another trait isYetDifferent(x) you'll have to both bring in the package defining that trait as a hard dependency (it can't be in a package extension because it needs to be written in the depot function directly) as well as have to modify the depot function to take that argument, leading to N trait-calls in the limit. In contrast, with a focus on abstract types, you don't have the depot function and just add regular method dispatches for your new trait, wherever the new struct/trait is defined - that can be a package extension, a submodule, etc.

My point is that dynamic trait/multiple inheritance may require old code to be thrown away regardless of the implementation

Right - my question is about what you mean with "dynamic trait". If you're talking about WhereTraits style "match any boolean predicate" style of traits, I wholeheartedly agree, that's not going to be scaling well - particulalry as it'll require SAT and/or SMT solving to be able to even say if a given method matches. That's certainly not what I want, hence my focus on making our existing abstract types more expressive/semantically meaningful, with the smallest possible addition to the type system that allows the same behavior.

[Seelengrab]

I'm converting this to a discussion, since there isn't really anything actionable in here - well, and that's what the OP was opened with :)