Skip to content

Type Tags for Extendable Type Information #37790

Open

Description

@ChrisRackauckas

Wrapper types are a mess. The classic example is that TrackedArray{Adjoint{Array}} is no longer seen as an Adjoint and Tracker.jl needed special overloads for handling adjoint arrays. Two deep you can start to special case: three deep is a nightmare. Why?

The issue is that wrapper types aren't exactly new types, they are extended pieces of information. A lot of information stays the same, like length, but only some changes. In the parlance of Julia, this is the kind of behavior that you normally have with type parameters, where Array{T,1} shares dispatches with Array{T,2} but then overrides what changes when you change dimensions. So, we could in theory do Array{T,1,Adjoint}, but now you can see the issue: it's an extendability problem since these tags need to all be pre-specified in the type definition! What if new packages add new tag verbs? SOL, redefine/pirate the type if you want to use it! This is why people use wrapper types: you can always extend, but of course you get the dispatch problem.

What about traits you say? They extend the way things dispatch, but only do so at compile time for a given type. isadjoint(::Array) can't be a thing, unless you have a type parameter for it.

So you need some extra information. That's why I am proposing type tags. A type tag is an extension like Array{Float64,1}{Adjoint}, where the tag adds information to an existing type. It could be like, tag(A,IsAdjoint()). By default, any tags would have the default behavior unless tagged, so Array{Float64,1} would have tag(Adjoint) == DefaultTag(). Then dispatches could be written on the tags, allowing for extension while keeping dispatches.

The one thing to really think about though is ambiguity handling...

Activity

  1. DilumAluthge commented on Sep 28, 2020

    @DilumAluthge
    Member

    This sounds great!

  2. DilumAluthge commented on Sep 28, 2020

    @DilumAluthge
    Member

    The one thing to really think about though is ambiguity handling...

    Could you elaborate on this a little bit?

  3. ChrisRackauckas commented on Sep 28, 2020

    @ChrisRackauckas
    MemberAuthor

    Doing some stuff like mul!(::IsAdjoint,..) would cause a lot of dispatch ambiguities if in Array{Float64,1}{Adjoint} the Adjoint has equal precedence to the other types. In this sense, I almost thing that a separate rule needs to be applied, like "tags supersede type parameters" in order to break ambiguities in a nice way. One of the type gods could probably point to some literature on an idea like this with its trade-offs.

  4. MasonProtter commented on Sep 28, 2020

    @MasonProtter
    Contributor

    Just to copy my thoughts so they don't get lost in the Slack-hole:

    I feel like the idea to have "something that extends the idea of type parameters" a-la Array{Float64,1}{Adjoint} is just re-discovering concrete inheritance.

    That is, Array{Float64,1}{Adjoint} are types that are distinct from Array{Float64,1}, but have the same memory layout and will be accepted by all the same methods. This is essentially what people are talking about when they discuss inheriting from concrete types.

    My understanding is that concrete inheritance is looked at with a lot of skepticism and scorn but it does feel like our abstract array ecosystem is hitting the limits of simple type composition and it may be time to revisit concrete inheritance and see if we can do it sanely.

  5. DilumAluthge commented on Sep 29, 2020

    @DilumAluthge
    Member

    Instead of concrete inheritance, is it possible to accomplish this with only abstract multiple inheritance (a la #5)?

    E.g. for the tracker example, something like this:

    abstract type AbstractArray{T, N}
    end
    
    abstract type AbstractAdjoint{A, T, N} <: AbstractArray{T, N}
    end
    
    struct Adjoint{A, T, N} <: AbstractAdjoint{A, T, N}
        data::A
    end 
    
    abstract type AbstractTrackedArray{B, T, N} <: AbstractArray{T, N}
    end
    
    struct TrackedArray{B, T, N} <: AbstractTrackedArray{B, T, N}
        data::B
        sensitivities
    end
    
    struct TrackedAdjointArray{B, T, N} <: AbstractTrackedArray{B, T, N}, AbstractAdjoint{B, T, N} # requires abstract multiple inheritance
        data::B
        sensitivities
    end

    We have a method adjoint with the following signature:

    adjoint(x::A) where A <: AbstractArray{T, N} -> AbstractAdjoint{A, T, N}

    And we have two methods for track with the following signatures:

    track(x::B) where B <: AbstractArray{T, N} -> TrackedArray{B, T, N}
    
    track(x::B) where B <: AbstractAdjoint{A, T, N} -> TrackedArray{B, T, N}
  6. ChrisRackauckas commented on Sep 29, 2020

    @ChrisRackauckas
    MemberAuthor

    That will hit this issue that we want something to be Tracked, Adjoint, and Symmetric. I think the tagging needs to be "from the outside" instead of being defined at type definition time.

  7. DilumAluthge commented on Sep 29, 2020

    @DilumAluthge
    Member

    Yeah I see what you mean. It's not sustainable to have different structs struct TrackedAdjointSymmetricArray for each possible combination.

    Also we don't currently have abstract multiple inheritance, so that would be another barrier.

  8. DilumAluthge commented on Sep 29, 2020

    @DilumAluthge
    Member

    I agree that the tag approach is probably more sustainable.

  9. yuyichao commented on Sep 29, 2020

    @yuyichao
    Contributor

    They extend the way things dispatch, but only do so at compile time for a given type. isadjoint(::Array) can't be a thing, unless you have a type parameter for it.

    Why is this? Trait doesn't have to happe all at compile time, much like dispatch doesn't have to happen at compile time. It's generally even just a way to compute a property from an object for dispatch. Why can't isadjoint(::Array) be a thing?

    Is this tag supposed to be per object or per type. When and how is it attached? If it's attached to the type then it seems to be exactly what traits are fore. If it's attached to the object, then this is essentially something that is used for dispatch that can be changed at runtime so it cannot be statically inferred. I don't think that's desireable effect.

  10. DilumAluthge commented on Sep 29, 2020

    @DilumAluthge
    Member

    If I understand correctly, the information is attached to instances, not types.

    julia> f(x::IsSymmetric) = 1
    
    julia> f(x::IsNotSymmetric) = 2
    
    julia> x = [3.0 4.0; 4.0 3.0]
    
    julia> f(x)
    2
    
    julia> tag!(x, IsSymmetric)
    
    julia> f(x)
    1
  11. DilumAluthge commented on Sep 29, 2020

    @DilumAluthge
    Member

    Why can't isadjoint(::Array) be a thing?

    To do this at compile time, you need a type parameter. E.g. a Boolean type parameter Adjointness:

    struct Array{T, N, Adjointness}
        data
    end

    Then we define:

    isadjoint(x::Array{T, N, Adjointness} = Adjointness

    To do it at run time, we need a field in the Array struct:

    struct Array{T, N} <: AbstractArray{T, N}
        data
    adjointness::Bool

    Then we define:

    isadjoint(x::Array{T, N} = x.adjointness

    In either case, we need to decide ahead of time which properties (adjointness, symmetricness, etc) we will support, and we have to hardcode those as either type parameters or fields.

    As Chris says:

    So, we could in theory do Array{T,1,Adjoint}, but now you can see the issue: it's an extendability problem since these tags need to all be pre-specified in the type definition!

  12. DilumAluthge commented on Sep 29, 2020

    @DilumAluthge
    Member

    That will hit this issue that we want something to be Tracked, Adjoint, and Symmetric. I think the tagging needs to be "from the outside" instead of being defined at type definition time.

    @ChrisRackauckas This would also mean that concrete inheritance would also not solve this issue, right?

  13. yuyichao commented on Sep 29, 2020

    @yuyichao
    Contributor

    If I understand correctly, the information is attached to instances, not types.

    If that is the case, then essentially f(x) will almost never be statically dispatched. It'll be really hard for type inference to know if the type tag is changed. Is this the desired effect? This is roughly equivalent to putting values into type domain, which is a standard form of dispatch abuse for exactly this reason.

    Still, it's unclear why this should be per-object.

    To do this at compile time, you need a type parameter. E.g. a Boolean type parameter Adjointness:

    Hmmm, no? What you need, if this property is not defined/determined by yourself, is a way to delegate this check to the part that should determine this, i.e. TrackedArray{A} should implement an isadjoint to call that on A instead. If there's nothing else to delegate to, a explicit implementation should be provided or the fallback is used and there's no way around this, no matter what you do.

    If the complaint is that TrackedArray has to implement isadjoint and unknown number of other properties and you would rather like a way to transfer all properties to the wrapped object without enumerating over all of them, well,

    1. I think passing all properties may not be the desired behavior since if you can't enumerate over all the possible properties, you very much also can't enumerate over all the ones you do/don't want to inherit in order to white/black list them.
    2. You can do that very easily with a standard trait interface, i.e. by spelling isadjoint as sth like trait(..., ::IsAdjoint) and simply implement trait(::TrackedArray, ::Any) = trait(<delegate to the wrapped array>)
  14. yuyichao commented on Sep 29, 2020

    @yuyichao
    Contributor

    Or if you replace the trait function I have above with tag, you already get an explicit version of what this is doing without the tag mutation part of the API that kills inference. In another word, instead of doing f(a) you do f(a, tag(a, IsAdjoint())) and that's exactly how all traits works with various different spelling. The tag function can rely on the type or rely on the object which may affect how inference friendly it is but at least it is not guaranteed to kill inference.... And note that even though IsAdjoint() is explicitly used here, this is on the consumer of the information which has to know about it and will not be dealing with unknown number of traits.

    And such a system is already there. Improving it/them and adopting a standard one are certainly things to be worked on. @vtjnash had even made a comment of replacing most complex dispatch with it. Changing the syntax so that this is done automatically without additional argument necessary can potentially be on the table once there's a clear winner/one that's widely adopted.


    Another way to look at this is that methods are already a way to attach compiler friendly info to arbitrary object/types (i.e. methods are mappings from object/type to an output). They also already allow very flexible manipulate/inheritance which seems to cover the need here. Attaching other generic metadata to object could have other use and it is basically what WeakKeyDict are for, but I highly doubt it's the right solution here.

  15. MasonProtter commented on Sep 29, 2020

    @MasonProtter
    Contributor

    If I understand correctly, the information is attached to instances, not types.

    No, I don't think that's correct @DilumAluthge. We want the information to be a part of the type. Basically, Chris is asking for an un-ordered set of extra parameters we can stick on the end of types to attach additional meaning.


    So, we could in theory do Array{T,1,Adjoint}, but now you can see the issue: it's an extendability problem since these tags need to all be pre-specified in the type definition! What if new packages add new tag verbs? SOL, redefine/pirate the type if you want to use it! This is why people use wrapper types: you can always extend, but of course you get the dispatch problem.

    @ChrisRackauckas Can't we do something like

    using LinearAlgebra: Adjoint
    struct MyArray{T, N, Tag <: Tuple}
        data::Array{T, N}
    end
    MyArray(x::Array{T, N}) where {T,N} = MyArray{T, N, Tuple{}}(x)
    
    function Base.adjoint(x::MyArray{T, N, Tag}) where {T, N, Tag}
        AdjointTag =  Adjoint in Tag.parameters ? Tuple{(T for T in Tag.parameters if T != Adjoint)...} : Tuple{Adjoint, Tag.parameters...}
        MyArray{T, N, AdjointTag}(x.data)
    end

    now we have

    julia> MyArray([1,2,3])
    MyArray{Int64,1,Tuple{}}([1, 2, 3])
    
    julia> MyArray([1,2,3])'
    MyArray{Int64,1,Tuple{Adjoint}}([1, 2, 3])
    
    julia> (MyArray([1,2,3])')'
    MyArray{Int64,1,Tuple{}}([1, 2, 3])

    In principal, other packages should be able to add their own tags as they please without any baking in ahead of time. Of course, the biggest downside to this approach that I see would be that basically all Tag functions would need to be @generated for performance which is a total nightmare.

    However, if we had something that was analogous to Tuple which could have a variable number of parameters and was un-ordered, then perhaps we could make it work.

    That is, you would need a type Tag such that

    julia> Tag{Int, Float64} === Tag{Float64, Int}
    true

    and you would need to be able to write methods like

    f(x::Array{T, N, Tag{Adjoint}}) = ...

    and have it apply to a x::Array{T, N, Tag{Sorted, Adjoint}} (and thus causing a new host of ambiguities)

  16. 56 remaining items

  17. MasonProtter commented on Sep 29, 2020

    @MasonProtter
    Contributor
    ```julia
    f(x::TaggedArray{T, 2, Tag}) where {T, Tag, Union{Adjoint, Symmetric} <: Tag} = 3

    Do you mean

    f(x::TaggedArray{T, 2, Tag}) where {T, Tag >: Union{Adjoint, Symmetric}} = 3

    ?

    Huh, interesting I had definitely tried the angry-face operator there and failed to make it work for me, but I probably just did something silly. Thanks for pointing that out. As I said earlier today on Zulip,

    Sometimes you just gotta do something badly and then hope someone smart comes and tells you why it's bad.

  18. Tokazama commented on Sep 30, 2020

    @Tokazama
    Contributor

    I think that this is a step in the right direction for traits in general. For example, an issue brought up on Zulip was that someone wanted to add the table interface (i.e. istable) to a bunch of types that did not have a common supertype but had another related trait. A simple solution would be doing istable(x) = istable(tags(x)) instead.

  19. martinholters commented on Oct 6, 2020

    @martinholters
    Member

    This is an interesting discussion and being able to attach extra tags to types seems like it might be quite useful, but I'm not convinced it solves the problem it's intended to here. What made me skeptical is the appearance of Adjoint and Sorted in the same example somewhere above. That makes sense for vectors, but if you think about higher-dimensional arrays, you see the problem coming. Imagine (and assume for the sake of discussion) the Sorted tag is also defined for higher-dimensional arrays, meaning the array is sorted in iteration order. Now one could add the Sorted tag when verifying the contents are sorted, and adjoint would attach the Adjoint tag. But wait! the adjoint of a sorted matrix is in general no longer sorted, so adjoint would need to be aware of the Sorted tag and remove it. Would it also need to remove other tags from other packages? Also the order of operations matters: Taking the adjoint first, and then verifying the result is sorted, one could safely attach the Sorted tag. But that's more or less where we're at now: Sorted{<:Adjoint} is sorted, Adjoint{<:Sorted} is not (necessarily) sorted, for SomeOtherWrapper{<:Sorted} we don't know.

    So I'm afraid for the problem of array wrappers, we replace the combinatorial explosion of wrapper combinations with the same explosion of tag combinations. Or can someone give a brief example, how, say, Sorted, Adjoint, TrackedArray, and SizedArray, all worked together more easily using tags? Ideally with no code using definition from TrackedArray and SizedArrayat once.

  20. pablosanjose commented on Oct 8, 2020

    @pablosanjose
    Contributor

    @martinholters: good points. Let me rephrase your comments to digest their consequences. You are essentially saying that if tags are commutative (i.e. their order doesn't matter, so we encode them with a Union instead of a Tuple) they need to encode orthogonal information. At the very least they should never conflict with each other. One should be able to add any tag in any order without contradicting (i.e. having to modify) any other existing tag.

    If we instead want to keep tags general (so we can have both Sorted and Adjoint tags, which are not orthogonal), then tags should necessarily be non-commutative. Wrapper types are already explicitly non-commutative. What would we gain with tags over wrapper types then? As presented in the OP, the advantage would be to be able to dispatch on a given tag regardless of the order it occupies in the tag list. But that has little value, I think, as the order is crucial. Matrix{Int}{Sorted, Adjoint} does not allow the same algorithms as Matrix{Int}{Adjoint, Sorted}, as @martinholters notes. Or another example: it doesn't make sense to dispatch on an Matrix with the UpperTriangular tag if Adjoint was added after it, because then it is actually LowerTriangular. So, it would seem that if we are adamant about allowing general, non-orthogonal tags, we might just as well stick to wrappers.

    What we could perhaps attempt in relation to the problem in the OP is to allow defining equivalence classes of types in terms of commutation rules of wrappers (or tags), such as

    Adjoint{UpperTriangular{T}} == LowerTriangular{Adjoint{T}} where {T<:AbstractMatrix}
    Adjoint{Diagonal{T}} == Diagonal{Adjoint{T}} where {T<:AbstractMatrix}
    Adjoint{Adjoint{T}} == T where {T<:AbstractMatrix}
    TrackedArray{Adjoint{T}} == Adjoint{TrackedArray{T}} where {T<:AbstractMatrix}
    

    Then, if we write a dispatch like f(::LowerTriangular{T}) where {T<:AbstractArray}, the type machinery could work out, based on repeated application of the commutation rules, that it also applies to f(::TrackedArray{Adjoint{UpperTriangular{Matrix{Int}}}}). But I wonder whether that is a hopelessly complex task in practice.

  21. pablosanjose commented on Oct 8, 2020

    @pablosanjose
    Contributor

    Uhm, I guess there is another aspect in which tags as proposed in the OP are prefereable to wrappers: they don't require unwrapping to access the fields of a type. The interface of a tagged type would not change as a result of the added types. So the above stuff on commutation rules should actually be reformulated in terms of non-commutative tags instead of wrappers so that the same code of the f method could work on types with equivalent sets of tags.

  22. martinholters commented on Oct 19, 2020

    @martinholters
    Member

    But that only holds if the tag (née wrapper) does not interfere with how the internals are to be interpreted. So most algorithms would have to differentiate whether Adjoint is present or not, which would also mean that those that don't would have to opt-in to ignoring it. (Otherwise, functions unprepared for the Adjoint tag could silently give wrong results.) That doesn't sound that much better than having to peel off a wrapper. So maybe the tag idea shines for additional information, as in a Sorted tag? Not really, the problem persists for mutating functions. And as we don't distinguish mutating/non-mutating functions on the language level, that doesn't really gain us anything.

  23. MasonProtter commented on Oct 23, 2020

    @MasonProtter
    Contributor

    Yes, I agree this is a bad fit for Adjoint. I think the most attractive use for this is things like Sorted and that it gives people a way to 'alias' existing structs and have a little copy that they can put their own methods on and treat differently without having to do all the extra stuff involved with making wrappers.

  24. MasonProtter commented on Oct 23, 2020

    @MasonProtter
    Contributor

    Good discussion. I see this as basically moving type parameters from being positional to being like keyword arguments, which is something I've thought about. That would make parameters more meaningful as well as more extensible. A lot of things to think about here. For example, Vector{Int} could become an abstract type in some sense --- we know the layout, and we know enough to do compile-time dispatch for most functions, but there could be more parameters. Or we could insist that types written as they are now are exact, and you need something like Vector{Int; ...} to mean the abstract type that includes types with additional parameters. That's very explicit but seems less useful.

    @JeffBezanson I think at least for the purposes of labelling the fields of structs, we'd be forced for sanity reasons to say that e.g.

    struct Foo
        a::Vector{Int}
    end

    can only store the un-tagged version. However, I wonder if in all other circumstances, we could get away with having it be 'abstract'? i.e. people can write

    struct Foo{V <: Vector{Int}}
        a::V
    end

    and then that would be compatible with Vector{Int; MyTag}? This has the downside of being an extra rule that people have to remember about spelling types though.

  25. bramtayl commented on Nov 18, 2020

    @bramtayl
    Contributor

    One way to deal with the whole trait/multiple inheritance thing would just be to have a macro for appending to union types. Like if you have

    struct BlueRough <: AbstractRough end
    struct BlueSmooth <: AbstractSmooth end
    const Blue = Union{BlueRough, BlueSmooth}
    

    A user could bestow a trait on a new thing

    struct BlueShiny end
    @bestow BlueShiny Blue
    
  26. ParadaCarleton commented on Nov 5, 2023

    @ParadaCarleton
    Contributor

    Question; is this proposal related to the tagged unions implemented in other programming languages, and in some Julia packages?

    More broadly, in the time since this was first proposed, has anyone written a possible ecosystem or package solution (not in base) that fixes this/provides this feature (like how there are packages fixing the lack of multiple inheritance, traits, etc. in Julia)?

  27. MasonProtter commented on Nov 5, 2023

    @MasonProtter
    Contributor

    Question; is this proposal related to the tagged unions implemented in other programming languages, and in some Julia packages?

    No.

  28. ParadaCarleton commented on Nov 8, 2023

    @ParadaCarleton
    Contributor

    Question; is this proposal related to the tagged unions implemented in other programming languages, and in some Julia packages?

    No.

    To clarify, I'm not asking because of the array--I'm asking because the original example in this thread, Adjoint, looks like it could be implemented by having arrays behave like unions with two basic cases (Adjoint and not Adjoint). @ChrisRackauckas would this solve the original problem?

  29. ChrisRackauckas commented on Nov 8, 2023

    @ChrisRackauckas
    MemberAuthor

    That addresses the problem of Adjoint, but not Tracked, Symmetric, and every other property someone would want to add.

  30. added
    types and dispatchTypes, subtyping and method dispatch
    featureIndicates new feature / enhancement requests
    on Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureIndicates new feature / enhancement requeststypes and dispatchTypes, subtyping and method dispatch

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions