Skip to content

Proposal: Generic Argument Wildcards #1992

Unanswered
mikernet asked this question in Language Ideas
Proposal: Generic Argument Wildcards #1992
Nov 10, 2018 · 52 answers · 2 replies

IMPORTANT: This proposal is very different than Java wildcards in terms of its implementation and behavior. It looks similar on the surface but it's a much more elegant solution than extending variance. It follows all existing variance rules and takes advantage of the CLR generic type system to transform the block of code using the unknown type into a generic method that is JITed to use the ACTUAL types of the generic parameters. "Extracted" types can therefore be safely used just like any other generic parameter and everything is compile-time verified so you can't do something wacky like try to pass an object into a method that takes a string without getting a compiler error.

Details:

I've run into many situations where this would be extremely useful but I'll give a couple to start things off:

A) Eliminate the need for a non-generic interface in addition to a suitable generic interface

It would help avoid the giant mess often created when generic types need to inherit non-generic versions of the same type in order to be usable when the generic type argument is unknown, and if implemented in an ideal manner where the JIT would emit different versions of the method as needed then it would also eliminate boxing in lots of situations.

Hypothetical example - non-generic IEnumerable interface would no longer be needed:

void IterateIfPossible(object obj)
{
    if (obj is IEnumerable<?> items) 
    {
        foreach (var item in items)
            Console.WriteLine(item);
    }
}

This would remove the need to implement a non-generic interface with a bunch of methods that just cast the generic argument typed return value/parameter to object.

In this case var would be treated as an anonymous generic parameter. The next example shows how to extract that parameter so it can be used like any other generic parameter.

B) Allow extraction of the unknown type:

void Type GetItemType(IEnumerable obj)
{
    if (obj is IEnumerable<?T>) 
    {
        return typeof(T);
    }
    return typeof(object);
}

C) Test if an object inherits a generic type or implements a generic interface and if so use the value in its strongly typed form:

I don't believe there is currently any way to avoid very messy and slow reflection code to test if an object implements, say, IList<T>, and if so then pass it to a method that requires an IList<T>. In order to do this:

private IItemsProcessor GetItemsProcessor(object items)
{
    // Optimize for List<T>
    if (items is List<?T> list)                
        return new ListProcessor<T>(list);

    if (items is IEnumerable<?T> enumerable)
        return new EnumerableProcessor<T>(enumerable);
}

I currently have to do this:

private IItemsProcessor GetItemsProcessor(object items)
{
    var itemsType = items.GetType();
    // GetBaseTypes() is an extension method
    var listType = itemsType.GetBaseTypes().FirstOrDefault(i => i.IsGenericType && i.GetGenericTypeDefinition() == typeof(List<>));

    // Optimize for indexable collections
    if (listType != null) 
    {
        var itemType = listType.GetGenericArguments()[0];
        var processorType = typeof(ListProcessor<>).MakeGenericType(itemType);

        return (IItemsProcessor)Activator.CreateInstance(processorType, items);
    }

    var enumerableType = itemsType.GetInterfaces().FirstOrDefault(i => i.IsGenericType && i.GetGenericTypeDefinition() == typeof(IEnumerable<>));

    if (enumerableType != null)
    {
        var itemType = listType.GetGenericArguments()[0];
        var processorType = typeof(EnumerableProcessor<>).MakeGenericType(itemType);

        return (IItemsProcessor)Activator.CreateInstance(processorType, items);
    }
}

D) Allow generic constraints:

void GetDisplayValue(Control control)
{
    if (control is LabelControl<?T> label where T : IDisplay) 
    {
        // LabelControl.Value is a property of type T
        // GetDisplayText() is a method on IDisplay
        return label.Value.GetDisplayText(); 
    }
}

This would let you avoid a lot of potential casting and allow method overload resolution to use methods that have compatible constraints on their generic parameters when the unknown value is passed in:

void Serialize<T>(Field<T> field) where T : class
{
    Serializer.Serialize<T>(field);
}

// ERROR: can't differentiate just on generic constraints
void Serialize<T>(Field<T> field) where T : struct
{
    // But I need to be able to call a method with a T : struct constraint!
    Serializer.FastSerializeValueType<T>(field);
}

// But this would work:

void Serialize(Field field)
{
    if (field is Field<?TValue> valueField where TValue : struct)
    {
        Serializer.FastSerializeValueType<TValue>(field);
    }
    else if (field is Field<?TRef> refField where TRef : class)
    {
        Serializer.Serialize<TRef>(field);
    }
}

Implementation

This is a simple demonstration of a roughly equivalent code transform (behavior-wise) that could be applied to the last example by the compiler to make it work:

void Serialize<T>(Field field)
{
    try 
    {
        Serialize_struct((dynamic)field);
        return;
    }
    catch (RuntimeBinderException) { } 

    try 
    {
        Serialize_class((dynamic)field);
    }
    catch (RuntimeBinderException) { }   
}

void Serialize_struct<T>(Field<T> field) where T : struct
{
    Serializer.FastSerializeValueType<T>(field);
}

void Serialize_class<T>(Field<T> field) where T : class
{
    Serializer.Serialize<T>(field);
}

Dynamic is quite quick here but the dynamic callsite code it outputs could definitely be optimized more (i.e. cutting out the overhead of the try/catch). There may be other optimizations that apply for this specific situation (known single method to call, only varies by generic parameters).

If there are any CLR enhancements that can make it a bit quicker but those can just be stubbed in by newer runtimes. Even without any CLR changes this can be very fast using the existing dynamic features in the runtime.

Replies

52 suggested answers
·
2 replies

This only works in Java as the generic type arguments don't exist at runtime. The JRE only recognizes the type Iterable, it is entirely unaware of whether it was Iterable<String> or Iterable<Integer> or Iterable<?>. In the CLR that is not the case. The generic type arguments are reified into the actual type, so IEnumerable<string> and IEnumerable<int> are entirely different and incompatible types.

I will agree that the reflection APIs in the BCL leave a lot to be desired when it comes to generics. I've advocated for better APIs here: https://github.com/dotnet/corefx/issues/644

0 replies

non-generic IEnumerable interface is not needed:

using System;
using System.Collections.Generic;

public class Program
{
    static void Main()
    {
        Console.WriteLine(IsEnumerable(new string[1])); // true
        Console.WriteLine(IsEnumerable(new Program[1])); // true
        Console.WriteLine(IsEnumerable(new List<string>())); // true
        Console.WriteLine(IsEnumerable(new List<Program>())); // true
    }

    static bool IsEnumerable(object obj) => obj is IEnumerable<object> items;
}
0 replies

@ufcpp

Console.WriteLine(IsEnumerable(new int[1])); // false
Console.WriteLine(IsEnumerable(new List<int>())); // false
0 replies

@HaloFour I'm pointing to Java as an example of a similar feature. Regardless of how or why it works on Java, there are ways to make this work cleanly on the CLR as well. When I have a bit more time later today or tomorrow I'll follow up with a demonstration of how it could be done.

0 replies

@ufcpp That only works in that particular case because the generic part of IEnumerable is defined as <out T>. Try that with an interface without the out part :)

As I stated, IEnumerable was just a hypothetical example. Sadly I doubt IEnumerable and the mess of other collection interfaces is going anywhere anytime soon, but my framework would benefit immensely from this.

0 replies

@mikernet

I'm pointing to Java as an example of a similar feature. Regardless of how or why it works on Java

The how is important. The Java language has this ability specifically because of how generics work in that language. C# can't work around the variance limitations very intentionally imposed by the CLR. Nor should it.

there are ways to make this work cleanly on the CLR as well.

Possibly, but it's exceptionally difficult to justify CLR modifications to support a language change unless the benefit is absolutely enormous and it can't be solved any other way. This is especially true given that it seems that the .NET Framework will never see such modifications.

0 replies

@HaloFour I think eliminating the need for redundant non-generic interfaces in 99% of situations where they are currently needed would be a huge win. That's obviously subjective. There are ways to make this work fairly well without CLR cooperation but optimally perhaps it might help a bit. Wildcard generic arguments would be useful for more than just C#. Regardless, I presume the runtime isn't doomed to never evolve out of its current state for all eternity just because the .NET Framework isn't evolving anymore. That would be kind of silly.

I never suggested C# should work around the variance limitations imposed by the CLR. I don't think anything I've suggested conceptually requires breaking any variance rules.

I'll go into more detail about potential ways to implement this soon.

0 replies

@HaloFour Updated my post with a simple implementation to demonstrate viability. I'll play around with ways to optimize this further on an IL level when I have more time.

Can we agree that my example transformation shows that an implementation is possible which doesn't involve breaking any "variance rules" and everything behaves like you would intuitively expect?

I haven't 100% determined how well it can be optimized yet but my gut tells me that specializing the call site caching code dynamic generates with a custom implementation to handle this particular case optimally (i.e. calling the generated generic method with the correct generic argument) could be "very fast", and any increase in execution time caused by this "generic wildcard cast" (compared to a plain non-generic interface cast) would be made up by avoiding boxing for value types for most non-trivial cases.

The real benefit would come from cleaning up interface related code though. Even if it's at the expense of a tiny amount of performance in some situations that would likely only be practically measurable in microbenchmarks, I still think this is how most people would opt to handle generic interfaces with unknown type arguments in a large majority of cases considering that the only real alternative is a bunch of mindless casting boilerplate interface bloat.

0 replies

if the dynamic-based implementation is acceptable, then I'm not sure this feature is needed, because I think this code:

void IterateIfPossible(object obj)
{
    if (obj is IEnumerable<?T> items where T : IFriendlyDisplay) 
    {
        foreach (T item in items)
            Console.WriteLine($"'{item.FriendlyDisplayText}' is type '{typeof(T)}'");
    }
}

is not that much better than this code, which works today:

void IterateIfPossible(object obj) => IterateIfPossibleImpl((dynamic)obj);

void IterateIfPossibleImpl<T>(IEnumerable<T> items) where T : IFriendlyDisplay
{
    foreach (T item in items)
        Console.WriteLine($"'{item.FriendlyDisplayText}' is type '{typeof(T)}'");
}

void IterateIfPossibleImpl(object _) { }
0 replies

@svick I tend to disagree that it's not much better. That example was rather simple just to demonstrate the idea and doesn't require capturing locals between any of the methods and only has a single unknown type with a single unknown parameter, doesn't handle mismatched constraints, etc. This gets very messy very quickly and the alternate pattern you've presented is rather obtuse for anything but the simplest case and I can't imagine very many people adopting it for many reasons, especially for more complicated flows. I can give you some examples if you like but most practical usages would require significantly more obtuse implementations. Even in the simple example your code fails if passed an object that implements IEnumerable<T> but the constraint is not met.

The alternate syntax I'm proposing is terse, simple, easily understood and would encourage people to use what I would argue is a much better design that avoids writing redundant non-generic interfaces where they aren't needed. It can be tuned to minimize any performance impact further than the dynamic implementation which is important for encouraging its use even in situations where you're only making one to a few calls on the object you're working with.

If I have to choose between writing code code like the transformation you've shown everywhere (along with all the additional boilerplate you've left off, such as checking for constraint violations) vs creating and implementing a non-generic base interface, I'll do the latter almost 100% of the time. Dynamic has been around for a while but how often do you see code like this? More importantly, I would never write a library that required consumers of my library to write code like this so non-generic interfaces are still added everywhere, resulting in a similar mess as the collection interfaces in .NET.

Conversely, if I got to choose between writing the proposed new syntax vs writing and implementing a non-generic base interface, I would do the former almost 100% of the time. If this became a language feature then it would become the standard way of handling unknown type parameters and we would all be better off for it, dealing with non-generic base interfaces sucks. I've seen many posts complaining about having to implement non-generic versions of IEnumerable or ICollection along with the generic versions. All that backwards compatibility stuff means that will probably never change but at least we can avoid the same problems for new interface hierarchies we're designing.

As I indicated before, that was just a demonstration to show how it could conceptually work. I don't think dynamic is how this would actually be implemented, there are too many issues with it. I know this can be emulated with dynamic and some leg work, but that's not the point - this is more about enabling a shift in how developers design interfaces so that we get nicer code to write and nicer code to consume. Solutions to the "problem" this is solving exist, as we've both shown, but nobody is going to stop writing redundant non-generic interfaces or stop requiring me to implement them with these existing solutions.

This is akin to async/await. We could always write asynchronous code before it, but who actually wanted to? Moving state between each method and splitting the logic between multiple methods was tedious, annoying and made reasoning about the flow difficult. The method you've proposed has all the same problems as async programming did which is why nobody will use it. This proposal fixes that and makes for a more enjoyable C# coding experience :)

0 replies

Here's an example of one more common problem that this transformation could solve nice and cleanly:

        void DoSomething<T>(IEnumerable<T> items) where T : class
        {
        }

        // ERROR: can't differentiate just on generic constraints
        void DoSomething<T>(IEnumerable<T> items) where T : struct
        {
            // But I need to be able to call a method that only takes value types in this case!
            ValueTypeOps.MethodWithValueTypeConstraint<T>(items.First());
        }

        // But this would work:

        void DoSomething<T>(IEnumerable<T> items)
        {
            if (items is IEnumerable<?TValue> valueItems where TValue : struct)
            {
                ValueTypeOps.MethodWithValueTypeConstraint<TValue>(items.First());
                return;
            }

            // do something with reference type items
        }

I believe that would solve this problem in an ideal way, performance and design wise.

0 replies

@svick P.S. your version of the transform which eliminates the type checking line is for all intents and purposes just as fast as the non-generic code in my test. This tells me that an optimized implementation of the new syntax could be effectively indistinguishable in performance vs using non-generic interfaces.

0 replies

One last point before I dip out for a while.

This would fundamentally change how generic hierarchies are designed and consumed in many cases and eliminate a lot of necessary redundancy and casting boilerplate which propagates throughout some massive class and interface hierarchies. That seems like a much more meaningful proposal than all this pattern matching stuff yet it seems like we are getting pattern matching so I don't understand the negativity towards this. It seems to me this would have wider reaching improvements than pattern matching and the workarounds are more complicated and more tedious than the workarounds for achieving what pattern matching does so I really don't see why this wouldn't be strongly considered.

0 replies

@mikernet I'm not convinced that this is as big a problem as you make it out to be. Sure, a non-generic version of a generic interface is sometimes useful, but creating and using them doesn't require that much boilerplate code (and it will be even less with default interface methods #52).

If you actually need type parameter extraction, then that is harder to do, but I'm not sure that's actually common.

As for negativity, I think that most proposals here will receive a negative initial reaction. If you want to have the proposal to progresses further, it's up to you to convince us, or rather, someone from the Language Design Committee, that it's a change worth making.

0 replies

Fair enough regarding the convincing, haha. Default interface implementations are an orthogonal issue to this - we can't create a default non-generic interface implementation that somehow forwards the calls to the unknown generic interface method, so it does nothing to solve this problem.

I think this is a perfect extension to the pattern matching work going on (and could probably help with making that even more powerful) and solves problems and design tradeoffs that I literally face every day building performance critical "lower level" library/framework code.

Even if it's not a big deal (which I disagree about), neither is declaring a variable before you use it for an out parameter or 90% of the other things being added to make programmer's lives easier.

I'll give you an example of some code in my library. I have a bunch of field types, and this is the hierarchy:

FieldComputedFieldComputedField<T>ReferenceFieldEntityCollectionFieldEntityListFieldEntityListField<T>EntitySetFieldEntitySetField<T>EntityFieldEntityField<T>TypeConstantFieldTypeConstantField<T>ValueFieldValueField<T>ValueListFieldValueListField<T>

All the non-generic types (except for Field) along with their associated abstract method definitions and overrides in the generic versions would go away if I could just say if (field is ValueField<?T> valueField) instead of having to do if (field is ValueField valueField) and then using abstract calls that do casting and all sorts of nasty boilerplate to work. It would probably cut the total number of lines in those files by half and make it way easier to reason about what is going on by eliminating the need to jump around between abstract method implementations in the generic type and common functionality in the base type when it could all be contained in the generic type. Lots of abstract method calls would be avoided as well as casting and boxing.

0 replies

@mikernet How do You propose it should work when type implements generic interface multiple times?

class MultipleEnumerable : IEnumerable<string>, IEnumerable<int> { ... }
...
var obj = new MultipleEnumerable()
if (obj is IEnumerable<?T>) 
    {
        return typeof(T); //what does it return?
    }
2 replies
@hez2010

This can work with C# union types proposal. In the case 'T' will be 'string | int' and further pattern matching is needed here to handle string and int respectively.

@mikernet

@hez2010 That would only work if you knew all the possible types T could be at compile-time,

@zippec Probably the same way dynamic works now in the same situation / how method overload resolution works when trying to match it to a method - throw error "The type arguments cannot be inferred from the usage."

EDIT: Nevermind, I changed my mind further down.

0 replies

Or simply pass over it so that you don't have to worry about exceptions being thrown? That probably makes more sense. I don't know - I'm open to whatever, I think this situation is exceedingly rare.

0 replies

@zippec Good point. This is logically a foreach kind of operation, not an if kind of operation. This could be glossed over, but should it be?

0 replies

Given that doing this causes all sorts of problems in lots of situations, I don't think anyone ever actually does this, and if they do then they can expect wacky behavior all over the place. Personally, I don't think allowing a type to implement the same generic interface multiple times with different type parameters should have ever been allowed but hey here we are, so whatever behavior is standardized here is fine by me.

I don't think throwing an error is appropriate because you are logically just asking "can I use the object as this type", and that operation should never really throw an error. Either just pick the first matching interface that works, or don't match and pass over it.

Mind you this is different than this situation:

class SingleEnumerable : IEnumerable<string> { ... }
class MultipleEnumerable : SingleEnumerable, IEnumerable<int> { ... }

var obj = new MultipleEnumerable()
if (obj is IEnumerable<?T>) 
    {
        return typeof(T); //what does it return?
    }

In that case obviously just return int.

0 replies
IEnumerable<Type> GetEnumerableTypes(IEnumerable collection)
{
    foreach (IEnumerable<?T> items in obj)
    {
        Debug.WriteLine($"Collection has {items.Count()} number of {typeof(T)} items.");
        yield return typeof(T);
    }
}

🤣 🤣 🤣

0 replies

I'm laughing, but that's actually not too bad, hahaha. I don't think the rarity of that situation would ever make it worthwhile to implement but hey it's fun to think about.

0 replies
        void Iterate<T>(IEnumerable<T> items)
        {
            if (typeof(T).IsClass || typeof(T).IsInterface)
            {
                foreach (var item in items)
                {
                    Console.WriteLine($"{item} : class or interface");
                }
            }
            else
            {
                foreach (var item in items)
                {
                    Console.WriteLine($"{item} : class or interface");
                }
            }
        }

        void Iterate(object items)
        {
            //Anything else
        }
0 replies

@VanKrock You didn't explain what you're trying to show and I can't tell from your code.

0 replies

@HaloFour @gafter Sorry for digging up old post and this might not be related, but I think I have seen issue that might be related, about complex generic inference

Currently now we might write complex generic like this

public IEnumerable<T> Flatten<E,T>(E items) where E : IEnumerable<T>
{
}

// Currently now error : cannot be inferred from the usage
object[][] objs;
var flatten = Flatten(objs);

I have seen proposal that said it was solve error above. I cannot find it anymore but I think it might also solve this issue too

0 replies

@Thaina That would be a very nice solution to many (but unfortunately not all) of the use cases presented, especially if it could be used in local functions, and probably has a much higher chance of actually being implemented. If someone could point to that proposal so I can upvote it that would be great :)

0 replies

#3271

0 replies

@gafter Thank you very much. But actually I think I have been seen this idea a lot more older. Maybe it was just a discussion, or all the way since roslyn

Mention roslyn make me remember I could find it in roslyn

dotnet/roslyn#5023
dotnet/roslyn#15166

0 replies

Any updates here? This proposal makes perfect sense to me. Now I have to do smth like this

((dynamic) _serviceProvider
                    .GetService(typeof(IDropdownProvider<>).MakeGenericType(t)))
                ?.GetDropdownOptions();

or introduce a non-generic interface which is somewhat awkward.

0 replies

I'd absolutely love to see a feature like this. Personally, I could do without being able to explicitly extract the matched type, but certainly invoking generic functions based on generic matches would be enormously useful.

I quite frequently run into similar issues, and need to go down the reflection route - which is messy, and potentially error-prone, in particular if you need to test for complex generic constraints.

I do agree ending up with multiple matches is likely an unavoidable issue (Yes, multiple IEnumerable<> implementations are a bit odd, but would probably still need to be supported - and either way, there are more practical cases of implementing the same generic interface for different types, think of IComparable<>, IEquatable<> and such). I think all these wildcard APIs would therefore need to work on a (potential) collection of matches, rather than assuming there will be 0 or 1 match. That'd make it a little clunky - something like if (items is List<?T> list) wouldn't really work anymore.

Alternatively, I wonder if you could process matches as some unspecified constrained shape, rather than a specific type to avoid this. I reckon that isn't supported by the framework just yet though (if anything, there might a degree of overlap with the Shapes proposal?)

0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Converted from issue