Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exploration: Shapes and Extensions #164

Open
MadsTorgersen opened this Issue Feb 22, 2017 · 263 comments

Comments

Projects
None yet
@MadsTorgersen
Copy link
Contributor

MadsTorgersen commented Feb 22, 2017

Shapes and Extensions

This is essentially a merger of two other proposals:

  1. Extension everything, which allows types to be extended with most kinds of members in the manner of extension methods, and
  2. Type Classes, which provide abstraction over sets of operations that can be added to a type separate from the type itself.

These two features have good synergy, and would benefit from being designed together. This proposal is a concrete shot at doing so, knowing full well that there are a myriad of different decisions that could be made. This is not to be particularly opinionated about those choices - it's just easier to understand and discuss a general proposal when it has a concrete shape.

Extensions

The idea behind "extension everything" in most proposals is to use a different approach to declaration syntax from today's "static methods with a this modifier", instead providing a type-like declaration with a name, an indication of the type to be extended, and a set of member declarations for that type. This syntactic approach generalizes more easily to other member kinds, including properties, static members and even operators.

Here is an example adding and using a static property:

public extension IntZero of int
{
    public static int Zero => 0;
}

WriteLine(5 + int.Zero); // in the scope of the extension, int has a Zero property

The name of the extension declaration (like that of the static class containing an extension method today) is useful primarily for disambiguation purposes. We'll get back to that later.

What should extension declarations compile into? The straightforward answer is a static class with static members (taking an extra parameter for the receiver if necessary). However, as we'll see below, this proposal suggests a different approach.

Shapes

Interfaces abstract over the shape of objects and values that are instances of types. The idea behind type classes is essentially to abstract over the shapes of the types themselves instead. Furthermore, where a type needs to opt in through its declaration to implement an interface, somebody else can make it implement a type class in separate code.

In C#, let's call type classes "shapes":

public shape SGroup<T>
{
    static T operator +(T t1, T t2);
    static T Zero { get; }
}

This declaration says that a type can be an SGroup<T> if it implements a + operator over T, and a Zero static property.

As an example, the int type is halfway to implementing SGroup<int>, since it has a + operator over int, and above we showed how to use an extension to add a static int-valued property Zero. Let's make it so that an extension declaration can also declare that the extended type implements a given shape:

public extension IntGroup of int : SGroup<int>
{
    public static int Zero => 0;
}

This declaration extends int not only with the Zero property, but with SGroup<int>-ness. In the scope of this extension, int is known to be an SGroup<int>.

In general, a "shape" declaration is very much like an interface declaration, except that it:

  • Can define almost any kind of member (including static members)
  • Can be implemented by an extension
  • Can be used like a type only in certain places

That last restriction is important: a shape is not a type. Instead, the primary purpose of a shape is to be used as a generic constraint, limiting type arguments to have the right shape, while allowing the body of the generic declaration to make use of that shape:

public static AddAll<T>(T[] ts) where T : SGroup<T> // shape used as constraint
{
    var result = T.Zero;                   // Making use of the shape's Zero property
    foreach (var t in ts) { result += t; } // Making use of the shape's + operator
    return result;
}

So as an important special case, shapes address the long-desired goal of abstracting numeric and computational code over the specific data types being manipulated, while allowing clean use of operators.

Let's call the AddAll method with some ints:

int[] numbers = { 5, 1, 9, 2, 3, 10, 8, 4, 7, 6 };
WriteLine(AddAll(numbers)); // infers T = int

Clearly we need to check the constraint at the call site. If this is called within the scope of the IntGroup extension declaration above, the compiler does indeed know that T = int satisfies the SGroup<T> constraint. However, there is more going on: how does the AddAll method know how int is an Sgroup<int> - at the call site? There needs to be more information passed in than just the (inferred) int type argument and the numbers array.

Implementation

There is an implementation trick at play here, which is stolen straight out of the type classes proposal referenced above. The trick starts as follows:

  • Shapes are translated into interfaces, with each member (even static ones) turning into an instance member on the interface
  • Extensions are translated into structs, with each member (even static ones) turning into an instance member on the struct
  • If the extension implements one or more shapes, then the underlying struct implements the underlying interfaces of those shapes

In our example, the shape and extension declarations translate into this:

public interface SGroup<T>
{
    T op_Addition(T t1, T t2); // Can't use "operator +" here
    T Zero { get; }
}

public struct IntGroup : SGroup<int>
{
    public int op_Addition(int i1, int i2) => i1 + i2;
    public int Zero => 0;
}

(Instance declarations of operators aren't allowed in C#, so those + "methods" are encoded as instance methods, just as today's operator declarations are actually encoded as static methods in IL).

Note that the struct encoding of IntGroup has a declaration for the + operator even though the original extension declaration doesn't. It captures what it thinks + means on ints, and thus fulfills the SGroup<int> interface.

The generic method taking shape-constrained type parameters is translated as follows:

  • For each type parameter that is constrained by one or more shapes, the generic method actually gets an extra type parameter constrained by struct and by the underlying interfaces of those shapes
  • The method creates and keeps an instance of each of those extra type parameters (it can because of the struct constraint)
  • Whenever an operation from the shape is used in the body, the translation instead calls it on the corresponding instance

Let's see that on the AddAll method from above:

public static T AddAll<T, Impl>(T[] ts) where Impl : struct, SGroup<T>
{
    var impl = new Impl();

    var result = impl.Zero;
    foreach (var t in ts) { result = impl.op_Addition(result, t); }
    return result;
}

See how the extra Impl type parameter carries the knowledge of how to do + and Zero into the method. The benefit of doing this with a struct type parameter, rather than, say, extra delegate parameters, is that the runtime does a really good job of optimizing it: it will specialize the generic method code for each different struct it gets called with, so that the method body can inline and optimize the specific + and Zero implementations. Measurements on the linked type classes proposal show incredibly good performance, with near-zero cost to the abstraction.

In general I show the translations instantiating the Impl structs once when possible, but it is essentially free to create an instance of an empty struct, so we could also consider instantiating it every single time we need to call a member on it. That's less readable though.

Finally, there's a bit of extra work for the call site: It needs to infer and pass that extra type argument:

int[] numbers = { 5, 1, 9, 2, 3, 10, 8, 4, 7, 6 };
WriteLine(AddAll<int, IntGroup>(numbers));

It infers T = int the normal way, and then looks to find exactly one declaration in scope that implements SGroup<int> on int. Finding the IntGroup extension, it passes its underlying struct type. In case of ambiguities, the original code needs to disambiguate, just as when more than one extension applies elsewhere. We'll get to that later.

Implementing shapes directly

Once shapes are in the world, new types will want to implement them directly, instead of via an extension declaration:

public struct Z10 : SGroup<Z10>
{
    public readonly int I;
    public Z10(int i) => I = i % 10;
    public static Z10 operator +(Z10 z1, Z10 z2) => new Z10(z1.I + z2.I);
    public static Z10 Zero => new Z10(0);    
}

This is easily supported by simply a) checking that the type does indeed conform to the shape, and b) generating an extension next to the type declaration, or rather its underlying struct, witnessing the implementation:

public struct Z10
{
    public readonly int I;
    public Z10(int i) => I = i % 10;
    public static Z10 operator +(Z10 z1, Z10 z2) => new Z10(z1.I + z2.I);
    public static Z10 Zero => new Z10(0);
}

public struct __Z10_SComparable : SGroup<Z10>
{
    public Z10 op_Addition(Z10 t1, Z10 t2) => t1 + t2;
    public Z10 Zero => Z10.Zero;
}

Whenever the Z10 type is in scope, so is the fact that it is an SGroup<Z10>.

Instance members

So far we only explored shapes and extensions for static members, but they should apply equally to instance members.

public shape SComparable<T>
{
    int CompareTo(T t);
}

public extension IntComparable of int : SComparable<int>
{
    public int CompareTo(int t) => this - t;
}

In order to create the underlying interface for SComparable<T> we need to take a page out of the current extension methods feature and add an extra parameter to convey the receiver of the CompareTo call. What should be the type of that receiver? Well that depends on what type the shape is ultimately implemented on. In other words, we need to give the interface an extra type parameter representing the "this type", and let implementers fill that in:

public interface SComparable<This, T>
{
    int CompareTo(This @this, T t);
}

public struct IntComparable : SComparable<int, int>
{
    public int CompareTo(int @this, int t) => @this - t;
}

Essentially, any shape that defines instance members needs to also have an extra This type parameter.

From there on, the translation of generic methods over these shapes is unsurprising. This method:

public static T Max<T>(T[] ts) where T : SComparable<T>
{
    var result = ts[0];
    foreach (var t in ts) { if (result.CompareTo(t) < 0) result = t; }
    return result;
}

Translates to this:

public static T Max<T, Impl>(T[] ts) where Impl : struct, SComparable<T, T>
{
    var impl = new Impl();

    var result = ts[0];
    foreach (var t in ts) { if (impl.CompareTo(result, t) < 0) result = t; }
    return result;
}

The instance method call result.CompareTo(t) "on" the result gets translated into an instance method call impl.CompareTo(result, t) on the impl struct, taking the "receiver" as a first parameter.

Extending interfaces with shapes

Note that the shape SComparable<T> is almost identical to the existing interface IComparable<T>. Obviously there's a completely trivial implementation of SComparable<T> on any T that implements IComparable<T>, and so we can write that implementation once and for all by extending the interface itself:

public extension Comparable<T> of IComparable<T> : SComparable<T> ;

We don't even need to provide a body; the compiler can just figure it out. We just have to say it to make it true (and to declare the underlying struct to "witness" the SComparableness to generic methods).

Under the hood, the compiler translates to:

public struct Comparable<T> : SComparable<T, T> where T: IComparable<T>
{
    public int CompareTo(T @this, T t) => @this.CompareTo(t);
}

Shapes in generic types

So far we've seen generic methods with type parameters constrained by shapes. We can do the same for generic classes, where a given type argument gets to come in with its own way of doing certain things.

As an example let's build a SortedList<T> where T needs to be SComparable<T>. This will then work both for T's that inherently implement IComparable<T> (and hence SComparable<T> if the previous section is applied), but for other T's the instantiator of SortedList<T> can apply an extension and imbue T with a suitable comparison to apply inside of the list (please forgive algorithmic errors! 😀):

public class SortedList<T> where T : SComparable<T>
{
    List<T> ts = new List<T>();
    
    public void Add(T t)
    {
        int l = 0, r = ts.Count;
        while (l < r)
        {
            int m = (l + r) / 2;
            if (t.CompareTo(ts[m]) < 0) { r = m; }
            else { l = m + 1; }
        }
    }
}

We can implement this much like we do with generic methods, adding an extra type parameter to pass in the implementation struct. We can even store an instance of that struct in a static field if we want.

public class SortedList<T, Impl> where Impl : struct, SComparable<T, T>
{
    static Impl impl = new Impl();

    List<T> ts = new List<T>();
    
    public void Add(T t)
    {
        int l = 0, r = ts.Count;
        while (l < r)
        {
            int m = (l + r) / 2;
            if (impl.CompareTo(t, ts[m]) < 0) { r = m; }
            else { l = m + 1; }
        }
    }
}

One problem here is that the Impl type argument becomes part of the type identity of the constructed SortedList type. So if SortedList<T> is constructed with the same explicit type argument in two different places that implement SComparable<T> with different extensions, those are different constructed SortedList<T> types! The shape implementation becomes part of the type identity, and if it differs, those types are not interchangeable.

Also, generic types can be overloaded on arity, so introducing secret extra type parameters can potentially throw a wrench into families of generic types all differing only on arity.

The type classes proposal linked above actually makes the "implicit" type parameters explicit. This comes with its own problems, but does have the advantage that the number of type parameters shown in source code corresponds to the number in IL.

Extensions on shapes

Using an approach similar to the shape-parametized types above, we can let extensions extend shapes, not just types. Let's say we want to write an extension that offers the trivial implementation of all the comparison operators on everything that implements SComparable<T>:

public extension Comparison<T> of SComparable<T>
{
    public bool operator ==(T t1, T t2) => t1.CompareTo(t2) == 0;
    public bool operator !=(T t1, T t2) => t1.CompareTo(t2) != 0;
    public bool operator > (T t1, T t2) => t1.CompareTo(t2) >  0;
    public bool operator >=(T t1, T t2) => t1.CompareTo(t2) >= 0;
    public bool operator < (T t1, T t2) => t1.CompareTo(t2) <  0;
    public bool operator <=(T t1, T t2) => t1.CompareTo(t2) <= 0;
}

Just like the generic methods and types explored above, the underlying struct for this extension needs to have an extra type parameter for the implementation of the SComparable<T, T> interface:

public struct Comparison<T, Impl> where Impl : struct, SComparable<T, T>
{
    static Impl impl = new Impl();

    public bool op_Equality(T t1, T t2) => impl.CompareTo(t1, t2) == 0;
    public bool op_Inequality(T t1, T t2) => impl.CompareTo(t1, t2) != 0;
    public bool op_GreaterThan(T t1, T t2) => impl.CompareTo(t1, t2) > 0;
    public bool op_GreaterThanOrEqual(T t1, T t2) => impl.CompareTo(t1, t2) >= 0;
    public bool op_LessThan(T t1, T t2) => impl.CompareTo(t1, t2) < 0;
    public bool op_LessThanOrEqual(T t1, T t2) => impl.CompareTo(t1, t2) <= 0;
}

If that extension is in scope at the declaration of the Max method above, the comparison operators can now be used directly:

public static T Max<T>(T[] ts) where T : SComparable<T>
{
    var result = ts[0];
    foreach (var t in ts) { if (result < t) result = t; }
    return result;
}

This gets straightforwardly implemented by passing the method's Impl type parameter (implementing SComparable) to the Comparison struct above, instantiating that, and calling its operator implementations:

public static T Max<T, Impl>(T[] ts) where Impl : struct, SComparable<T, T>
{
    var impl = new Comparison<T, Impl>();

    var result = ts[0];
    foreach (var t in ts) { if (impl.op_LessThan(result, t)) result = t; }
    return result;
}

Explicit implementation and disambiguation

This section is a potentially useful tangent, that one can choose to go down only a certain part of the way.

We can consider explicit implementation, akin to what interfaces have, where the shape's members don't show up on the extended types themselves, but only when accessed through the shape directly. For instance, integers can also be viewed as a group under multiplication, but since that would mean implementing + as * and Zero as 1, we would not have those versions show up directly on the int type:

public extension IntMulGroup of int : SGroup<int>
{
	static int operator Sgroup<int>.+(int i1, int i2) => i1 * i2;
	static int SGroup<int>.Zero => 1;
}

Thus, if both IntGroup and IntMulGroup were in scope, int.Zero would still yield 0, not 1.

When passing an SGroup constrained type argument, however, we'd still want to be able to disambiguate whether we meant "int with addition" or "int with multiplication".

Specifying which shape or extension to use

When there is more than one declaration in scope providing a given member or shape implementation, the compiler cannot automatically infer which one to use. We may be able to give sensible resolution rules that deal with a lot of cases, but there's going to be situations where you want to specify which extension declaration you meant to use.

AddAll(numbers); // use IntGroup or IntMulGroup?
AddAll<int>(numbers); // Doesn't help, it's Impl that can't be inferred, not T

An approach to this could be to simply allow the name of the extension declaration itself as a type name, with the rough meaning of "same type as the extended type, but give priority to this extension." It's sort of similar to base meaning "this type, but start member lookup in the base type":

AddAll<IntMulGroup>(numbers); // becomes AddAll<int, IntMulGroup>(numbers)

This would also work as an approach to get at explicitly implemented members:

IntMulGroup.Zero; // 1;

When accessing instance members on a receiver, to get at an explicitly implemented member, or to choose an extension to "view it as", cast the instance to the shape or extension name:

((SComparable<Point>)p1).CompareTo(p2); // Access an explicitly implemented but unambiguous member
((PointComparable))p1).CompareTo(p2);   // Access an ambiguous member by naming the declaring extension

Or maybe it looks better with and as expression:

(p1 as SComparable<Point>).CompareTo(p2);
(p1 as PointComparable).CompareTo(p2);

Using extensions as types

The number of places where you can use shapes as types is very limited: we've only seen them as constraints and in disambiguating uses. That is because they do not correspond to a single underlying type.

Extensions however, really do correspond to a single underlying type: the one that they extend. We could therefore imagine allowing them to be used as types of fields, parameters, etc. They would then denote, at runtime, the underlying type, but the compiler would know to "view it as" the extension.

Let's again imagine that PointComparable explicitly implements SComparable<Point> on the type Point. But now I want to write code that compares Points all the time, and I don't want to have to cast every single time. Instead, can I just declare that I want to view these particular ints as PointComparable's?:

PointComparable[] ps = GetPoints();
...
ps[i].CompareTo(ps[j]);

This translates into:

var impl = new PointComparable();

Point[] ps = GetPoints();
...
impl.CompareTo(ps[i], ps[j]);

For public interfaces we would have a way to signal the "overlay" extension type in metadata, e.g. through an attribute.

Extensions as wrapper types

One potentially useful further step to this, is to allow extensions to explicitly implement their own members, not just ones from shapes. What it would mean is, they don't actually expose the member on the underlying extended type, but only when the extension itself is used as the type.

This can be used to create compile time "wrapper types", that compile down to using the underlying type at runtime, but give it an extra face at compile time:

public extension JPoint of JObject
{
	public int JPoint.X => (int)this["X"];
	public int JPoint.Y => (int)this["Y"];
}

JObject o = GetObject[];
WriteLine(o.X); // Error: X is not exposed on JObject, because it is explicitly implemented
JPoint p = o;
WriteLine(p.X); // Now the JObject is seen as a JPoint, so X is there

This is an example of giving a typed overlay to something less typed. That appears to be a common scenario, and is the whole basis for e.g. TypeScript's type system. Whether or not this is the right mechanism for it is probably debatable, but it is certainly a mechanism.

Discussion

This is a very high level proposal - it is more than a proof on concept that a design exists, and many details would need to be locked down (and changed) if we want to pursue this, e.g.:

  • How is an extension brought into scope? Does it need to be usinged, or is it in effect just through its presence?
  • How exactly are instance extension members encoded, so that they can have the extra @this parameter?
  • Which rules should be used to pick which extension members are more specific, so that there aren't ambiguities all the time?
  • Etc...

Some issues with the proposal as it currently stands:

  • Two new "type declaration" forms to the language make it heavy on "concept".
  • Shapes and extensions are only "halfway" types, which may be a confusing notion to wrap your head around.
  • Hidden type parameters introduce a split between source and IL level generics, that may be ugly to pave over

Other directions one might explore to achieve some of the same goals:

  • Find a way to extend interfaces to play the role of shapes here: declare static members, apply after the fact, etc. This would likely require runtime changes, but maybe that's better all-up.
  • Something more dynamic: structural typing, duck typing, whatever the term. This has the potential to fail at runtime, if something "turns out" not to fit the shape you assumed at compile time, and also doesn't clearly address some of the more generic scenarios.

Looking forward to further discussion of the pros and cons!

Mads

@gafter

This comment has been minimized.

Copy link
Member

gafter commented Feb 22, 2017

A type class (shape) can declare conversions. I suspect you don't intend to allow extension declarations of conversions. Do you?

@gafter

This comment has been minimized.

Copy link
Member

gafter commented Feb 22, 2017

The only reason I can think of that a declaration such as

public extension Comparable<T> of IComparable<T> : SComparable<T> ;

is necessary is that not every type that implements IComparable<T> has members with the same signature (because if some type explicitly implements the interface, it doesn't have those members directly in its type).

@svick

This comment has been minimized.

Copy link
Contributor

svick commented Feb 22, 2017

This looks really interesting. My thoughts:

  1. Why do extension members need to be marked as public? Is there any other option? (The proposal examples also seems inconsistent about whether explicit extension members should be marked public.)

  2. Also, generic types can be overloaded on arity, so introducing secret extra type parameters can potentially throw a wrench into families of generic types all differing only on arity.

    Generic types are not overloaded on arity in IL, instead the number of type parameters is included in the name, after a backtick.

    So, what would happen if the actual type name contained the number of generic parameters in source? E.g. C# SortedList<T> → IL SortedList`1<T, Impl> (and not SortedList`2<T, Impl>).

    I think this would resolve the conflict, but it could throw off some tools that don't expect this (and it's also not CLS compatible, in case that matters).

  3. Can extensions as types propagate somehow? Consider this code:

    IntMulGroup a = 2;
    var b = a + a;         // a is IntMulGroup, so + is IntMulGroup.+
    IntMulGroup c = b + b; // b is int (?), so + is int.+ (??)

    Based on the proposal, I would expect that they don't propagate, so b is just an int and c would be 8. But I think the user would expect the result to be 16 (using IntMulGroup.+ in both cases), and so this would be an easy to make bug.

    Maybe declaring IntMulGroup.+ like this could work?

    static IntMulGroup operator Sgroup<int>.+(int i1, int i2) => i1 * i2;
@Joe4evr

This comment has been minimized.

Copy link

Joe4evr commented Feb 22, 2017

  • Shapes and extensions are only "halfway" types, which may be a confusing notion to wrap your head around.

Well, I have to say that this explanation of "Type classes" as shapes and how it could extend anything of a type was what I needed to wrap my head around the concept, at least, instead of needing a Ph. D in functional language jargon. This is now making me pretty excited for the possibilities this could bring once the kinks are worked out.

@alrz

This comment has been minimized.

Copy link
Contributor

alrz commented Feb 22, 2017

Find a way to extend interfaces to play the role of shapes here: declare static members, apply after the fact, etc. This would likely require runtime changes, but maybe that's better all-up.

I'd rather to wait for proper clr support for "virtual extension methods" (#52) and further, apply after the fact (dotnet/roslyn#8127). This looks too magical and honestly I don't see much value added for "generic numeric operations" specially that code generators can seemlessly support that scenario without performance penalty enforced via this proposal. Meanwhile, a portion of "extension everything" can be implemented right now as it doesn't need voodoo under the hood.

@Thaina

This comment has been minimized.

Copy link

Thaina commented Feb 22, 2017

This proposal seem like completely summarize functionality we requested so far

Still there are something I disagree with

  • Just personal but I wish there would be keyword better than shape
    • some keyword plus interface such as static interface maybe?
  • Please don't introduce extension of keyword, At least drop of keyword

Also I still want to propose static extend syntax

public static class IntExt : int,SGroup<int>
// must place type to extend before any shapes
// same rule as class : class,interface,interface
{
    // same new implementation
    // allow static class extended from type can implement non static member
}

However this is very interesting transpile implementation

@gafter

This comment has been minimized.

Copy link
Member

gafter commented Feb 22, 2017

@alrz I don't know what performance penalty you're referring to. The invocations of shape methods get specialized by the JIT so they become simple non-virtual invocations of static methods.

@MgSam

This comment has been minimized.

Copy link

MgSam commented Feb 22, 2017

@ArlZ Could you elaborate as to how you foresee code generators as seamlessly supporting writing generic numeric operations? What would that even look like?

@alrz

This comment has been minimized.

Copy link
Contributor

alrz commented Feb 22, 2017

Built-in operators don't emit methods, but shapes need a method call anyways. With generators one can overload a method over numeric types,

[NumericOverload]
int AddAll(int[] nums) {..}

I'd admit that wouldn't be an straightforward generator (and doesn't provide much flexibility) but still possible. I think other use cases like duck typing etc, would be addressed by #52 in which case it doesn't need complex code generation on the compiler side.

@MgSam

This comment has been minimized.

Copy link

MgSam commented Feb 22, 2017

So you'd have to use int as a proxy for all numeric types? Seems pretty messy to me. Working outside of the type system rather than with it.

I think the "just do it with generators instead" argument is kind of a rabbit hole. You could make an argument that many of the features already in C# shouldn't be there because they could be done with generators instead. Of course, doing this means the tooling support is vastly inferior and the code much harder to follow.

I think generators are a great feature, but they need to be used carefully and thoughtfully. Speaking from experience using PostSharp, though its powerful it makes it much harder to reason about what's happening with your code.

Also, I'm not sure why you keep making a perf argument against this feature. Mads/Gafter already said that tests showed nearly zero perf hit. If that ultimately changes I think they should take that into account but as it stands I'm satisfied to take their word on this.

@ufcpp

This comment has been minimized.

Copy link

ufcpp commented Feb 22, 2017

@alrz

The JIT optimizes struct generics very well. In Release build, virtual calls are replaced to non-virtual calls. And in many cases, non-virtual calls are expanded inline. As a result, the performance of the op_Addition is almost the same as the build-in add instruction.

Here is the benchmark:

https://gist.github.com/ufcpp/15af6d3d7606fb3771a91c81898dcfa3

@YaakovDavis

This comment has been minimized.

Copy link

YaakovDavis commented Feb 22, 2017

Regarding disambiguation, some new syntactic support could be handy:

AddAll<int as IntMulGroup>(numbers);
AddAll<int as IntGroup>(numbers);

The as Shape clause is necessary only in ambiguous cases.

@SamPruden

This comment has been minimized.

Copy link

SamPruden commented Feb 22, 2017

I imagine and hope the answer is yes, but would the following be a valid extension?

public extension PointlessExtension<T> of T
    where T: class
{
    public T PointlessReferenceToSelf => this;
}

In short, would the extension be applicable directly to the generic argument?

@eyalsk

This comment has been minimized.

Copy link
Contributor

eyalsk commented Feb 22, 2017

@TheOtherSamP How would it work? by reading this proposal it would get implemented like this:

public struct PointlessExtension<T> : T where T: class
{
}

Disregard the fact that it is a struct but can you derive from T today? something like the following might work:

public extension PointlessExtension<T> of Object
    where T: class
{
    public T PointlessReferenceToSelf => this;
}
@SamPruden

This comment has been minimized.

Copy link

SamPruden commented Feb 22, 2017

@eyalsk That's a fair question, and one I'll confess I gave little thought. I honestly don't know how to best make that work internally, but it feels like a fairly essential capability to me. That's something you can do with current extension methods, and something I use often enough that I know I'd miss it if it were lacking in these new extensions.

To give a slightly more useful (while still simple) example:

public static T[] ToArrayContaining<T>(this T item) => new []{item};

This is an extension method using the current syntax that uses this capability. Yes that would continue to work, but the ability to do things like this would be nice in the new syntax too. Without it, the new extension features would feel incomplete to me because they're taking a step back, and I'd end up mixing and matching between the two extension method syntaxes in my code. That feels wrong.

@eyalsk

This comment has been minimized.

Copy link
Contributor

eyalsk commented Feb 22, 2017

@TheOtherSamP I think this case is covered by my second code snippet but I could be wrong. :)

@SamPruden

This comment has been minimized.

Copy link

SamPruden commented Feb 22, 2017

@eyalsk Perhaps I'm not fully understanding how type parameters work on extensions. My reading of your second example is an extension method that applies to all Objects, but where T is not specified. You're suggesting that this would be of type T in this scenario? My interpretation was that this would be Object.

@eyalsk

This comment has been minimized.

Copy link
Contributor

eyalsk commented Feb 23, 2017

@TheOtherSamP Good point, however, I feel like these are two different kinds of extensions.

In the following case:

public static T[] ToArrayContaining<T>(this T item) => new []{item};

You don't derive from the type you pass the instance of the type whereas with this proposal you actually derive from the type to extend it.

@SamPruden

This comment has been minimized.

Copy link

SamPruden commented Feb 23, 2017

@eyalsk Perhaps you're right that there should be a distinction there, and frankly as of yet I haven't really studied the proposal in enough detail to comment on the specifics any further.

I would however restate that for me personally the lack of ability to represent public static T[] ToArrayContaining<T>(this T item) => new []{item}; in the new syntax and to do similar things on other types of extension members makes this an incomplete feeling version of extension everything. There's a lot of power lost in not being able to generically capture the type of the type being extended.

@eyalsk

This comment has been minimized.

Copy link
Contributor

eyalsk commented Feb 23, 2017

@TheOtherSamP

in the new syntax and to do similar things on other types of extension members makes this an incomplete feeling version of extension everything. There's a lot of power lost in not being able to generically capture the type of the type being extended.

You can extend any type except types that are "untyped", you're trying to derive from T where T has no meaning.

Again, I'm basing my assumptions on the proposal above and really guessing, it all depends on the design and the implementation of this feature and as @MadsTorgersen stated it's a very high level proposal. :)

@gafter

This comment has been minimized.

Copy link
Member

gafter commented Feb 23, 2017

FYI, extension everything has a working prototype at https://github.com/dotnet/roslyn/tree/features/extensionEverything

@Joe4evr

This comment has been minimized.

Copy link

Joe4evr commented Feb 23, 2017

So the opening post says this:

  • Can be used like a type only in certain places
    ....a shape is not a type. Instead, the primary purpose of a shape is to be used as a generic constraint....

Random thought, can a shape be used as a target to cast? Something like

public shape SIndexable<T>
{
    public T this[int index];
}

Now imagine this could be used in LINQ's ElementAt() method:

public static TSource ElementAt<TSource>(this IEnumerable<TSource> source, int index)
{
    if (source is SIndexable<TSource> list) //instead of the current cast to IList<TSource>
    {
        return list[index];
    }
    //....
}
@gafter

This comment has been minimized.

Copy link
Member

gafter commented Feb 23, 2017

No, shapes are not runtime types. The CLR cannot convert an object to a shape as the conversion is dependent on the compile-time context.

@bondsbw

This comment has been minimized.

Copy link

bondsbw commented Feb 23, 2017

If an interface and struct exist which look like a shape/extension:

public interface SGroup<T>
{
    T Zero { get; }
}

public struct IntGroup : SGroup<int>
{
    public int Zero => 0;
}

Would the compiler allow constraints and call sites to utilize them as if they were an actual set of shape/extension?

The motivation is a third-party library perspective. I might like to build a forward-compatible library in C# 6 which supports this feature for anyone using C# vNext.

(I purposely removed the operator since it introduces signature conversions which may not be well defined.)

@ufcpp

This comment has been minimized.

Copy link

ufcpp commented Feb 23, 2017

Can I use nested shapes? like:

shape SEnumerable<T>
{
    // The SEnumerator<T> shape is not a type. This method is not allowed. right?
    SEnumerator<T> GetEnumerator();
}

shape SEnumerator<T>
{
    bool MoveNext();
    T Current { get; }
    void Dispose();
}

I explored about whether I can use Shapes for no-allocation foreach. The resuit is not very good, but anyway I'll share the result:

https://gist.github.com/ufcpp/8085250f7e01d1cc0bb655a5eb1ef748

@ufcpp

This comment has been minimized.

Copy link

ufcpp commented Feb 23, 2017

The other usecase:

shape SAwatable
{
    SAwaiter GetAwaiter();
}

shape SAwaiter
{
    bool IsComplete { get; }
    void GetResult();
    void OnCompleted(Action continuation);;
}

// I want to use this method for Task, ValueTask, and any other awaitables
static async void FireAndForget<TTask>(TTask t)
    where TTask : SAwaitable
{
    try
    {
        await t;
    }
    catch(Exception ex)
    {
        System.Diagnostics.Debug.Write(ex);
    }
}
@MadsTorgersen

This comment has been minimized.

Copy link
Contributor Author

MadsTorgersen commented Feb 23, 2017

@TheOtherSamP Good point about the extended type itself being a type parameter. I hadn't thought about it, but I don't think there's anything in my proposal that outright prohibits that.

The extended type is used in two ways in the proposal:

  1. to determine whether the extension applies to a given type or instance receiver
  2. as the type of the @this parameter in the translation of extension instance members

While 2 is trivially achieved, 1 is a bit more involved. The case where a type parameter T of the extension is the extended type is a special case of where such T parameters occur in the extended type E. What we need to do there is a type inference between the receiver type R and E to find the type arguments for extension. It's as is we were calling some

void M<T1...Tn>(E @this);

With the receiver: M(receiver), inferring T1...Tn and checking that the method applies and that all the constraints are fulfilled.

All the machinery is in the language for this today, and indeed is used in the resolution of today's extension methods, as you point out. So I believe it should work just fine.

@MadsTorgersen

This comment has been minimized.

Copy link
Contributor Author

MadsTorgersen commented Feb 23, 2017

@gafter about conversions: It is likely that user defined conversions (especially implicit ones) are too scary to allow in extensions: after all it is very subtle that assignment between two types suddenly works in a given scope.

Similarly, we might want to consider whether to allow the == and != operators. Since they are already defined for all types, defining them would not add equality to a type, but would change it.

If we choose to have these restrictions, we might be more lenient when those are declared as explicit implementations, since they wouldn't then just automatically apply to the extended types.

For user defined operators in general, today most of them come with a number of rules on the types involved, etc, and we should continue to apply the same or similar rules in extension operators.

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

Could somebody please share the status of this feature? Is it planned, and when for, or just being discussed?

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

AFAIK nothing is planned, and it is unlikely this will even be looked at seriously till after C# 8 is out.

At the moment I think the status of this and similar proposals is: let's get the discussion rolling, so that when we do want to look into creating a concrete design, we've already sounded the waters on what would be valuable and what not, as well as what it might look like.

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

Thank you @YairHalberstadt. I guess I'll add my share to the discussion then. What I need is simply extensions for everything unsupported at present:

  • constructors;
  • destructors;
  • properties;
  • static methods.
@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

Destructors are probably impossible, since they are called by the GC, so it would require serious changes to the runtime to get that to work.

Also destructors are generally not very useful, and require serious knowledge about how the runtime works to be used effectively. For example, the spec makes no guarantees a destructor will run at all.

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

Assuming it's guaranteed destructors cannot be extended, I could do without them. But not without any of the rest.

@yaakov-h

This comment has been minimized.

Copy link
Contributor

yaakov-h commented Feb 6, 2019

Constructors might be tricky, too. I assume extension constructors would get compiled into factory methods or similar.

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

Yep, I think extension constructors would just be syntactic sugar over a factory method.

@masonwheeler

This comment has been minimized.

Copy link

masonwheeler commented Feb 6, 2019

I'm not sure there's any way to implement an extension static constructor, particularly on a type in another assembly. Static constructors involve heavy doses of Runtime magic to work the way they do, and just thinking about the implications is enough to make my head hurt...

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

Normal constructors are all I need, thanks 😀.

@masonwheeler

This comment has been minimized.

Copy link

masonwheeler commented Feb 6, 2019

As long as we're talking about constructors, would anyone else really love to see a way to make initializer syntax and readonly properties / fields work together?

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

@masonwheeler
Yes, but I haven't seen any design I like for this. I think there is a championed proposal for this somewhere, but it didn't provide any details about the design.

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

@HaloFour

This comment has been minimized.

Copy link
Contributor

HaloFour commented Feb 6, 2019

@YairHalberstadt @masonwheeler

#1667
#1683

Relevant to where the current thinking on this may be:

#1667 (comment)

@jnm2

This comment has been minimized.

Copy link
Contributor

jnm2 commented Feb 6, 2019

@ddobrev

Assuming it's guaranteed destructors cannot be extended, I could do without them. But not without any of the rest.

This seems like a stretch. A significant amount of software has been implemented using C# since 1.0, so someone has figured out how to do without these things.

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

@jnm2 there's a strictly defined situation I need them for which cannot be handled by anything else. It's not confidential so feel free to ask but I warn that it's a lengthy and complicated explanation.

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

@ddobrev
If you genuinely feel you've come across a situation which the language is currently incapable of handling, I suggest you open an issue here, or you could discuss it on gitter if you'd prefer to do so more informally.

@masonwheeler

This comment has been minimized.

Copy link

masonwheeler commented Feb 6, 2019

@ddobrev OK, now this I gotta hear. Please share?

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

@YairHalberstadt I don't think there's any need to open a new issue since this one would solve the problem in question.
@masonwheeler of course, please read below.
I develop along with @tritao https://github.com/mono/CppSharp (which is, by the way, a Microsoft project since it's under Mono, and unfortunately largely ignored and unappreciated by Microsoft itself). It generates bridges from C++ to C#. I want to represent templates and this is only partially possible automatically. The reason is that the binary code to actually call does not exist in C++ until compilation. This limits usage of templates to just C++ types and which are already in use in the wrapped native library. In short, the generic representation which I want for more familiar to the C++ users syntax, would only work with a limited set of specializations. This along with type safety gave me the idea of using extension methods. This way if Template<int> is supported, a user would get its methods at compile time but if it isn't, he won't because there would be no extensions. By now you already see the problem that I can only represent instance methods as extensions, while I also need statics and constructors, and also properties as I use heuristics to turn certain methods or pairs of them into properties.

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

@ddobrev
I see where you are coming from.

I can't imagine seamless interop between C# and C++ can ever happen, but certainly the more tricks you have at your disposal, the better you can get.

The current proposal is purely a compile time trick over the witness pattern.

Would simply implementing the witness pattern itself be sufficient, or is the syntax sugar this provides necessary?

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

@YairHalberstadt complete interop is as impossible as unnecessary in the overwhelming majority of cases. Macros, for example, are really difficult to map but most decent C++ API-s rely on functions and classes instead, and the only feature there which can be partially represented is precisely templates.
I haven't heard of the witness pattern though you might mean the observer pattern but even if I did, it wouldn't matter because, as you've prudently asked, I need the syntactic sugar. I want the generated bridges to feel as natural as possible to users, without the latter constantly reading documentation about the internals of C++# or asking maintainers such as myself. That is, generate an API as clean and simple as possible, build the documentation into it by mimicking the original C++ as closely as the two pieces of technology (C++ and C#) allow me to.

@masonwheeler

This comment has been minimized.

Copy link

masonwheeler commented Feb 6, 2019

@ddobrev there's nothing unfortunate about diminishing the influence of C++ in the programming world. If Microsoft is paying less attention to that, it's admirable!

@YairHalberstadt

This comment has been minimized.

Copy link
Contributor

YairHalberstadt commented Feb 6, 2019

@ddobrev
That's fair enough.

I made up the term witness pattern to describe the implementation this proposal uses.

You can read the proposal if you want to know the details. I don't have time right now to describe it properly, but as you've said, it probably wouldn't help you.

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

@YairHalberstadt got it, I'll read it later, it might contain some useful insight after all.
@masonwheeler I've felt enough disappointment with Microsoft giving its own code the cold shoulder to put up with your admiration of the fact. Mostly because you have completely misunderstood the purpose of CppSharp. It generates bridges from C++, it does not extend or even encourage and condone C++. On the opposite, CppSharp has the potential to rid the entire IT industry of C++ as new code would be written in higher level languages, such as C#, because existing C++ code would only be consumed through these bridges. Your comment is therefore both harmful (to CppSharp) and hurtful (to me) and I'd like to let you know I expect your apology.

@CyrusNajmabadi

This comment has been minimized.

Copy link

CyrusNajmabadi commented Feb 6, 2019

@ddobrev Feel free to discuss things more over at gitter.it/dotnet/csharplang. It's usually better for these sort of free-form exploratory discussions.

@ddobrev

This comment has been minimized.

Copy link

ddobrev commented Feb 6, 2019

@CyrusNajmabadi thank you, I'll have it in mind.

@Marko-Petek

This comment has been minimized.

Copy link

Marko-Petek commented Apr 10, 2019

When you get to implementing generic arithmetic into the language, keep in mind that the performance should be the same as it is now with concrete types. Otherwise, scientists and engineers will keep using workarounds: https://www.codeproject.com/Articles/8531/Using-generics-for-calculations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.