Support target-typed new #25196

alrz · 2018-03-02T20:52:56Z

Proposal: https://github.com/dotnet/csharplang/blob/master/proposals/target-typed-new.md
Test plan: #28489

alrz · 2018-03-02T20:57:11Z

src/Compilers/CSharp/Portable/Binder/Binder_Conversions.cs

+            {
+                // TODO(target-typed-new): Use uncommon data to pass over the already computed 
+                // TODO(target-typed-new): overload resolution results from succeeded conversion
+                // TODO(target-typed-new): to manually populate a BoundObjectCreationExpression


I held this off until the design is finalized. There is certainly room for improvements i.e. we can avoid some overload resolution pass in some codepathes. I guess we should forbid dynamic arguments for now. When that's confirmed I'll refactor to eliminate redundant analysis and reuse precomputed overload resolution results.

jcouv · 2018-03-02T20:59:34Z

Thanks. Marked as blocked until LDM reviews the language design proposal. #Closed

jcouv · 2018-04-04T23:20:09Z

Marking as "for personal review" for now, so it doesn't show in our "active reviews" query.
LDM is out of session for next 4 weeks or so.
I've added a note for this issue in https://github.com/dotnet/roslyn/projects/27 to remind me to push this onto the agenda once LDM restarts. #Closed

jcouv · 2018-05-21T19:25:47Z

@alrz Sorry for the delay. LDM discussed this today and supports the feature (as usual, modulo unforseen major issues we might discover later). 🎉

As a result, we can start the review process for the PR. It would probably be good to refresh/rebase as a starting point.

We discussed the overload resolution rules and compat implications (see upcoming notes).
In short, an argument that is a target-typed new behaves like an out var in that it doesn't contribute a type to overload resolution. That behavior mitigates some of the compat implications of the feature.

We didn't get to cover other open issues, but I think we can get them on LDM agenda soon. Let's document them in the championed issue.
#Closed

CyrusNajmabadi · 2018-05-21T21:05:42Z

We discussed the overload resolution rules and compat implications (see upcoming notes).
In short, an argument that is a target-typed new behaves like an out var in that it doesn't contribute a type to overload resolution. That behavior mitigates some of the compat implications of the feature.

Interesting! Though somewhat a pity as it means if you have overloads that are otherwise identical you'll have an ambiguity. But that's likely a good place to start from . In the future, it seems like actual rules for deciding how the new call could be part of overload resolution could be added solely to solve those ambiguous issues in a non-breaking fashion.

Neme12 · 2018-05-21T21:14:35Z

Though somewhat a pity as it means if you have overloads that are otherwise identical you'll have an ambiguity.

Why? There are already perfectly good ways to resolve an ambiguity, the best one being just specifying the type of what you're creating. It's not clear at the call point what it is anyway. Overloading based on the arguments to new() seems really like a bad idea. Overload resolution is already complicated as it is. Why open a whole new can of worms?

Isn't the main scenario for target typed new just:

private readonly Dictionary<(string a, blah b), IEnumerable<Dictionary<string, (int blah, blahlabh b)>))> d = new();

?

I don't see how sprinkling new() all over method calls especially considering how extensively C# uses overload resolution is a good idea or makes the code clearer.

Though somewhat a pity

Can you give an example of code where this would be worth pitying over?

Neme12 · 2018-05-21T21:21:49Z

I'm sorry, that's just how I see it. Please give me an example of where new() contributing to overload resolution would be useful and not make the code unreadable and error-prone.

I thought this would work just like default - as a shorthand when it's absolutely clear what the target type is. If there's any sort of ambiguity, just specify the type you need. Or if you prefer, use named argument. I don't see any problem with that.

jaredpar · 2018-05-21T22:47:56Z

I'm sorry, that's just how I see it. Please give me an example of where new() contributing to overload resolution would be useful and not make the code unreadable and error-prone.

Consider this scenario:

// Util.dll 
public class C1 { } 
public class C2 {
  public C2(int count) { } 
}
public class C3 {
  public static M(C1 p) { } 
  public static M(C2 p) { }
}

Now consider I consume the code from a different library in the following manner:

C3.M(new (1));

So far so good. If new contributed to type inferencee then this would compile fine and everything would be great. Now imagine time passes and the author of Util.dll adds the following constructor:

public C1(int i) { }

Now suddenly the consume is broken because new (1) is ambiguous between C1 and C2. Expanding this out it essentially means that evyr new constructur is potentially a source breaking change in the face of target typed new. This is the core scenario we're trying to guard against.

jcouv · 2018-05-21T22:53:33Z

@jaredpar I think you answered a different question, and instead supported @Neme12's concern about ambiguities :-) #Closed

Neme12 · 2018-05-21T23:23:52Z

@jaredpar Thanks for the response, although I'm a little confused. I was arguing against new contributing to type inference. But I didn't even think of this problem with compatibility. Thanks for the explanation.

I probably didn't make myself clear enough, which is my fault. By "why" I was asking why the proposed behavior is pity and was surprised that changing new to affect overload resolution would be considered a welcome change for a future version.

jaredpar · 2018-05-21T23:59:49Z

Thanks for the response, although I'm a little confused. I was arguing against new contributing to type inference

Sorry. I was in a rush to finish my comment before another meeting started and misread your feedback.

But I didn't even think of this problem with compatibility.

My mind is twisted to think about evil things after years of compatibility bugs 😄

CyrusNajmabadi · 2018-05-22T01:03:47Z

Why open a whole new can of worms?

I explicitly did not say that. I said precisely that: "that's likely a good place to start from"

:)

I simply said if there was any limitations here they could be considered in the future, not that htey should or that there was any problem here. It was a show of support for this approach precisely because it's likely good enough and because it likely doesn't shut down potential improvements in the future. Both of these are important for me when designing language features. I very much want to ensure that when decisions are made now, they don't shut down possible areas of interest that may arise later.

CyrusNajmabadi · 2018-05-22T01:05:32Z

I thought this would work just like default - as a shorthand when it's absolutely clear what the target type is. If there's any sort of ambiguity, just specify the type you need. Or if you prefer, use named argument. I don't see any problem with that.

That seems totally fine to me. Importantly, i think that's a great place to start at. And, if it ends up potentially being limiting, is something that could potentially addressed in the future in a way that i do not think would cause breaking changes.

CyrusNajmabadi · 2018-05-22T01:07:40Z

Now suddenly the consume is broken because new (1) is ambiguous between C1 and C2. Expanding this out it essentially means that evyr new constructur is potentially a source breaking change in the face of target typed new. This is the core scenario we're trying to guard against.

Well, this is a situation with "default" as well. If i have a method:

void M(int i) { }

And someone calls: M(default);

it's now source-breaking if i add:

void M(bool b) { }

because the 'default' is ambiguous.

I think that's an acceptable place to be in.

jaredpar · 2018-05-22T01:12:13Z

@CyrusNajmabadi

Well, this is a situation with "default" as well.

The situation is a bit different with default. In the case of default the source breaking change comes when an overload of an existing method is added. That is an understood place where source breaks can happen which default does make worse.

In the case of target typed new the break comes when a constructor is added to a type. This ends up breaking an existing method overload that the type author was quite possible completely unaware of.

The way I think about target typed new is that for the purpose of overload resolution defining a new constructor is like defining a new implicit conversion on the type.

CyrusNajmabadi · 2018-05-22T01:21:20Z

In the case of default the source breaking change comes when an overload of an existing method is added. That is an understood place where source breaks can happen which default does make worse.

I guess i long internalized that adding methods could trivially lead to source-breaks. So having a way for adding constructors to break things doesn't really bother me :)

That said, i'm not pushing for this to be supported. As i said, i think this is a good place to start from. And only at some future point would this need to be revisited if there was particular value in expanding support here.

jaredpar · 2018-05-22T03:39:26Z

@CyrusNajmabadi

I guess i long internalized that adding methods could trivially lead to source-breaks.

Agree. I think the difference here for me is the proximity of the change to the break. When adding an overload I'm both:

Aware I'm introducing a potential break. The other overload is generally in the same source file and same section as the new code I'm adding.
Able to look at the set of overloads in totality and evaluate the risk of a breaking source change.

Those aren't absolutes: extension methods, partial classes, etc ... can make it harder. But generally speaking true. The same is not true with target type new. The new constructor and the overload it affects don't have to be in the same source file or even the same library.

jcouv · 2018-05-22T19:12:49Z

In short, an argument that is a target-typed new behaves like an out var in that it doesn't contribute a type to overload resolution.

From discussion with @gafter, my summary is incorrect.

Here's a better summary: A target-typed new is never a better "conversion from expression" to one type versus another.

Neal is recommending to wait for LDM to decide on the initializer scenario before moving ahead with this PR.
Here's the example:
M(new () { x = 3 }) where two overloads exist (M(C) and M(D)). Suppose that D doesn't have a property/field named x, does expression new () { x = 3 } convert to D or not?

For the record, some scenarios Neal explained to me:

[Updated] If you have Base and Derived, both with empty constructors, then M(new ()) would ~~pick Derived because it is more specific.~~ be ambiguous.
M(new (i: 1, j: 2)) when we have two overloads (one with C and the other with D) and two constructors C(int i, int j) and D(int x, int y). Only the overload with C is applicable because expression new (i: 1, j: 2) does not convert to D. #Closed

jcouv · 2018-07-31T22:19:51Z

src/Compilers/CSharp/Portable/BoundTree/BoundNodes.xml

+  <Node Name="UnboundObjectCreationExpression" Base="BoundExpression">
+    <!-- Type is not significant for this node type; always null -->
+    <Field Name="Type" Type="TypeSymbol" Override="true" Null="always"/>
+    <Field Name="AnalyzedArguments" Type="AnalyzedArguments"/>


AnalyzedArguments [](start = 17, length = 17)

I don't think we should hold onto a pooled object in the bound tree. It's unclear who's responsible for freeing it... #Closed

So you're suggesting we postpone binding arguments (potentially until we create the conversion)? In that case, we should be responsible for binding the argument list in error cases when there's no destination type to convert to e.g. new(a, b);. #Closed

Some alternatives for discussion:

we could bind arguments twice (once in BindObjectCreationExpression as today, and a second time, in CreateImplicitNewConversion, which would discard diagnostics), but that is somewhat wasteful

we could use a non-pooled instance of AnalyzedArguments, but that is a lot of plumbing/infrastructure change to do cleanly (we'd want to split AnalyzedArguments into a non-poolable base type and a poolable derived type, modify consumers to work with non-poolable one, etc).

I'd suggest we go with (1) for now, with a PROTOTYPE marker.

In reply to: 206795081 [](ancestors = 206795081)

As yet another alternative, I think we could bind the arg list in CreateImplicitNewConversion and another time in some other phase (likely DiagnosticsPass?) when we encounter a UnboundObjectCreationExpression. This way, we only bind it once, for the error cases or otherwise.

jcouv · 2018-07-31T22:22:26Z

src/Compilers/CSharp/Portable/Parser/LanguageParser.cs

            }
            finally
            {
                this.Reset(ref resetPoint);
                this.Release(ref resetPoint);
            }
+
+            isPossibleArrayCreation =


isPossibleArrayCreation [](start = 12, length = 23)

Just curious: What prompted you to move the initialization of isPossibleArrayCreation outside of the try/finally? #Closed

we capture all the information required in try block, so I thought this would be more clear. #Closed

jcouv · 2018-07-31T23:01:42Z

        comp.VerifyDiagnostics(

Nice. Thanks! #Closed

Refers to: src/Compilers/CSharp/Test/Emit/CodeGen/CodeGenTupleTest.cs:939 in 141f778. [](commit_id = 141f778, deletion_comment = False)

jcouv · 2018-07-31T23:04:10Z

src/Compilers/CSharp/Test/Semantic/IOperation/IOperationTests_IObjectCreationExpression.cs

@@ -9,6 +9,1404 @@ namespace Microsoft.CodeAnalysis.CSharp.UnitTests
 {
    public partial class IOperationTests : SemanticModelTestBase
    {
+        [CompilerTrait(CompilerFeature.IOperation)]
+        [Fact]
+        public void TargetTypedObjectCreationWithMemberInitializers()


Thanks for adding those IOperation tests!
Tagging @333fred @mavasani in case they want to take a look at those tests too. #Closed

jcouv · 2018-07-31T23:12:06Z

src/Compilers/CSharp/Test/Semantic/IOperation/IOperationTests_IObjectCreationExpression.cs

+              OutConversion: CommonConversion (Exists: True, IsIdentity: True, IsNumeric: False, IsReference: False, IsUserDefined: False) (MethodSymbol: null)
+            IArgumentOperation (ArgumentKind.DefaultValue, Matching Parameter: o) (OperationKind.Argument, Type: null, IsImplicit) (Syntax: '2')
+              ILiteralOperation (OperationKind.Literal, Type: System.Object, Constant: null, IsImplicit) (Syntax: '2')
+              InConversion: CommonConversion (Exists: True, IsIdentity: True,


} [](start = 8, length = 1)

If you don't mind, could you add one IOperations test with arguments in new(...) as well? Thanks

jcouv

Done with review pass (iteration 35).
Only blocking concern is holding onto pooled object (AnalyzedArguments) in the bound tree. Aside from that a couple minor questions/suggestions.
Thanks!

jcouv

LGTM (iteration 37).
Thanks for sorting the AnalyzedArguments situation out.

gafter · 2018-08-02T23:44:21Z

I'm reviewing this.

gafter · 2018-08-03T17:35:02Z

src/Compilers/CSharp/Portable/BoundTree/BoundNodes.xml

@@ -1292,7 +1292,14 @@
    <!-- BinderOpt is added as a temporary solution for IOperation implementation and should probably be removed in the future -->
    <Field Name="BinderOpt" Type="Binder" Null="allow"/>
  </Node>
-
+
+  <Node Name="UnboundObjectCreationExpression" Base="BoundExpression">


Since the object initializer is logically part of the (un)bound object creation expression, it should be captured here. Probably as an optional InitializerExpressionSyntax. The caller should not assume that the syntax of this bound node is of the proper form, as that would prevent us from using it as an intermediary in translations (of perhaps new, different syntax forms) later.

Done.

Please review the follow-up PR at #29167 after this goes in. thanks.

gafter · 2018-08-03T17:36:18Z

src/Compilers/CSharp/Portable/Binder/Binder_Conversions.cs

@@ -128,6 +133,126 @@ internal partial class Binder
            { WasCompilerGenerated = wasCompilerGenerated };
        }

+        private BoundExpression CreateImplicitNewConversion(SyntaxNode syntax, BoundExpression source, Conversion conversion, bool isCast, TypeSymbol destination, DiagnosticBag diagnostics)
+        {
+            var node = (ObjectCreationExpressionSyntax)source.Syntax;


ObjectCreationExpressionSyntax [](start = 24, length = 30)

As I mentioned in BoundNodes.xml, we should not require a relationship between an UnboundObjectCreationExpression and its syntax node. Please store the initializer (or anything else needed) in the UnboundObjectCreationExpression.

gafter · 2018-08-03T17:39:44Z

src/Compilers/CSharp/Portable/Binder/Binder_Conversions.cs

+                var unboundObjectCreation = (UnboundObjectCreationExpression)source;
+
+                // PROTOTYPE(target-typed-new): Reconstruct AnalyzedArguments to avoid binding twice
+                arguments.Arguments.AddRange(unboundObjectCreation.Arguments);


I think this comment can be deleted now?

gafter · 2018-08-03T17:45:57Z

src/Compilers/CSharp/Portable/Binder/Binder_Conversions.cs

+                        @checked: false,
+                        explicitCastInCode: isCast,
+                        constantValueOpt: null, // A "target-typed new" would never produce a constant.
+                        type: destination);


This should be explicit in the spec, if it isn't already, as the following corresponding code is accepted today:

const int N = new int();

I think there is a rule that a value type's default constructor can't be used; that is what causes this (commented fact) to be true. #ByDesign

gafter · 2018-08-03T19:51:03Z

src/Compilers/CSharp/Portable/Binder/Semantics/Conversions/UserDefinedImplicitConversions.cs

@@ -609,6 +609,9 @@ private static bool IsEncompassingImplicitConversionKind(ConversionKind kind)
                case ConversionKind.ImplicitTupleLiteral:
                case ConversionKind.ImplicitTuple:
                case ConversionKind.ImplicitThrow:
+
+                // Added for C# 8.
+                case ConversionKind.ImplicitNew:
                    return true;


Do we have a test that exercises this branch?

Looks like it's not reachable for ImplicitThrow as well. My understanding is that since both new and throw are convertible to any type, there's no conversion left for these to encompass. Is that correct?

gafter · 2018-08-03T19:54:00Z

src/Compilers/CSharp/Portable/Lowering/LocalRewriter/LocalRewriter_Conversion.cs

+
+                case ConversionKind.ImplicitNew:
+                    return rewrittenOperand;
+


Is this branch exercised by tests? I'm wondering if it is reachable.

Now that we directly return the bound object creation, it's not reachable. will revert.

gafter · 2018-08-03T20:04:17Z

src/Compilers/CSharp/Test/Semantic/Semantics/TargetTypedNewTests.cs

+                );
+        }
+    }
+}


Can you please add tests to show how the semantic model works? In normal (non-error) cases, e.g. for the whole expression and its argument expressions, but also inside the object initializer, and inside the object initializer when there is no target type (error scenario), for example

M(new() { X = <bind>N()</bind> }); // M not found, but N is found. Also try when M is found but no X is found

How does GetTypeInfo work on the subexpression N()? Even if it doesn't work, it would be nice to have a test documenting the current (expected?) behavior.

If it's ok I'll address these in the follow-up PR.

gafter · 2018-08-03T20:05:20Z

Done with review pass (Iteration 37). Generally it is looking quite good, except as noted.

In reply to: 410101220 [](ancestors = 410101220)

alrz · 2018-08-08T16:08:47Z

The next PR based on these changes is opened at #29167

gafter

alrz requested a review from a team as a code owner March 2, 2018 20:52

alrz commented Mar 2, 2018

View reviewed changes

jcouv added Area-Compilers Blocked labels Mar 2, 2018

jcouv added this to the 16.0 milestone Mar 2, 2018

jcouv added the Community The pull request was submitted by a contributor who is not a Microsoft employee. label Mar 2, 2018

alrz force-pushed the features/target-typed-new branch from 4c1b011 to 54cc99c Compare March 2, 2018 21:03

jcouv added the Area-Language Design label Mar 17, 2018

jcouv added the PR For Personal Review Only The PR doesn’t require anyone other than the developer to review it. label Apr 4, 2018

jcouv removed Blocked PR For Personal Review Only The PR doesn’t require anyone other than the developer to review it. labels May 21, 2018

alrz force-pushed the features/target-typed-new branch from 54cc99c to 3dd3a26 Compare May 22, 2018 11:21

jcouv self-assigned this May 22, 2018

jcouv reviewed Jul 31, 2018

View reviewed changes

Reconstruct AnalyzedArguments to avoid binding twice

df4764a

alrz force-pushed the features/target-typed-new branch from 6628fda to 85e0a01 Compare August 2, 2018 10:31

Add additional tests.

aaa3b35

alrz force-pushed the features/target-typed-new branch from 85e0a01 to aaa3b35 Compare August 2, 2018 10:38

jcouv approved these changes Aug 2, 2018

View reviewed changes

gafter removed the Area-Language Design label Aug 2, 2018

gafter reviewed Aug 3, 2018

View reviewed changes

PR feedback

b1ef174

alrz mentioned this pull request Aug 8, 2018

Address test plan for target-typed new #29167

Merged

gafter approved these changes Aug 8, 2018

View reviewed changes

gafter merged commit 1e246ca into dotnet:features/target-typed-new Aug 8, 2018

alrz deleted the features/target-typed-new branch August 8, 2018 17:16

alrz mentioned this pull request Nov 8, 2018

Update target-typed-new.md dotnet/csharplang#1989

Merged

gafter mentioned this pull request Dec 15, 2018

Need SemanticModel tests for target-typed new #31807

Closed

gafter added the New Language Feature - Target-Typed New `new (args)` gets the created type inferred from context label Dec 15, 2018

Support target-typed new #25196

Support target-typed new #25196

Conversation

alrz commented Mar 2, 2018 • edited by jcouv

Choose a reason for hiding this comment

jcouv commented Mar 2, 2018 • edited

jcouv commented Apr 4, 2018 • edited

jcouv commented May 21, 2018 • edited

CyrusNajmabadi commented May 21, 2018

Neme12 commented May 21, 2018 • edited

Neme12 commented May 21, 2018 • edited

jaredpar commented May 21, 2018

jcouv commented May 21, 2018 • edited

Neme12 commented May 21, 2018 • edited

jaredpar commented May 21, 2018

CyrusNajmabadi commented May 22, 2018

CyrusNajmabadi commented May 22, 2018

CyrusNajmabadi commented May 22, 2018

jaredpar commented May 22, 2018

CyrusNajmabadi commented May 22, 2018

jaredpar commented May 22, 2018

jcouv commented May 22, 2018 • edited

jcouv Jul 31, 2018 • edited

Choose a reason for hiding this comment

alrz Aug 1, 2018 • edited by jcouv

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcouv Jul 31, 2018 • edited

Choose a reason for hiding this comment

alrz Aug 1, 2018 • edited by jcouv

Choose a reason for hiding this comment

jcouv commented Jul 31, 2018 • edited

jcouv Jul 31, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcouv left a comment

Choose a reason for hiding this comment

jcouv left a comment

Choose a reason for hiding this comment

gafter commented Aug 2, 2018 • edited

gafter Aug 3, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gafter Aug 3, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gafter commented Aug 3, 2018

alrz commented Aug 8, 2018

gafter left a comment

Choose a reason for hiding this comment

alrz commented Mar 2, 2018 •

edited by jcouv

jcouv commented Mar 2, 2018 •

edited

jcouv commented Apr 4, 2018 •

edited

jcouv commented May 21, 2018 •

edited

Neme12 commented May 21, 2018 •

edited

Neme12 commented May 21, 2018 •

edited

jcouv commented May 21, 2018 •

edited

Neme12 commented May 21, 2018 •

edited

jcouv commented May 22, 2018 •

edited

jcouv Jul 31, 2018 •

edited

alrz Aug 1, 2018 •

edited by jcouv

jcouv Jul 31, 2018 •

edited

alrz Aug 1, 2018 •

edited by jcouv

jcouv commented Jul 31, 2018 •

edited

jcouv Jul 31, 2018 •

edited

gafter commented Aug 2, 2018 •

edited

gafter Aug 3, 2018 •

edited

gafter Aug 3, 2018 •

edited