Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
C# Design Notes for Apr 14, 2015 #2134
C# Design Meeting Notes for Apr 14
Bart De Smet visited from the Bing team to discuss their use of Expression Trees, and in particular the consequences of their current shortcomings.
The Expression Tree API today is not able to represent all language features, and the language supports lambda conversions even for a smaller subset than that.
Certain Bing services, such as parts of Cortana, rely on shipping queries between different machines, both servers in the cloud and client devices.
In a typical scenario a lambda expression tree is produced in one place, ideally via the conversion from lambda expressions to expression trees that exists in C#. The expression tree is then serialized using a custom serialization format, and sent over the wire. At the receiving end it will often be stitched into a larger expression tree, which is then compiled with the
Along the way, several transformations are often made on the trees, for efficiency reasons etc. For instance, rather than invoke a lambda its body can often be inlined in the enclosing tree that the lambda gets stitched into.
The serialization format is able to carry very specific type information along with the code, but can also represent the code loosely. A looser coupling makes for code that is more resilient to "schema" differences between the nodes, and also allows for use of types and functions that aren't present where the lambda is concocted. However, it also makes it harder to stich things back up right on the other side.
Shortcomings in the Expression Tree API
The expression tree API was introduced with Linq and C# 3.0, and was extended to support the implementation infrastructure of
This is a list of ones that are confounding to the Bing team.
Even though the API was extended for the benefit of the dynamic feature, the feature itself ironically is not well represented in the API. This is not a trivial undertaking: in Expression Trees today, all invoked members are represented using reflection structures such as
In the Bing scenario, dynamic would probably help a lot with representing the loosely coupled invocations that the serialization format is able to represent.
Alternatively, if the C# language added something along the "lightweight dynamic" lines that were discussed (but ultimately abandoned) for C# 6, the representation of that in Expression Trees would probably yield similar benefit with a fraction of the infrastructure.
Representing await would probably be easy in the Expression Tree API. However, the implementation of it in
Needless to say, distributed scenarios such as that of Bing services have a lot of use for
Null conditional operators and string interpolation
These should also be added as Expression Tree nodes, but could probably be represented as reducible nodes. Reducible nodes are a mechanism for describing the semantics of a node in terms of reduction to another expression tree, so that the
Higher level statements
While expression trees have support for statements, those tend to be pretty low level. Though there is a
Shortcomings in the languages
C# and VB are even more limited in which lambdas can be converted to expression trees. While statements are part of the Expression Tree API, the languages will not convert them. Also, assignment operators will not be converted.
This is a remnant of the first wave of Linq, which focused on allowing lambdas for simple, declarative queries, that could be translated to SQL.
There has traditionally been an argument against adding more support to the languages based on the pressure this would put on existing Linq providers to support the new nodes that would start coming from Linq queries in C# and VB.
This may or may not ever have been a very good argument. However, with Roslyn analyzers now in the world, it pretty much evaporates. Any Linq provider that wants to limit at design time the kinds of lambdas allowed, can write a diagnostic analyzer to check this, and ship it with their library.
None of this support would require new syntax in the language, obviously. It is merely a matter of giving fewer errors when lambdas are converted to expression trees, and of mapping the additional features to the corresponding nodes in what is probably a straightforward fashion.
There is no remaining design reason we can think of why we shouldn't a) bring the Expression Tree API up to date with the current state of the languages, and b) extend the language support accordingly.
The main issue here is simply that it is likely to be a huge undertaking. We are not sure that the sum of the scenarios will warrant it, when you think about the opportunity cost for other language evolution. Bing is a very important user, but also quite probably an atypical one.
So in summary, as a language design team, we certainly support completing the picture here. But with respect to priorities, we'd need to see a broader call for these improvements before putting them ahead of other things.
Mads asked me to post my mail to him here ...
Not digging super deep into this, the intro to the ETs v2 spec/docs says that it is not a goal of ETs to reflect C#. There are multiple examples where they do not 1:1 model C# or VB; there just happens to be a lot of close modeling and semantics to C# because unless we had to be different, it was a good place to be :-).
I'm not dismissing Bart's requests, just saying that a motivator and expectation should NOT be that ETs must keep track of C# features. ETs v2 were conceived to model linguistic constructs driven by implementing three dynamic languages on .NET using the DLR and to model the bound expressions in the polymorphic inline caches for dynamic CallSites.
I'm mostly mentioning this so that there is awareness of the history that didn't seem reflected in the notes, and that there is explicit design choices to veer from the original v2 goals. Veering may be very reasonable, we clearly veered from v1 to v2 :-).
Spec is at (I'll edit with a github link when I find out who moved this where :-)): http://dlr.codeplex.com/wikipage?title=Docs%20and%20specs&referringTitle=Documentation
In the meantime, I've taken an early stab at providing a runtime library with C#-specific expression nodes that are implemented as reducible expressions. More information at https://github.com/bartdesmet/ExpressionFutures/tree/master/CSharpExpressions. I'll discuss this with the team in the weeks to come.
Design notes have been archived at https://github.com/dotnet/roslyn/blob/future/docs/designNotes/2015-04-14%20C%23%20Design%20Meeting.md but discussion can continue here.