Proposal: Add new GetValue method to System.Reflection #14021

Open
Esidar opened this Issue Nov 26, 2016 · 29 comments

Projects

None yet

7 participants

@Esidar
Esidar commented Nov 26, 2016 edited

This new method would accept an "object" to which a Value could be copied instead of being unnecessarily boxed. Example of use would look like this:

static object boxedFloat = null;
....
boxedFloat = fieldInfo.GetValue( myObject, boxedFloat );

Boxing of the value would occur only once. This would eliminate unnecessarily memory allocations in situations when GetValue is called many times. When method is used on a class types, copying would not occur and stored object would be returned instead.

One of the uses of GetValue method is User Interface based on MVVM (Model View Viewmodel) where values used to control an UI are pulled directly from the data. In realtime applications (like games) this boxing makes hundreds if not thousands of unnecessary memory allocations every second.

Another use of GetValue is serialization, and having ability to serialize hundreds of values without the need of boxing/unboxing, would greatly improve memory usage and performace.

There was also an idea dotnet/coreclr#8277 that this new GetValue method would take a form of generic method, but I think this would limit the use as one of the advantages of working with System.Object is this abstraction that makes some parts of the code simpler.

So, ideally this new method should remove the unnecessary boxing while keeping convenience of System.Object. Additionally please bear in mind this should work with both FieldInfo and PropertyInfo, as well as give option to get values from nested Value as in following example:

struct Vector3
{
   float x,y,z;
}
struct Transform
{
    Vector3 position { get; set }
}

transform = positionProperty.GetValue( transformObject, transform );
boxedX = xField.GetValue( transform, boxedX );
DisplayValue( boxedX );

(note intermixing fields with property)

@svick
Contributor
svick commented Nov 26, 2016

Interesting, I didn't realize you can reassign values inside boxes.

If this is going to work with properties, I think it should be made completely general and work with any method that returns a value.

And a couple questions:

  1. What should the behavior be if the passed-in boxed value is of the wrong type? The same as null, i.e. allocate a new box?
  2. Wouldn't it be simpler if the input and output were specified at the same time using a ref parameter, i.e.: fieldInfo.GetValue(myObject, ref boxedFloat);? No need to specify boxedFloat twice.
@jkotas
Member
jkotas commented Nov 27, 2016 edited

FieldInfo.GetValue/SetValue are part of reflection invoke APIs.

All current reflection invoke APIs have a general problem that they expected everything boxed. It is not just a performance problem mentioned here, but also a functional deficiency problem.

Among other things, it means that it is not possible to write high fidelity IL interpreter using reflection. Let's say that you would like to interpret static void m1(ref int x) { m2(ref x); }. There is no way to do it using reflection today because of there is no reflection invoke method that can pass x along as byref.

If we were to add a new reflection invoke APIs, we should aim to solve both of these problems.

CoreRT has internal Internal.Runtime.CallInterceptor.MakeDynamicCall that is basically a box-free version of the MemberInfo.Invoke. It would need to be cleaned up and argument validation added for it to become good public API.

@MichalStrehovsky
Member

There is no way to do it using reflection today because of there is no reflection invoke method that can pass x along as byref

MethodBase.Invoke will actually modify the elements of the parameters array if one of the parameters is a byref (a ref or out parameter). A better example would be a method that returns a byref (this will be exposed as a new language feature in C# soon). There's no way to reflection-invoke those.

Another thing to consider would be finding a replacement for FieldInfo.GetValueDirect and FieldInfo.SetValueDirect. These APIs let you access fields on a value type without boxing the struct that owns the field. Since TypedReference never was a first class citizen (and won't be part of .NET Core) a replacement API would be a nice addition.

@bartdesmet
Contributor

MethodBase.Invoke will actually modify the elements of the parameters array if one of the parameters is a byref (a ref or out parameter).

While that's true, it's rarely the case that the storage locations of the objects or values being passed by reference are truly in the object[] being passed. Typically one copies the value into the object[] and performs a write-back after the Invoke method returns. This causes issues such as #4411 and #13458.

The expression interpreter runs into a number of other issues as well. For example, we can't pass Type.Missing in a verbatim manner, see #4112. Also, the object[] allocation and the copy made by the reflection API is expensive, see #4125. An issue to discuss addressing just these two limitations is opened at dotnet/coreclr#7253.

Having boxes passed to reflection APIs has also led to behavioral changes in the expression interpreter compared to the compiler, such as #13007 (though one can argue the interpreter is doing the right thing in this case, merely by accident).

@Esidar
Esidar commented Nov 28, 2016 edited

Looking through available features (to avoid major changes in FX/CLR) I've stumbled upon arglist. C# have this undocumented keyword that is supported by CLR which allow for passing varying number of arguments and the use looks like this:

void Foo( __arglist )
{
   var it = new ArgIterator( __arglist );
}
void Test()
{
    float myFloat = 1;
    Foo( __arglist( 10, ref myFloat ) );
}

As you can see, you can even pass float as a ref.

From what I can see by disassembling a code (x86-64), values are passed clean, without any boxing.

New ReflectionAPI could utilize this to have both: box free GetValue as in fieldInfo.GetValue( obj, ref myFloat ) and a pretty neat, box free way of calling methods fooMethodInfo.Invoke( 10, ref myFloat )

Of course there would be a need to make a syntax nicer than a keyword __arglist, also, I'm not sure how this API would work with other languages. There is also a limit on how __arglist can be used. Currently ArgIterator, to read a value/argument, returns TypedReference and the only way to retrieve value is via TypedReference.ToObject() which does boxing. So that limits the use in managed environment. It is not a problem in native environment because GetValue and Invoke (both are implemented in native code), can read values and pointers directly from ArgIterator without boxing.

@Esidar
Esidar commented Nov 28, 2016 edited

Having boxes passed to reflection APIs

Problem shown in #13007 cannot be fixed by removing boxing. Boxing ensures that value types are valid during delegate invocation. In given example, variable o is valid only when delegate is created. Actual call to function f can occur much later when variable o is no longer available, thus copy must be made.

@MichalStrehovsky
Member

TypedReference, ArgIterator and RuntimeArgumentHandle are a no-go. None of those types are included in .NET Core or .NET Standard - they only exist in the legacy runtimes and AFAIK there are no plans to bring them back. I don't think we really need them. MakeDynamicCall mentioned above can get by without them, for example.

@jkotas
Member
jkotas commented Nov 28, 2016

MakeDynamicCall mentioned above can get by without them

The LocalVariableSet type used by MakeDynamicCall is very close to TypedReference. It is basically array of TypedReferences. It may be interesting to bring TypedReference back and making it useful, instead of creating a new same thing under a different name.

@Esidar
Esidar commented Nov 29, 2016

Is there a specific reason why TypedReference is being avoided? I guess there can be stability issue (reference to a local that is no longer available), but is there anything else?

Another solution would be use of new Tuples feature. methodInfo.Invoke could accept tuple struct instead of object[] array. CoreCR/VM could then use the same tuple if invoked method has any ref/out arguments

var args = ( name: "Mike", value: 10.0f );
methodInfo.Invoke( obj, args );
Console.Write( args.value );

I don't know the implementation details of the tuples, but afaik tuple is a struct that still needs to be boxed but we're boxing one struct instead of N-arguments and N-out-results. This wouldn't solve most of the problems, but GetValue could also use tuple to return a value (again, tuple is boxed only once and can be reused), could solve problems with properly resolving Missing arguments or unboxing values in Closures and so on.

@jkotas
Member
jkotas commented Nov 29, 2016 edited

Is there a specific reason why TypedReference is being avoided?

TypedReference has not been very useful: C# does not have a first class support for it, it requires non-trivial custom logic in the codegenerator and runtime, not all .NET runtimes support it. It is missing some methods that would make it useful and easy to work with, e.g. it should have a method to cast the value to strong typed value, like ref T As<T>() - this method is equivalent to the undocumented __refvalue C# keyword.

The non-boxing reflection method to get the value can then be: TypedReference GetValueReference(). To get the field value without boxing, one would write: fieldInfo.GetValueReference(myObject).As<float>().

new Tuples feature

I do not think Tuples are a good fit for this.

@jkotas
Member
jkotas commented Nov 29, 2016

I have opened #14088 to bring back TypedReference that should be the first on this.

cc @davidwrighton

@Thaina
Thaina commented Jan 13, 2017 edited

I think it should support return ref on FieldInfo directly, without TypedReference

I think TypedReference add more complexity to implementation. It make reference that would need to be fixed can be passed around. While return ref are compatible with dotnet behaviour

GameObject obj;

FieldInfo fi = typeof(obj).GetField("Transform");
ref Matrix4x4 transform = fi.GetRefValue<Matrix4x4>(ref obj);
// ref transform live in this stack
transform.M43 = 1; // obj.Transform.M43 = 1

Also, GetMethod should be able to use by cast it into generic delegate that can be used with ref object

Matrix4x4 matrix;

MethodInfo mi = typeof(matrix).GetMethod("RotateY");
RefAction<Matrix4x4,float> rotate = mi.GetRefAction<Matrix4x4,float>();
rotate(ref matrix,90); // Internally call matrix.RotateY(90);

////

namespace System
{
    public delegate void RefAction<T>(ref T obj);
    public delegate void RefAction<T,P1>(ref T obj,P1 param1);
    public delegate void RefAction<T,P1...>(ref T obj,P1 param1...);

    public delegate R RefFunc<T,R>(ref T obj);
    public delegate R RefFunc<T,P1>(ref T obj,P1 param1);
    public delegate R RefFunc<T,P1...>(ref T obj,P1 param1...);

    public delegate ref R RefFuncRef<T,R>(ref T obj);
    public delegate ref R RefFuncRef<T,P1>(ref T obj,P1 param1);
    public delegate ref R RefFuncRef<T,P1...>(ref T obj,P1 param1...);
}

class System.Reflection.MethodInfo
{
    public RefAction<T> GetRefAction<T,P>(P param) where T : class
    {
        // return void action(ref T t,param) that wrap void t.action(param);
    }
}
@jkotas
Member
jkotas commented Jan 13, 2017 edited

The implementation that returns TypedReference is more efficient and more powerful.

It is more efficient because of it does not suffer from generic code bloat.

It is more powerful because of you can easily add a strong typed generic convenience wrapper extension method around it that just returns ref (this wrapper can be part of the API proposal too); but the other direction is not possible.

@jkotas
Member
jkotas commented Jan 13, 2017 edited

Also, GetMethod should be able to use by cast it into generic delegate that can be used with ref object

There are existing methods that do that like Delegate.CreateDelegate(Type type, Object firstArgument, MethodInfo method). If you wish to have more convenient wrappers around them, it would be best to split it into a separate API proposal issue.

@Thaina
Thaina commented Jan 13, 2017

@jkotas You are right that it more powerful, which, at the same time, overkill. It not safe to pass TypedReference around. And TypedReference is hard to have it implement everywhere while return ref is more promising to be standard

as you say

it requires non-trivial custom logic in the codegenerator and runtime, not all .NET runtimes support it.

So I want to trade off here. We can live with generic in the most case

@Thaina
Thaina commented Jan 13, 2017

Also with Delegate.CreateDelegate the returned delegate does not work with mutable struct (without boxing?). So I propose something to make it work

The point here is to avoid bringing back TypedReference. Which I think generic is satisfied what we need and it make code always be strong type

@jkotas
Member
jkotas commented Jan 13, 2017

TypedReference is coming back for other reasons. It is used in non-trivial percentage of .NET libraries. See dotnet/standard#20.

@jkotas
Member
jkotas commented Jan 13, 2017 edited

It not safe to pass TypedReference around.

There is nothing inherently unsafe about it. The safety rules for TypedReference are exactly same as the rules for ref and other byref-like types.

@Thaina
Thaina commented Jan 13, 2017 edited

@jkotas any ref T is syntactically error in async and lambda expression/function callback. Was the TypedReference get the same behaviour in compile time?

I mean

void DoSomething(ref int i)
{
    i++; // normally fine
    Action plusplus = () => i++; // error, cannot access ref
}

is it the same for this?

void DoSomething(TypedReference tr)
{
    Action action = () => {
        // would it error if we use tr here?
    };
}
@Thaina
Thaina commented Jan 13, 2017 edited

It is used in non-trivial percentage of .NET libraries

I quite suspect that most of those could be replace with ref return. To support old code is ok. but I don't think we need to use in the future anymore

I said "not bringing it back" is misused, I was actually mean not encourage to using it more in the future or make any new library rely on it. It fine to have it support like volatile. But should not use it to implement the new reflection

@jkotas
Member
jkotas commented Jan 13, 2017

Was the TypedReference get the same behaviour in compile time?

TypedReference was in .NET since forever. byref-like type safety rules has been always enforced for it by the C# compiler. You example works as expected today - there is no extra work required to make it work.

@jkotas
Member
jkotas commented Jan 13, 2017

But should not use it to implement the new reflection

It is not possible to build things like good high fidelity IL interpreter on top of reflection without having the TypedReference-based reflection. I think it is important that any reflection APIs give you full power. We do not need yet another partial solution.

@Thaina
Thaina commented Jan 13, 2017

@jkotas Well, that's why I propose generic + return ref based API. And I think it could done all of things TypedReference based api could do, with strong typed

Could you please guide me what TypedReference could do but generic + return ref could not?

@jkotas
Member
jkotas commented Jan 13, 2017

If you are building things like IL interpreter, the actual types are not known. You need a type that can store any value that can appear as method argument or on IL stack. TypedReference is such type.

@Thaina
Thaina commented Jan 13, 2017 edited

OK, I quite get some image for the reason. So now I would like to agree with you

Thank you very much for explanation

2 last things I concern is
1 - Is TypedReference box things? Could we get ref of struct without boxing/unboxing with it?
2 - Will it cls compatible in the future?

@jkotas
Member
jkotas commented Jan 13, 2017
  1. TypedReference does not box things.
  2. The CLS compliance guidelines were set 15+ years ago and have not change since then. I doubt anybody will bother with updating them. It does not really help anything.
@Thaina
Thaina commented Jan 13, 2017

Thank you very much

@Thaina
Thaina commented Jan 13, 2017

So actually what I should want is ref T field = fieldInfo.GetValueDirect<T>(typedReference) ?

@jnm2
jnm2 commented Jan 14, 2017

The ability to assign to a field reference obtained via reflection with equal performance to setting a field directly would be amazing. I could stop emitting IL for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment