NullReferenceException only dotcover test coverage runner

[mbrookson]

I am currently faced with a very confusing issue. I have some unit tests which utilise an implementation of IAsyncEnumerable and IAsyncQueryProvider to enable testing of EntityFrameworkCore async extensions. I've implemented DelegateDecompiler and am seeing weird issues where some of my unit tests are failing only when running code coverage with dotnet dotcover test (https://www.jetbrains.com/help/dotcover/Running_Coverage_Analysis_from_the_Command_LIne.html#using-dotnet-exe-to-run-coverage-analysis-of-unit-tests)

I have tried running the tests normally with just dotnet test and with the built in unit test and coverage tools within JetBrains Rider IDE, and the test run successfully on

• Windows and MacOS
• .NET 5.0
• Debug and Release configurations

System.NullReferenceException: Object reference not set to an instance of an object.
  Stack Trace:
      at DelegateDecompiler.Processor.BuildAssignment(Expression instance, MemberInfo member, Expression value, Boolean& push)
   at DelegateDecompiler.Processor.Process()
   at DelegateDecompiler.Processor.Process(VariableInfo[] locals, IList`1 args, Instruction instruction, Type returnType)
   at DelegateDecompiler.MethodBodyDecompiler.DecompileConcrete(MethodInfo method, IList`1 args)
   at DelegateDecompiler.MethodBodyDecompiler.Decompile(MethodInfo method, Type declaringType)
   at DelegateDecompiler.DecompileExtensions.<>c__DisplayClass6_0.<.cctor>b__1()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at DelegateDecompiler.DecompileExtensions.Decompile(MethodInfo method, Type declaringType)
   at DelegateDecompiler.DecompileExpressionVisitor.Decompile(MethodInfo method, Expression instance, IList`1 arguments)
   at DelegateDecompiler.DecompileExpressionVisitor.VisitMember(MemberExpression node)
   at System.Linq.Expressions.MemberExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitBinary(BinaryExpression node)
   at System.Linq.Expressions.BinaryExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitMemberAssignment(MemberAssignment node)
   at System.Linq.Expressions.ExpressionVisitor.VisitMemberBinding(MemberBinding node)
   at System.Linq.Expressions.ExpressionVisitor.Visit[T](ReadOnlyCollection`1 nodes, Func`2 elementVisitor)
   at System.Linq.Expressions.ExpressionVisitor.VisitMemberInit(MemberInitExpression node)
   at System.Linq.Expressions.MemberInitExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitLambda[T](Expression`1 node)
   at System.Linq.Expressions.Expression`1.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Linq.Expressions.ExpressionVisitor.VisitUnary(UnaryExpression node)
   at System.Linq.Expressions.UnaryExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitArguments(IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at DelegateDecompiler.DecompileExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at DelegateDecompiler.DecompileExpressionVisitor.Decompile(Expression expression)
   at DelegateDecompiler.EntityFrameworkCore.AsyncDecompiledQueryProvider.CreateQuery[TElement](Expression expression)
   at DelegateDecompiler.EntityFrameworkCore.DecompileExtensions.DecompileAsync[T](IQueryable`1 self)

Any ideas very welcome! Thanks.

[mbrookson]

Further investigation seems to show that, when running in normal debug unit test mode, I can put a breakpoint on Processor:268 which appears in the above stack trace and it does not get hit and the test passes.

However, when running with the dotCover code coverage tool that causes the test to fail, it does seem to run that line of code because it's in the stacktrace (can't use breakpoints with dotCover though) and then calls into BuildAssignment which is where the NullReferenceException occurs.

Really struggling to understand what the dotCover tool could be doing different when running the code which make it run differently. Can expressions differ based on runtime or configuration?

[hazzik]

Can you provide a test case which is failing?

[hazzik]

My suspicion is that dotcover changes IL which is not recognized by decompiler

[mbrookson]

Managed to get some further information out by adding a call to Debug.WriteLine at Processor:268 as seen below. I also noticed DelegateDecompiler already logs the state.Instruction to the debug console.

else if (state.Instruction.OpCode == OpCodes.Stfld)
{
    var value = state.Stack.Pop();
    var instance = state.Stack.Pop();
    var field = (FieldInfo) state.Instruction.Operand;
    Debug.WriteLine("!!! RUNNING LINE 268 FOR INSTRUCTION {0}", state.Instruction);
    var expression = BuildAssignment(instance.Expression, field, value, out var push);
    if (push)
        state.Stack.Push(expression);
    else
        instance.Expression = expression;
}

Running dotCover outputs the following which I think may explain the issue I'm seeing.
It logs an Instruction of JetBrains.Profiler.Core.Instrumentation.DataOnStack and it seems like it also logs an extra StatementCount instruction which is added by the JetBrains dotCover tool. I believe that's the one that's causing it to step into Processor:268

Debug Trace:
IL_0000: ldloca.s JetBrains.Profiler.Core.Instrumentation.DataOnStack (0)
 IL_0002: ldc.i4.1
 IL_0003: stfld UInt32 StatementCount
 !!! RUNNING LINE 268 FOR INSTRUCTION IL_0003: stfld UInt32 StatementCount

The code I'm currently running this on I am unable to share but if I cannot resolve the issue then I will try to reproduce it with a small POC code example tomorrow.

[mbrookson]

Wonder if there is a way to detect running within dotcover and exclude certain dotcover statements from being processed?

[hazzik]

Should be fixed in 0.29.0

[mbrookson]

@hazzik I'm afraid to say that it is not quite fixed 😞I'm seeing a different error based on the changes in 0.29.0 as follows

System.ArgumentException: Type must not be ByRef (Parameter 'type')
  Stack Trace:
      at System.Dynamic.Utils.TypeUtils.ValidateType(Type type, String paramName, Boolean allowByRef, Boolean allowPointer)
   at System.Dynamic.Utils.TypeUtils.ValidateType(Type type, String paramName)
   at System.Linq.Expressions.Expression.Convert(Expression expression, Type type, MethodInfo method)
   at System.Linq.Expressions.Expression.Convert(Expression expression, Type type)
   at DelegateDecompiler.Processor.AdjustType(Expression expression, Type type)
   at DelegateDecompiler.Processor.GetArguments(ProcessorState state, MethodBase m)
   at DelegateDecompiler.Processor.Call(ProcessorState state, MethodInfo m)
   at DelegateDecompiler.Processor.Process()
   at DelegateDecompiler.Processor.Process(VariableInfo[] locals, IList`1 args, Instruction instruction, Type returnType)
   at DelegateDecompiler.MethodBodyDecompiler.DecompileConcrete(MethodInfo method, IList`1 args)
   at DelegateDecompiler.MethodBodyDecompiler.Decompile(MethodInfo method, Type declaringType)

[hazzik]

I've created #172 to track it.