AssemblyRunner's "OnTestFailed" does not give enough information for AggregateExceptions.

[danrozenberg]

Hello, we have some tests running through the AssemblyRunner class, and it works fine, in general.

However, when a test run fails with an AggregateException, it becomes hard to gather more information about the failure... Consider the example:

            runner.OnTestFailed = info =>
            {
               _logger.LogInformation($"The following test has failed: {info.TestDisplayName} after {info.ExecutionTime} seconds, with error: {info.ExceptionMessage} and stack trace {info.ExceptionStackTrace}. The exception type is: {info.ExceptionType}.");
            };

When a test fails, here is what would be output:

The following test has failed: MyTest after 30 seconds, with error: One or more errors occurred. and stack trace . The exception type is: System.AggregateException.

Sadly, I couldn't find anything in TestFailedInfo that could display the inner exceptions. Is there any way to see all inner exceptions? If not, could something like this be added in the future?

Thank you!

[danrozenberg]

Can we have this for target 3.0? :)

[bradwilson]

I have not yet decided what to do with all the existing open issues w/rt 3.0. I don't even know if AssemblyRunner is something that's going to exist in 3.0. I'm leaving them all as-is for now, to be re-evaluated once 3.0 starts to take shape.

So merely being open is good enough to be a candidate for 3.0. :)

[ohadschn]

It should be noted that AggregateException is only one such case our of many. For example, HttpRequestException can say something like The SSL connection could not be established, see inner exception. And it's not necessarily the inner exception(s) that's missing, for example in some type load exceptions it's some other property if I recall correctly. And of course inner exception could also have inner exceptions, and other properties, and so on. The point it, only Exception.ToString will work correctly.

Fortunately, the solution is very simple, add something like TestFailedInfo.FullExceptionInfo and populate it with exception.TosSring().

I don't even know if AssemblyRunner is something that's going to exist in 3.0

@bradwilson I hope there would be an equivalent though? We heavily rely on the ability to execute tests from a loaded assembly without launching an additional process.