Throwing exception if the underlying rsp fails with an exception

[Anipik]

The benchmarks involving rsps don't fail even when the underlying rsp has thrown an exception

[Ivanidzo4ka]

Out of curiosity, why we even use rsp?

[Anipik]

Out of curiosity, why we even use rsp?

Because when we introduced benchmarking, we wanted to have them asap. And converting them to api required someone on the ml team and it was not possible at that time

[Anipik]

cc @danmosemsft @justinormont @adamsitnik

[TomFinley]

Hi @Anipik thanks for writing this.

Minor point... so, I see this appears to be part of something called a Dispose method. Could we name it something else? I think naming it this when it's not actually a dispose method violates some sort of .NET design guideline (despite the fact that this does not implement IDisposable). It's at least needlessly confusing.

While we're at it, is there a reason to not make this base class abstract?

[TomFinley]

Thanks @Anipik !

[Anipik]

@justinormont @Ivanidzo4ka can you review this one ?

[adamsitnik]

I propose a different approach: instead of introducing a base class type with [IterationCleanup] method that checks the exit code we could add a new helper method, which would call Maml.MainCore, check the exit code and throw in the benchmark when the exception happens.

sth like:

void ExecuteMamlCommand(this string command, IHostEnvironment environment)
{
    if (Maml.MainCore(environment, command, alwaysPrintStacktrace: false) < 0)
        throw new Exception($"Command {command} returned error code"); // + some details if possible
}

[sfilipi]