[TMVA] Add new Evaluation Metric ( meanAbsoluteError between two matrices ) #2376
Conversation
…ices ) needed for evaluating the optimizers.
|
Can one of the admins verify this patch? |
| */ | ||
|
|
||
| //______________________________________________________________________________ | ||
| template <typename Matrix1, typename Matrix2> |
There was a problem hiding this comment.
Is it needed to template the type of both matrices? Shouldn't they be of the same type?
There was a problem hiding this comment.
I see that the implementation of the maximumRelativeError has done the some... Nevertheless, is it needed?
There was a problem hiding this comment.
Yeah, I guess we need the template arguments. Because in certain cases, the matrices can be of different types like TMatrixT or TCpuMatrix or TCudaMatrix.
|
@phsft-bot build |
|
Starting build on |
|
These changes are fine with me. I can merge it |
Need:
The need for a new evaluation metric for testing the convergence of the optimizer is essential. The already existing metric was maximumRelativeError() between two matrices which takes the maximum of all the relative errors between its individual elements. But the relative error between these elements depends on the element values. i.e
Relative error between a and b = abs(a-b)/(abs(a)+abs(b)). Let use consider 2 cases,
case a) If two values are a = 0.0001 , b = 0.0002, relative error = 0.3333
case b) If two values are a = 10.0001 b = 10.0002 relative error = 4.99992e-6
Since the unit tests for optimizer is written in a way so that a sample 3 layer DNN will learn this function Y = K * X. So, If X = I ( Identity matrix ), then Y = K * I = K. This should be equivalent to the output of the trained DNN when I is feed as Input. Let Y' be the output of the trained DNN. So I need to compare the matrices K and Y' for approximate equality with a certain threshold.
So If I use maximumRelativeError for comparing the approximate equality for two matrices, then even though the difference is small for two cases, the relative error is significantly different. So there is a need for a new evaluation metric.
Goal:
The goal of this PR is to implement new evaluation metric meanAbsoluteError() between two matrices which takes the mean of all the absolute errors of individual elements.
Absolute error between a and b = abs(a-b).
So both the cases described above will have the same absolute error. So I propose this would be a good choice of metric for comparing two matrices for approximate equality as needed for testing optimizers.