Join GitHub today
[TMVA] Add new Evaluation Metric ( meanAbsoluteError between two matrices ) #2376
The need for a new evaluation metric for testing the convergence of the optimizer is essential. The already existing metric was maximumRelativeError() between two matrices which takes the maximum of all the relative errors between its individual elements. But the relative error between these elements depends on the element values. i.e
Relative error between a and b = abs(a-b)/(abs(a)+abs(b)). Let use consider 2 cases,
case a) If two values are a = 0.0001 , b = 0.0002, relative error = 0.3333
Since the unit tests for optimizer is written in a way so that a sample 3 layer DNN will learn this function Y = K * X. So, If X = I ( Identity matrix ), then Y = K * I = K. This should be equivalent to the output of the trained DNN when I is feed as Input. Let Y' be the output of the trained DNN. So I need to compare the matrices K and Y' for approximate equality with a certain threshold.
So If I use maximumRelativeError for comparing the approximate equality for two matrices, then even though the difference is small for two cases, the relative error is significantly different. So there is a need for a new evaluation metric.
The goal of this PR is to implement new evaluation metric meanAbsoluteError() between two matrices which takes the mean of all the absolute errors of individual elements.
Absolute error between a and b = abs(a-b).
So both the cases described above will have the same absolute error. So I propose this would be a good choice of metric for comparing two matrices for approximate equality as needed for testing optimizers.