{{ message }}

# [TMVA] Add new Evaluation Metric ( meanAbsoluteError between two matrices )#2376

Merged
merged 1 commit into from Aug 2, 2018
Merged

# [TMVA] Add new Evaluation Metric ( meanAbsoluteError between two matrices )#2376

merged 1 commit into from Aug 2, 2018

## Conversation

### ravikiran0606 commented Jul 27, 2018 • edited

 Need: The need for a new evaluation metric for testing the convergence of the optimizer is essential. The already existing metric was maximumRelativeError() between two matrices which takes the maximum of all the relative errors between its individual elements. But the relative error between these elements depends on the element values. i.e Relative error between a and b = abs(a-b)/(abs(a)+abs(b)). Let use consider 2 cases, case a) If two values are a = 0.0001 , b = 0.0002, relative error = 0.3333 case b) If two values are a = 10.0001 b = 10.0002 relative error = 4.99992e-6 Since the unit tests for optimizer is written in a way so that a sample 3 layer DNN will learn this function Y = K * X. So, If X = I ( Identity matrix ), then Y = K * I = K. This should be equivalent to the output of the trained DNN when I is feed as Input. Let Y' be the output of the trained DNN. So I need to compare the matrices K and Y' for approximate equality with a certain threshold. So If I use maximumRelativeError for comparing the approximate equality for two matrices, then even though the difference is small for two cases, the relative error is significantly different. So there is a need for a new evaluation metric. Goal: The goal of this PR is to implement new evaluation metric meanAbsoluteError() between two matrices which takes the mean of all the absolute errors of individual elements. Absolute error between a and b = abs(a-b). So both the cases described above will have the same absolute error. So I propose this would be a good choice of metric for comparing two matrices for approximate equality as needed for testing optimizers. The text was updated successfully, but these errors were encountered:
We are unable to convert the task to an issue at this time. Please try again.
``` [TMVA] Add new Evaluation Metric ( meanAbsoluteError between two matr… ```
``` 9cd057e ```
`…ices ) needed for evaluating the optimizers.`

### phsft-bot commented Jul 27, 2018

 Can one of the admins verify this patch?

### ravikiran0606 commented Jul 27, 2018

 @lmoneta @stwunsch Can you review this PR? reviewed
 */ //______________________________________________________________________________ template

### stwunsch Jul 27, 2018

Is it needed to template the type of both matrices? Shouldn't they be of the same type?

### stwunsch Jul 27, 2018

I see that the implementation of the `maximumRelativeError` has done the some... Nevertheless, is it needed?

### ravikiran0606 Jul 27, 2018

Yeah, I guess we need the template arguments. Because in certain cases, the matrices can be of different types like TMatrixT or TCpuMatrix or TCudaMatrix.

### stwunsch Jul 27, 2018

alright, fine then.

### stwunsch commented Jul 27, 2018

 @phsft-bot build

### phsft-bot commented Jul 27, 2018

 Starting build on `slc6`/`gcc48`, `slc6-i686`/`gcc49`, `centos7`/`clang39`, `centos7`/`gcc62`, `centos7`/`gcc7`, `fedora28`/`native`, `ubuntu16`/`native`, `mac1013`/`native`, `windows10`/`vc15` with flags `-Dimt=ON -Dccache=ON` How to customize builds

### lmoneta commented Aug 2, 2018

 These changes are fine with me. I can merge it