add confusion matrix layer for classification task.

[melody-rain]

Signed-off-by: Jin Ma xiah_sunny@hotmail.com

[Noiredd]

Copying @shaibagon's question from the previous version of this PR (#5999): will there be any test for this layer? Additionally, I restarted the suspicious Travis builds - but it still fails on some lint errors.

Also, why does this have to modify solver.cpp and caffe.cpp? Isn't it the very idea of Caffe to abstract away the specific layer classes?
It is my personal opinion that printing the output to log is not very useful. It might be practical for datasets with 10, maybe 15 classes, but for more than that I would prefer to use Python (and directly copy the confusion matrix from the output blob) instead of parsing the text logs.

[melody-rain]

Hi @Noiredd
Yes I am writing the test for confusion_matrix_layer.cpp.
However as you said I modified solver.cpp and caffe.cpp. The reason why I did so is because I calculate the confusion matrix at each iteration and want to output the final confusion matrix after all the iterations are tested. Thus the problem comes that the confusion matrix layer does not known which is the last iteration to stop calculating and to print the confusion matrix and to reset the buffer that stores the confusion matrix.

In the last version of my implementation, the layer must be input a parameter called num_iterations to let the layer know when to stop and print the confusion matrix, but I think this way is tedious, so I gave it up.

As for the python implementation, there is indeed a python implementation. But my program cannot use it when importing the python one because of the environment path setting.

At last if you use output blob to store confusion matrix, the elements of the output blob will be output at each iteration, like:

at iteration 0,
#output0 = ...
#output1 = ...
...
#output199= ...

at iteration 1,
#output0 = ...
...
#output199= ...

which is ugly.

The overall implementation design is to calculate the confusion matrix for all testing iterations and show the confusion matrix at the end of the test.

I think I will not commit the code to Caffe now as I have to modify solver.cpp and caffe.cpp, which might bring unknown bugs to Caffe.

Thanks for your comments.

[Noiredd]

I understand the problem with writing the output to log. But remember that we have the HDF5 Output layer - maybe it could be used to write the matrix directly to disk, instead of having to deal with log parsing or, worse yet, modifying solver.cpp?

CM layer would be useful, and your implementation looks promising. The output issue needs to be sorted out, same with those lint errors, maybe also some optimization (why do we need to partial_sort the whole thing - look at the new accuracy layer to see how we're doing classification now) and documentation touches (we're having predictions in rows and ground truth in columns, or vice-versa? would be nice to have this bit in doxygen). But I don't see a reason to throw this code away.

[melody-rain]

@Noiredd Thanks for your suggestions. I will find a better way to do that.