output module LR/SVM

[berlino]

Thank you for this awesome repo.
This is not actually a code issue, I'm just curious to ask. Do you have any idea why do we need an extra linear model or SVM for the prediction? I mean this module doesn't go through the backpropagation at all.
Or do you find some improvements using this LR or SVM compared with its fully connected output layer?

Thanks

[berlino]

I read through the paper, still don't get the point.

[galsang]

In the paper, 5.1.3 may help.

5.1.3 Classifier
We found that performance increases if we do
not use the output of the LR layer as the final decision,
but instead train linear SVM or logistic regression
with default parameters directly on the input
to the LR layer (i.e., on the kn similarity scores
that are generated by the k-block stack after network
training is completed). Direct training of SVMs/LR
seems to get closer to the global optimum than gradient
descent training of CNNs.

I think this is a kind of empirical(or heuristic) optimizations.
Actually, the performance seems slightly better when applying those additional modules.

[berlino]

It's sort of weird to me since the fully connected output layer is equivalent to linear model theoretically.
Thanks.