[WIP] Generalize ZeroOneLoss for multiple classes #119
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an attempt to generalise the
ZeroOneLoss
to multi-class problems. As described in the existing doc string, the loss is more general than just checking the sign of the product ofy
andyhat
. It is more general in the sense that we check an "agreement" betweeny
andyhat
and return either0
or1
, i.e. the misclassification indicator function.Overall, the conceptual changes in this PR consist of:
ZeroOneLoss
is now a subtype ofSupervisedLoss
as opposed to the more restrictiveMarginLoss
typeNumber
as before, and I will continue this PR to make it work with pair ofCategoricalValue
fromCategoricalArrays.jl
Could you please provide feedback in this breaking change? Do you agree that we could move this very specific (seldom used as the doc string says) loss to the "other" family of losses along with
CrossentropyLoss
andPoissonLoss
? In the future these "other" losses could be classified into more specific subtype of SupervisedLoss. For example,ZeroOneLoss
has a functionpsi(y, yhat)
that is simplypsi(y, yhat) = y == yhat
unlike the functionpsi(y, yhat) = y*yhat
for margin-based losses, and unlike the functionpsi(y, yhat) = y - yhat
for distance-based losses.I appreciate if you can take a look at this PR, and provide feedback quickly. We are writing a paper that depends on this, and it would be nice to use a stable version of LossFunctions.jl in the Manifest.toml for the accompanying scripts.