Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Generalize ZeroOneLoss for multiple classes #119

Closed
wants to merge 1 commit into from

Conversation

juliohm
Copy link
Member

@juliohm juliohm commented Mar 20, 2020

This is an attempt to generalise the ZeroOneLoss to multi-class problems. As described in the existing doc string, the loss is more general than just checking the sign of the product of y and yhat. It is more general in the sense that we check an "agreement" between y and yhat and return either 0 or 1, i.e. the misclassification indicator function.

Overall, the conceptual changes in this PR consist of:

  1. ZeroOneLoss is now a subtype of SupervisedLoss as opposed to the more restrictive MarginLoss type
  2. It works for a pair of Number as before, and I will continue this PR to make it work with pair of CategoricalValue from CategoricalArrays.jl

Could you please provide feedback in this breaking change? Do you agree that we could move this very specific (seldom used as the doc string says) loss to the "other" family of losses along with CrossentropyLoss and PoissonLoss? In the future these "other" losses could be classified into more specific subtype of SupervisedLoss. For example, ZeroOneLoss has a function psi(y, yhat) that is simply psi(y, yhat) = y == yhat unlike the function psi(y, yhat) = y*yhat for margin-based losses, and unlike the function psi(y, yhat) = y - yhat for distance-based losses.

I appreciate if you can take a look at this PR, and provide feedback quickly. We are writing a paper that depends on this, and it would be nice to use a stable version of LossFunctions.jl in the Manifest.toml for the accompanying scripts.

@juliohm
Copy link
Member Author

juliohm commented Mar 20, 2020

An alternative approach to this breaking PR is to introduce a new name like MisclassificationLoss with the behaviour above, and leave ZeroOneLoss untouched. Please let me know which is the most interesting approach and I can keep working on it.

@juliohm
Copy link
Member Author

juliohm commented Mar 22, 2020

Replaced by #122

@juliohm juliohm closed this Mar 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant