Weighted loss

[al13n321]

It would be nice to be able to pass node weights for Hamming loss function, i.e. make some labels more important than the others. For example, if we are labeling superpixels of an image instead of pixels, we want to minimize the number of wrong pixels, not superpixels.

[amueller]

You are right, that would be a nice enhancement. It would be quite easy if we changed the interface of loss to also contain x. Then you could store the weighting factor in x and use that in the loss.

[al13n321]

Yep, I'm adding x argument to all loss() functions all over the code right now.

[amueller]

Cool. Do you think it is feasible to do this in a backward-compatible way?
I think I would add it as a last and optional argument for the models, so that they would also work with learners that don't do it. But I don't see an easy way to make the new learners work with models that don't take an x.

[al13n321]

I don't know how to add weights to all models without going through all the code, so I'll probably only add support for weights to EdgeFeatureGraphCRF (since I'm currently only interested in it). I'm thinking to just check the length of x tuple to decide if the weights are given and the loss should be weighted.

[amueller]

I would rather create a new model that inherits from EdgeFeatureGraphCRF which always wants weighted nodes. If that is too complicated, I'd at least add a boolean parameter.
If you inherit and create a new model, you only have to write a new loss and overwrite the function that checks x to be in the correct format.

[zaxtax]

+1 for boolean parameter that can be checked if loss is weighted.
On Oct 26, 2013 7:33 PM, "Andreas Mueller" notifications@github.com wrote:

I would rather create a new model that inherits from EdgeFeatureGraphCRFwhich always wants weighted nodes. If that is too complicated, I'd at least
add a boolean parameter.
If you inherit and create a new model, you only have to write a new lossand overwrite the function that checks
x to be in the correct format.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/94#issuecomment-27158238
.

[amueller]

Ideally, this should be implemented in a way that also includes the class_weight option. That is currently hacked in a bit. Maybe the class_weight should be implemented in terms of instance weights?

[al13n321]

Yeah, class_weight seems redundant when nodes can have weights, but still more convenient. EdgeFeatureGraphCRF could translate class weights to node weights so that lower levels don't have know about class weights.

[amueller]

exactly :)