Imbalanced learning: mu-parameter not used, leads to unweighted crossentropy-function in "mildly" unbalanced cases

[JulianRein]

Hi,
The utils-function '''get_class_weighted_cross_entropy(y_train, mu=0.15)''' does not actually use the mu-parameter, but sets it to 0.15 regardless.
See line 29: "weights = _make_smooth_weights_for_balanced_classes(y_train, mu=0.15)"

pytorch_tabular/pytorch_tabular/utils.py

Line 29 in 9092543

weights = _make_smooth_weights_for_balanced_classes(y_train, mu=0.15)

In my binary-classification case with a 1:10 imbalance, this leads to "weights" of 1 to 1 for the two classes...
Also, you might want to think about setting mu higher by default, to get actual weights for non-extreme imbalances like mine.
I am using a mu > 1 to actually get different weights, does not work due to the bug (setting weights manually for now)

To Reproduce
Run "get_class_weighted_cross_entropy(y_train, mu=2)" with an 1:10 imbalanced, binary y_train.

Expected behavior
Get different weights for the two classes. Returns crossentropy with weight=[1,1] instead.

[manujosephv]

That is definitely an error I made while coding. Would you be able to raise a PR? I'll merge it right away on to the main branch

[JulianRein]

Sure, will do. Do you want to keep the current default mu=0.15?
Out of curiosity, any guidelines how to best choose it?

[manujosephv]

Sorry for the late reply. Got caught up with some other commitments.

So, the method is actually from this stackoverflow post. There is no explanation to why 0.15, but there is on kaggle notebook shows why 0.14 is an okay default.

But, this is strictly empirical and should be treated as such.

[JulianRein]

Thanks for the links.
Makes sense, the stackoverflow example is for multi-class, which usually has a total sample number not nearly only filled by one majority class.
For binary-classification, 0.15 is too small IMO, only starts working on imbalances of 1:18 (18=e/mu) , which is too late.
PR is out, leaves the default untouched.

[manujosephv]

I agree... I was playing with this a bit yesterday and 0.15 is too small for binary classification... How about we keep mu=1 as the default.. That will make it unsmoothed by default and users can tune the smoothing if required?

[JulianRein]

Completely agree, that is for the more common binary case the better default. Will change the PR.
The log should still do some smoothing, if I don't misunderstand the formula.

[JulianRein]

PR done

[JulianRein]

merged by @manujosephv