LightGBM: IsUnbalanced is relevant for OVA multi class

[rauhs]

Version: 1.0

Setup:

• LightGBM Multiclass trainer
• NOT softmax (thus: "objective" = "mutliclassova").

Before 1.0 we had UnbalancedSets available for the three booster configs, but this option was removed it seems.

Though, that option very much had an effect on performance in this case.

You can see that the OVA objective just calls the binary one:

https://github.com/microsoft/LightGBM/blob/0a4a7a86f5a1d3146c36c7d8c082154a193d4893/src/objective/multiclass_objective.hpp#L173-L182

which will happily use the config parameter:

https://github.com/microsoft/LightGBM/blob/0a4a7a86f5a1d3146c36c7d8c082154a193d4893/src/objective/binary_objective.hpp#L20-L36

[glebuk]

Perhaps we can expose this parameter in Multi-class LightGBM?

[abgoswam]

I believe the bug was caused by the following commit

• From what I can tell, the PR tried to ensure that each trainer had its own respective trainer class.

• In particular the UnbalancedSets parameter was removed from the booster options.

  • It was added to binary trainer options.
  • But was not added to the multiclass trainer. As such there is currently no way for the user to use "multiclassova" with UnbalancedSets.
  • this seems to be the root cause of the issue the customer reports in the bug description

Proposal: Add UnbalancedSets option to lightgbm multiclass trainer

• this would probably be similar to how it was added to the binary trainer
• this would be an incremental change. non-breaking
• estimate : 2 days

@codemzs @artidoro