[MRG+1] ENH add a parameter to SVC and NuSVC to break ties in predict if desired

[adrinjalali]

Fixes #8277

This is a proposal to fix the issue in SVC and NuSVC where the predict does not break ties but decision_function does, when decision_function_shape='ovr' and n_classes > 2

The proposal is to add a parameter to BaseSVC, and return argmax of decision_function instead of calling libsvm's predict, which breaks ties according to the confidence values.

The default value of the parameter is False to keep it backward compatible, and if needed, in the future, switch the default value to True in an appropriate deprecation cycle.

[agramfort]

@adrinjalali travis is not happy.

[jnothman]

Thanks @adrinjalali

[adrinjalali]

I'm not sure what whats_new tags to use here though.

[jnothman]

I'm not sure what whats_new tags to use here though.

It's an enhancement, IMO.

[jnothman]

Nitpicks

[jnothman]

Tests failing?

[rth]

Looks good apart for one comment below.

[GaelVaroquaux]

LGTM, aside from my two comments. The naming is a nitpick which can be ignored, but I feel that the random_state is important.

[jnothman]

I can't see the difference between those images.

[azrdev]

I can't see the difference between those images.

@jnothman It's the area "inside" the intersection of the three borders, at around (1.5, 8) which has a concave extension of the top-left green class, iff tie breaking is used

[adrinjalali]

@jnothman I increased the plot size a bit, I hope that helps.

[NicolasHug]

I find it hard to see the difference between the two plots (took me more than a minute to see that the regions weren't the same at the center).

The description should explain what differences to look for IMO.

[adrinjalali]

@thomasjpfan I'm not sure why the two azure pipelines are failing, the failing test is not touched by this PR (and I can't figure what the error is anyway).

EDIT: is/was fixed on master

[adrinjalali]

@NicolasHug does this look good now?

[NicolasHug]

Looks good, a few comments still.

Not sure if we want to raise an error if break_ties is True and decision_function_shape is not 'ovr'. I'd say yes but apparently the default is subject to change?

[adrinjalali]

Not sure if we want to raise an error if break_ties is True and decision_function_shape is not 'ovr'. I'd say yes but apparently the default is subject to change?

I'm not sure. I'd leave that to another discussion maybe.

[NicolasHug]

I'm not sure. I'd leave that to another discussion maybe.

If we merge this before 0.21, it will be too late. I think we should raise an error, we've been very conservative so far on what users can do.

[adrinjalali]

@NicolasHug done :)

[NicolasHug]

Last comment, LGTM otherwise!

[NicolasHug]

Thanks Adrin!