RFE: more average_method's for normalized_mutual_info_score

[sam-s]

Description

It would be nice if normalized_mutual_info_score could compute uncertainty coefficient, AKA proficiency.

Implementation

E.g., average_method can be labels_true to compute proficiency.

Expected Results

normalized_mutual_info_score(
    labels_true=[0,1,0,1], 
    labels_pred=[0,1,0,2], 
    average_method='labels_true')

should evaluate to 1.

[aishgrt1]

@jnothman Shall I take this?

[jnothman]

It seems reasonable enough given that we are supporting various normalizers, and seeing as this makes the normalization independent of labels_pred. @sam-s, is this used often in practice for clustering evaluation?

[sam-s]

I use it all the time.
The proficiency metric measures the proportion of information in the target distribution recovered by the classifier, so, unlike all the other normalizations, it has a clear interpretable meaning.

[sam-s]

BTW, it would be nice to be able to specify the average_method for scoring in cross_validate. Something like 'normalized_mutual_info_score/average_method=labels_true' instead of make_scorer(lambda y,yh: normalized_mutual_info_score(y,yh,average_method='labels_true')).

[amueller]

make_scorer(normalized_mutual_info_score, average_method='labels_true') works, right?
we could add a named scorer, we have done that for the different precision/recall averages.

[sam-s]

@amueller right, thanks, missed that.