Add method to find probability for each class in case of multi-class classification

[amansrivastava17]

Added feature to get the confidence score of each class in case of Multi-class text classification
Added new param multi_class_prob to function predict to return the confidence score of all classes instead of just returning class with the highest confidence.

Example to use this method

>>> from flair.models import TextClassifier
>>> from flair.data import Sentence

# Loading English pretrained sentiment classification model
>>> classifier = TextClassifier.load('en-sentiment')

>>> sentence = Sentence('I have added a new feature in flair')
>>> classifier.predict(sentence)
>>> print('Sentiment Score: ', sentence.labels)
Sentiment Score:  [POSITIVE (0.9674275517463684)]

# Now if you want confidence score for both the class, you need to pass `multi_class_prob` as `True` to predict method

>>> classifier.predict(sentence, multi_class_prob=True)
>>> print('Sentiment Score: ', sentence.labels)
Sentiment Score:  [POSITIVE (0.9674275517463684), NEGATIVE (0.032572414726018906)]

[amansrivastava17]

@alanakbik @tabergma @kashif @pranjalsrajput @khituras @stefan-it
Please have a look

[amansrivastava17]

@alanakbik Please have a look.

[alanakbik]

@amansrivastava17 thanks for adding this - lots of people will find this useful. Just to clarify: this only modifies the private function _obtain_labels so most users that just use the predict function will not be able to set a flag to get these probabilities, correct? Could you paste a quick code example of how to use this functionality?

[amansrivastava17]

@alanakbik I have added code usage in the description above. You were right, the user who uses just predict function will not be able to use this feature, for this, I have added one multi_class_prob, which when set as True will return probability for each class.

[amansrivastava17]

@stefan-it Please have a look

[stefan-it]

Looks good so far, I'll add a review now :)

[stefan-it]

It would be awesome if you could also add a relevant integration test for that new method.

For example you could build such an integration test on-top of the imdb test:

https://github.com/zalandoresearch/flair/blob/master/tests/test_model_integration.py#L378

[amansrivastava17]

@stefan-it Hi, I have added integration test for new method

Please check here
https://github.com/zalandoresearch/flair/blob/66c6cb9af06dae4e67ffbab58eca6f10bb4766ed/tests/test_model_integration.py#L416

[alanakbik]

@amansrivastava17 it looks like the new integration test is throwing an error. Could you check? In particular, it looks like you are using the method load_from_file in the test, which you should replace with the load method.

[amansrivastava17]

@alanakbik Have fixed integration issue.
Please have a look.

[alanakbik]

Looks good, thanks!

[alanakbik]

👍

[kashif]

👍

[amansrivastava17]

@stefan-it have made changes, Please approve.

[stefan-it]

Thanks for adding this feature :)

[Rajat-Mehta]

hi @stefan-it, @amansrivastava17,

Is this feature already available to be used in flair?

I tried but failed, it says: "TypeError: predict() got an unexpected keyword argument 'multi_class_prob'".

I am using flair version 0.4.1.

Thanks

[alanakbik]

It's merged into the master branch and so will be part of upcoming v0.4.2, but currently the only way to use the feature is to use the master branch.

[tsu3010]

Hi @alanakbik, When is v0.4.2 scheduled for release?

[alanakbik]

I am hoping to get v0.4.2 out before NAACL, i.e. end of next week.