Clarify semantics of Model.predict_real

[lsc36]

Currently Model.predict_real is connected to predict_proba in scikit-learn, which returns an array of n_classes floats standing for probabilities of corresponding labels. But decision_function is another candidate whose returning shapes vary from model to model, for example (in our case n_samples = 1):

• LogisticRegression: (n_samples,) if n_classes == 2 else (n_samples, n_classes)
• C-SVC: (n_samples, n_classes * (n_classes-1) / 2)

We have to make sure what we want in order to well-define the interface. @hsuantien can you give us some advice on this?

[hsuantien]

I suggest that we consider the logistic regression type only. Technically,
the C-SVC one can be converted to the LogisticRegression one in some way.
Thanks.

--HT

SC Lee notifications@github.com 於 2015年12月17日 週四 下午9:22寫道：

Currently Model.predict_real is connected to predict_proba in
scikit-learn, which returns an array of n_classes floats standing for
probabilities of corresponding labels. But decision_function is another
candidate whose returning shapes vary from model to model, for example (in
our case n_samples = 1):

• LogisticRegression: (n_samples,) if n_classes == 2 else (n_samples,
  n_classes)
• C-SVC: (n_samples, n_classes * (n_classes-1) / 2)

We have to make sure what we want in order to well-define the interface.
@hsuantien https://github.com/hsuantien can you give us some advice on
this?

—
Reply to this email directly or view it on GitHub
#21.

[yangarbiter]

Actually the output of C-SVC differs with different multi class method (OVO, OVR).

[yangarbiter]

I tried to fix it in this branch https://github.com/ntucllab/libact/tree/predict_real_interface

Though I am not entirely sure the implementation of the largest margin method for now.

[lsc36]

We should determine the interface before writing code. Is the "LogReg-style conversion" generally applicable?

[yangarbiter]

For binary classification case, svm and logReg-style are able to convert.

For multiclass case logReg-style supports only OVR method for SVM, but not OVO (it seems sklearn's logReg didn't support OVO).

As for other classifier, we might have to discuss case by case.

[hsuantien]

Let's use OVR-style for the interface now, I suggest. Thanks.

On Fri, Dec 18, 2015 at 5:13 PM, yangarbiter notifications@github.com
wrote:

For binary classification case, svm and logReg-style are able to convert.

For multiclass case logReg-style supports only OVR method for SVM, but not
OVO (it seems sklearn's logReg didn't support OVO).

As for other classifier, we might have to discuss case by case.

—
Reply to this email directly or view it on GitHub
#21 (comment).

---

Hsuan-Tien Lin htlin@csie.ntu.edu.tw

http://www.csie.ntu.edu.tw/~htlin

Associate Professor
Dept. of Computer Science and Information Engineering
& Graduate Institute of Networking and Multimedia

National Taiwan University

[yangarbiter]

I think for now we can make predict_real output ndarray with shape (n_sample, n_classes) (even n_classes=2)

but another thing might be defining the meaning of predict_real. For LogisticRegression and SVM like algorithm, their value may be more positive more towards label 1 and negative towards label -1.

How about other algorithms? Will they always be in this case?
@hsuantien

[lsc36]

Consider as solved. Closing.