For probabilistic scorers, LogisticRegressionCV(multi_class='multinomial') uses OvR to calculate scores #8720
Comments
Yes, that sounds like a bug. Thanks for the report. A fix and a test is welcome. |
Sounds good |
Yes, I thought I replied to this. A pull request is very welcome.
…On 12 April 2017 at 03:13, Tom Dupré la Tour ***@***.***> wrote:
It seems like altering L922 to read
log_reg = LogisticRegression(fit_intercept=fit_intercept,
multi_class=multi_class)
so that the LogisticRegression() instance supplied to the scoring function
at line 955 inherits the multi_class option specified in
LogisticRegressionCV() would be a fix, but I am not a coder and would
appreciate some expert insight!
Sounds good
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#8720 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz62ZEqTnYubanTrD-Xl7Elc40WtAsks5ru7TKgaJpZM4M2uJS>
.
|
I would like to investigate this. |
please do |
_log_reg_scoring_path: https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/logistic.py#L771 Constructor of LogisticRegression: _log_reg_scoring_path method: It can be seen that they have similar parameters with equal default values: penalty, dual, tol, intercept_scaling, class_weight, random_state, max_iter, multi_class, verbose; As @njiles suggested, adding multi_class as argument when creating logistic regression object, solves the problem for multi_class case. |
please submit a PR ideally with a test for correct behaviour of reach
parameter
…On 19 Apr 2017 8:39 am, "Shyngys Zhiyenbek" ***@***.***> wrote:
_log_reg_scoring_path: https://github.com/scikit-learn/scikit-learn/blob/
master/sklearn/linear_model/logistic.py#L771
It has a bunch of parameters which can be passed to logistic regression
constructor such as "penalty", "dual", "multi_class", etc., but to the
constructor passed only fit_intercept on https://github.com/scikit-
learn/scikit-learn/blob/master/sklearn/linear_model/logistic.py#L922.
Constructor of LogisticRegression:
def __init__(self, penalty='l2', dual=False, tol=1e-4, C=1.0,
fit_intercept=True, intercept_scaling=1, class_weight=None,
random_state=None, solver='liblinear', max_iter=100, multi_class='ovr',
verbose=0, warm_start=False, n_jobs=1)
_log_reg_scoring_path method:
def _log_reg_scoring_path(X, y, train, test, pos_class=None, Cs=10,
scoring=None, fit_intercept=False, max_iter=100, tol=1e-4,
class_weight=None, verbose=0, solver='lbfgs', penalty='l2', dual=False,
intercept_scaling=1., multi_class='ovr', random_state=None,
max_squared_sum=None, sample_weight=None)
It can be seen that they have similar parameters with equal default
values: penalty, dual, tol, intercept_scaling, class_weight, random_state,
max_iter, multi_class, verbose;
and two parameters with different default values: solver, fit_intercept.
As @njiles <https://github.com/njiles> suggested, adding multi_class as
argument when creating logistic regression object, solves the problem for
multi_class case.
After that, it seems like parameters from the list above should be passed
as arguments to logistic regression constructor.
After searching by patterns, I didn't find similar bug in other files.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#8720 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz677KSfKhFyvk7HMpAwlTosVNJp6Zks5rxTuWgaJpZM4M2uJS>
.
|
I would like to tackle this during the sprint Reading through the code, if I understand correctly Scoring indeed could call scikit-learn/sklearn/linear_model/logistic.py Lines 948 to 953 in 46913ad so I think we do not even need to set the To sum up if this is correct, as suggested by @njiles I would need to inheritate from the This would eventually amount to replace this line : scikit-learn/sklearn/linear_model/logistic.py Line 925 in 46913ad by this one log_reg = LogisticRegression(multi_class=multi_class) I am thinking about the testing now but I hope these first element are already correct |
Description:
For scorers such as
neg_log_loss
that use.predict_proba()
to get probability estimates out of a classifier, the predictions used to generate the scores forLogisticRegression(multi_class='multinomial')
do not seem to be the same predictions as those generated by the.predict_proba()
method ofLogisticRegressionCV(multi_class='multinomial')
. The former uses a single logistic function and normalises (one-v-rest approach), whereas the latter uses the softmax function (multinomial approach).This appears to be because the
LogisticRegression()
instance supplied to the scoring function at line 955 of logistic.py within the helper function_log_reg_scoring_path()
,(https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/logistic.py#L955)
scores.append(scoring(log_reg, X_test, y_test))
,is initialised,
(https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/logistic.py#L922)
log_reg = LogisticRegression(fit_intercept=fit_intercept)
,without a multi_class argument, and so takes the default, which is
multi_class='ovr'
.It seems like altering L922 to read
log_reg = LogisticRegression(fit_intercept=fit_intercept, multi_class=multi_class)
so that the
LogisticRegression()
instance supplied to the scoring function at line 955 inherits themulti_class
option specified inLogisticRegressionCV()
would be a fix, but I am not a coder and would appreciate some expert insight! Likewise, I do not know whether this issue exists for other classifiers/regressors, as I have only worked with Logistic Regression.Minimal example:
Versions:
Linux-4.4.0-72-generic-x86_64-with-Ubuntu-14.04-trusty
Python 2.7.6
NumPy 1.12.0
SciPy 0.18.1
Scikit-learn 0.18.1
The text was updated successfully, but these errors were encountered: