New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bayesian ridge regression - alpha and lambda values don't correspond to calculated coef #8224

Closed
gedeck opened this Issue Jan 23, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@gedeck
Contributor

gedeck commented Jan 23, 2017

The coefficients coef_ are not calculated based on the alpha_ and lambda_ value returned in in the Bayesian Ridge regression model as these two properties are modified after the coef_ calculation. This can be seen in this reduced excerpt from the code in sklearn/linear_model/bayes.py.

        # Convergence loop of the bayesian ridge regression
        for iter_ in range(self.n_iter):
            # Compute mu and sigma
            # sigma_ = lambda_ / alpha_ * np.eye(n_features) + np.dot(X.T, X)
            # coef_ = sigma_^-1 * XT * y
            ** lambda_ and alpha_ are used to calculate coef_**
            coef_ = ...
            ...

            # Update alpha and lambda
            ...
            **lambda_ and alpha_ are modified**
            lambda_ = ((gamma_ + 2 * lambda_1) /
                       (np.sum(coef_ ** 2) + 2 * lambda_2))
            alpha_ = ((n_samples - gamma_ + 2 * alpha_1) /
                      (rmse_ + 2 * alpha_2))

            # Compute the objective function
            ...

            # Check for convergence
            ...

        self.alpha_ = alpha_
        self.lambda_ = lambda_
        self.coef_ = coef_

A possible solution would be keeping a copy of alpha_ and lambda_ at the point of calculation of coef_ and assign these copies to self.alpha_ and self.lambda_. Alternatively, move the assignment to self.alpha_ and self.lambda_ to the point of calculation of coef_.

@amueller

This comment has been minimized.

Show comment
Hide comment
@amueller

amueller Mar 4, 2017

Member

Thanks. Do you want to provide a regression test and a fix?

Member

amueller commented Mar 4, 2017

Thanks. Do you want to provide a regression test and a fix?

@gedeck

This comment has been minimized.

Show comment
Hide comment
@gedeck

gedeck Mar 4, 2017

Contributor

Yes, I'm happy to do this. It's straightforward.

Contributor

gedeck commented Mar 4, 2017

Yes, I'm happy to do this. It's straightforward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment