Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bayesian ridge regression - alpha and lambda values don't correspond to calculated coef #8224

Closed
gedeck opened this issue Jan 23, 2017 · 2 comments
Labels
Bug

Comments

@gedeck
Copy link
Contributor

@gedeck gedeck commented Jan 23, 2017

The coefficients coef_ are not calculated based on the alpha_ and lambda_ value returned in in the Bayesian Ridge regression model as these two properties are modified after the coef_ calculation. This can be seen in this reduced excerpt from the code in sklearn/linear_model/bayes.py.

        # Convergence loop of the bayesian ridge regression
        for iter_ in range(self.n_iter):
            # Compute mu and sigma
            # sigma_ = lambda_ / alpha_ * np.eye(n_features) + np.dot(X.T, X)
            # coef_ = sigma_^-1 * XT * y
            ** lambda_ and alpha_ are used to calculate coef_**
            coef_ = ...
            ...

            # Update alpha and lambda
            ...
            **lambda_ and alpha_ are modified**
            lambda_ = ((gamma_ + 2 * lambda_1) /
                       (np.sum(coef_ ** 2) + 2 * lambda_2))
            alpha_ = ((n_samples - gamma_ + 2 * alpha_1) /
                      (rmse_ + 2 * alpha_2))

            # Compute the objective function
            ...

            # Check for convergence
            ...

        self.alpha_ = alpha_
        self.lambda_ = lambda_
        self.coef_ = coef_

A possible solution would be keeping a copy of alpha_ and lambda_ at the point of calculation of coef_ and assign these copies to self.alpha_ and self.lambda_. Alternatively, move the assignment to self.alpha_ and self.lambda_ to the point of calculation of coef_.

@amueller
Copy link
Member

@amueller amueller commented Mar 4, 2017

Thanks. Do you want to provide a regression test and a fix?

@gedeck
Copy link
Contributor Author

@gedeck gedeck commented Mar 4, 2017

Yes, I'm happy to do this. It's straightforward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

2 participants
You can’t perform that action at this time.