Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: attach Null model params #6229

Open
josef-pkt opened this issue Nov 9, 2019 · 1 comment
Open

ENH: attach Null model params #6229

josef-pkt opened this issue Nov 9, 2019 · 1 comment

Comments

@josef-pkt
Copy link
Member

josef-pkt commented Nov 9, 2019

We had question about computing r-squared and loglike for out-of-sample data for cross-validation. (I don't find any related issues right now)

Problem is that r-squared requires assumption on the intercept, or more generally params of the null model.

I think now that we can get a consistent interpretation of R-squared and similar measures if we evaluate the new exog at the null model for comparison. To be able to do that, we need to keep the null model or at least some sufficient statistics like params around.

In the simple OLS rsquared case, this would be using the intercept estimate from the training/estimation data, i.e. model.endog.mean(), or zero if no constant has been added.

Some statistics are in tools.eval_measures

@josef-pkt
Copy link
Member Author

maybe a useful example, I didn't look at the details
negative rsquared for out-of-sample data
https://stackoverflow.com/questions/59919285/i-obtain-negative-r2-regardless-of-what-regression-model-i-use

my guess is it uses the test data mean in the out of sample computation, but I never looked at the sklearn code for it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant