Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: perfect prediction check in DiscreteModel only applies to binary? no offset #8477

Open
josef-pkt opened this issue Oct 28, 2022 · 2 comments

Comments

@josef-pkt
Copy link
Member

while browsing and skimming some code

The perfect prediction check in DiscreteModel ignores offset (new enhancement) and code uses properties that apply only for binary models, Logit and Probit.
If we would check for perfect prediction in countmodels like Poisson, then we would also need to include exposure.

    def _check_perfect_pred(self, params, *args):
        endog = self.endog
        fittedvalues = self.cdf(np.dot(self.exog, params[:self.exog.shape[1]]))
        if (self.raise_on_perfect_prediction and
                np.allclose(fittedvalues - endog, 0)):
            msg = "Perfect separation detected, results not available"
            raise PerfectSeparationError(msg)

I guess currently this method is only called by binary models.

@josef-pkt
Copy link
Member Author

simple fix is to replace the explicit computation by call to predict

@josef-pkt josef-pkt modified the milestones: 0.15, 0.14 Feb 11, 2023
@josef-pkt
Copy link
Member Author

josef-pkt commented Feb 11, 2023

bump, this is still a bug, now that we have offset in discrete binary models

josef-pkt added a commit to josef-pkt/statsmodels that referenced this issue Feb 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant