Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

genmod Links numerical improvements #2024

Closed
josef-pkt opened this issue Oct 3, 2014 · 2 comments

Comments

Projects
None yet
3 participants
@josef-pkt
Copy link
Member

commented Oct 3, 2014

reported by Brent Pedersen on mailing list

In doing so, I noticed that using families.links.Logit.inverse gives
nan with these Z's:

z = [1123.4910007309222, 1483.952316802719, 1344.86033748641,
706.339159002542, 1167.9986375146532, 663.8345826933115,
1496.3691686913917, 1563.0763842182257, 1587.4309332296314,
697.1173174974248, 1333.7256198289665, 1388.7667560586933,
819.7605431778434, 1479.9204150555015, 1078.5642245164856,
480.10338454985896, 1112.691659145772, 534.1061908007274,
918.2011296406588, 1280.8808515887802, 758.3890788775948,
673.503699841035, 1556.7043357878208, 819.5269028006679,
1262.5711060356423, 1098.7271535253608, 1482.811928490097,
796.198809756532, 893.7946963941745, 470.3304989319786,
1427.77079226037, 1365.2050226373822, 1492.4193201661922,
871.9922191949931, 768.4735925445908, 732.9222777654679,
812.2382651982667, 495.06449978924525]

If I use: 1 / (1. + np.exp(-z))
as the inverse, it works fine.

there are a few more places similar to t/(1+t), for example logit deriv2 and negative binomial link

Related: scipy.special has now some new special functions for these kinds of expressions with better corner case behavior. (several changes in scipy.stats benefit currently from improvements.)

@kshedden

This comment has been minimized.

Copy link
Contributor

commented Oct 5, 2014

Improvements here would be helpful. I have been working with a Poisson regression in which 40% of the deviance residuals are NaN.

@josef-pkt

This comment has been minimized.

Copy link
Member Author

commented Oct 5, 2014

About Poisson deviance residuals:
scipy.special got a xlogy function that handles 0*log(0) I don't remember which version it was added.

Poisson.deviance has a specific code for the endog==0 case, but Poisson.resid_dev doesn't

@jseabold jseabold added this to the 0.6 milestone Oct 6, 2014

josef-pkt added a commit to josef-pkt/statsmodels that referenced this issue Jan 31, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.