proportion_confint beta fails.

[vvoracek]

Describe the bug

statsmodels.stats.proportion.proportion_confint is failing when 0 successes and very small alph.

Code Sample, a copy-pastable example if possible

In the following example one can see that 0/1000 is problematic while 1/1000 is ok. It also shows that for 1000/1000 there is no problem. The last line is the exact computation.

# Your code here that produces the bug
>>> from statsmodels.stats.proportion import proportion_confint
>>> proportion_confint(0,1000,method='beta', alpha=10**-7)
(0, 0.9999998642895137)
>>> proportion_confint(1,1000,method='beta', alpha=10**-7)
(5.0000001248700306e-11, 0.019662593716583528)
>>> 1-proportion_confint(1000,1000,method='beta', alpha=10**-7)[0]
0.0166707224315088
>>> 1-(10**-7/2)**(1/1000)
0.01667072243152068

I suggest to add the following logic to the function which solves the problematic case.

if(count_a == 0):
    return 1-alpha_2**(1/nobs) 

Expected Output

I'm new to this and I'm willing to implement the posted fix, so I ask for approval if this makes sense.

Output of import statsmodels.api as sm; sm.show_versions()

import statsmodels.api as sm; sm.show_versions()

INSTALLED VERSIONS

Python: 3.8.8.final.0
OS: Linux 5.14.0-1054-oem #61-Ubuntu SMP Fri Oct 14 13:05:50 UTC 2022 x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

statsmodels

Installed: 0.12.2 (/home/vasik/anaconda3/lib/python3.8/site-packages/statsmodels)

Required Dependencies

cython: 0.29.23 (/home/vasik/anaconda3/lib/python3.8/site-packages/Cython)
numpy: 1.22.4 (/home/vasik/anaconda3/lib/python3.8/site-packages/numpy)
scipy: 1.6.2 (/home/vasik/anaconda3/lib/python3.8/site-packages/scipy)
pandas: 1.2.4 (/home/vasik/anaconda3/lib/python3.8/site-packages/pandas)
dateutil: 2.8.2 (/home/vasik/anaconda3/lib/python3.8/site-packages/dateutil)
patsy: 0.5.1 (/home/vasik/anaconda3/lib/python3.8/site-packages/patsy)

Optional Dependencies

matplotlib: 3.5.2 (/home/vasik/anaconda3/lib/python3.8/site-packages/matplotlib)
backend: QtAgg
cvxopt: Not installed
joblib: 1.0.1 (/home/vasik/anaconda3/lib/python3.8/site-packages/joblib)

Developer Tools

IPython: 7.22.0 (/home/vasik/anaconda3/lib/python3.8/site-packages/IPython)
jinja2: Not installed
sphinx: 4.0.1 (/home/vasik/anaconda3/lib/python3.8/site-packages/sphinx)
pygments: 2.16.1 (/home/vasik/anaconda3/lib/python3.8/site-packages/pygments)
pytest: 6.2.3 (/home/vasik/anaconda3/lib/python3.8/site-packages/pytest)
virtualenv: Not installed

[josef-pkt]

either your statsmodels or your scipy version is too old

with current statsmodels (master, PR version) and scipy 1.7. 3, I get the correct

proportion_confint(0,1000,method='beta', alpha=10**-7)
(0.0, 0.016670722431520707)

method beta is essentially two calls to scipy.stats.beta

[josef-pkt]

and there is continuity close to 1:

proportion_confint(1e-10,1000,method='beta', alpha=10**-7)
(0.0, 0.016670722431861334)

[vvoracek]

Thanks, updating scipy 1.6.2 -> 1.7.2 solved the problem; sorry for bothering you.